WO2017208619A1

WO2017208619A1 - Information processing device, information processing method, and program

Info

Publication number: WO2017208619A1
Application number: PCT/JP2017/014461
Authority: WO
Inventors: 麻紀井元; 健太郎井田; 拓也池田; 陽方川名; 龍一鈴木
Original assignee: ソニー株式会社
Priority date: 2016-05-31
Filing date: 2017-04-07
Publication date: 2017-12-07

Abstract

[Problem] To provide a device with which a user can cause the intended processing to be executed in the device, and the burden on the user related to operating the device can be reduced. [Solution] Provided is an information processing device which is provided with: a processing unit which executes processing on the basis of movement information related to an operation body; and a control unit which, on the basis of a first direction determined on the basis of the positional change in the movement of the operation body, and the movement path length of the operation body, controls the movement information used in the processing. Also provided is an information processing method which includes: a step in which a processor is used to execute processing on the basis of movement information related to an operation body; and a step in which the movement information used in the processing is controlled on the basis of a first direction determined on the basis of the positional change in the movement of the operation body, and the movement path length of the operation body. Furthermore, provided is a program for causing a computer to implement the function of the information processing device.

Description

Information processing apparatus, information processing method, and program

This disclosure relates to an information processing apparatus, an information processing method, and a program.

In recent years, with the development of information processing technology, various technologies have been researched and developed for input for operating the device. Specifically, there is a technique related to input in a two-dimensional space using a touch screen or the like, or a technique related to input in a three-dimensional space based on a recognized user gesture.

Here, the operation intended by the user may not be recognized by the device. For example, if the detected gesture does not match the gesture associated with the process, the detected gesture is not recognized as an operation. As a result, the processing desired by the user is not executed.

On the other hand, in Patent Document 1, when a gesture is detected and the detected gesture is an unrecognizable gesture, the invention relates to an information processing terminal that provides an interface for associating the detected gesture with a process to the user. Is disclosed. According to the present invention, a gesture that could not be recognized can be corrected to a recognizable gesture.

JP 2012-256099 A

However, it is desired that the burden on the operation of the apparatus is further reduced. For example, in Patent Literature 1, a user's gesture and a process corresponding to the gesture are associated with each other by the user, so that the process intended by the user can be reliably executed based on the gesture. On the other hand, since the user performs the association, a burden is generated in order for the apparatus to recognize the gesture.

Therefore, the present disclosure proposes a mechanism capable of causing the apparatus to execute processing intended by the user and reducing the burden on the user for operating the apparatus.

According to the present disclosure, the processing unit that performs processing based on movement information of the operating tool, the first direction determined based on a change in position in the movement of the operating tool, and the movement path length of the operating tool, And a control unit that controls movement information used in the processing based on the information processing apparatus.

In addition, according to the present disclosure, using a processor, processing is performed based on movement information of the operating body, and the first direction determined based on a change in position in the movement of the operating body and the operation There is provided an information processing method including controlling movement information used in the processing based on a movement path length of a body.

In addition, according to the present disclosure, a processing function for executing processing based on movement information of the operating body, a first direction determined based on a change in position in the movement of the operating body, and a movement path of the operating body A program for causing a computer to realize a control function for controlling movement information used in the processing based on the length is provided.

As described above, according to the present disclosure, a mechanism capable of causing the apparatus to execute a process intended by the user and reducing the burden on the user for operating the apparatus is provided. Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is a figure showing an example of composition of an information processing system concerning one embodiment of this indication. 2 is a block diagram schematically illustrating an example of a functional configuration of an information processing apparatus according to an embodiment of the present disclosure. FIG. 5 is a diagram for describing movement information correction processing in the information processing apparatus according to an embodiment of the present disclosure. FIG. It is a figure which shows the example of the change of the position of the operating body to move in the information processing apparatus which concerns on one Embodiment of this indication. 5 is a flowchart conceptually showing an example of overall processing of an information processing apparatus according to an embodiment of the present disclosure. 14 is a flowchart conceptually showing an example of details of correction processing of movement information in the information processing apparatus according to an embodiment of the present disclosure. It is a figure showing the 1st example of application of correction control of movement information by the information processor concerning the 1st modification of one embodiment of this indication. It is a figure which shows the 2nd application example of correction | amendment control of the movement information by the information processing apparatus which concerns on the 1st modification of one Embodiment of this indication. It is a figure showing the 3rd example of application of amendment control of movement information by the information processor concerning the 1st modification of one embodiment of this indication. It is a figure showing the 4th example of application of amendment control of movement information by the information processor concerning the 1st modification of one embodiment of this indication. It is a figure showing the 5th example of amendment control of movement information by the information processor concerning the 1st modification of one embodiment of this indication. 14 is a diagram for describing an example of correction control in an information processing device according to a second modification of an embodiment of the present disclosure. FIG. 14 is a flowchart conceptually showing an example of overall processing of an information processing apparatus according to a third modification of an embodiment of the present disclosure. It is a figure which shows the example of the choice of the presence or absence of correction | amendment of the movement information which a display is controlled by the information processing apparatus which concerns on the 4th modification of one Embodiment of this indication. 16 is a diagram for describing an example in which a preview indicating movement information after correction is displayed as an option object in the information processing apparatus according to the fourth modification of an embodiment of the present disclosure. FIG. It is a figure showing an example of mode control of a choice object in an information processor concerning a 4th modification of one embodiment of this indication. 16 is a flowchart conceptually showing an example of details of correction processing of movement information in an information processing apparatus according to a fifth modification of an embodiment of the present disclosure. 16 is a flowchart conceptually showing an example of overall processing including movement information correction cancellation processing in an information processing apparatus according to a sixth modification of an embodiment of the present disclosure; FIG. 3 is an explanatory diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment of the present disclosure.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. One Embodiment of the Present Disclosure 1.1. System configuration 1.2. Configuration of apparatus 1.3. Device processing 1.4. Summary of one embodiment of the present disclosure 1.5. Modification 2 2. Hardware configuration of information processing apparatus according to an embodiment of the present disclosure; Conclusion

<1. One Embodiment of the Present Disclosure>
An information processing system according to an embodiment of the present disclosure and an information processing apparatus for realizing the information processing system will be described.

<1.1. System configuration>
First, a configuration of an information processing system according to an embodiment of the present disclosure will be described with reference to FIG. FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment of the present disclosure.

As shown in FIG. 1, the information processing system includes an information processing apparatus 100, a measurement apparatus 200, and a projection apparatus 300. The information processing apparatus 100, the measurement apparatus 200, and the projection apparatus 300 are connected for communication.

The information processing apparatus 100 controls the projection of the projection apparatus 300 using the measurement result of the measurement apparatus 200. Specifically, the information processing apparatus 100 recognizes the operating tool from the measurement result provided from the measurement apparatus 200. Then, the information processing apparatus 100 controls the mode of projection by the projection apparatus 300 based on the movement information about the recognized operating body. For example, the information processing apparatus 100 controls the projection position of the virtual object 20 to be projected on the projection apparatus 300 according to the movement of the user's hand measured by the measurement apparatus 200.

The measuring device 200 measures the situation around the measuring device 200. Specifically, the measuring apparatus 200 measures a phenomenon in which the position or state of an object (for example, a user or an operating body) existing around the measuring apparatus 200 is grasped. Then, the measuring apparatus 200 provides information obtained by the measurement (hereinafter also referred to as measurement information) to the information processing apparatus 100 as a measurement result. For example, the measurement apparatus 200 is a depth sensor, and a positional relationship between a body part (for example, a finger or a hand) of a user on which a marker is mounted and a surrounding object (that is, a three-dimensional space of the body part and the surrounding object). Position) can be measured. The measurement information may be 3D image information. Measurement device 200 may be an inertial sensor worn by a user or a ring-type or bracelet-type wearable device having the inertial sensor. Moreover, the measuring apparatus 200 may be installed in any way as long as an operation area can be secured. For example, the measuring device 200 may be provided on the floor in addition to the ceiling as shown in FIG.

Projection apparatus 300 projects an image based on instruction information from information processing apparatus 100. Specifically, the projection apparatus 300 projects an image provided from the information processing apparatus 100 onto a designated location. For example, the projection apparatus 300 projects the virtual object 20 onto the projection area 10 shown in FIG. In FIG. 1, the projection apparatus 300 is a 2D (Dimension) projector, but the projection apparatus 300 may be a 3D projector.

Here, in recent years, input devices that use the user's body as an operating body have been developed and are in widespread use. For example, there is an input device that is operated by directly touching a display screen such as a smartphone with a finger or the like, or an input device that is operated by performing a gesture such as a user waving an arm toward a projected screen. . These input devices have the advantage that the operation is intuitive and easy to use.

On the other hand, there is a risk that a user who operates such an input device is burdened with the operation. For example, when a user holds a smartphone with one hand and performs a scroll operation using the thumb of the hand holding the smartphone, even if the user intends to move the thumb linearly, the thumb actually does not arc. Sometimes it is drawn and moved. In addition, for example, when the user operates the apparatus by performing a linear gesture using a hand or the like on an image projected on a table or the like, the user moves linearly as the hand moves away from his / her body. It becomes difficult, and it moves like an arc around its own body. Similarly, when the user operates the device by performing a gesture in the air, the user becomes less likely to move linearly as the hand moves away from the user's body, and the arc-shaped movement centered on the user's body becomes difficult. Resulting in. This is considered to be due to a difference between the somatic sensation of a human hand or finger and the range of motion of an actual hand or finger. Therefore, the operation actually recognized by the apparatus does not match the operation intended by the user, and the user performs the operation again. Furthermore, there is a possibility that processing corresponding to an operation not intended by the user is executed. Further, when attempting an operation intended by the user, the user may be forced into an unreasonable posture, and there is a risk that the body may be burdened.

In contrast, there is a technique for preventing processing according to an operation that is not intended by the user. For example, when a scroll operation is recognized in a specific direction, there is a technique in which a scroll process is executed only in the specific direction and a scroll process in a direction other than the specific direction is not executed. According to the technique, scrolling in a direction not intended by the user is suppressed, and suppression of re-operation is expected.

However, the technology may still not reduce the burden on the user. For example, in this technique, since a user operation in a direction other than a specific direction related to the recognized scroll operation is ignored, only a part of the user operation amount is reflected in the scroll process. In other words, only the component in the specific direction related to the recognized scroll operation among the operation amount of the user is reflected in the scroll process. Therefore, the scroll amount is reduced compared to the actual user operation amount. As described above, when the processing amount with respect to the operation amount decreases, the operation amount increases even for an operation for executing the same processing, and as a result, the number of operations may increase. This may increase the operation burden. In recent years, since the size of smartphones and tablet terminals is increasing, the operation burden can be further increased in order to operate a wide display screen.

Therefore, in the present disclosure, an information processing system capable of causing a device to execute a process intended by a user and reducing a user's burden on the operation of the device, and an information processing device 100 for realizing the information processing system. Propose.

<1.2. Configuration of device>
Next, the configuration of the information processing apparatus 100 according to an embodiment of the present disclosure will be described with reference to FIG. FIG. 2 is a block diagram schematically illustrating an example of a functional configuration of the information processing apparatus 100 according to an embodiment of the present disclosure.

2, the information processing apparatus 100 includes a communication unit 102, a recognition unit 104, a correction control unit 106, and a processing control unit 108.

(Communication Department)
The communication unit 102 communicates with a device external to the information processing device 100. Specifically, the communication unit 102 receives a measurement result from the measurement apparatus 200 and transmits projection instruction information to the projection apparatus 300. For example, the communication unit 102 communicates with the measurement apparatus 200 and the projection apparatus 300 using a wired communication method. Note that the communication unit 102 may communicate using a wireless communication method.

(Recognition part)
The recognition unit 104 performs recognition processing based on the measurement result of the measurement device 200. Specifically, the recognition unit 104 recognizes the operating tool based on the measurement information received from the measurement device 200. The operation body includes a part of the user's body. For example, the recognition unit 104 recognizes the user's finger or hand based on the three-dimensional image information obtained from the measurement apparatus 200. Note that the operation body recognized by the recognition unit 104 may be a pen-type or stick-type operation device.

Also, the recognition unit 104 recognizes the position of the operating body. Specifically, the recognition unit 104 recognizes the position of the operation body in the operation area where the operation body is recognized. For example, the recognition unit 104 recognizes the position of the operation body in the three-dimensional space in the operation region based on the distance between the measurement apparatus 200 and the operation body measured by the measurement apparatus 200. Note that the position of the operating tool in the depth direction from the measuring apparatus 200 toward the operating tool may be recognized based on the recognized size of the operating tool. The recognizing unit 104 recognizes the movement of the operating body by recognizing the position of the operating body. For example, the recognizing unit 104 recognizes the presence / absence of movement of the operating body, the moving direction, the speed and acceleration of movement, the moving path, and the like based on the change in the position of the operating body.

(Correction control unit)
The correction control unit 106 controls correction for an input operation. Specifically, the correction control unit 106 controls movement information of the operating tool for an operation of moving the operating tool. More specifically, the correction control unit 106 functions as a control unit in a first direction (hereinafter, also referred to as an estimated movement direction) determined based on a change in position in the movement of the operating body, and the movement path of the operating body. Based on the length, the movement information used in the process executed based on the movement information of the operating body (hereinafter also referred to as an operation corresponding process) is controlled. For example, the movement information includes time-series discrete coordinate information or continuous movement route information. Further, the correction control unit 106 determines the control mode of the movement information based on the information related to the movement mode of the operating tool. The movement information correction will be described in detail with reference to FIG. FIG. 3 is a diagram for describing movement information correction processing in the information processing apparatus 100 according to an embodiment of the present disclosure.

(Determination of correction control mode)
The correction control unit 106 acquires movement information of the operating tool. For example, the correction control unit 106 acquires information related to the position of the user's hand as the moving operation body recognized by the recognition unit 104.

Next, the correction control unit 106 determines a movement information control mode based on a change in the movement direction of the operating tool. Specifically, the correction control unit 106 determines whether or not to correct the movement information based on a change in the movement direction of the operating tool. Furthermore, the moving direction of the operating tool will be described in detail with reference to FIG. FIG. 4 is a diagram illustrating an example of a change in the position of the moving operating body in the information processing apparatus 100 according to an embodiment of the present disclosure. FIG. 4 discretely shows changes in the hand position corresponding to the movement of the user's hand in FIG.

First, the correction control unit 106 estimates the movement path of the operating tool. For example, the correction control unit 106 acquires information identifying the operating body positions P50A to P50E as illustrated in FIG. Then, the correction control unit 106 calculates the movement path R60 of the operating body that passes through these positions P50A to P50E based on the information. In addition, when the information regarding a movement route is provided from the recognition unit 104, the processing may be omitted.

Next, the correction control unit 106 calculates a tangent for the position of the moving operating body. For example, the correction control unit 106 calculates a tangent to the calculated movement route R60 for each of the operating body positions P50A to P50E. It should be noted that the position of the operating body that is a tangent calculation target may be sprinkled.

Then, the correction control unit 106 determines whether or not the movement information is corrected based on the determination of whether the calculated tangent direction has changed. For example, as shown in FIG. 4, the correction control unit 106 determines that the movement information is to be corrected because the tangential direction for the position P50D is different from the tangential direction for the positions P50A to P50C.

(Execution of correction control)
When it is determined to correct the movement information, the correction control unit 106 calculates the estimated movement direction of the operating tool. Specifically, the correction control unit 106 determines the direction determined from the tangent to the position of the moving operating body as the estimated movement direction. More specifically, the correction control unit 106 determines a direction in which the operating body is moving in a direction parallel to the calculated tangent as an estimated movement direction. Specifically, the correction control unit 106 is a direction parallel to a tangent to the position of the operating body between the time when the operating body is recognized and after a predetermined period has elapsed, and the direction in which the operating body is moving. Is determined as the estimated moving direction. For example, the direction that is parallel to the tangent to the positions P50A to P50C as shown in FIG. 4 and that goes from the position P50A to P50C is determined as the estimated movement direction.

Note that the method of determining the estimated movement direction is not limited to the above method. For example, the correction control unit 106 may determine the direction parallel to the direction with the largest number of coincidence of the tangential directions with respect to the position of the operating body as the estimated movement direction. Specifically, the tangent directions are different at each of the positions P50A to P50C, the position P50D, and the position P50E as shown in FIG. 4, but the direction in which the tangent directions coincide at the three positions P50A to P50C is estimated movement. Determined in the direction.

Returning to the description of the movement information correction process with reference to FIG. 3, the correction control unit 106 calculates the movement path length of the operating tool. Specifically, the correction control unit 106 calculates the length of the movement path R60 of the operating tool as shown in FIG.

Next, the correction control unit 106 determines movement information related to movement in the estimated movement direction of a length corresponding to the movement path length of the operating body as movement information used in the operation handling process. For example, the correction control unit 106 determines the determined estimated movement direction as the movement direction of the operation, and determines the calculated movement path length as the operation amount. In other words, instead of the actual movement path R60 of the operating body, the movement path R70 as shown in FIG. 3 is used for the operation handling process.

(Processing control unit)
The processing control unit 108 controls the processing of the information processing apparatus 100 as a whole. Specifically, the process control unit 108 executes a process based on the movement information of the operating tool. More specifically, the processing control unit 108 performs display control processing based on movement information related to movement of the operating tool recognized by the recognition unit 104 or movement information of the operating tool corrected by the correction control unit 106. To do. For example, the process control unit 108 determines the operation direction and the operation amount in the process from the movement direction and the movement amount of the operating body. Then, the processing control unit 108 changes the projection position of the virtual object to be projected on the projection device 300 according to the determined operation direction and operation amount.

<1.3. Device processing>
Next, processing of the information processing apparatus 100 will be described.

(Overall processing)
First, the overall processing of the information processing apparatus 100 will be described with reference to FIG. FIG. 5 is a flowchart conceptually showing an example of overall processing of the information processing apparatus 100 according to an embodiment of the present disclosure.

The information processing apparatus 100 starts an application (step S401). Specifically, the process control unit 108 activates an application in response to a user application activation operation recognized by the recognition unit 104. Note that the application may be automatically started.

Next, the information processing apparatus 100 determines whether an end operation has been recognized (step S402). Specifically, the process control unit 108 determines whether the user operation recognized by the recognition unit 104 is an application end operation.

If it is determined that the end operation has not been recognized (step S402 / NO), the information processing apparatus 100 determines whether the movement of the operating tool has been recognized (step S403). Specifically, the correction control unit 106 determines whether the recognition unit 104 has recognized the movement of the operating tool.

If it is determined that the movement of the operating tool has been recognized (step S403 / YES), the information processing apparatus 100 determines whether the moving direction of the operating tool has changed (step S404). Specifically, when it is determined that the movement of the operating tool has been recognized by the recognition unit 104, the correction control unit 106 estimates the moving direction of the operating tool, and the estimated moving direction of the operating tool changes during the movement. Determine if you did.

If it is determined that the moving direction of the operating tool has changed (step S404 / YES), the information processing apparatus 100 determines the estimated moving direction based on the change in the position of the operating tool (step S405). Specifically, when it is determined that the movement direction of the operating tool has changed during movement, the correction control unit 106 determines the movement direction before the change as the estimated movement direction.

Next, the information processing apparatus 100 calculates the movement path length of the operating tool (step S406). Specifically, the correction control unit 106 calculates the length of the movement path estimated based on the recognized change in the position of the operating tool.

Next, the information processing apparatus 100 executes movement information correction control (step S407). Specifically, the correction control unit 106 corrects the movement direction and the movement amount related to the movement information based on the estimated movement direction and the movement path length.

Next, the information processing apparatus 100 executes processing based on the movement information (step S408). Specifically, the process control unit 108 executes projection control of the projection apparatus 300 based on the corrected movement direction and movement amount.

If it is determined that the end operation has been recognized (step S402 / YES), the information processing apparatus 100 ends the application (step S409) and ends the process. Hereinafter, the processes in steps S404 to S407 are collectively referred to as step S500A.

(Movement information correction process)
Next, details of the movement information correction process (step S500A) in the information processing apparatus 100 will be described with reference to FIG. FIG. 6 is a flowchart conceptually showing an example of details of the movement information correction processing in the information processing apparatus 100 according to an embodiment of the present disclosure.

The information processing apparatus 100 stores the position of the operating tool within a predetermined period (step S501). Specifically, the correction control unit 106 causes the storage unit to store information indicating the position of the operating tool during a period corresponding to a predetermined number of frames provided from the recognition unit 104.

Next, the information processing apparatus 100 calculates a tangent for the movement path of the operating body for each of the stored positions of the operating body (step S502). Specifically, the correction control unit 106 estimates the movement route based on the stored information indicating the position of the operating tool. Next, the correction control unit 106 calculates a tangent to the estimated movement path for each position of the operating tool.

Next, the information processing apparatus 100 determines whether the calculated direction of the tangent has changed (step S503). Specifically, the correction control unit 106 determines whether the slope of the tangent with respect to the position of the operating tool has changed by a predetermined value or more between the positions before and after the time series.

If it is determined that the direction of the tangent has changed (step S503 / YES), the information processing apparatus 100 determines the direction of the tangent before the direction is changed as the estimated movement direction (step S504). Specifically, when the correction control unit 106 determines that the inclination of the tangent has changed by a predetermined value or more, the correction control unit 106 is in a direction parallel to the tangent with respect to the position of the operating body before the inclination changes, The change direction of the position (that is, the movement direction) is determined as the estimated movement direction.

Further, the information processing apparatus 100 calculates the movement path length of the operating tool (step S505). Specifically, the correction control unit 106 calculates the length of the travel route estimated as described above.

Next, the information processing apparatus 100 corrects the movement direction related to the movement information to the estimated movement direction (step S506). Specifically, the correction control unit 106 changes the movement direction related to the movement information to the estimated movement direction determined from the movement direction related to the estimated movement route (that is, the actual movement direction).

Further, the information processing apparatus 100 corrects the movement amount related to the movement information to an amount corresponding to the calculated movement route length (step S507). Specifically, the correction control unit 106 changes the movement amount related to the movement information to a movement path length calculated from the length of the estimated movement path in a specific direction.

In the following, the processes of steps S501 to S505 are collectively referred to as step S600.

<1.4. Summary of Embodiment of Present Disclosure>
As described above, according to the embodiment of the present disclosure, the information processing apparatus 100 executes the process based on the movement information of the operating tool, and is determined based on the change in position in the movement of the operating tool. The movement information used in the process is controlled based on the direction and the movement path length of the operating tool.

Conventionally, it has been difficult for a user who operates an apparatus using an operating tool to perform an operation intended by the user in a range that exceeds the range of motion of the body having the operating tool. Therefore, there is a possibility that processing unintended by the user is executed. In addition, there is a technology that uses only a component in a specific direction among operations performed by the user for processing, but in this technology, the operation amount used for the processing is the actual user operation amount (that is, the movement amount of the operating body). It will be dull than. Therefore, the user may be forced to move the operating body for a longer time in order to perform a desired operation.

On the other hand, according to an embodiment of the present disclosure, the movement information used in the process is corrected to the movement information related to the movement of the operation tool intended by the user, so that the operation in a range beyond the user's range of motion is performed. Also for the above, it is possible to execute processing in accordance with the user's intention. Further, the amount of operation can be reduced by correcting the operation amount based on the movement path length. Therefore, it is possible to cause the apparatus to execute processing intended by the user and to reduce the burden on the user for operating the apparatus. Thereby, the user can perform a desired operation without changing the operation method or the operation posture of the operation tool. Therefore, the user can concentrate on the purpose of the operation without paying attention to the operation of the apparatus.

Further, the information processing apparatus 100 determines movement information related to movement in the first direction having a length corresponding to the movement path length as movement information used in the processing. For this reason, a process can be performed based on the length equivalent to the distance which the user actually moved the operation body. Therefore, it is possible to prevent a part of the user's operation from being reflected in the processing (that is, wasted), and it is possible to more reliably reduce the burden on the operation. Note that the length of the movement according to the movement information may be less than the movement path length or may be longer than the movement path length. Further, as will be described later, the length of the movement information may be controlled based on various information.

Further, the information processing apparatus 100 determines the control mode of the movement information based on the information related to the movement mode of the operating body. The movement information control mode includes the presence or absence of the movement information control. Here, it may not be preferable for the user to always perform the correction control for the movement information of the operating tool. For example, if the process related to the correction control is executed even in a situation where the user can operate correctly, the processing load of the apparatus increases unnecessarily. Thus, the processing load of the apparatus can be reduced by executing correction control according to how the operating tool is moved. In particular, from the movement mode of the operating body, it is easy to estimate an operation exceeding the range of motion of the body part where the user operates the operating body as described above. For example, it is easy to determine that correction of movement information is desired because the movement path changes in an arc shape. Therefore, the accuracy or adaptability of the correction control can be improved by determining the correction control mode based on the movement mode of the operating body.

Further, the movement mode of the operating body includes a change in the moving direction of the operating body. Here, in the operation beyond the movable range of the body part where the user manipulates the operating body as described above, the moving direction of the user operating body generally changes during the movement. Accordingly, the correction control can be effectively performed by executing the correction control when a change in the moving direction of the operating tool is detected.

Further, the first direction includes a direction determined from a tangent to the position of the moving operating body. For this reason, the moving direction of the operating body can be grasped more accurately. Therefore, it is possible to execute processing according to the user's intention by correcting the movement direction according to the movement information in a direction closer to the direction intended by the user.

<1.5. Modification>
The embodiment of the present disclosure has been described above. Note that an embodiment of the present disclosure is not limited to the above-described example. Hereinafter, first to seventh modifications of the embodiment of the present disclosure will be described.

(First modification)
As a first modification of an embodiment of the present disclosure, the information processing apparatus 100 may execute correction control for operations other than the gestures of waving hands in the air as described above. Examples in which the correction control is applied will be described with reference to FIGS.

FIG. 7 is a diagram illustrating a first application example of the movement information correction control by the information processing apparatus 100 according to the first modification example of the embodiment of the present disclosure. FIG. 7 shows an example in which the user performs an operation by moving the user's hand, which is the operating body, from the upper part of the thigh to the side part. The recognition unit 104 recognizes the user's hand 50 and the movement of the hand 50. The correction control unit 106 determines the presence / absence of correction based on the recognized change in the position of the hand 50. As shown in FIG. 7, since the hand 50 is moved in an arc as in the movement route R61, it is determined that correction control is performed here. Next, the correction control unit 106 determines an estimated movement direction for the hand 50 based on a change in the position of the hand 50, and calculates the length of the movement route R61. Then, the correction control unit 106 changes the movement direction related to the movement information of the operation tool to the determined estimated movement direction (that is, the direction of the movement route R71), and calculates the movement amount related to the movement information of the operation tool. The length is changed to the length of the route R61. In other words, the correction control unit 106 changes the movement information of the operating tool from the movement information related to the movement route R61 to the movement information related to the movement route R71.

FIG. 8 is a diagram illustrating a second application example of the movement information correction control by the information processing apparatus 100 according to the first modification example of the embodiment of the present disclosure. FIG. 8 shows an example in which the user performs an operation of tracing the palm of the left hand with the finger of the right hand. The recognition unit 104 recognizes the user's finger 51 and the movement of the finger 51. The correction control unit 106 determines the presence or absence of correction based on the recognized change in the position of the finger 51. As shown in FIG. 8, the finger 51 is moved in an arc as in the movement path R62, and therefore it is determined here that correction control is performed. Next, the correction control unit 106 determines an estimated movement direction for the finger 51 based on a change in the position of the finger 51, and calculates the length of the movement route R62. Then, the correction control unit 106 changes the movement direction related to the movement information of the operation tool to the determined estimated movement direction (that is, the direction of the movement route R72), and calculates the movement amount related to the movement information of the operation tool. The length is changed to the length of the route R62. In other words, the correction control unit 106 changes the movement information of the operating tool from the movement information related to the movement route R62 to the movement information related to the movement route R72.

FIG. 9 is a diagram illustrating a third application example of the movement information correction control by the information processing apparatus 100 according to the first modification example of the embodiment of the present disclosure. FIG. 9 shows an example in which the user performs an operation of tracing the forearm from the wrist of the right arm with the finger of the left hand. The recognition unit 104 recognizes the user's finger 51 and the movement of the finger 51. The correction control unit 106 determines the presence or absence of correction based on the recognized change in the position of the finger 51. As shown in FIG. 9, since the finger 51 is moved in an arc as in the movement path R63, it is determined here that correction control is performed. Next, the correction control unit 106 determines an estimated movement direction for the finger 51 based on a change in the position of the finger 51, and calculates the length of the movement route R63. Then, the correction control unit 106 changes the movement direction related to the movement information of the operation tool to the determined estimated movement direction (that is, the direction of the movement route R73), and calculates the movement amount related to the movement information of the operation tool. The length is changed to the length of the route R63. In other words, the correction control unit 106 changes the movement information of the operating tool from the movement information related to the movement route R63 to the movement information related to the movement route R73.

FIG. 10 is a diagram illustrating a fourth application example of the movement information correction control by the information processing apparatus 100 according to the first modification example of the embodiment of the present disclosure. FIG. 10 shows an example in which the user performs an operation to move the finger pointing at the projection area from left to right toward the projection area. The recognition unit 104 recognizes the user's finger 51 directed to the projection area 10 and the movement of the finger 51. The correction control unit 106 determines the presence or absence of correction based on the recognized change in the position of the finger 51. As shown in FIG. 10, the finger 51 is moved in an arc on the projection area 10 like a movement path R64, and therefore it is determined that correction control is performed here. Next, the correction control unit 106 determines an estimated movement direction for the finger 51 based on a change in the position of the finger 51, and calculates the length of the movement route R64. Then, the correction control unit 106 changes the movement direction related to the movement information of the operation tool to the determined estimated movement direction (that is, the direction of the movement route R74), and calculates the movement amount related to the movement information of the operation tool. The length is changed to the length of the route R64. In other words, the correction control unit 106 changes the movement information of the operating tool from the movement information related to the movement route R64 to the movement information related to the movement route R74. The change in the position of the operating tool may be a change in position estimated from the position of the operating tool. For example, the estimated movement direction may be determined based on a change in the position on the projection region 10 indicated by the user's finger 51 as shown in FIG. The estimated change in position may be a change in the position of the operation target operated by the operating tool.

FIG. 11 is a diagram illustrating a fifth application example of the movement information correction control by the information processing apparatus 100 according to the first modification example of the embodiment of the present disclosure. FIG. 11 shows an example in which the user performs an operation of sliding the thumb of the right hand holding a device having a touch screen such as a smartphone on the touch screen. The recognition unit 104 recognizes the user's finger 51 on the touch screen 30 and the movement of the finger 51. The correction control unit 106 determines the presence or absence of correction based on the recognized change in the position of the finger 51. As shown in FIG. 11, the finger 51 is moved on the touch screen 30 in an arc as in the movement path R <b> 65, and therefore it is determined that correction control is performed here. Next, the correction control unit 106 determines an estimated movement direction for the finger 51 based on a change in the position of the finger 51, and calculates the length of the movement route R65. Then, the correction control unit 106 changes the movement direction related to the movement information of the operating tool to the determined estimated moving direction (that is, the direction of the movement route R75), and calculates the movement amount related to the movement information of the operating tool. The length is changed to the length of the route R65. In other words, the correction control unit 106 changes the movement information of the operating tool from the movement information related to the movement route R65 to the movement information related to the movement route R75. Note that the correction control process of the information processing apparatus 100 may be applied to an apparatus in which a touch panel and a display unit are separately provided instead of an apparatus including a touch screen.

Thus, according to the first modification, the information processing apparatus 100 can be applied to various types of operations. Therefore, for various operations, it is possible to reduce a burden on operations while causing the apparatus to execute processing intended by the user.

(Second modification)
As a second modification of an embodiment of the present disclosure, the information processing apparatus 100 may determine the control mode based on the degree of change in the moving direction of the operating tool. Specifically, the correction control unit 106 determines to perform correction control when the degree of change in the moving direction of the operating tool is equal to or greater than a predetermined value. With reference to FIG. 12, the process of this modification is demonstrated in detail. FIG. 12 is a diagram for describing an example of correction control in the information processing apparatus 100 according to the second modification of an embodiment of the present disclosure.

The recognition unit 104 recognizes the movement of the operating body. For example, the recognition unit 104 recognizes the movement of the operating body by recognizing the positions P51A to P51D of the operating body that are moved so as to draw a circle as illustrated in FIG.

The correction control unit 106 calculates the amount of change in the movement direction related to the movement of the recognized operating tool. For example, the correction control unit 106 calculates a tangent to the movement route R66 for each of the operating body positions P51A to P51D as shown in FIG. Then, the correction control unit 106 calculates a difference in tangent slope between the calculated tangents.

Then, the correction control unit 106 determines a correction control mode based on whether the calculated change amount exceeds a predetermined value. For example, the correction control unit 106 determines that the correction control is to be executed when the difference in the slope of the tangent between the calculated tangents exceeds a predetermined value. The predetermined value includes a value determined from a difference in tangent slope between tangents. For example, the predetermined value may be an average value, a mode value, a median value, or the like of the tangent slope difference between the tangent lines. The predetermined value may be a value determined before the operation.

Thus, according to the second modification, the information processing apparatus 100 determines the control mode based on the degree of change in the moving direction of the operating tool. For this reason, it is possible to correct the movement information for an operation (for example, a rotation operation) in which the moving direction of the operating body changes by a predetermined degree. Therefore, it is possible to further reduce the operation burden on the user by increasing the variation of the operation to be corrected.

(Third Modification)
As a third modification example of an embodiment of the present disclosure, the information processing apparatus 100 may determine a movement information correction control mode using pattern matching of a movement mode of an operating tool. Specifically, the correction control unit 106 determines the movement information control mode based on a comparison between the movement mode of the operating tool and the pattern of the moving mode of the operating tool stored in advance. As the movement mode of the operating body for which the pattern is prepared, there are the speed of movement of the operating body, acceleration, or a moving path. For example, the recognition unit 104 performs matching between the movement mode of the operating tool and a movement mode pattern stored in advance. Then, the correction control unit 106 determines whether to control the movement information based on the pattern matching result of the recognition unit 104. Furthermore, the pattern of the movement mode of the operating body may be obtained by machine learning. For example, by accumulating the moving speed, acceleration, or moving path of the operating tool for an operation that is repeatedly performed, a pattern of the operating mode of the operating tool is derived.

Next, the processing of this modification will be described in detail with reference to FIG. FIG. 13 is a flowchart conceptually showing an example of overall processing of the information processing apparatus 100 according to the third modification example of the embodiment of the present disclosure. Note that description of processing that is substantially the same as the processing described above is omitted.

If it is determined that the end operation has not been recognized after the application is started (step S411) (step S412 / NO), the information processing apparatus 100 determines whether the movement of the operating tool has been recognized (step S413).

If it is determined that the movement of the operating tool has been recognized (step S413 / YES), the information processing apparatus 100 determines whether the movement mode of the operating tool matches a predetermined pattern (step S414). Specifically, the recognizing unit 104 performs matching with a learning pattern stored in advance for the recognized movement speed, acceleration, or movement path of the operating tool. Then, when the matching with the learning pattern is successful, the correction control unit 106 determines to execute the movement information correction process. On the other hand, if the matching with the learning pattern fails, it is determined not to execute the movement information correction process.

If it is determined that the movement mode of the operating body matches the predetermined pattern (step S414 / YES), after the movement information correction process is executed (steps S415 to S417), the process is performed based on the corrected movement information. This is executed (step S418).

As described above, according to the third modification, the information processing apparatus 100 determines the control mode of the movement information based on the comparison between the movement mode of the operating tool and the movement mode pattern stored in advance. . For this reason, it is possible to improve the certainty that the correction process of the movement information is performed when the movement that is the mode of the pattern in which the correction of the movement information is desired. Therefore, it becomes easy for the user to learn a pattern to be corrected, and usability can be improved.

Further, the movement mode of the operating body includes the speed, acceleration, or moving path of the operating body. Here, the higher the speed or acceleration of movement of the operating body, the more likely the movement of the operation is distorted. Therefore, by controlling the presence / absence of the correction process based on the speed or acceleration of the movement of the operating tool, the correction process can be more reliably executed when correction is desired. Further, by performing pattern matching on the movement path, it is possible to execute the correction process when correction is desired more reliably than the speed or acceleration of movement.

Further, the pattern of the movement mode of the operating body is obtained by machine learning of the movement mode of the operating body. Here, the operation of the operation body generally differs among individual users. Therefore, a machine-learned pattern for each user is used for the determination process of whether or not correction is performed, so that a correction process suitable for each user can be provided. A learning pattern may be prepared for each individual user, and a learning pattern may be prepared for each user attribute.

(Fourth modification)
As a fourth modification example of the embodiment of the present disclosure, whether or not the movement information is corrected may be selected by the user. Specifically, the correction control unit 106 determines the control mode of the movement information based on the information related to the selection operation of the control mode of the movement information by the subject of the operation using the operation tool. For example, the process control unit 108 causes the display device to display an option (hereinafter also referred to as an option object) for selecting whether or not to correct the movement information. Then, the correction control unit 106 determines whether or not to correct the movement information based on recognition of an operation for selecting the displayed option. Furthermore, with reference to FIG. 14, the process of this modification is demonstrated in detail. FIG. 14 is a diagram illustrating an example of options for whether or not to correct the movement information whose display is controlled by the information processing apparatus 100 according to the fourth modification example of the embodiment of the present disclosure.

The correction control unit 106 determines whether or not the movement information of the operating tool is corrected. For example, the correction control unit 106 determines the presence / absence of correction based on the movement mode of the operating tool as described above.

When it is determined that the movement information of the operating tool is to be corrected, the correction control unit 106 causes the processing control unit 108 to display an option object. For example, when the operation object 21 projected to a position on the projection area corresponding to the position of the operation body is moved as shown in the movement path R67 as shown in FIG. 14, the correction control unit 106 executes correction processing. It is determined. Then, the processing control unit 108 causes the projection device 300 to project the option object 40A that selects correction and the option object 40B that selects not correction. Here, the position where the choice objects 40A and 40B are projected may be a position adjacent to the projection position of the operation object 21. For example, the choice objects 40 </ b> A and 40 </ b> B are positions adjacent to the projected operation object 21 and are projected on an extension line of the movement path of the operation object 21.

Next, the correction control unit 106 controls whether or not the correction process is executed based on the selection result of the choice object. For example, when the recognition unit 104 recognizes an operation for selecting the choice object 40A, the correction control unit 106 performs a correction process. On the contrary, when an operation for selecting the option object 40B is recognized, the correction process is not executed. The choice object is selected by an operation body (for example, the left hand) different from the operation body (for example, the right hand) that is operating the operation object. Further, the choice object may be selected according to the number of fingers operating the operation object. The selection of whether or not the movement information is corrected may be selected by another selection method. For example, the option may be selected by voice input, and the option may be selected based on the line of sight of the user who performs the operation.

Note that the displayed option object may be a preview showing the corrected movement information. With reference to FIG. 15, a case where the option object is a preview indicating the corrected movement information will be described. FIG. 15 is a diagram for describing an example in which a preview indicating movement information after correction is displayed as an option object in the information processing apparatus 100 according to the fourth modification of an embodiment of the present disclosure.

When it is determined that the movement information of the operating tool is to be corrected, the correction control unit 106 calculates the movement direction related to the corrected movement information and the movement direction related to the movement information before correction. For example, the correction control unit 106 calculates the estimated movement direction based on the movement information before correction. Further, the correction control unit 106 is also referred to as a movement direction (hereinafter, also referred to as the latest movement direction) estimated from a position after it is determined that the movement direction among the positions related to the movement information before correction has changed in time series. ) Is calculated.

Next, the process control unit 108 causes the projection apparatus 300 to project the option object indicating the movement direction related to the calculated corrected movement information and the option object indicating the movement direction related to the calculated movement information before correction. For example, the processing control unit 108 causes the projection device 300 to project the option object 41A indicating the estimated movement direction and the option object 41B indicating the latest movement direction calculated by the correction control unit 106.

Further, the correction control unit 106 may control the mode of options. Specifically, the correction control unit 106 controls the mode of options displayed based on the evaluation information regarding the control of movement information before execution. With reference to FIG. 16, the control of the choice object mode will be described in detail. FIG. 16 is a diagram illustrating an example of the choice object mode control in the information processing apparatus 100 according to the fourth modification example of the embodiment of the present disclosure.

The correction control unit 106 determines whether or not the movement information of the operation tool is corrected based on the movement mode of the operation tool, and calculates an estimated correction proposal level. The proposal level of correction may be an index indicating the appropriateness (in other words, reliability) of the correction to be performed with respect to the correction estimated to be desired by the user. Specifically, the correction control unit 106 calculates a correction proposal level according to the degree of change in the movement direction. For example, when the movement path of the operating body is a meandering movement path R68 as shown in FIG. 16, the correction control unit 106 reduces the degree of change in the movement direction, so the correction proposal level is the meandering movement path. Without being calculated as compared with the case where the moving direction changes. Note that the suggested level of correction may be calculated by an external device.

Next, when it is determined that the movement information of the operating tool is to be corrected, the correction control unit 106 calculates the movement direction related to the corrected movement information and the movement direction related to the movement information before correction.

Then, the processing control unit 108 causes the projection apparatus 300 to project the option object indicating the movement direction related to the corrected movement information and the option object indicating the movement direction related to the movement information before correction in a manner corresponding to the proposal level. For example, the processing control unit 108 causes the projection device 300 to project the option object 42A indicating the estimated movement direction whose mode has been changed according to the proposal level and the option object 42B indicating the latest movement direction. Here, since the proposal level is low, the outlines and transparency of the choice objects 42A and 42B are made thinner and thinner than the choice objects 41A and 41B shown in FIG. 15, and the presence is weakened. Note that the mode of the option object to be controlled may be another visual mode such as the color, shading, or luminance of the option object.

As described above, according to the fourth modification, the information processing apparatus 100 determines the movement information control mode based on the information related to the selection operation of the movement information control mode performed by the subject of the operation using the operating tool. . Here, even when the user desires to input the operation that is the correction control target as it is, if the movement information is corrected, processing that does not conform to the user's intention may be executed. On the other hand, whether the correction of the movement information is selected by the user can control whether the correction process is performed according to the user's intention. Therefore, it is possible to prevent the processing based on the movement information corrected against the user's intention from being executed.

Also, the selection operation includes an operation for selecting a displayed option. For this reason, when the option is visually presented to the user, the possibility that the user misses the option or erroneously selects the option can be suppressed. Note that the options may be presented to the user by voice.

Also, the displayed options include a preview showing movement information after control. For this reason, it is possible to present to the user material for determining whether or not the movement information should be corrected by the apparatus. Therefore, the user can more accurately determine whether or not the movement information is corrected.

Further, the information processing apparatus 100 controls the mode of options displayed based on the evaluation information regarding the control of the movement information before execution. For this reason, when the strength of the recommendation is presented to the user, it is possible to increase materials for determining whether the apparatus should correct the movement information. Therefore, the user can more accurately determine whether or not the movement information is corrected.

(Fifth modification)
As a fifth modification of an embodiment of the present disclosure, the movement information correction mode may be controlled based on other information. Specifically, the correction control unit 106 determines the control mode of the movement information based on the attribute information or mode information to be operated. The operation target includes an application or a displayed virtual object. The operation target attribute information includes the type, format, or identifier of the operation target. For example, the correction control unit 106 determines the degree of correction (for example, a correction parameter) of the movement information of the operating tool according to the type of application operated by the operating tool. Further, the operation target aspect information includes the size, shape, or movement speed of the operation target. For example, the correction control unit 106 determines the degree of correction of the movement information of the operating tool according to the size of the virtual object operated by the operating tool.

Further, the correction control unit 106 determines the control mode of the movement information based on the mode information of the place where the operation is performed. The form information of the place where the operation is performed includes the shape, texture or moisture of the place where the operation is performed. For example, the correction control unit 106 determines the degree of correction of the movement information of the operating tool according to the undulation of the surface on which the operation is performed.

Further, the correction control unit 106 determines the control mode of the movement information based on the attribute information or mode information of the operation subject. The attribute information of the subject of operation includes the age, sex or health status of the user who operates the operating tool. For example, the correction control unit 106 increases the correction amount of the movement information of the operating tool as the user is older. Further, the mode information of the subject of the operation includes the physique, position, posture or action of the user who operates the operating tool. For example, the correction control unit 106 decreases the correction amount of the movement information of the operating tool as the length of the user's arm is longer.

Furthermore, with reference to FIG. 17, the process of this modification is demonstrated. FIG. 17 is a flowchart conceptually showing a detailed example of the movement information correction process (step S500B) in the information processing apparatus 100 according to the fifth modification example of the embodiment of the present disclosure. Note that description of processing that is substantially the same as the processing described above is omitted.

The information processing apparatus 100 executes correction preparation processing (step S600). Specifically, the processes of steps S501 to S505 described above are executed.

Next, the information processing apparatus 100 determines whether attribute information or aspect information to be operated has been acquired (step S521). Specifically, the correction control unit 106 acquires attribute information from the application. The recognizing unit 104 recognizes the aspect of the virtual object and provides the aspect information to the correction control unit 106.

If it is determined that the attribute information or aspect information of the operation target has been acquired (step S521 / YES), the information processing apparatus 100 determines a correction amount according to the attribute or aspect of the operation target (step S522). Specifically, the correction control unit 106 calculates a correction amount corresponding to the attribute indicated by the acquired attribute information or the mode indicated by the mode information.

In addition, the information processing apparatus 100 determines whether or not the operation location mode information has been acquired (step S523). Specifically, the recognizing unit 104 recognizes an operation surface on which the user moves the operation body, and provides the correction control unit 106 with mode information related to the mode of the recognized operation surface.

If it is determined that the operation location mode information has been acquired (step S523 / YES), the information processing apparatus 100 determines a correction amount according to the operation location mode (step S524). Specifically, the correction control unit 106 calculates a correction amount to a degree according to the mode indicated by the acquired mode information of the operation place.

In addition, the information processing apparatus 100 determines whether the attribute information or the mode information of the operation subject has been acquired (step S525). Specifically, the recognizing unit 104 recognizes the user who performs the operation, and provides the correction control unit 106 with information that identifies the recognized user or aspect information related to the recognized user aspect. And the correction control part 106 acquires the attribute information of the said user from a memory | storage part etc. based on the information by which the provided user is specified. Note that the user attribute information may be acquired from an external device via communication.

If it is determined that the attribute information or aspect information of the operating subject has been acquired (step S525 / YES), the information processing apparatus 100 determines a correction amount according to the attribute or aspect of the operating subject (step S526). Specifically, the correction control unit 106 calculates a correction amount corresponding to the attribute indicated by the acquired user attribute information or the mode indicated by the mode information.

In addition, when none of the above-described information is acquired, the information processing apparatus 100 determines the correction amount according to the standard (step S527). Specifically, the correction control unit 106 sets the correction amount to an initial value.

Next, the information processing apparatus 100 corrects the movement direction related to the movement information based on the correction amount and the estimated movement direction (step S528). Specifically, the correction control unit 106 changes the movement direction related to the movement information to a direction obtained by adding the correction amount calculated to the estimated movement direction determined from the movement direction related to the estimated movement route.

Further, the information processing apparatus 100 corrects the movement amount related to the movement information based on the correction amount and the calculated movement route length (step S529). Specifically, the correction control unit 106 sets the movement amount related to the movement information to a length obtained by adding the correction amount calculated to the movement route length calculated from the length of the estimated movement route in a specific direction. change.

Thus, according to the fifth modification, the information processing apparatus 100 determines the control mode of the movement information based on the attribute information or the mode information of the operation target. Generally, the operation method or the ease of operation differs depending on the object of operation. Therefore, by controlling the presence / absence or degree of correction according to the operation target, it is possible to control the correction processing according to the desired presence / absence of correction, that is, user needs.

Further, the information processing apparatus 100 determines the control mode of the movement information based on the mode information of the place where the operation is performed. In general, the ease of operation differs depending on the mode of operation. In view of this, the presence / absence or degree of correction is controlled in accordance with the operation location, so that it is possible to control the correction processing according to the desired presence / absence of correction, that is, user needs.

Also, the information processing apparatus 100 determines the control mode of the movement information based on the attribute information or mode information of the operation subject. In general, the method of operation or the range of operation differs depending on the operation subject. Therefore, by controlling the presence / absence or degree of correction according to the operation subject, it is possible to control the correction process according to the desired presence / absence of correction, that is, user needs.

Also, the movement information control mode includes the degree of movement information control. For this reason, more precise correction control can be performed by controlling not only the presence / absence of correction but also the degree of correction. Therefore, it becomes possible to respond to finer user needs.

(Sixth Modification)
As a sixth modification example of the embodiment of the present disclosure, the correction of the movement information may be canceled. Specifically, the correction control unit 106 controls cancellation of movement information control in accordance with the presence or absence of a predetermined operation (hereinafter also referred to as cancellation operation) after the start of movement information control. More specifically, when the cancel operation is recognized, the correction control unit 106 changes the movement information used for the operation handling process to the movement information before the control. Canceling operations include shaking hands, stopping the operation for a predetermined time, and increasing or decreasing the number of fingers. For example, after the movement information is corrected and a virtual object or the like is moved in the direction related to the corrected movement information according to the user's operation, when the user performs a cancel operation, the correction control unit 106 The virtual object is moved to the position when the virtual object is moved in the direction related to the movement information before correction in accordance with the operation.

Further, with reference to FIG. 18, the movement information correction cancellation process will be described in detail. FIG. 18 is a flowchart conceptually showing an example of overall processing including movement information correction cancellation processing in the information processing apparatus 100 according to the sixth modification of an embodiment of the present disclosure. Note that description of processing that is substantially the same as the processing described above is omitted.

If it is determined that the end operation is not recognized after the application is started (step S431) (step S432 / NO), the information processing apparatus 100 determines whether the movement of the operating tool is recognized (step S433).

If it is determined that the movement of the operating tool has been recognized (step S433 / YES), the information processing apparatus 100 determines whether the moving direction of the operating tool has changed (step S434).

If it is determined that the movement direction of the operating tool has changed (step S434 / YES), the movement information correction process is executed (steps S435 to S437), and then the process is executed based on the corrected movement information. (Step S438).

Next, the information processing apparatus 100 determines whether a cancel operation for correcting the movement information has been recognized (step S439). Specifically, the correction control unit 106 determines whether the cancel operation has been recognized by the recognition unit 104 after the process is executed or during the process based on the corrected movement information.

If it is determined that the cancel operation for the correction of the movement information has been recognized (step S439 / YES), the information processing apparatus 100 changes the movement information to the movement information before correction (step S440). Specifically, when the cancel operation is recognized, the correction control unit 106 causes the processing control unit 108 to cancel or cancel the processing based on the corrected movement information, and causes the processing control unit 108 to transmit the movement information before correction. provide.

Next, the information processing apparatus 100 executes processing based on the changed movement information (step S441). Specifically, the process control unit 108 cancels or cancels the process based on the corrected movement information and executes the process based on the movement information before correction.

Thus, according to the sixth modification, the information processing apparatus 100 controls cancellation of movement information control according to the presence or absence of a predetermined operation after the start of movement information control. For this reason, even if it is a case where movement information is corrected contrary to a user's intention, it can control that processing based on movement information after amendment is performed. Therefore, usability can be improved.

Further, the cancellation of the control of the movement information includes changing the movement information used for the processing to the movement information before the control. For this reason, not only the correction is canceled, but the processing based on the movement information before the correction is automatically executed, so that it is possible to prevent the user's operation from being wasted. Therefore, the burden on the user can be further reduced. Note that the movement information correction may be canceled by canceling the operation. For example, the process control unit 108 cancels or cancels the process based on the corrected movement information and does not execute an alternative process.

(Seventh Modification)
As a seventh modified example of an embodiment of the present disclosure, the determination of the movement information control mode may be performed before the operation is started. Specifically, the correction control unit 106 determines a correction mode based on information related to an operation determined before the operation is started. More specifically, the correction control unit 106 determines a correction mode based on the determined operation subject, operation location, or operation target. For example, when the operation target is determined to be a map application or an application that displays products in a three-dimensional space, the correction control unit 106 determines that correction is not automatically performed.

Further, the determination of the movement information control mode may be performed at the start of the operation. Specifically, the correction control unit 106 determines a correction mode based on information related to the operation determined at the start of the operation. For example, by touching a slide bar that can be moved only in a one-dimensional direction or a seek bar of a player that plays video or music, these virtual objects are determined as operation targets. When these virtual objects are determined as operation targets, the correction control unit 106 determines to perform correction. Note that whether or not correction is actually performed may be selected by the user as described above.

Thus, according to the seventh modification, the determination timing of the movement information control mode includes the time before the start of the operation or the time when the operation starts. For this reason, the processing result (for example, movement of a virtual object etc.) based on the corrected movement information from the beginning of the operation can be presented to the user. Therefore, the user can perform an operation while confirming the processing result based on the corrected movement information.

<2. Hardware Configuration of Information Processing Device According to One Embodiment of Present Disclosure>
Heretofore, the information processing apparatus 100 according to an embodiment of the present disclosure has been described. The processing of the information processing apparatus 100 described above is realized by cooperation of software and hardware of the information processing apparatus 100 described below.

FIG. 19 is an explanatory diagram illustrating a hardware configuration of the information processing apparatus 100 according to an embodiment of the present disclosure. As illustrated in FIG. 19, the information processing apparatus 100 includes a processor 132, a memory 134, a bridge 136, a bus 138, an interface 140, an input device 142, an output device 144, a storage device 146, a drive 148, a connection port 150, and a communication device. 152.

(Processor)
The processor 132 functions as an arithmetic processing unit, and realizes the functions of the recognition unit 104, the correction control unit 106, and the processing control unit 108 in the information processing apparatus 100 in cooperation with various programs. The processor 132 operates various logical functions of the information processing apparatus 100 by executing a program stored in the memory 134 or another storage medium using the control circuit. For example, the processor 132 may be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a system-on-a-chip (SoC).

(memory)
The memory 134 stores a program used by the processor 132 or an operation parameter. For example, the memory 134 includes a RAM (Random Access Memory), and temporarily stores a program used in the execution of the processor 132 or a parameter that changes as appropriate in the execution. The memory 134 includes a ROM (Read Only Memory), and the function of the storage unit is realized by the RAM and the ROM. Note that an external storage device may be used as part of the memory 134 via the connection port 150 or the communication device 152.

Note that the processor 132 and the memory 134 are connected to each other by an internal bus including a CPU bus or the like.

(Bridge and bus)
The bridge 136 connects the buses. Specifically, the bridge 136 connects an internal bus to which the processor 132 and the memory 134 are connected and a bus 138 to be connected to the interface 140.

(Input device)
The input device 142 is used for a user to operate the information processing apparatus 100 or input information to the information processing apparatus 100. For example, the input device 142 includes input means for a user to input information, an input control circuit that generates an input signal based on an input by the user, and outputs the input signal to the processor 132. The input means may be a mouse, keyboard, touch panel, switch, lever, microphone, or the like. A user of the information processing apparatus 100 can input various data or instruct a processing operation to the information processing apparatus 100 by operating the input device 142.

(Output device)
The output device 144 is used to notify the user of information. For example, the output device 144 may be a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, a projector, a speaker, a headphone, or the like, or a module that outputs to the device.

Note that the input device 142 or the output device 144 may include an input / output device. For example, the input / output device may be a touch screen.

(Storage device)
The storage device 146 is a device for storing data. The storage device 146 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 146 stores programs executed by the CPU 132 and various data.

(drive)
The drive 148 is a storage medium reader / writer, and is built in or externally attached to the information processing apparatus 100. The drive 148 reads information stored in a mounted removable storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and outputs the information to the memory 134. The drive 148 can also write information on a removable storage medium.

(Connection port)
The connection port 150 is a port for directly connecting a device to the information processing apparatus 100. For example, the connection port 150 may be a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, or the like. The connection port 150 may be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like. Data may be exchanged between the information processing apparatus 100 and the external device by connecting the external device to the connection port 150.

(Communication device)
The communication device 152 mediates communication between the information processing device 100 and the external device, and realizes the function of the communication unit 102. Specifically, the communication device 152 executes communication according to a wireless communication method or a wired communication method. For example, the communication device 152 performs wireless communication according to a cellular communication method such as WCDMA (registered trademark) (Wideband Code Division Multiple Access), WiMAX (registered trademark), LTE (Long Term Evolution), or LTE-A. Note that the communication device 152 may be a short-range wireless communication method such as Bluetooth (registered trademark), NFC (Near Field Communication), wireless USB or TransferJet (registered trademark), or a wireless LAN (Local trademark) such as Wi-Fi (registered trademark). Wireless communication may be executed according to an arbitrary wireless communication method such as an area network method. The communication device 152 may execute wired communication such as signal line communication or wired LAN communication.

Note that the information processing apparatus 100 may not have a part of the configuration described with reference to FIG. 19 or may have any additional configuration. In addition, a one-chip information processing module in which all or part of the configuration described with reference to FIG. 19 is integrated may be provided.

<3. Conclusion>
As described above, according to an embodiment of the present disclosure, the movement information used in the process is corrected to the movement information related to the movement of the operation tool intended by the user, so that an operation in a range exceeding the user's range of motion is performed. Also for the above, it is possible to execute processing in accordance with the user's intention. Further, the amount of operation can be reduced by correcting the operation amount based on the movement path length. Therefore, it is possible to cause the apparatus to execute processing intended by the user and to reduce the burden on the user for operating the apparatus. Thereby, the user can perform a desired operation without changing the operation method or the operation posture of the operation tool. Therefore, the user can concentrate on the purpose of the operation without paying attention to the operation of the apparatus.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

For example, in the above embodiment, the user of the information processing apparatus 100 is single, but the present technology is not limited to such an example. Specifically, there may be a plurality of users. For example, the movement information correction control mode is determined and managed for each of a plurality of users. Further, the movement information correction control mode may be collectively managed for a plurality of users.

In the above-described embodiment, an example in which the operation target is projected by the projection device 300 has been described. However, the operation target may be displayed by another display device. For example, the operation target may be displayed by a display device such as a tablet terminal, a smartphone, or a stationary display. Further, for example, the operation target is a HUD (Head Up Display) in which the light of the external image is transmitted and the image is displayed on the display unit or the image light related to the image is projected on the user's eyes, or the captured external image and image May be displayed by HMD (Head Mount Display) or the like. In the former case, the position of the operating body is recognized by using a touch sensor (for example, a capacitive or pressure-sensitive sensor) integrated with the display unit. In the latter case, recognition of the position of the operating tool is realized using analysis processing of an image obtained by imaging by the imaging device.

In the above embodiment, an example in which the degree of correction of movement information is automatically controlled by the information processing apparatus 100 has been described, but may be selected by a user. For example, each of the results of the processing based on the movement information corrected based on the plurality of correction parameters is displayed as a preview, and the user can select any one of the results of the plurality of processing displayed in the preview by selecting one of the results. Select whether to use correction parameters.

The information processing system according to an embodiment of the present disclosure may be a server client type system or a cloud service type system. For example, the information processing apparatus 100 may be a server installed at a place different from the place where the measurement apparatus 200 and the projection apparatus 300 are installed.

In addition, the information processing system according to an embodiment of the present disclosure may be applied to other cases. For example, the information processing apparatus 100 inputs an operation using the hand of a user who performs product assembly work as an operating body, and automatically corrects the operation based on the input operation or proposes correction of the operation. You may do it.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

In addition, the steps shown in the flowcharts of the above-described embodiments are executed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the order described. Including processing to be performed. Further, it goes without saying that the order can be appropriately changed even in the steps processed in time series.

Also, it is possible to create a computer program for causing the hardware built in the information processing apparatus 100 to exhibit functions equivalent to the functional configurations of the information processing apparatus 100 described above. A storage medium storing the computer program is also provided.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A processing unit that executes processing based on movement information of the operating body;
A control unit for controlling movement information used in the processing based on a first direction determined based on a change in position in the movement of the operating body and a moving path length of the operating body;
An information processing apparatus comprising:
(2)
The control unit determines movement information related to movement in the first direction having a length corresponding to the movement path length as movement information used in the processing.
The information processing apparatus according to (1).
(3)
The control unit determines a control mode of the movement information based on information related to a movement mode of the operating body.
The information processing apparatus according to (1) or (2).
(4)
The movement mode of the operating body includes a change in the moving direction of the operating body.
The information processing apparatus according to (3).
(5)
The control unit determines a control mode of the movement information based on a comparison between a movement mode of the operation body and a pattern of the movement mode of the operation body stored in advance.
The information processing apparatus according to (3) or (4).
(6)
The movement mode of the operation body includes a speed of movement, acceleration, or a movement path of the operation body.
The information processing apparatus according to (5).
(7)
The movement mode pattern of the operation body is obtained by machine learning of the movement mode of the operation body.
The information processing apparatus according to (5) or (6).
(8)
The control unit determines a control mode of the movement information based on information related to a selection operation of the control mode of the movement information by a subject of the operation using the operation body;
The information processing apparatus according to any one of (1) to (7).
(9)
The selection operation includes an operation of selecting a displayed option.
The information processing apparatus according to (8).
(10)
The displayed options include a preview showing movement information after control,
The information processing apparatus according to (9).
(11)
The control unit controls an aspect of the displayed option based on evaluation information about control of the movement information before execution.
The information processing apparatus according to (9) or (10).
(12)
The control unit determines a control mode of the movement information based on attribute information or mode information of an operation target by the operating body.
The information processing apparatus according to any one of (1) to (11).
(13)
The control unit determines a control mode of the movement information based on mode information of a place where an operation by the operating body is performed.
The information processing apparatus according to any one of (1) to (12).
(14)
The control unit determines a control mode of the movement information based on attribute information or mode information of an operation subject by the operating body.
The information processing apparatus according to any one of (1) to (13).
(15)
The movement information control mode includes the presence or absence of the movement information or the degree of control.
The information processing apparatus according to any one of (1) to (14).
(16)
The control unit controls cancellation of the control of the movement information according to the presence or absence of a predetermined operation after the start of the control of the movement information;
The information processing apparatus according to any one of (1) to (15).
(17)
Canceling control of the movement information includes changing movement information used for the processing to movement information before control.
The information processing apparatus according to (16).
(18)
The first direction includes a direction determined from a tangent to the position of the operating body that moves.
The information processing apparatus according to any one of (1) to (17).
(19)
Using a processor
Executing processing based on movement information of the operating body;
Controlling movement information used in the processing based on a first direction determined based on a change in position in movement of the operating body and a moving path length of the operating body;
An information processing method including:
(20)
A processing function for executing processing based on movement information of the operating tool;
A control function for controlling movement information used in the processing based on a first direction determined based on a change in position in movement of the operating body and a movement path length of the operating body;
A program to make a computer realize.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 102 Communication part 104 Recognition part 106 Correction control part 108 Processing control part 200 Measuring apparatus 300 Projection apparatus

Claims

A processing unit that executes processing based on movement information of the operating body;
A control unit for controlling movement information used in the processing based on a first direction determined based on a change in position in the movement of the operating body and a moving path length of the operating body;
An information processing apparatus comprising:
The control unit determines movement information related to movement in the first direction having a length corresponding to the movement path length as movement information used in the processing.
The information processing apparatus according to claim 1.
The control unit determines a control mode of the movement information based on information related to a movement mode of the operating body.
The information processing apparatus according to claim 1.
The movement mode of the operating body includes a change in the moving direction of the operating body.
The information processing apparatus according to claim 3.
The control unit determines a control mode of the movement information based on a comparison between a movement mode of the operation body and a pattern of the movement mode of the operation body stored in advance.
The information processing apparatus according to claim 3.
The movement mode of the operation body includes a speed of movement, acceleration, or a movement path of the operation body.
The information processing apparatus according to claim 5.
The movement mode pattern of the operation body is obtained by machine learning of the movement mode of the operation body.
The information processing apparatus according to claim 5.
The control unit determines a control mode of the movement information based on information related to a selection operation of the control mode of the movement information by a subject of the operation using the operation body;
The information processing apparatus according to claim 1.
The selection operation includes an operation of selecting a displayed option.
The information processing apparatus according to claim 8.
The displayed options include a preview showing movement information after control,
The information processing apparatus according to claim 9.
The control unit controls an aspect of the displayed option based on evaluation information about control of the movement information before execution.
The information processing apparatus according to claim 9.
The control unit determines a control mode of the movement information based on attribute information or mode information of an operation target by the operating body.
The information processing apparatus according to claim 1.
The control unit determines a control mode of the movement information based on mode information of a place where an operation by the operating body is performed.
The information processing apparatus according to claim 1.
The control unit determines a control mode of the movement information based on attribute information or mode information of an operation subject by the operating body.
The information processing apparatus according to claim 1.
The movement information control mode includes the presence or absence of the movement information or the degree of control.
The information processing apparatus according to claim 1.
The control unit controls cancellation of the control of the movement information according to the presence or absence of a predetermined operation after the start of the control of the movement information;
The information processing apparatus according to claim 1.
Canceling control of the movement information includes changing movement information used for the processing to movement information before control.
The information processing apparatus according to claim 16.
The first direction includes a direction determined from a tangent to the position of the operating body that moves.
The information processing apparatus according to claim 1.
Using a processor
Executing processing based on movement information of the operating body;
Controlling movement information used in the processing based on a first direction determined based on a change in position in movement of the operating body and a moving path length of the operating body;
An information processing method including:
A processing function for executing processing based on movement information of the operating tool;
A control function for controlling movement information used in the processing based on a first direction determined based on a change in position in movement of the operating body and a movement path length of the operating body;
A program to make a computer realize.