WO2022014609A1

WO2022014609A1 - Information processing device and information processing method

Info

Publication number: WO2022014609A1
Application number: PCT/JP2021/026355
Authority: WO
Inventors: 惇一清水; 慧高橋
Original assignee: ソニーグループ株式会社
Priority date: 2020-07-14
Filing date: 2021-07-13
Publication date: 2022-01-20
Also published as: JPWO2022014609A1

Abstract

This information processing device comprises: a gesture detection unit that detects a user gesture performed in a detection region of a detection unit that detects objects; and a correction unit that, on the basis of information relating to the gesture and/or information relating to the state of a prescribed device comprising the detection unit, corrects a detectable range of the gesture in the detection region and/or a reference direction for detecting the gesture.

Description

Information processing equipment and information processing method

This disclosure relates to an information processing device and an information processing method.

Various technologies related to device operation have been developed. For example, in recent years, there has been active development of technology that enables device operation by gestures.

Japanese Unexamined Patent Publication No. 2018-109873

While device operation by gesture has the advantage that the user can easily operate the device, there is a problem that false detection is likely to occur. For example, device operation by gesture may cause problems such as erroneously detecting a user's unconscious movement as a gesture, or detecting a user's gesture as a gesture different from the gesture intended by the user. be.

Therefore, in this disclosure, we propose an information processing device and an information processing method that can realize highly accurate gesture detection.

In order to solve the above-mentioned problems, the information processing apparatus according to the present disclosure includes a gesture detection unit for detecting a user's gesture performed in the detection area of the detection unit for detecting an object, information on the gesture, and information on the gesture. Based on at least one of the information regarding the state of the predetermined device including the detection unit, at least one of the detectable range of the gesture in the detection area and the reference direction for detecting the gesture is corrected. It is provided with a correction unit and a correction unit.

It is a figure which shows a mode that a user performs a gesture. It is a figure which shows a mode that a user performs a gesture. It is a figure which shows how the position of a detection area changes depending on the mounting direction of an information processing apparatus. It is a figure which shows how the position of a detection area changes depending on the mounting direction of an information processing apparatus. It is a figure which shows how the orientation of an information processing apparatus changes depending on the mounting orientation of an information processing apparatus. It is a figure which shows how the orientation of an information processing apparatus changes depending on the mounting orientation of an information processing apparatus. It is a figure which shows the appearance that the detectable range of a gesture is limited to a part area of a detection area. It is a figure which shows a mode that an information processing apparatus corrects a reference direction. It is a figure which shows a mode that an information processing apparatus corrects a reference direction. It is a figure which shows the structural example of the information processing apparatus which concerns on embodiment of this disclosure. It is a figure which shows the structural example of the input detection part. It is a figure which showed the correspondence relationship between the function which an information processing apparatus has, and a gesture. It is a figure for demonstrating a pinch operation. It is a figure for demonstrating the hold operation. It is a figure for demonstrating the left-right swipe. It is a figure for demonstrating the left-right swipe. It is a figure for demonstrating the up and down swipe. It is a figure for demonstrating the up and down swipe. It is a figure for demonstrating the operation of holding a hand. It is a figure for demonstrating the operation of holding a hand. It is a figure for demonstrating a touch operation. It is a figure for demonstrating the operation which keeps a hand away. It is a flowchart which shows the command execution process which concerns on this embodiment. It is a figure which shows an example of the case where a gesture detection unit does not detect a gesture. It is a figure which shows an example of the case where a gesture detection unit does not detect a gesture. It is a figure which shows an example of the case where the gesture detection unit detects a gesture. It is a figure which shows an example of the case where the gesture detection unit detects a gesture. It is a figure which shows the configuration example of an information processing system. It is a figure which shows the structural example of the output device which concerns on embodiment of this disclosure. It is a figure which shows the structural example of the terminal apparatus which concerns on embodiment of this disclosure. It is a figure which shows a mode that a user manually sets the size of a detectable range. It is a figure which shows a mode that a user manually sets a position of a detectable range. It is a figure which shows the appearance which the gesture often used by a user and the success rate are displayed. It is a figure which shows the structural example of the information processing apparatus which concerns on Embodiment 3. It is a figure which shows a mode that a user performs a gesture by physical contact. It is a flowchart which shows an example of the command execution processing which concerns on Embodiment 3. It is a figure which shows a mode that a user is performing a gesture by an aerial operation. It is a flowchart which shows the other example of the command execution process which concerns on Embodiment 3. It is a figure which shows the state of inputting the command which accompanies the operation amount. It is a figure which shows how the detectable range is changed according to the distance to an obstacle. It is a figure which shows an example of the GUI for setting the detectable range. It is a figure which shows an example of the GUI for setting the detectable range.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each of the following embodiments, the same parts are designated by the same reference numerals, so that overlapping description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numerals. For example, a plurality of elements having substantially the same functional configuration are distinguished as needed, such as

output devices

20A and 20B. However, if it is not necessary to distinguish each of a plurality of elements having substantially the same functional configuration, only the same reference numerals are given. For example, when it is not necessary to distinguish between the

output devices

20A and 20B, the

output devices

20A and 20B are simply referred to as output devices 20.

In addition, the present disclosure will be described according to the order of items shown below.
1. 1. Outline of the present disclosure 1-1. Issues 1-2. Solution 2. Embodiment 1
2-1. Configuration of information processing device 2-2. Gestures that can be detected by the information processing device 2-3. Operation of information processing device 3. Embodiment 2
3-1. Information processing system configuration 3-2. Output device configuration 3-3. Configuration of terminal equipment 3-4. Operation of information processing system 4. Embodiment 3
4-1. Outline of Embodiment 3 4-2. Configuration of information processing device 4-3. Operation of information processing device 5. Modification example 6. Conclusion

<< 1. Summary of the present disclosure >>
First, the outline of the present disclosure will be described.

<1-1. Challenges>
In recent years, the development of technology that enables device operation by gesture has been actively carried out. For example, in recent years, a technique has been developed that enables a user to perform a command input operation (for example, Next / Prev operation) to a device that can be worn by the user, such as headphones and earphones, using a gesture. There is.

Many devices capable of gesture detection are equipped with a detection unit (for example, a proximity sensor) that can detect an object in a non-contact manner, and detect a user's gesture based on the detection result (for example, a sensor value) of the detection unit. In the following description, a device capable of gesture detection may be simply referred to as an information processing device.

1 and 2 are diagrams showing how the user makes a gesture. FIG. 1 shows an earphone 10A as an example of the information processing apparatus of the present embodiment. Further, FIG. 2 shows the headphones 10B as an example of the information processing apparatus of the present embodiment. The device operation by the gesture is performed by the user performing a predetermined gesture in the space near the information processing device. In the examples of FIGS. 1 and 2, the earphone 10A and the headphone 10B are attached to the user's ear, and the user U makes a gesture for operating the device in the space near his / her ear. In the example of FIG. 1, the user U makes a gesture of moving the hand H from the front to the back, and in the example of FIG. 2, the user U makes a gesture of moving the hand H from the back to the front.

In the present embodiment, the motion of moving the hand includes the motion of moving the finger. The description of "hand" appearing in the following explanation can be replaced with "finger" as appropriate.

Also, in the following explanation, the XYZ coordinate system may be used for the explanation in order to facilitate understanding. Here, the X-axis direction, the Y-axis direction, and the Z-axis direction are all directions determined with reference to the user. The X-axis plus direction is the front direction of the user U, and the X-axis minus direction is the back direction of the user U. Further, the Y-axis plus direction is the left-hand side direction of the user U, and the Y-axis minus direction is the right-hand side direction of the user U. Further, the Z-axis plus direction is the upward direction, and the Z-axis minus direction is the downward direction. In the present embodiment, the user U is assumed to be in a standing or sitting position, the X-axis and the Y-axis are in the horizontal direction, and the Z-axis is in the vertical direction (gravity direction). The X-axis direction, Y-axis direction, and Z-axis direction appearing in the following description may be appropriately read according to the posture of the user. For example, when the user is lying on his back, the X-axis direction is the vertical direction, and the Y-axis and the X-axis directions are the horizontal direction.

Further, in the following explanation, the gesture in which the user U moves the hand H in the horizontal direction (X-axis direction) may be referred to as a left / right swipe. Further, in the following description, the gesture in which the user U moves the hand H in the vertical direction (Z-axis direction) may be referred to as a vertical swipe.

As described above, the information processing device detects the user's gesture based on the detection result of the object in the detection unit. When the information processing apparatus detects a gesture, it executes a function corresponding to the gesture (for example, Next / Prev or Vol + / Vol−).

While device operation by gesture has the advantage that the user can easily operate the device, there is a problem that false detection is likely to occur. Here, it is assumed that the information processing device is a device that can be worn by the user. In this case, it is assumed that the cause of the erroneous detection is (1-1-1) a change in the position of the detection area of the detection unit and (1-1-2) a change in the orientation of the information processing apparatus. Hereinafter, false positives caused by these will be described in detail. In the following description, the device that can be worn by the user may be abbreviated as the device that can be worn.

(1-1-1) Change in the position of the detection area of the detection unit When the information processing device is a mountable device, the position of the detection area (also referred to as the operating area) of the detection unit may change depending on the mounting direction of the information processing device. Change. 3A and 3B are diagrams showing how the position of the detection region changes depending on the mounting direction of the information processing apparatus. FIG. 3A is a diagram showing a state in which the earphone 10A is attached in a standard orientation, and FIG. 3B is a diagram showing a state in which the earphone 10A is attached in an irregular orientation.

In the example of FIG. 3A, since the earphone 10A is mounted in the standard orientation, the detection region OR is located in a certain range centered on the position of the ear of the user U when viewed from the Y-axis plus direction. There is. That is, in the example of FIG. 3A, the detection region OR is located in the range intended by the developer of the apparatus. On the other hand, in the example of FIG. 3B, since the earphone 10A is worn in an irregular direction, the detection region OR is located in a range extending from slightly below the ear of the user U to the temporal region. That is, in the example of FIG. 3B, the detection region OR is not located in the range intended by the developer of the apparatus. In other words, when the user wants to make a gesture near the ear, in the example of FIG. 3A, the detection area OR is located at a position suitable for the user's sensation, whereas in the example of FIG. 3B, the user's sensation is present. There is a detection area OR at a position that does not match.

If the detection area OR is located at a position that does not suit the user's sense, even an operation that the user U does not intend as a gesture will be detected as a gesture. For example, the action of the user U trying to scratch his head and bringing his hand to the temporal region is likely to be an action within the detection region OR in the example of FIG. 3B. In that case, the operation may be detected as a gesture by the information processing apparatus.

Further, when the detection area OR is located at a position that does not suit the user U's sense, the operation that the user U intentionally performs the gesture becomes an operation outside the detection area OR in some cases. For example, in the example of FIG. 3B, the gesture performed slightly below the earphone 10A is likely to be an operation near the user's ear, but outside the detection region OR. In this case, the information processing device cannot detect the user's action as a gesture. In order to prevent the gesture of the user U from operating outside the detection area OR, it is recalled to increase the sensitivity of the detection unit and widen the detection area OR. However, in this case, although the downward direction of the user U's ear is good, in the other direction, the detection region OR extends to a position that does not suit the user U's sense. In that case, the same problem as the above-mentioned problem occurs.

Although the earphone type information processing device is shown in FIGS. 3A and 3B, the change in the position of the detection area of the detection unit may occur even when the information processing device is a device other than the earphone type. For example, the change in the position of the detection area of the detection unit may occur even when the information processing device is a headphone type device as shown in FIG. 2, for example. Therefore, when the information processing device is a device other than the earphone type, the same problem as described in this (1-1-1) occurs.

(1-1-2) Change in orientation of information processing device When the information processing device is a mountable device, the orientation of the information processing device with respect to the orientation of the user U changes depending on the mounting orientation of the information processing device. 4A and 4B are diagrams showing how the orientation of the information processing apparatus changes depending on the mounting orientation of the information processing apparatus. FIG. 4A is a diagram showing a state in which the headphones 10B are worn in a standard wearing direction, and FIG. 4B is a diagram showing a state in which the headphones 10B are worn in an irregular direction.

Here, D1 to D4 in the figure are directions with respect to the device. More specifically, D1 to D4 in the figure indicate the upward, downward, forward, and backward directions when the information processing apparatus is mounted in the standard mounting orientation. D1 is upward, D2 is downward, D3 is forward, and D4 is backward. In the following description, D1 to D4 are referred to as device directions.

In the example of FIG. 4A, since the headphones 10B are mounted in the standard direction, the vertical direction (Z-axis direction) and the device directions D1 and D2 match, and the horizontal direction (X-axis direction) and the device. The directions D3 and D4 also match. On the other hand, in the example of FIG. 4B, since the headphones 10B are mounted in an irregular direction, the vertical direction (Z-axis direction) and the device directions D1 and D2 do not match, and the horizontal direction (X-axis direction). ) And the device directions D3 and D4 do not match either. As described above, when the information processing device is a wearable device, the direction of the information processing device with respect to the direction of the user U (for example, the X-axis direction and the Z-axis direction) (for example, the device directions D3 and D4, and the device directions D1 and D2). Changes.

Here, it is assumed that the user swipes left or right as an input operation to the information processing device. Generally, the information processing apparatus determines whether or not the gesture performed by the user is a left / right swipe based on the horizontal direction (device directions D3, D4) at the time of standard mounting. In the example of FIG. 4A, the horizontal direction (X-axis direction) and the horizontal direction (device directions D3, D4) of the information processing apparatus at the time of standard mounting coincide with each other. Therefore, the information processing apparatus can detect that the user's gesture is a left / right swipe without any problem.

However, in the example of FIG. 4B, the horizontal direction (X-axis direction) and the horizontal direction of the information processing device at the time of standard mounting (device directions D3 and D4) do not match. Therefore, even if the user intentionally makes a gesture with a left / right swipe, the information processing apparatus cannot detect that the gesture is a left / right swipe. In the example of FIG. 4B, the direction of the movement of the user's hand is close to the vertical direction (device directions D1 and D2) of the information processing apparatus at the time of standard mounting. Therefore, in the example of FIG. 4B, the information processing apparatus erroneously detects that the user has swiped up and down.

Although a headphone type information processing device is shown in FIG. 4B, a change in the orientation of the information processing device may occur even when the information processing device is a device other than the headphone type. For example, the change in the orientation of the information processing device may occur even when the information processing device is an earphone type device. Therefore, when the information processing device is a device other than the headphone type, the same problem as described in this (1-1-2) occurs.

<1-2. Solution>
Therefore, in the present embodiment, the above-mentioned problems (1-1-1) and (1-1-2) are solved by the following means. As a solution, (1-2-1) correction of the detectable range of the gesture and (1-2-2) correction of the reference direction for detecting the gesture are recalled. Hereinafter, these solutions will be described. The information processing apparatus may execute a plurality of solutions shown below in parallel or in combination.

(1-2-1) Correction of the detectable range of the gesture As described above, when the detection area OR is located at a position that does not suit the user's sense, the information processing apparatus makes a gesture until the operation that the user does not intend as a gesture. Will be detected as. Therefore, the information processing apparatus of the present embodiment limits the detectable range of the gesture to a part of the detection area OR.

FIG. 5 is a diagram showing a state in which the detectable range of the gesture is limited to a part of the detection area OR. In the example of FIG. 5, the detection region OR is formed in a certain range around the information processing apparatus 10 when viewed from the Y-axis plus direction. The information processing device 10 may be an earphone 10A or a headphone 10B. In the example of FIG. 5, the detection area OR is divided into a detectable range AR and a non-detectable range NR. The detectable range AR is the range in which the information processing apparatus 10 can detect the gesture, and the non-detectable range NR is the range in which the information processing apparatus 10 does not detect the gesture. Both the detectable range AR and the non-detectable range NR are within the detection area OR.

The information processing device 10 corrects the detectable range AR based on the information regarding the gesture of the user U. Here, the information about the gesture may be the information of the operation of the user U regarding the gesture. More specifically, the information regarding the gesture may be information on an operation (hereinafter referred to as an approach operation) in which the user U enters the hand H into the detectable range AR. If the information processing device 10 is a device worn on the ear of the user U, the approaching motion may be an motion in which the user U swings the hand H toward the ear (hereinafter, referred to as a swinging motion). The swing-up motion is, for example, an motion in which the user U swings the hand H of the user U toward the ear like a pendulum with the elbow joint as a fulcrum while moving the elbow joint stationary or slowly forward. In the following description, it is assumed that the approaching motion is the swinging motion of the hand H of the user U.

When the user U swings up the hand H, the approach direction AD of the user U's hand H into the detectable range AR is a certain direction due to the structure of the human body. Although there are individual differences and differences between gestures, in many cases, the approach direction AD is assumed to fall within a range of a certain vertical angle from the back direction (X-axis minus direction). In the example of FIG. 5, the user's hand H has entered the detectable range AR from a position slightly downward from the front of the user U. Therefore, in the example of FIG. 5, the approach direction AD is a direction slightly inclined upward from the back surface direction (X-axis minus direction).

If the approach direction AD is known, the information processing apparatus 10 can infer where the user U makes the gesture. Therefore, the information processing apparatus 10 corrects the detectable range AR to be narrower than the detection area OR based on the information of the approach direction AD. In the example of FIG. 5, as a result of the correction, the detectable range AR is set in a state of being biased in the detection area OR so that a part of the edge of the detectable range AR touches a part of the edge of the detection area OR. More specifically, the detectable range AR is biased to a position where the user U is presumed to have entered the detection region OR (hereinafter referred to as an approach position). Here, the approach position will be described using the example of FIG. 5, for example, an intersection of an arrow indicating the approach direction AD and a broken line indicating the edge of the detection region OR.

The information on the approaching motion is not limited to the approaching direction AD, and may be, for example, information on the approaching position. In this case, the information processing apparatus 10 may correct the detectable range AR to be narrower than the detection area OR based on the information of the approach position.

According to the present solution, the information processing apparatus 10 corrects the detectable range of the gesture based on the information of the swinging motion of the hand H of the user U. As a result, it is less likely that an action that the user does not intend as a gesture is detected as a gesture. Even if the information processing apparatus 10 increases the sensitivity of the detection unit and widens the detection area OR, the possibility of erroneous detection is low.

(1-2-2) Correction of reference direction for detecting gesture As described above, the orientation of the information processing apparatus 10 with respect to the orientation of the user U changes depending on the mounting orientation of the information processing apparatus 10. Therefore, the information processing apparatus 10 of the present embodiment corrects the reference direction for gesture detection according to the inclination of the apparatus. In addition, "detection" in the detection of a gesture is a "detection" in a broad sense including recognition of a gesture. Here, the reference direction for gesture detection is the reference direction for the information processing apparatus 10 to detect the gesture. For example, if the information processing apparatus 10 is trying to detect a left / right swipe, the reference direction is, for example, the horizontal direction.

FIG. 6 is a diagram showing how the information processing apparatus 10 corrects the reference direction. In the example of FIG. 6, B1 to B4 are the corrected reference directions. In the following description, B1 to B4 may be simply referred to as "reference direction". Since the device directions D1 to D4 are directions with respect to the device, when the information processing device 10 is tilted and mounted, it is tilted from the horizontal direction or the vertical direction. When the information processing apparatus 10 detects a gesture with the device directions D1 to D4 as reference directions, erroneous detection such as detecting a left / right swipe as an up / down swipe will occur.

Various methods are recalled as methods for correcting the reference direction. For example, the information processing apparatus 10 corrects the reference direction based on the information about the gesture. Here, the information about the gesture may be the information of the operation of the user U regarding the gesture. More specifically, the information about the gesture may be the information of the approaching motion in which the user U enters the hand H into the detectable range AR. If the information processing device 10 is a device worn on the ear of the user U, the approaching motion may be an motion in which the user U swings his / her hand H toward his / her ear (swinging motion).

As described above, when the user U swings up the hand H, the approach direction AD of the user U's hand H into the detectable range AR is a certain direction. The information processing apparatus 10 corrects the reference direction based on the information of the approach direction AD. FIG. 7 is a diagram showing how the information processing apparatus 10 corrects the reference direction. Since the device directions D1 to D4 are directions with respect to the device, the information processing device 10 naturally knows about the device directions D1 to D4. The information processing device 10 corrects the reference direction based on the information in the relative device directions D1 to D4 with respect to the approach direction AD. As an example, the information processing apparatus 10 can correct the reference direction by the following method.

For example, the information processing device 10 calculates an angle R formed by at least one of the device directions D1 to D4 (device direction D2 in the example of FIG. 7) and the approach direction AD. The approach direction AD is, for example, an angle at the position C where the user U's hand H enters the detectable range AR. As described above, it is assumed that the approach direction AD is a direction slightly inclined upward from the back surface direction (X-axis minus direction) of the user U. Therefore, the information processing device 10 knows the angle of the device direction reference of the approach direction AD that should be when the information processing device 10 is installed as standard. The information processing device 10 can estimate the mounting inclination of the information processing device 10 from the difference between the angle of the approach direction AD that should be when the information processing device 10 is mounted as standard and the angle of the approach direction AD that is actually detected. The information processing apparatus 10 may use the difference in angles as the estimation result as it is. Then, the reference direction is corrected to the reference directions B1 to B4 based on the estimation result of the mounting inclination of the information processing apparatus 10.

In the example of FIG. 7, the reference directions B1 to B4 are in the horizontal direction or the vertical direction, but the reference directions B1 to B4 do not necessarily have to be in the horizontal direction or the vertical direction. The reference direction can be any direction that the information processing apparatus 10 uses as a reference for detecting the gesture. For example, if the information processing apparatus 10 detects a swipe in the diagonal 45 ° direction, the diagonal 45 ° direction may be regarded as the reference direction.

According to the present solution, the information processing apparatus 10 does not use the device directions D1 to D4 as the reference direction, but detects the gesture based on the corrected reference directions B1 to B4. This reduces erroneous detection, for example, detecting a left / right swipe as an up / down swipe.

(1-2-3) Other Solutions In the above-mentioned (1-2-1) and (1-2-2), the detectable range of the gesture and / or the gesture is based on the information about the gesture of the user U. Corrected the reference direction for gesture detection. However, the information processing apparatus 10 can also correct the detectable range and / or the reference direction by using information other than the information regarding the gesture.

For example, the information processing apparatus 10 may correct at least one of the detectable range and the reference direction based on the information regarding the state of the information processing apparatus 10. For example, the information processing apparatus 10 includes a sensor unit that detects the state of the information processing apparatus 10, and corrects at least one of the detectable range and the reference direction based on the information from the sensor unit.

Here, the sensor unit may be composed of one or a plurality of acceleration sensors, or one or a plurality of gyro sensors. In this case, the information processing apparatus 10 may be able to correct at least one of the detectable range and the reference direction based on the mounting state of the information processing apparatus 10 to the user U estimated based on the information from the sensor unit. ..

Since the gravitational acceleration is constantly applied to the information processing device 10, the information processing device 10 can estimate the gravitational direction based on the information from the sensor unit. The information processing apparatus 10 estimates the inclination of the information processing apparatus 10 from the standard mounted state from the estimated gravity direction. Then, the information processing apparatus 10 corrects at least one of the detectable range and the reference direction based on the estimation result of the inclination. The information processing apparatus 10 detects the gesture based on the corrected detectable range and / or the reference direction.

Also by this method, it is possible to reduce the false detection of the gesture of the user U as in the means described in (1-2-1) and (1-2-2).

<< 2. Embodiment 1 >>
The outline of the present embodiment has been described above, but the information processing apparatus 10 according to the first embodiment will be described below.

<2-1. Information processing device configuration>
First, the configuration of the information processing apparatus 10 will be described.

The information processing device 10 is an information processing terminal that executes various functions based on a user's input operation. For example, the information processing device 10 is an acoustic output device that can be worn by a user such as earphones and headphones. The information processing device 10 may be a display device such as an AR (Augmented Reality) glass or an MR (Mixed Reality) glass that can be worn by a user. In the present embodiment, the information processing apparatus 10 is assumed to be a device that can be worn by the user U, and is provided with a portion located at least in the ear of the user U when worn. Then, it is assumed that at least a detection unit for detecting an object is arranged in this portion. The information processing apparatus 10 is configured to be able to detect a user's gesture, and executes a function such as music playback based on the user's gesture.

FIG. 8 is a diagram showing a configuration example of the information processing apparatus 10 according to the embodiment of the present disclosure. The information processing apparatus 10 includes an input detection unit 11, a state detection unit 12, an output unit 13, a communication unit 14, a storage unit 15, and a control unit 16. The configuration shown in FIG. 8 is a functional configuration, and the hardware configuration may be different from this. Further, the functions of the information processing apparatus 10 may be distributed and implemented in a plurality of physically separated configurations.

The input detection unit 11 is a detection unit that detects a user's input operation. For example, the input detection unit 11 is a non-contact type detection unit that detects an object in a non-contact manner. The input detection unit 11 includes one or a plurality of proximity sensors (also referred to as non-contact sensors), and detects an object located in the detection area. The proximity sensor may be an optical sensor or a capacitance type sensor. An optical proximity sensor is a sensor that detects the light reflected by an object. The capacitance type proximity sensor is a sensor that detects a change in capacitance that occurs between an object and the sensor.

When the input detection unit 11 is a non-contact type, the input detection unit 11 may be a 2D sensor or a 3D sensor. When the input detection unit 11 is a 2D sensor, the detection area formed by the input detection unit 11 is a two-dimensional area, and when the input detection unit 11 is a 3D sensor, the detection area formed by the input detection unit 11 is a three-dimensional area. It becomes.

The input detection unit 11 is not limited to the non-contact type. The input detection unit 11 may be a contact type detection unit. In this case, the input detection unit 11 includes one or a plurality of touch sensors, and detects an object that comes into contact with a predetermined place of the information processing apparatus 10. If the information processing device 10 is an earphone type device, the predetermined location is, for example, the side surface of the earphone (opposite the surface of the speaker). If the information processing device 10 is a headphone type device, the predetermined location is, for example, the side surface of the headphones (opposite the surface of the speaker). In the following description, the side surface of the earphone or the side surface of the headphone may be referred to as the housing surface.

FIG. 9 is a diagram showing a configuration example of the input detection unit 11. FIG. 9 shows a state in which the earphone-type information processing apparatus 10 is viewed from the side surface of the earphone. The configuration shown in FIG. 9 is merely an example, and the configuration of the input detection unit 11 is not limited to the configuration shown in FIG.

In the example of FIG. 9, the input detection unit 11 includes an infrared emitter IR, four photo sensors PD1, PD2, PD3, and PD4 as non-contact sensors. The infrared emitter ID is arranged in the center of the housing surface, and the four photosensors PD1, PD2, PD3, and PD4 are arranged at equal intervals in the circumferential direction so as to surround the infrared emitter ID. The input detection unit 11 detects an object by detecting infrared light emitted from the infrared emitter ID and reflected on the surface of the object by four photosensors PD1, PD2, PD3, and PD4.

Further, the input detection unit 11 is provided with a touch pad TP as a contact type sensor. The touch pad TP is composed of a plurality of touch sensors arranged in a plane. The infrared emitter ID is arranged on the housing surface. The input detection unit 11 detects an object in contact with the surface of the touch pad TP.

The state detection unit 12 is a sensor unit that detects the state of the information processing device 10. The state detection unit 12 is composed of one or a plurality of sensors that detect the state of the information processing device 10. The state detection unit 12 includes, for example, one or a plurality of acceleration sensors. Further, the state detection unit 12 may include one or a plurality of gyro sensors. Further, the state detection unit 12 may include one or a plurality of geomagnetic sensors. The state detection unit 12 may include a motion sensor in which these plurality of types of sensors are combined. Further, a biosensor for acquiring user's biometric information such as a blood pressure sensor or a heartbeat sensor may be provided.

The state detection unit 12 detects the state of the information processing apparatus 10 based on the detection results of these sensors. For example, the state detection unit 12 detects the direction of gravity based on the sensor values of these sensors. For example, since the information processing device 10 is constantly subjected to gravitational acceleration, the information processing device 10 can detect the gravitational direction by averaging the directions of the accelerations detected by the acceleration sensor for a certain period of time. ..

The output unit 13 is an output interface that outputs information to the user. For example, the output unit 13 may be an acoustic device such as a speaker or a buzzer, or may be a vibration device such as a vibration motor. Further, the output unit 13 may be a display device such as a liquid crystal display (Liquid Crystal Display) or an organic EL display (Organic Electroluminescence Display), or may be a lighting device such as an LED (Light Emitting Diode) lamp. .. The output unit 13 functions as an output means of the information processing apparatus 10. The output unit 13 outputs various information to the user according to the control of the control unit 16.

The communication unit 14 is a communication interface for communicating with other devices. The communication unit 14 may be a network interface or a device connection interface. Further, the communication unit 14 may be a wired interface or a wireless interface. The communication unit 14 functions as a communication means of the information processing device 10. The communication unit 14 communicates with other devices according to the control of the control unit 16. Other devices are, for example, terminal devices such as music players and smartphones.

The storage unit 15 is a storage device capable of reading and writing data such as a DRAM (Dynamic Random Access Memory), a SRAM (Static Random Access Memory), a flash memory, and a hard disk. The storage unit 15 functions as a storage means for the information processing device 10. The storage unit 15 stores, for example, the settings related to the detectable range AR of the gesture.

The control unit 16 is a controller that controls each unit of the information processing device 10. The control unit 16 is realized by, for example, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). For example, the control unit 16 is realized by the processor executing various programs stored in the storage device inside the information processing device 10 using a RAM (Random Access Memory) or the like as a work area. The control unit 16 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The CPU, MPU, ASIC, and FPGA can all be regarded as controllers.

As shown in FIG. 8, the control unit 16 includes an acquisition unit 161, a gesture detection unit 162, a command execution unit 163, an output control unit 164, a guessing unit 165, and a correction unit 166. Each block (acquisition unit 161 to correction unit 166) constituting the control unit 16 is a functional block indicating the function of the control unit 16. These functional blocks may be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks may be one software module realized by software (including a microprogram), or may be one circuit block on a semiconductor chip (die). Of course, each functional block may be one processor or one integrated circuit. The method of configuring the functional block is arbitrary.

The control unit 16 may be configured in a functional unit different from the above-mentioned functional block. Further, another device may perform a part or all of the operations of each block (acquisition unit 161 to correction unit 166) constituting the control unit 16. For example, a terminal device such as a music player or a smartphone may perform some or all operations of each block constituting the control unit 16. The operation of each block constituting the control unit 16 will be described later.

<2-2. Gestures that can be detected by information processing equipment>
The configuration of the information processing apparatus 10 has been described above, but before the operation of the information processing apparatus 10 according to the present embodiment is described in detail, the gestures that can be detected by the information processing apparatus 10 will be described.

(2-2-1) Correspondence between functions and gestures FIG. 10 is a diagram showing a correspondence between functions and gestures of the information processing apparatus 10. The information processing device 10 has a function related to answering a telephone in addition to a function related to playing music. For example, the information processing apparatus 10 has the functions shown in the following (1) to (7). In addition, "function" can be paraphrased as "command".
(1) Play / Pause
(2) AnswerCall / End
(3) Next / Prev
(4) Vol + / Vol-
(5) Candle
(6) Quick Attention
(7) Ambient Control

Hereinafter, the functions of (1) to (7) and the gestures associated with each of (1) to (7) will be described. The user's actions constituting one gesture may include a first action that triggers the start of the gesture and a second action following the first action. The first operation is, for example, a swing-up operation of the hand H of the user U, and the second operation is, for example, an operation such as a swipe following the first operation. It is also possible to regard each of the first motion and the second motion as one gesture.

The functions and gestures described here are just examples. The functions and gestures of the information processing apparatus 10 are not limited to the functions and gestures shown below. The first operation and the second operation are not limited to the operations shown below. Further, the combination of the function and the gesture is not limited to the combination shown below. For example, although the above (1) to (7) are not described, the Noise Canceling function may be operated by a gesture.

In the following description, the information processing device 10 is assumed to be an earphone-type device worn on the user's ear, but the information processing device 10 is not limited to the earphone-type device. For example, the information processing device 10 may be a headphone type device. Further, the information processing apparatus 10 may have a portion worn on the user's ear and another portion. Further, in the following description, it is assumed that the information processing apparatus 10 is attached to the user's ear, but the apparatus attached to the user's ear may be a predetermined device other than the information processing apparatus 10. The predetermined device is, for example, an output device wirelessly connected to the information processing device 10 (for example, an earphone or headphone, a hearing aid, a sound collector, etc., regardless of whether it is a wired, wireless, canal type, open ear type, etc.). You may. If the information processing device 10 is worn on the user's ear, the predetermined device is the information processing device 10 itself. A predetermined device can be paraphrased as a predetermined device.

Hereinafter, with reference to FIG. 10, the functions of the information processing apparatus 10 and the gestures associated with the functions will be described.

(1) Play / Pause
"Play" is a function for playing a song, and "Pause" is a function for pausing the playback of a song. The "Play / Pause" is associated with the swing-up motion of the hand H as the first motion, and is associated with the pinch motion as the second motion. FIG. 11 is a diagram for explaining a pinch operation. As shown in FIG. 11, the pinch operation is an operation of picking an object with a finger. The information processing apparatus 10 can detect that the user U has performed a pinch operation even if the user U has not actually picked up an object (for example, the information processing apparatus 10).

(2) Answer Call / End
"Answer Call" is a function for answering a telephone call, and "Answer End" is a function for ending a telephone call. The "Answer Call / End" is associated with a swing-up motion of the hand H as a first motion, and a pinch motion or a hold motion as a second motion. FIG. 12 is a diagram for explaining the hold operation. As shown in FIG. 12, the hold operation is an operation of grasping an object with the hand H. The information processing apparatus 10 can detect that the user U has performed the hold operation even if the user U does not actually grasp an object (for example, the information processing apparatus 10).

(3) Next / Prev
"Next" is a function for cueing the next song, and "Prev" is a function for cueing the previous song or the song being played. The "Next / Prev" is associated with the swinging motion of the hand H as the first motion, and is associated with the left and right swipe as the second motion. Here, "Next" is associated with the left swipe (front swipe) of the left and right swipes as the second operation, and "Prev" is associated with the right swipe of the left and right swipes as the second operation (prev). After swipe) is associated. It should be noted that "Next / Prev" may be associated with a gesture that performs the second operation without the first operation.

13A and 13B are diagrams for explaining a left-right swipe. FIG. 13A is a left swipe and FIG. 13B is a right swipe. The left / right swipe is an operation (gesture) involving a movement width, unlike the pinch operation and the hold operation. Here, the left swipe is an operation in which the user U slides the hand H in the forward direction (X-axis plus direction) by a predetermined movement width W1 as shown in FIG. 13A. Further, the right swipe is an operation in which the user U slides the hand H in the backward direction (X-axis minus direction) by a predetermined movement width W2 as shown in FIG. 13B.

Note that both FIGS. 13A and 13B are diagrams showing how the user U is performing a gesture near the left ear. Unlike the examples of FIGS. 13A and 13B, when the user U makes a gesture near the right ear, the right swipe is the action of sliding the hand H forward, and the left swipe is the action of sliding the hand H backward. So be careful. In this case, "Next" is associated with, for example, a right swipe (front swipe) as a second action, and "Prev" is associated with, for example, a left swipe (back swipe) as a second action.

(4) Vol + / Vol-
"Vol +" is a function to raise the volume, and "Vol-" is a function to lower the volume. "Vol + / Vol-" is associated with the swinging motion of the hand H as the first motion, and is associated with the up / down swipe as the second motion. Here, "Vol +" is associated with the up swipe of the up and down swipes as the second action, and "Vol-" is associated with the down swipe of the up and down swipes as the second action. There is. It should be noted that "Vol + / Vol-" may be associated with a gesture that performs the second operation without the first operation.

14A and 14B are diagrams for explaining the up / down swipe. FIG. 14A is a left swipe and FIG. 14B is a right swipe. The up / down swipe is an operation (gesture) involving a movement width, unlike the pinch operation and the hold operation. Here, the upper swipe is an operation in which the user U slides the hand H upward (Z-axis plus direction) by a predetermined movement width W3, as shown in FIG. 14A. Further, the lower swipe is an operation in which the user U slides the hand H downward (Z-axis minus direction) by a predetermined movement width W4, as shown in FIG. 14B.

Note that "Vol + / Vol-" is a function for adjusting the volume. Therefore, in order to execute this function, the information processing apparatus 10 can acquire not only the information on raising and lowering the sound but also the information on the amount of raising and lowering sound (the amount of raising or lowering the sound) from the user U. desirable. Therefore, in the present embodiment, "Vol + / Vol-" is assumed to be a function accompanied by an operation amount (in the case of this function, an amount of raising or lowering the sound). In this function, for example, the movement amount or movement speed of the hand H when swiping up and down is associated with the operation amount of "Vol + / Vol-". By configuring the information processing apparatus 10 so as to raise or lower a certain amount of sound with one input operation, it is possible to make this function a function without an operation amount.

(5) Candle
"Cancel" is a function of canceling the operation performed by the user U. The "Cancel" is associated with the swinging motion of the hand H as the first motion, and is associated with the motion of holding the hand as the second motion. 15A and 15B are diagrams for explaining the operation of holding the hand H. FIG. 15A is a view of the user U viewed from the left side, and FIG. 15B is a view of the user U viewed from the front. As shown in FIGS. 15A and 15B, the operation of holding the hand H is an operation in which the user U holds the spread hand H toward the information processing apparatus 10.

(6) Quick Attention
"Quick Attention" is a function for the user U to quickly hear the surrounding sound. Specifically, "Quick Attention" is a function that makes it easier to hear surrounding sounds by quickly lowering or stopping the volume of the sound being output (for example, music, call voice, or ringtone). .. The "Quick Attention" is associated with the swinging motion of the hand H as the first motion, and is associated with the touch motion as the second motion. FIG. 16 is a diagram for explaining a touch operation. As shown in FIG. 16, the touch operation is an operation in which the user U touches a part of the information processing apparatus 10 (for example, the housing surface).

(7) Ambient Control
The "Ambient Control" is a function for the user U to listen to music while checking the surrounding sounds. Specifically, "Ambient Control" is a function of capturing external sounds during music reproduction. The "Ambient Control" is associated with a swing-up motion of the hand H as a first motion, and is associated with a motion of moving the hand H away as a second motion. FIG. 17 is a diagram for explaining an operation of moving the hand H away. The motion of moving the hand H away is a motion (gesture) with a movement width, unlike the pinch motion, the hold motion, the touch motion, and the motion of holding the hand. Here, in the operation of moving the hand H away, as shown in FIG. 17, after the user U holds the hand H over the information processing apparatus 10, the hand H is moved in a predetermined separation direction by a predetermined movement width W4. In the example of FIG. 17, it is an operation of moving in the Y-axis plus direction).

(2-2-2) Correspondence between function and correction processing The correspondence between function and gesture has been described above. Next, the correspondence between function and correction processing will be described. In FIG. 10 shown above, in addition to the correspondence between the function and the gesture, the correspondence between the function and the correction process is also shown. Here, the correction process is a correction process for the detectable range of the gesture and a correction process for the reference direction for detecting the gesture. In the example shown in FIG. 10, the correction process differs for each function or each gesture.

The information processing apparatus 10 may correct the detectable range and the reference direction immediately after the detection of the first motion is completed, or the gesture after the acquisition of the information on the series of movements of the user related to the gesture is completed. The detectable range and the reference direction may be corrected at the stage of performing the recognition process of.

For example, the information processing apparatus 10 corrects the detectable range based on the information of the first operation after the acquisition of the information of both the first operation and the second operation is completed. Then, the information processing apparatus 10 determines whether or not the second operation has been performed within the detectable range based on the information of the second operation for which the acquisition has been completed. If the second operation is not performed within the detectable range, the information processing apparatus 10 excludes the information of the second operation for which acquisition has been completed from the target of gesture detection.

Further, the information processing apparatus 10 corrects the reference direction based on the information of the first operation, for example, after the acquisition of the information of both the first operation and the second operation is completed. The information processing apparatus 10 interprets the information of the second operation for which the acquisition has been completed, based on the information in the reference direction corrected based on the information of the first operation.

Hereinafter, the correspondence between the functions of the information processing apparatus 10 and the correction processing will be described with reference to FIG.

(1) Play / Pause
The "Play / Pause" is associated with the swing-up motion of the hand H as the first motion, and is associated with the pinch motion as the second motion. In this function, the information processing apparatus 10 limits the detectable range based on, for example, the first operation. The information processing device 10 does not have to make corrections in the reference direction. This is because the pinch motion is a gesture without a movement width, so information in the reference direction is not always necessary for detecting the gesture.

(2) Answer Call / End
The "Answer Call / End" is associated with a swing-up motion of the hand H as a first motion, and a pinch motion or a hold motion as a second motion. In this function, the information processing apparatus 10 limits the detectable range based on, for example, the first operation. The information processing device 10 does not have to make corrections in the reference direction. This is because the pinch operation and the hold operation are gestures without a movement width, and therefore the information in the reference direction is not always necessary for the gesture detection.

(3) Next / Prev
The "Next / Prev" is associated with the swing-up motion of the hand H as the first motion, and is associated with the left / right swipe as the second motion. In this function, the information processing apparatus 10 limits the detectable range based on, for example, the first operation. After limiting the detectable range based on the first operation, the information processing apparatus 10 may make a correction to widen the limited detectable range after detecting the start of the operation of the second operation, for example. As a result, the detection accuracy of the second operation is improved. Further, in this function, the information processing apparatus 10 corrects the reference direction based on, for example, the first operation.

(4) Vol + / Vol-
"Vol + / Vol-" is associated with the swinging motion of the hand H as the first motion, and is associated with the up / down swipe as the second motion. In this function, the information processing apparatus 10 limits the detectable range based on, for example, the first operation. After limiting the detectable range based on the first operation, the information processing apparatus 10 may make a correction to widen the limited detectable range after detecting the start of the operation of the second operation, for example. As a result, the detection accuracy of the second operation is improved. Further, in this function, the information processing apparatus 10 corrects the reference direction based on, for example, the first operation.

(5) Candle
The "Cancel" is associated with the swinging motion of the hand H as the first motion, and is associated with the motion of holding the hand H as the second motion. In this function, the information processing apparatus 10 sets a detectable range based on, for example, information about the state of the information processing apparatus 10. For example, the information processing apparatus 10 sets the detectable range based on the information from the state detection unit 12. In this function, the information processing apparatus 10 does not necessarily have to perform correction in the reference direction. This is because the movement of holding the hand H is a gesture without a movement width, and therefore information in the reference direction is not always necessary for detecting the gesture.

(6) Quick Attention
The "Quick Attention" is associated with the swinging motion of the hand H as the first motion, and is associated with the touch motion as the second motion. In this function, the information processing apparatus 10 does not have to correct the detectable range and the reference direction. This is because information on the detectable range and the reference direction is not always necessary for detecting the touch operation.

(7) Ambient Control
The "Ambient Control" is associated with a swing-up motion of the hand H as a first motion, and is associated with a motion of moving the hand H away as a second motion. In this function, the information processing apparatus 10 limits the detectable range based on, for example, the first operation. After limiting the detectable range based on the first operation, the information processing apparatus 10 may make a correction to widen the limited detectable range after detecting the start of the operation of the second operation, for example. As a result, the detection accuracy of the second operation is improved. Further, in this function, the information processing apparatus 10 corrects the reference direction based on, for example, the first operation.

<2-3. Information processing device operation>
The gestures that can be detected by the information processing apparatus 10 have been described above. Next, the operation of the information processing apparatus 10 that can detect such gestures will be described.

FIG. 18 is a flowchart showing a command execution process according to the present embodiment. The command execution process is a process in which the information processing apparatus 10 executes a command input by the user U using a gesture. Here, the "command" is a command from the inside or the outside of the device for causing the information processing device 10 to execute a function. It is also possible to regard the "command" as a word indicating the function itself of the information processing apparatus 10, such as Play / Pause. The description of "command" appearing in the following explanation can be replaced with "function".

The command execution process is executed by the control unit 16 of the information processing device 10. The command execution process is started, for example, when the information processing apparatus 10 is turned on.

First, the acquisition unit 161 of the control unit 16 acquires the sensor value from the input detection unit 11 and the state detection unit 12 (step S101). For example, the acquisition unit 161 acquires information about an object in the detection region OR from a non-contact type sensor such as a proximity sensor. Further, the acquisition unit 161 acquires information regarding contact with the housing surface from a contact-type sensor such as a touch sensor. Further, the acquisition unit 161 acquires information regarding the state of the information processing device 10 from sensors such as an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

Subsequently, the acquisition unit 161 acquires information regarding the gesture of the user U based on the sensor value acquired in step S101 (step S102). For example, the acquisition unit 161 acquires information on the operation of the user U regarding the gesture based on the sensor value from the input detection unit 11. More specifically, the acquisition unit 161 acquires information on the approaching motion in which the user U enters the hand H into the detectable range AR based on the sensor value of a non-contact type sensor such as a proximity sensor. If the information processing device 10 is a device worn on the ear of the user U, the approaching motion may be a swinging motion in which the user U swings the hand H toward the ear. In this case, the information on the approaching motion may be the information on the approaching direction AD of the user U's hand H into the detectable range AR.

Subsequently, the acquisition unit 161 acquires information regarding the state of the information processing apparatus 10 based on the sensor value acquired in step S101 (step S103). For example, the acquisition unit 161 acquires information about the state of the information processing apparatus 10 based on the sensor value from the state detection unit 12. For example, the acquisition unit 161 acquires information regarding the direction of gravity based on information from an acceleration sensor, a gyro sensor, or a geomagnetic sensor.

Next, the correction unit 166 of the control unit 16 executes the correction process related to the gesture detection (step 104). For example, the correction unit 166 corrects at least one of the detectable range of the gesture and the reference direction for detecting the gesture. The corrections made by the correction unit 166 are roughly classified into (1) corrections based on information on gestures and (2) corrections based on information on device states.

(1) Correction based on information on gestures First, corrections based on information on gestures will be described.

The correction unit 166 corrects at least one of the detectable range AR and the reference direction based on the information acquired in step S102. For example, the correction unit 166 corrects at least one of the detectable range AR and the reference direction based on the information of the operation of the user U. <1-2. As described in Solution>, the action of the user U may be the swing-up action of the hand H. In addition, <1-2. As described in Solution>, the swing-up motion may be an approach motion in which the user U enters the hand H into the detectable range AR. At this time, the correction unit 166 may correct at least one of the detectable range AR and the reference direction based on the information of the approach direction AD of the user U's hand H to the detectable range AR.

The information on the approaching motion may be the information on the approaching direction AD of the user U's hand H. At this time, the correction unit 166 may correct at least one of the detectable range AR and the reference direction based on the information of the approach direction AD. More specifically, the correction unit 166 corrects at least one of the detectable range AD and the reference direction based on the information regarding the mounting state of the information processing apparatus 10 to the user U estimated based on the information of the approach direction AD. You may. The information regarding the mounting state may be, for example, information regarding the mounting inclination. The mounting tilt is, for example, the tilt of the current information processing apparatus 10 from the standard mounting state of the information processing apparatus 10. The estimation of the mounting inclination may be performed by the estimation unit 165 of the control unit 16 based on the information of the approach direction AD. Then, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on the estimation result of the estimation unit 165.

Note that the user's actions constituting one gesture may include a first action that triggers the start of the gesture and a second action following the first action. Here, the first operation is a swing-up operation of the hand H of the user U. Then, after the swing-up motion is detected in the detectable range AR, the correction unit 166 may widen the detectable range for detecting the second motion to be wider than the detectable range AR when the swing-up motion is detected. good. More specifically, when the correction unit 166 detects the swing-up motion in the detectable range for detecting the second motion when the gesture to be detected (second motion) is a gesture with a movement width. It may be wider than the detectable range AR of. The gesture accompanied by the movement width is, for example, a swipe or an action of moving the hand H away.

This makes it possible to improve the accuracy of gesture detection while suppressing false detection.

(2) Correction based on information on device status Next, correction based on information on device status will be described.

The correction unit 166 corrects at least one of the detectable range AR and the reference direction based on the information acquired in step S103. For example, the correction unit 166 corrects at least one of the detectable range AR and the reference direction based on the information regarding the mounting state of the information processing apparatus 10 to the user U, which is estimated based on the information from the state detection unit 12. .. The mounting tilt is, for example, the tilt of the current information processing apparatus 10 from the standard mounting state of the information processing apparatus 10. The estimation of the mounting inclination may be performed by the estimation unit 165 of the control unit 16. Then, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on the estimation result of the estimation unit 165.

Further, the correction unit corrects at least one of the detectable range AR and the reference direction based on the information regarding the posture of the user U estimated based on the information from the state detection unit 12. The information regarding the posture of the user U is information indicating that the user is lying on his back or the like. The information regarding the posture of the user U may be information in the direction of gravity detected by the acceleration sensor or the gyro sensor. The estimation unit 165 of the control unit 16 may estimate the posture of the user U. Then, the correction unit 166 may correct at least one of the detectable range AD and the reference direction based on the estimation result of the estimation unit 165.

Next, the gesture detection unit 162 of the control unit 16 determines whether or not the gesture has been detected (step S105). For example, the gesture detection unit 162 determines whether or not the first operation is detected in the detection area OR and then the second operation is also detected in the detection area OR. When the gesture is not detected (step S105: No), the gesture detection unit 162 returns the process to step S101.

Subsequently, when the gesture is detected (step S105: Yes), the gesture detection unit 162 determines whether the detected gesture is executed within the detectable range AR (step S106). For example, the gesture detection unit 162 determines whether the first motion is detected in the detectable range AR and then the second motion is detected in the detectable range AR. If the second operation is a gesture with a movement width such as a left / right swipe, whether the gesture was executed within the detectable range AR depending on whether the start position and the end position are both within the detectable range AR. It may be determined.

Note that the detectable range AR for detecting the first operation and the detectable range AR for detecting the second operation may have different widths. For example, when the second motion is a gesture with a movement width, the control unit 16 makes the detectable range for detecting the second motion wider than the detectable range AR when the swing-up motion is detected. May be good.

Further, in order to facilitate the input operation by the gesture by the user U, when the first operation direction is detected in the detectable range AR, the output control unit 164 controls the output unit 13 and is accepting the gesture detection. You may notify the user that this is the case. At this time, the notification given by the output control unit 164 may be a sound.

If the gesture is not executed within the detectable range (step S106: No), the gesture detection unit 162 returns the process to step S101.

When the hand H of the user U enters the detectable range AR from the non-detectable range NR, the gesture detection unit 162 said that the second operation is performed within the detectable range AR regardless of whether or not the second operation is performed. Gestures may not be detected.

19A and 19B are diagrams showing an example of a case where the gesture detection unit 162 does not detect a gesture. As shown in FIGS. 19A and 19B, the detectable range AR is unevenly arranged in the detection region OR so that a part of the edge of the detectable range AR touches a part of the edge of the detection region OR. More specifically, the detectable range AR is unevenly arranged on the front side of the user U. As shown in FIG. 19A, when the user U's hand H enters from the detectable range AR from the non-detectable range NR, the user U's entry into the detectable range AR is not due to the swing-up operation. Is assumed. Therefore, even if the second operation is performed within the detectable range AR as shown in FIG. 19B, the gesture detection unit 162 does not detect the gesture.

On the other hand, the gesture detection unit 162 indicates that the second operation is performed within the detectable range AR when the user U's hand H directly enters the detectable range AR instead of from the non-detectable range NR. The gesture is detected as a condition.

20A and 20B are diagrams showing an example of a case where the gesture detection unit 162 detects a gesture. As shown in FIG. 20A, the user U directly enters the detectable range AR of the hand H, and then performs the second operation as shown in FIG. 20B. In the example of FIG. 20B, since the second motion is a gesture accompanied by a movement width, the control unit 16 sets the detectable range for detecting the second motion to the detectable range AR when the first motion is detected ( It is wider than the detectable range AR) shown in FIG. 20A. In the example of FIG. 20B, since the second operation is performed within the detectable range AR, the gesture detection unit 162 detects the gesture.

In this way, when the user U's hand H enters the detectable range AR from the detectable range NR, the gesture detection unit 162 prevents the gesture from being detected, and the user U's hand H does not detect the non-detectable range NR. When entering the detectable range AR directly instead of from the body, the possibility of erroneous detection of the gesture is further reduced by detecting the gesture.

Returning to FIG. 18, when the gesture is executed within the detectable range (step S106: Yes), the command execution unit 163 of the control unit 16 executes the command corresponding to the detected gesture (step S107). For example, the command execution unit 163 executes the function of the information processing apparatus 10 associated with the detected gesture.

Next, the correction unit 166 executes a correction process related to gesture detection (step S108). For example, the correction unit 166 corrects at least one of the detectable range and the reference direction based on the information of the gesture detected by the gesture detection unit 162. For example, the correction unit 166 widens or narrows the detectable range AR based on the information on the start position and the end position of the swipe. Further, the correction unit 166 finely adjusts the reference direction from the movement direction of the detected left / right swipe or up / down swipe.

When the correction process is completed, the control unit 16 returns the process to step S101 and executes the process of steps S101 to S108 again.

According to the present embodiment, since the information processing apparatus 10 corrects the detectable range, it is possible to reduce the situation where even an operation not intended by the user as a gesture is detected as a gesture. Further, since the information processing apparatus 10 corrects the reference direction, it is less likely that the gesture intended by the user is mistaken for another gesture.

<< 3. Embodiment 2 >>
In the first embodiment, the information processing apparatus 10 detects the movement of the user's hand H and executes a command. However, the device that detects the movement of the user's hand H and the device that executes the command may be different devices. Hereinafter, the information processing system 1 of the second embodiment will be described.

<3-1. Information processing system configuration>
FIG. 21 is a diagram showing a configuration example of the information processing system 1. The information processing system 1 is a system that executes various functions based on the gesture of the user. As shown in FIG. 21, the information processing system 1 includes an output device 20 and a terminal device 30. In the example of FIG. 21, the information processing system 1 includes an output device 20A and an output device 20B. In the example of FIG. 21, the output device 20 and the terminal device 30 are wirelessly connected, but the output device 20 and the terminal device 30 may be configured to be connectable by wire.

The information processing system 1 may be provided with only one output device 20 or may be provided with a plurality of output devices 20. For example, assuming that the output device 20 is an earphone, the information processing system 1 may include a pair of output devices 20 that are wirelessly connected to the terminal device 30 and are attached to the left and right ears of the user U, respectively. As an example, the output device 20A is an earphone worn on the left ear of the user U, and the output device 20B is an earphone worn on the right ear of the user U.

Note that one output device 20 does not necessarily have to be one integrated device. It is also possible to regard a plurality of separate devices that are functionally or practically related as one output device 20. For example, a pair of left and right earphones worn on the left and right ears of the user U may be regarded as one output device 20. Of course, one output device 20 may be one integrated earphone worn on one ear of the user U, or one integrated headphone worn on the left and right ears of the user U. good.

<3-2. Output device configuration>
First, the configuration of the output device 20 will be described.

The output device 20 is an acoustic output device that can be worn by a user such as earphones and headphones. The information processing device 10 may be a display device such as AR glass or MR glass that can be worn by the user. In the present embodiment, the output device 20 is a device that can be worn by the user U, and is provided with a portion located at least in the ear of the user U when worn. Then, at least a detection unit for detecting an object is arranged in this portion. Here, the detection unit is a functional block corresponding to the input detection unit 11 when the first embodiment is described as an example. If there are two left and right parts located in the ears of the user U, each of both parts may have a detection unit, or only one part may have a detection part.

FIG. 22 is a diagram showing a configuration example of the output device 20 according to the embodiment of the present disclosure. The output device 20 includes an input detection unit 21, a state detection unit 22, an output unit 23, a communication unit 24, and a control unit 26. The configuration shown in FIG. 22 is a functional configuration, and the hardware configuration may be different from this. Further, the functions of the output device 20 may be distributed and implemented in a plurality of physically separated configurations.

The input detection unit 21 is a detection unit that detects a user's input operation. The state detection unit 22 is a sensor unit that detects the state of the output device 20. The output unit 23 is an output interface that outputs information to the user. The communication unit 24 is a communication interface for communicating with other devices such as the terminal device 30. The control unit 26 is a controller that controls each unit of the output device 20.

In addition, the input detection unit 21, the state detection unit 22, the output unit 23, the communication unit 24, and the control unit 26 are configured by the input detection unit 11, the state detection unit 12, the output unit 13, and the communication unit included in the information processing device 10. The configuration is the same as that of the control unit 16 and the control unit 16. In this case, the description of the information processing device 10 may be replaced with the output device 20 as appropriate.

<3-3. Terminal device configuration>
Next, the configuration of the terminal device 30 will be described.

The terminal device 30 is an information processing terminal capable of communicating with the output device 20. The terminal device 30 is a kind of information processing device of the present embodiment. The terminal device 30 is, for example, a mobile phone, a smart device (smartphone or tablet), a PDA (Personal Digital Assistant), or a personal computer. Further, the terminal device 30 may be a device such as a commercial camera equipped with a communication function, or may be a mobile body equipped with a communication device such as an FPU (Field Pickup Unit). Further, the terminal device 30 may be an M2M (Machine to Machine) device or an IoT (Internet of Things) device. The terminal device 30 controls the output device 20 from the outside of the output device 20 via wire or wireless.

FIG. 23 is a diagram showing a configuration example of the terminal device 30 according to the embodiment of the present disclosure. The terminal device 30 includes an input unit 31, a state detection unit 32, an output unit 33, a communication unit 34, a storage unit 35, and a control unit 36. The configuration shown in FIG. 23 is a functional configuration, and the hardware configuration may be different from this. Further, the functions of the terminal device 30 may be distributed and implemented in a plurality of physically separated configurations.

The input unit 31 is an input interface that accepts user input operations. For example, the input unit 31 is a button or a touch panel. The state detection unit 32 is a sensor unit that detects the state of the terminal device 30. The output unit 33 is an output interface that outputs information to the user. The communication unit 34 is a communication interface for communicating with other devices such as the output device 20. The storage unit 35 is a storage device capable of reading and writing data. The storage unit 35 stores, for example, information about the detectable range AR of the gesture. The control unit 36 is a controller that controls each unit of the terminal device 30. The control unit 36 includes an acquisition unit 361, a gesture detection unit 362, a command execution unit 363, an output control unit 364, a guessing unit 365, and a correction unit 366.

In addition, the input detection unit 21, the state detection unit 22, the output unit 23, the communication unit 24, and the control unit 26 are configured by the input detection unit 11, the state detection unit 12, the output unit 13, and the communication unit included in the information processing device 10. The configuration is the same as that of the control unit 16 and the control unit 16. Further, in the configuration of the acquisition unit 361, the gesture detection unit 362, the command execution unit 363, the output control unit 364, the guessing unit 365, and the correction unit 366, the acquisition unit 361 communicates with the input detection unit 21 and the state detection unit 22. The configuration is the same as that of the acquisition unit 161, the gesture detection unit 162, the command execution unit 163, the output control unit 164, the guessing unit 165, and the correction unit 166 included in the control unit 16 of the information processing apparatus 10 except that the information is acquired from. be. In this case, the description of the information processing device 10 may be appropriately replaced with the output device 20 or the terminal device 30.

<3-4. Information processing system operation>
The configuration of the information processing system 1 has been described above. Next, the operation of the information processing system 1 having such a configuration will be described.

(Command execution processing)
The information processing system 1 can execute a command execution process in the same manner as the information processing apparatus 10 of the first embodiment. The command execution process of the information processing system 1 is the same as the command execution process of the first embodiment except that the terminal device 30 acquires the sensor value from the output device 20. Hereinafter, the command execution process will be described with reference to FIG. 18 as in the first embodiment.

FIG. 18 is a flowchart showing a command execution process according to the present embodiment. The command execution process is executed by the control unit 36 of the terminal device 30. The command execution process is executed, for example, when the terminal device 30 establishes communication with the output device 20.

First, the acquisition unit 361 of the control unit 36 acquires the sensor value from the input detection unit 21 and the state detection unit 22 of the output device 20 via the communication unit 34 (step S101). For example, the acquisition unit 361 acquires information about an object in the detection region OR from the non-contact type sensor of the output device 20. Further, the acquisition unit 361 acquires information regarding contact of the output device 20 with the housing surface from the contact-type sensor of the output device 20. Further, the acquisition unit 361 acquires information regarding the state of the output device 20 from a sensor such as an acceleration sensor of the output device 20.

Subsequently, the acquisition unit 361 acquires information regarding the gesture of the user U based on the sensor value acquired in step S101 (step S102). For example, the acquisition unit 361 acquires information on the operation of the user U regarding the gesture based on the sensor value from the input detection unit 21.

Subsequently, the acquisition unit 361 acquires information regarding the state of the output device 20 based on the sensor value acquired in step S101 (step S103). For example, the acquisition unit 361 acquires information regarding the state of the output device 20 based on the sensor value from the state detection unit 22.

Next, the correction unit 366 of the control unit 36 executes the correction process related to the gesture detection (step 104). For example, the correction unit 366 corrects at least one of the detectable range of the gesture and the reference direction for detecting the gesture.

Next, the gesture detection unit 362 of the control unit 36 determines whether or not the gesture has been detected (step S105). For example, the gesture detection unit 362 determines whether or not the first operation is detected in the detection area OR and then the second operation is also detected in the detection area OR. When the gesture is not detected (step S105: No), the gesture detection unit 362 returns the process to step S101.

Subsequently, when the gesture is detected (step S105: Yes), the gesture detection unit 362 determines whether the detected gesture is executed within the detectable range AR (step S106). For example, the gesture detection unit 362 determines whether the first motion is detected in the detectable range AR and then the second motion is detected in the detectable range AR.

If the gesture is not executed within the detectable range (step S106: No), the gesture detection unit 362 returns the process to step S101.

On the other hand, when the gesture is executed within the detectable range (step S106: Yes), the command execution unit 363 of the control unit 16 executes the command corresponding to the detected gesture (step S107). For example, the command execution unit 363 executes the function of the output device 20 associated with the detected gesture.

Next, the correction unit 366 executes a correction process related to gesture detection (step S107). For example, the correction unit 366 corrects at least one of the detectable range and the reference direction based on the information of the gesture detected by the gesture detection unit 362.

When the correction process is completed, the control unit 36 returns the process to step S101 and executes the processes of steps S101 to S108 again.

Also in this embodiment, since the terminal device 30 corrects the detectable range, it is possible to reduce the situation where even an operation not intended by the user as a gesture is detected as a gesture. Further, since the terminal device 30 corrects the reference direction, it is possible to reduce the situation where the gesture intended by the user U is mistaken for another gesture.

(Manual setting of detectable range)
In the above embodiment, the terminal device 30 automatically sets the detectable range AR based on the sensor value. However, the terminal device 30 may be configured so that the user U can manually set the detectable range AR.

FIG. 24 is a diagram showing how the user U manually sets the size of the detectable range AR. The terminal device 30 is configured so that the user U can change the setting of the size of the detectable range AR by using the GUI (Graphical User Interface). Hereinafter, the processing of the terminal device 30 that enables the manual setting of the size of the detectable range AR will be described with reference to FIG. 24. The following processing is executed by the control unit 36 of the terminal device 30.

First, the acquisition unit 361 of the control unit 36 acquires information on the detectable range AR from the storage unit 35. For example, the storage unit 35 stores information indicating the current detection range AR setting. The acquisition unit 361 acquires the information from, for example, the storage unit 35. If information indicating the current detectable range AR setting is stored in another device (for example, an output device 20 or a device on the cloud), the acquisition unit 361 may be used via the communication unit 34. The information may be obtained from the device of. Then, the output control unit 364 visualizes the current detectable range AR setting based on the acquired information so that the user can visually recognize it, and displays it on the output unit 33 (step S201).

In the example of FIG. 24, a part of the detection area OR is the detectable range AR. The acquisition unit 361 acquires the operation information of the user U from the input unit 31 of the terminal device 30 (step S202). For example, the acquisition unit 361 acquires information on the operation of the user U to change the size of the detectable range AR from the input unit 31. In the example of FIG. 24, the terminal device 30 includes a touch panel display as an input unit 31, and the acquisition unit 361 acquires operation information for expanding the detectable range AR of the user U. More specifically, in the example of FIG. 24, the acquisition unit 361 acquires information that the user U is pinching out the hand H on the detectable range AR displayed on the touch panel display. There is.

The output control unit 364 visualizes the detectable range AR related to the change operation of the user U based on the information acquired in step S202 so that the user U can see it, and displays it on the output unit 33 (step S203).

Then, when the control unit 36 detects an operation (for example, pressing the OK button) instructing the user U to change the detectable range AR, the control unit 36 stores the information based on the information of the detectable range AR displayed in step S203. The setting of the detectable range AR stored in the unit 35 is updated.

The terminal device 30 may be configured so that the user U can manually change the position of the detectable range AR in the detection area OR.

FIG. 25 is a diagram showing how the user U manually sets the position of the detectable range AR. The terminal device 30 is configured so that the user U can change the setting of the position of the detectable range AR by using the GUI. Hereinafter, the processing of the terminal device 30 that enables the manual setting of the position of the detectable range AR will be described with reference to FIG. 25. The following processing is executed by the control unit 36 of the terminal device 30.

First, the acquisition unit 361 of the control unit 36 acquires information on the detectable range AR from the storage unit 35. Then, the output control unit 364 visualizes the current detectable range AR setting based on the acquired information so that the user can visually recognize it, and displays it on the output unit 33 (step S301).

In the example of FIG. 25, a part of the detection area OR is the detectable range AR. The acquisition unit 361 acquires the operation information of the user U from the input unit 31 of the terminal device 30 (step S302). For example, the acquisition unit 361 acquires information on the operation of the user U to change the position of the detectable range AR from the input unit 31. In the example of FIG. 25, the terminal device 30 includes a touch panel display as an input unit 31, and the acquisition unit 361 acquires operation information to change the position of the detectable range AR of the user U. More specifically, in the example of FIG. 25, in the acquisition unit 361, the user U touches the detectable range AR displayed on the touch panel display, and the user U swipes the finger toward the position to be moved. The information of is acquired.

The output control unit 364 visualizes the detectable range AR related to the change operation of the user U based on the information acquired in step S302 so that the user U can see it, and displays it on the output unit 33 (step S303).

Then, when the control unit 36 detects an operation (for example, pressing the OK button) instructing the user U to change the detectable range AR, the control unit 36 stores the information based on the information of the detectable range AR displayed in step S303. The setting of the detectable range AR stored in the unit 35 is updated.

By allowing the user U to manually set the size and position of the detectable range AR, it is possible to set the detectable range AR according to the individuality of the user U. As a result, false positives of gestures are reduced. In addition to the GUI (Graphical User Interface), the detectable range AR or the non-detectable range NR may be adjusted in conjunction with, for example, the voice recognition function of the terminal device 30. For example, when the user recognizes the voice "widen the detectable range AR", the detectable range AR is changed step by step.

(Manual setting of gesture assignment)
In the above embodiment, the combination of gesture and function is predetermined. However, the terminal device 30 may be configured so that the user U can manually set the combination of the gesture and the function.

At this time, the terminal device 30 may be configured to store the operation history information related to the gesture of the user U in the storage unit 35. Then, the terminal device 30 may aggregate the gestures frequently used by the user U and their success rates based on the information in the operation history, and display the aggregated results so that the user U can visually recognize them. FIG. 26 is a diagram showing how the gestures frequently used by the user U and their success rates are displayed. In the example of FIG. 26, it can be seen that the user U uses the swipe operation a lot, but the success rate of the swipe operation is poor.

The terminal device 30 may be configured so that the user U can change the association between the gesture and the function. For example, the terminal device 30 may have a GUI for changing the association between the gesture and the function. In the example of FIG. 26, the user U modifies the function associated with the swipe operation (eg, Next / Prev) using the GUI to associate with the pinch operation, for example.

The terminal device 30 may be configured to propose the detectable range AR to the user based on the information in the operation history. Further, the terminal device 30 may be configured so that the user can change the detectable range AR based on the proposal.

By allowing the user U to manually set the association between the gesture and the function, it is possible to associate the gesture with the function according to the individuality of the user U. As a result, the user U can associate the gesture with a high success rate with the frequently used function, so that the false detection of the gesture is reduced.

<< 4. Embodiment 3 >>
In the first and second embodiments, the non-contact sensor is mainly used to determine the user's operation on the device. In the third embodiment, the contact sensor and the non-contact sensor are combined to determine the user's device operation. Hereinafter, the information processing apparatus 40 of the third embodiment will be described.

<4-1. Outline of Embodiment 3>
Gesture detection using a non-contact sensor changes the range in which gestures can be detected according to the orientation and posture of the user's face. Therefore, the detection accuracy of gesture detection using a non-contact sensor is low. This is because the position of the gesture is absolutely determined by the location of the sensor, but the user relatively recognizes the position of the gesture regardless of the posture state. To solve this problem, it is possible to expand the detectable range of gestures. However, if the detectable range of the gesture is unnecessarily expanded, the information processing apparatus may mistakenly recognize other misleading actions as the gesture. Therefore, it is desirable to limit the detectable range as much as possible.

Note that the device operation (aerial operation) by gesture has a small load on the user. However, since gestures vary greatly from person to person (that is, the gesture position is not reproducible), the accuracy of device operation is low. On the other hand, device operation (physical operation) by contact with the device or the human body has high accuracy of device operation. However, in physical operation, since the operation surface is limited, the operation area becomes narrow and the commands that can be input are reduced.

Therefore, in the third embodiment, the contact sensor and the non-contact sensor are combined. More specifically, the information processing apparatus 40 of the third embodiment makes a correction related to gesture detection or a correction related to device operation based on the sensor information from the contact sensor and the sensor information from the non-contact sensor.

For example, the information processing apparatus 40 detects the user's gesture using at least the sensor information from the contact sensor, and estimates the gesture position at that time based on the sensor information from the non-contact sensor. Then, the information processing apparatus 40 corrects the detectable range of the gesture based on the information of the estimated gesture position. As a result, the detectable range of the gesture is updated at any time according to the usage status of the user, so that the detection accuracy of the gesture is improved.

Further, the information processing device 40 may be configured to detect the user's gesture by using at least the sensor information from the non-contact sensor. Then, the information processing device 40 may correct the detectable range of the gesture based on the sensor information from the non-contact sensor when the user touches the face or the information processing device 40. As a result, the detectable range of the gesture is corrected based on the position of the physical contact, so that the detection accuracy of the gesture can be improved.

The information processing device 40 may be configured to detect the user's gesture using at least the sensor information from the contact sensor. Then, the information processing apparatus 40 may be configured to execute a command corresponding to the detected gesture. The command may include at least a command with an operation amount. At this time, the information processing apparatus 40 may determine the operation amount based on the sensor information from the non-contact sensor. As a result, the information processing apparatus 40 can determine the operation amount by using the non-contact sensor while realizing highly accurate gesture detection by using the contact sensor.

<4-2. Information processing device configuration>
The outline of the third embodiment has been described above, but next, the configuration of the information processing apparatus 40 will be described.

FIG. 27 is a diagram showing a configuration example of the information processing apparatus 40 according to the third embodiment. The information processing device 40 includes an input detection unit 41, a state detection unit 42, an output unit 43, a communication unit 44, a storage unit 45, and a control unit 46. The configuration shown in FIG. 27 is a functional configuration, and the hardware configuration may be different from this. Further, the functions of the information processing apparatus 40 may be distributed and implemented in a plurality of physically separated configurations. For example, as shown in the second embodiment, the functions of the information processing apparatus 40 may be distributed and mounted in the output apparatus 20 and the terminal apparatus 30.

The input detection unit 41 is a detection unit that detects a user's input operation. The input detection unit 11 includes a non-contact type detection unit (hereinafter, referred to as a non-contact sensor) that detects an object non-contactly, and a contact-type detection unit (hereinafter, contact) that detects contact with the user's human body or an object. It is called a sensor).

The non-contact sensor may be composed of one or a plurality of proximity sensors that detect an object located in the detection area. The proximity sensor may be an optical sensor or a capacitance type sensor. An optical proximity sensor is a sensor that detects the light reflected by an object. The capacitance type proximity sensor is a sensor that detects a change in capacitance that occurs between an object and the sensor. The non-contact sensor may be a 2D sensor or a 3D sensor.

The non-contact sensor includes one or a plurality of touch sensors, and detects an object that comes into contact with a predetermined place of the information processing apparatus 10. If the information processing device 40 is an earphone type device, the predetermined location is, for example, the side surface of the earphone (opposite the surface of the speaker). If the information processing device 10 is a headphone type device, the predetermined location is, for example, the side surface of the headphones (opposite the surface of the speaker). In the following description, the side surface of the earphone or the side surface of the headphone may be referred to as the housing surface.

The non-contact sensor is not limited to the sensor that detects contact with the device. For example, the non-contact sensor may be a sensor that detects contact with the human body of the user who wears the information processing device 40. For example, if the information processing device 40 is an earphone type or headphone type device, the non-contact sensor may be a sensor that detects contact with the face of the user wearing the information processing device 40. In this case, the non-contact sensor may be configured to detect contact with the user's face, for example, by detecting vibration when the user touches his / her face with his / her hand.

The state detection unit 42 is a sensor unit that detects the state of the information processing device 40. The state detection unit 42 may be configured to detect the user's state.

As shown in FIG. 27, the control unit 46 includes a first acquisition unit 461A, a second acquisition unit 461B, a gesture detection unit 462, a command execution unit 463, an output control unit 164, and a guessing unit 165. , And a correction unit 166. Each block (acquisition unit 161 to correction unit 166) constituting the control unit 46 is a functional block indicating the function of the control unit 46, respectively. These functional blocks may be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks may be one software module realized by software (including a microprogram), or may be one circuit block on a semiconductor chip (die). Of course, each functional block may be one processor or one integrated circuit. The method of configuring the functional block is arbitrary.

The control unit 16 may be configured in a functional unit different from the above-mentioned functional block. Further, another device may perform a part or all of the operations of each block (first acquisition unit 461A to correction unit 466) constituting the control unit 16. For example, a terminal device such as a music player or a smartphone may perform some or all operations of each block constituting the control unit 46. The operation of each block constituting the control unit 46 will be described later.

In addition, the input detection unit 41, the state detection unit 42, the output unit 43, the communication unit 44, and the control unit 46 are configured with the input detection unit 11, the state detection unit 12, the output unit 13, and the communication unit included in the information processing device 10. The configuration is the same as that of the control unit 16 and the control unit 16.

<4-3. Information processing device operation>
The configuration of the information processing apparatus 40 has been described above. Next, the operation of the information processing apparatus 40 having such a configuration will be described.

As described above, in the third embodiment, the contact sensor and the non-contact sensor are combined. More specifically, the information processing apparatus 40 makes a correction related to gesture detection or a correction related to device operation based on the sensor information from the contact sensor and the sensor information from the non-contact sensor. Some specific examples are shown below.

<4-3-1. Example 1>
First, Example 1 will be described.

(Contents of Example 1)
Example 1 assumes a gesture due to physical contact with the user's face or device. FIG. 28 is a diagram showing a user performing a gesture by physical contact. In the example of FIG. 28, the detectable range A1 of the contact sensor is set to a part of the left cheek of the user, and the detectable range A2 of the non-contact sensor is set to the vicinity of the left cheek including a part of the left cheek of the user. Has been done. In the example of FIG. 28, a part of the detectable range A1 and a part of the detectable range A2 overlap. In the example of FIG. 28, the user operates the device by tapping the cheek.

In the existing case, since the operation area for contact is set to a single unit, the same gesture is recognized regardless of the user tapping any position on the face or device. Therefore, the number of commands may be limited. Therefore, in Example 1, the information processing apparatus 40 estimates the operation area (gesture position) at the time of contact based on the sensor information from the non-contact sensor. As a result, the information processing apparatus 40 makes it possible to determine in which area within the detectable range A1 the user performed the gesture.

There are individual differences in the size and position of the operation area depending on the user. Therefore, the information processing apparatus 40 corrects the detectable range of the gesture based on the information of the estimated gesture position. For example, the information processing apparatus 40 increases the size of the detectable range A1 when the operating area (gesture position) covers a range equal to or larger than a predetermined ratio of the current detectable range A1. When the operation area (gesture position) is located at the end of the current detectable range A1, the position of the detectable range A1 is corrected so that the operation area is in the center. As a result, the detectable range A1 of the gesture is updated at any time according to the usage status of the user, so that the detection accuracy of the gesture is improved.

Note that the user's actions constituting one gesture may include a first action that triggers the start of the gesture and a second action following the first action. In this case, the information processing apparatus 40 may narrow the detectable range of the second operation based on the information of the operation area (gesture position) of the first operation. This makes it possible to prevent erroneous detection of gestures due to daily activities.

The information processing device 40 executes a command corresponding to the detected gesture. At this time, the information processing apparatus 40 may change the command to be executed even if the gestures detected based on the sensor information from the contact sensor are the same gestures but the gesture positions are different. That is, the information processing apparatus 40 determines a command to be executed based on the gesture detected based on the sensor information from the contact sensor and the gesture position information detected based on the sensor information from the non-contact sensor. You may. As a result, the information processing apparatus 40 can assign many commands to one gesture while realizing highly accurate gesture detection by physical contact.

As shown in FIG. 28, when a part of the detectable range A1 and a part of the detectable range A2 overlap, the information processing apparatus 40 detects the gesture due to physical contact with the non-contact sensor alone. It is also possible.

(Processing example)
FIG. 29 is a flowchart showing an example of the command execution process according to the third embodiment. The command execution process is executed by the control unit 46 of the information processing apparatus 40. The command execution process is executed, for example, when the information processing apparatus 40 is turned on.

First, the second acquisition unit 461B of the information processing apparatus 40 acquires the sensor value of the non-contact sensor (step S201). Then, the guessing unit 465 of the information processing apparatus 40 estimates the gesture position based on the sensor value of the non-contact sensor (step S202). Further, the first acquisition unit 461A of the information processing apparatus 40 acquires the sensor value of the contact sensor (step S203).

Next, the gesture detection unit 462 of the information processing apparatus 40 determines whether or not the gesture has been detected based on the sensor value of the contact sensor (step S204). When the gesture is not detected (step S204: No), the gesture detection unit 462 returns the process to step S201.

On the other hand, when the gesture is detected (step S204: Yes), the guessing unit 465 of the information processing apparatus 40 estimates the device state (step S205). Then, the gesture detection unit 462 of the information processing apparatus 40 determines the gesture based on the estimation result of the device state and the sensor value of the contact sensor (step S206). Then, the correction unit 466 of the information processing apparatus 40 corrects the detectable range of the gesture (in the example of FIG. 28, the detectable range A1) (step S207). After that, the command execution unit 463 executes the command assigned to the determined gesture (step S208). After executing the command, the command execution unit 463 returns the process to step S201.

<4-3-2. Example 2>
Next, Example 2 will be described.

(Contents of Example 2)
In Example 2, a user's non-contact gesture is assumed. FIG. 30 is a diagram showing a user performing a gesture by aerial motion. In the example of FIG. 30, the detectable range A1 of the contact sensor is set in the information processing device 40 and its surroundings, and the detectable range A2 of the non-contact sensor is set at a position separated to the left from the user's head. There is. In the example of FIG. 30, the detectable range A1 and the detectable range A2 are separated from each other and do not overlap. In the example of FIG. 30, the user operates the device by performing a predetermined aerial operation on the left side of the head.

When the non-contact sensor mainly detects gestures, the gesture position may vary greatly depending on the user's condition. Therefore, when the detection of the non-contact gesture is not successful due to the variation in the gesture position, the information processing apparatus 40 urges the user to come into contact with the human body, an object, or an arbitrary space point. For example, the information processing device 40 prompts the user to tap the face or physically touch the information processing device 40.

Then, the information processing apparatus 40 calibrates the detectable range A2 of the gesture based on the operation of the user's physical touch. For example, the information processing apparatus 40 corrects the detectable range A2 of the gesture based on the sensor information from the non-contact sensor when the user touches a human body, an object, or an arbitrary space point. More specifically, the information processing apparatus 40 detects contact with the user's human body, object, or arbitrary space point based on the sensor information from the contact sensor, and detects contact with the user's human body or object. The detectable range A2 of the gesture is corrected based on the sensor information from the non-contact sensor at that time. As a result, the detectable range A2 of the gesture is corrected based on the position of the physical contact, so that the detection accuracy of the gesture is improved.

Note that the information processing apparatus 40 may provide feedback to the user by sound or the like when the user's hand is located in the corrected detectable range A2 after detecting the contact with the user's body or an object. Further, when the gesture cannot be detected by the non-contact sensor due to the posture state or context of the user (for example, walking or running), the information processing apparatus 40 recognizes the user's hand in the detectable range A2 of the non-contact sensor. Occasionally, feedback may be given to encourage physical operation.

(Processing example)
FIG. 31 is a flowchart showing another example of the command execution process according to the third embodiment. The command execution process is executed by the control unit 46 of the information processing apparatus 40. The command execution process is executed, for example, when the information processing apparatus 40 is turned on.

First, the second acquisition unit 461B of the information processing apparatus 40 acquires the sensor value of the non-contact sensor (step S301). Then, the gesture detection unit 462 of the information processing apparatus 40 determines whether or not the gesture has been detected based on the sensor value of the non-contact sensor (step S302).

When the gesture is not detected (step S302: No), the first acquisition unit 461A of the information processing apparatus 40 acquires the sensor value of the contact sensor (step S303). Then, the correction unit 466 of the information processing apparatus 40 executes an operation related to calibration of the detectable range A2 of the gesture (step S304). For example, the correction unit 466 prompts the user to tap the face or physically touch the information processing device 40. The correction unit 466 corrects the detectable range A2 of the gesture based on the sensor information from the non-contact sensor when the user comes into contact with the human body or an object (step S305). When the correction is completed, the correction unit 466 returns the process to step S301.

On the other hand, when the gesture is detected (step S302: Yes), the guessing unit 465 of the information processing apparatus 40 estimates the device state (step S306). Then, the gesture detection unit 462 of the information processing apparatus 40 determines the gesture based on the estimation result of the device state and the sensor value of the contact sensor (step S307). Then, the command execution unit 463 executes the command assigned to the determined gesture (step S308). After executing the command, the command execution unit 463 returns the process to step 301.

<4-3-3. Example 3>
Next, Example 3 will be described.

(Contents of Example 3)
Some commands include operations. FIG. 32 is a diagram showing how a command accompanied by an operation amount is input. For example, if the command is a volume control command (Vol + / Vol-), the operation amount is volume. In the case of the volume control command, the movement direction of the user's hand corresponds to the change direction of the volume (whether the volume is increased or decreased), and the movement amount or movement speed of the user's hand becomes the change amount (input value) of the sound. handle.

In Example 3, the contact sensor and the non-contact sensor are combined to enable not only a command but also an operation amount (change direction and change amount) to be input. For example, the information processing apparatus 40 detects the user's gesture using at least the sensor information from the contact sensor. Then, the information processing apparatus 40 determines the operation amount based on the sensor information from the non-contact sensor. As a result, the information processing apparatus 40 can determine the operation amount by using the non-contact sensor while realizing highly accurate gesture detection by using the contact sensor.

Note that the user's actions constituting one gesture may include a first action that triggers the start of the gesture and a second action following the first action. If the command is a volume control command, the first action is, for example, a face tap, and the second action is, for example, a hand movement in a predetermined direction (for example, in the front-back direction) from the face tap. In this case, the information processing apparatus 40 may determine the volume to be changed based on the moving direction, the moving amount, and / or the moving speed of the second operation.

The size and speed of gestures differ from person to person. Therefore, in the case of a command involving an operation amount, the operation amount to be input may be low in accuracy. Therefore, the information processing apparatus 40 makes corrections related to the second operation based on information on the speed and magnitude of the first operation (for example, the speed of the hand until the face is tapped and the strength of the face tap).

For example, the information processing device 40 determines the speed of the first operation based on the information from the non-contact sensor. Then, the information processing apparatus 40 corrects a set value (for example, a threshold value) for associating the second operation with the operation amount based on the information of the speed of the first operation. After that, the information processing apparatus 40 determines the speed or the amount of movement of the second operation based on the sensor information from the non-contact sensor. Then, the information processing apparatus 40 determines the operation amount based on the corrected set value and the information of the speed or the movement amount of the second operation. As a result, the information processing apparatus 40 can improve the input accuracy of the user's operation amount.

A user with a fast gesture input speed may bring his / her hand to a distant place, so it is desirable that the range in which the gesture can be detected is wide. On the other hand, a user with a slow gesture input speed is unlikely to bring his / her hand to a distant place. Therefore, considering false detection, it is desirable that the range in which the gesture can be detected is narrow. Therefore, the information processing apparatus 40 makes a correction regarding the detection of the second operation based on the information of the first operation of the user.

For example, the information processing device 40 determines the speed of the first operation based on the sensor information from the non-contact sensor. Then, the information processing apparatus 40 corrects the detectable range of the second operation based on the information of the speed of the first operation. As a result, the detection accuracy of the second operation can be made high.

<< 5. Modification example >>
Each of the above embodiments shows an example, and various modifications and applications are possible.

(Modified example of gesture detection)
For example, the information processing apparatus 10 of the first embodiment may change the detectable range AR according to the distance to an obstacle. FIG. 33 is a diagram showing how the detectable range AR is changed according to the distance to the obstacle. For example, the information processing apparatus 10 includes a pair of

input detection units

11 _R and 11 _L. As shown in FIG. 33, the obstacle O1 is located in the right direction (Y-axis minus direction) _{of the input detection unit 11 R} , and in the left direction (Y-axis plus direction) of _{the input detection unit 11 L.} Obstacle O2 is located. The obstacle may be a structure such as a wall, or may be an object independent of the structure.

The input detection unit 11 _R and the input detection unit 11 _L each detect the distance to an obstacle. In the example of FIG. 33, the distance to the obstacle O1 is d1, and the distance to the obstacle O2 is d2. Then, based on this information, the information processing apparatus 10 adjusts the detectable range AR so that an obstacle does not enter the detectable range AR. In the example of FIG. 33, the information processing apparatus 10 corrects the detectable range AR _R _{of the input detection unit 11 R} so that the maximum distance in the separation direction (Y-axis minus direction) is d3, which is shorter than d1. .. Further, the information processing apparatus 10, the detectable range AR _L of the input detection unit 11L, the maximum distance separating direction (Y-axis plus direction) is corrected so as to be shorter than d2 d4.

The information processing apparatus 10 to d3 and d4 may be corrected detectable range AR _R and detectable range AR _L to be the same distance, the detectable range AR _R and detection to d3 and d4 is different distances range AR _L may be corrected. Further, the information processing apparatus 10 may correct only one of the _{detectable range AR R} and the detectable range AR _L. If the position of the obstacle and / or the user U is known in advance, the information processing apparatus 10 may correct the detectable range AR based on the position information.

From this, it is possible to reduce the situation where the information processing apparatus 10 cannot detect the gesture due to an obstacle, so that the gesture detection accuracy becomes high. This modification can also be applied to the information processing system 1 of the second embodiment.

(Transformation example related to function execution)
The information processing apparatus 10 of the first embodiment may include a pair of predetermined parts located in both ears of the user U when worn. In this case, the predetermined portion may be a portion where the sound of the earphone or the headphone is output. Then, when the swinging motion of the user U is detected in both of the pair of predetermined parts, the information processing apparatus 10 does not execute the function associated with the gesture even if the gesture is detected. You may do it.

Similarly, the output device 20 of the second embodiment may include a pair of predetermined parts located in both ears of the user U when worn. In this case, the predetermined portion may be a portion where the sound of the earphone or the headphone is output. Then, when the swing-up motion of the user U is detected in both of the pair of predetermined parts, the terminal device 30 of the second embodiment performs the function associated with the gesture even when the gesture is detected. You may not execute it.

If the swinging motion is detected on both the left and right sides at the same time, it is assumed that the user U is not performing a gesture but has released the device such as earphones or headphones. By not executing the function in such a case, the information processing device 10 or the terminal device 30 can reduce operations that do not meet the user's intention.

(Modification example related to operation amount)
The function of the information processing apparatus 10 of the first embodiment may include a predetermined function accompanied by an operation amount. At this time, the predetermined function accompanied by the operation amount may be a function related to the volume operation (Vol + / Vol−) or a function related to the reproduction speed. Further, the predetermined function accompanied by the operation amount is not limited to these, and may be, for example, a function related to fast forward, rewind, and slow playback. A gesture with a movement width, such as a swipe, may be associated with a predetermined function. When the gesture detected by the gesture detection unit 162 is a gesture with a movement width, the information processing apparatus 10 has a relative movement width of the gesture based on the size of the detectable range. The operation amount of a predetermined function (for example, the amount of sound to be raised or lowered) may be determined based on the above.

Similarly, the function of the terminal device 30 of the second embodiment may include a predetermined function accompanied by an operation amount. Then, a gesture with a movement width such as a swipe may be associated with the predetermined function. Then, when the gesture detected by the gesture detection unit 362 is a gesture with a movement width, the terminal device 30 determines the relative movement width of the gesture based on the size of the detectable range. Based on this, the operation amount of a predetermined function (for example, the amount of sound to be raised or lowered) may be determined.

This makes it possible to input the operation amount with one gesture, so that the usability of the information processing device 10 or the terminal device 30 is improved.

(Modified example of device mode)
In the above-described embodiment, the information processing device 10 and the output device 20 are devices that can be worn by the user (wearable devices), but are not necessarily wearable devices. For example, the information processing device 10 and the output device 20 may be devices installed and used in a structure or a moving body such as a television, a car navigation system, a driver's cab, and various operation panels. Further, the information processing device 10 and the output device 20 may be the mobile body itself.

Here, the mobile body may be a mobile terminal such as a smartphone, a mobile phone, a personal computer, a music player, or a portable TV, or may be a remote controller for operating a device. Further, the moving body may be a moving body moving on land (for example, a vehicle such as a car, a bicycle, a bus, a truck, a motorcycle, a train, a linear motor car, etc.) or in the ground (for example, in a tunnel). It may be a moving body (for example, a subway) that moves around. Further, the moving body may be a moving body moving on the water (for example, a ship such as a passenger ship, a cargo ship, a hovercraft, etc.), or a moving body moving underwater (for example, a submersible, a submarine, an unmanned submarine, etc.). It may be a submarine). Further, the moving body may be a moving body moving in the atmosphere (for example, an aircraft such as an airplane, an airship, or a drone), or a moving body moving outside the atmosphere (for example, an artificial celestial body such as a space station). There may be.

The structure is, for example, a high-rise building, a house, a steel tower, a station facility, an airport facility, a port facility, a stadium, or the like. The concept of structure includes not only buildings but also structures such as tunnels, bridges, dams, walls, and iron pillars, and equipment such as cranes, gates, and windmills. Further, the concept of a structure includes not only a structure on land or in the ground, but also a structure on water such as a pier and a mega float, and a structure underwater such as an ocean observation facility.

(Modification example of the sensor used)
In the third embodiment, the information processing apparatus 40 detects the user's gesture using both the non-contact sensor and the contact sensor, but it is not always necessary to continue using both the non-contact sensor and the contact sensor. For example, the information processing apparatus 40 may decide whether to continue using both the non-contact sensor and the contact sensor according to the remaining battery level, or to use either the non-contact sensor or the contact sensor. Further, the information processing apparatus 40 may change the settings related to the sensor to be used when there is a special environmental change such as wearing a hat.

(Modification example related to correction)
In the third embodiment, the information processing apparatus 40 corrects the detectable range of the sensor based on the sensor value, but the detectable range may be corrected by using information other than the sensor value. For example, the information processing apparatus 40 may correct the detectable range according to the type of the device of the information processing apparatus 40 (headphones / earphones / head-mounted display, etc.). For example, when the type of the device of the information processing apparatus 40 is headphones, there is a margin in the area where the user can physically contact, so the correction of the detectable range A1 is made small. Further, another sensor (for example, a biological sensor such as a heart rate sensor) may be used to estimate the user's state, and the detectable range may be corrected based on the estimation result.

(User detectable range setting)
In the third embodiment, the information processing apparatus 40 corrects the detectable range of the sensor based on the sensor value, but may be configured so that the user can set the detectable range. 34A and 34B are diagrams showing an example of a GUI (Graphical User Interface) for setting a detectable range. The GUI shown in FIG. 34A is a GUI for setting the detectable range in the plane direction (front-back and left-right directions toward the user's ear E), and the GUI shown in FIG. 34B is separated from the depth direction (user's ear E). It is a GUI for setting the detectable range of (direction). The area A3 corresponds to, for example, the detectable range A1 of the contact sensor, and the area A4 corresponds to, for example, the detectable range A2 of the non-contact sensor. Alternatively, the area A3 corresponds to, for example, the detectable range A2 of the non-contact sensor, and the area A4 corresponds to, for example, the detection area OR of the non-contact sensor. The user sets the detectable range of the sensor using the GUI shown in FIGS. 34A and 34B. The information processing device 40 may visualize the difference in the operation area between the headphones and the earphones. Further, the information processing apparatus 40 may be used as a tutorial on the GUI when switching devices.

(Other variants)
The control device for controlling the information processing device 10, the output device 20, or the terminal device 30 of the present embodiment may be realized by a dedicated computer system or a general-purpose computer system.

For example, a communication program for executing the above operation is stored and distributed in a computer-readable recording medium such as an optical disk, a semiconductor memory, a magnetic tape, or a flexible disk. Then, for example, the control device is configured by installing the program in a computer and executing the above-mentioned processing. At this time, the control device may be an information processing device 10, an output device 20, or an external device (for example, a personal computer) of the terminal device 30. Further, the control device may be an internal device (for example, control unit 16, control unit 26, or control unit 36) of the information processing device 10, the output device 20, or the terminal device 30.

Further, the above communication program may be stored in a disk device provided in a server device on a network such as the Internet so that it can be downloaded to a computer or the like. Further, the above-mentioned functions may be realized by the collaboration between the OS (Operating System) and the application software. In this case, the part other than the OS may be stored in a medium and distributed, or the part other than the OS may be stored in the server device so that it can be downloaded to a computer or the like.

Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or part of it can be done automatically by a known method. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.

Further, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

Further, the above-described embodiments can be appropriately combined in a region where the processing contents do not contradict each other. Further, the order of each step shown in the flowchart of the above-described embodiment can be changed as appropriate.

Further, for example, the present embodiment includes a device or any configuration constituting the system, for example, a processor as a system LSI (Large Scale Integration), a module using a plurality of processors, a unit using a plurality of modules, and a unit. It can also be implemented as a set or the like with other functions added (that is, a configuration of a part of the device).

In the present embodiment, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..

Further, for example, the present embodiment can have a cloud computing configuration in which one function is shared by a plurality of devices via a network and jointly processed.

<< 6. Conclusion >>
As described above, according to one embodiment of the present disclosure, the information processing apparatus 10 and the terminal apparatus 30 correct the detectable range of the gesture based on the information regarding the gesture and the information regarding the device state. Therefore, the information processing device 10 and the terminal device 30 can reduce the situation where the user detects an operation that is not intended as a gesture as a gesture.

Further, according to one embodiment of the present disclosure, the information processing apparatus 10 and the terminal apparatus 30 correct the reference direction for detecting the gesture based on the information regarding the gesture and the information regarding the device state. Therefore, the information processing device 10 and the terminal device 30 can reduce erroneous detection such as detecting the left / right swipe as the up / down swipe.

Although each embodiment of the present disclosure has been described above, the technical scope of the present disclosure is not limited to the above-mentioned embodiments as they are, and various changes can be made without departing from the gist of the present disclosure. be. In addition, components spanning different embodiments and modifications may be combined as appropriate.

Further, the effects in each embodiment described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can also have the following configurations.
(1)
A gesture detection unit that detects a user's gesture performed in the detection area of the detection unit that detects an object, and a gesture detection unit.
The detectable range of the gesture in the detection area and the reference direction for detecting the gesture based on at least one of the information about the gesture and the information about the state of the predetermined device including the detection unit. A correction unit that corrects at least one of the
Information processing device equipped with.
(2)
The correction unit corrects at least one of the detectable range and the reference direction based on the information of the user's movement regarding the gesture.
The information processing apparatus according to (1) above.
(3)
The user's action information regarding the gesture includes information on the first action that causes the user to enter the detectable range.
The correction unit corrects at least one of the detectable range and the reference direction based on the information of the first operation.
The information processing device according to (2) above.
(4)
The correction unit corrects the detectable range to be narrower than at least the detection region based on the information of the first operation.
The information processing apparatus according to (3) above.
(5)
The user's actions constituting one gesture include the first action and a second action following the first action.
After the first operation is detected in the detectable range, the correction unit sets the detectable range for detecting the second operation in the detectable range when the first operation is detected. Wider than
The information processing apparatus according to (4) above.
(6)
When the gesture to be detected is a gesture with a movement width, the correction unit sets the detectable range for detecting the second motion as the detectable range when the first motion is detected. Wider than
The information processing apparatus according to (5) above.
(7)
The information of the first operation includes information indicating the approach direction of the hand entering the detectable range.
The correction unit corrects at least one of the detectable range and the reference direction based on the information indicating the approach direction of the hand.
The information processing apparatus according to any one of (3) to (6).
(8)
The predetermined device can be worn by the user.
The correction unit corrects at least one of the detectable range and the reference direction based on the information regarding the wearing state of the predetermined device to the user, which is estimated based on the information indicating the approach direction of the hand.
The information processing apparatus according to (7) above.
(9)
The correction unit corrects at least one of the detectable range and the reference direction based on the information regarding the mounting inclination of the predetermined device estimated based on the information indicating the approach direction of the hand.
The information processing apparatus according to (8) above.
(10)
The predetermined device comprises a predetermined portion located in the user's ear when worn.
The detection unit is arranged at the predetermined portion, and the detection unit is arranged.
The correction unit corrects at least one of the detectable range and the reference direction based on the information of the swinging motion in which the user raises his / her hand toward his / her ear.
The information processing apparatus according to (8) or (9).
(11)
When the gesture is detected, the execution unit, which executes the function associated with the gesture, is provided.
The predetermined device comprises a pair of the predetermined parts located in both ears of the user when worn.
When the swing-up motion is detected in both of the pair of the predetermined parts, the execution unit does not execute the function associated with the gesture even if the gesture is detected.
The information processing apparatus according to (10) above.
(12)
The detectable range is arranged in the detection area so that a part of the edge of the detectable range is in contact with a part of the edge of the detection area.
The gesture detection unit does not detect the gesture when the user's hand enters from a range other than the detectable range in the detection area.
The information processing apparatus according to any one of (3) to (11).
(13)
When the gesture is detected, the execution unit, which executes the function associated with the gesture, is provided.
The above-mentioned functions include at least a predetermined function accompanied by an operation amount.
A predetermined gesture with a movement width is associated with the predetermined function.
When the gesture detected by the gesture detection unit is the predetermined gesture, the execution unit determines the operation amount based on the magnitude of the movement width of the predetermined gesture with respect to the detectable range. do,
The information processing apparatus according to any one of (1) to (12).
(14)
The correction unit corrects at least one of the detectable range and the reference direction based on the information from the state detection unit that detects the state of the predetermined device.
The information processing apparatus according to any one of (1) to (13).
(15)
The predetermined device can be worn by the user.
The correction unit corrects at least one of the detectable range and the reference direction based on the information regarding the wearing state of the predetermined device to the user, which is estimated based on the information from the state detection unit.
The information processing apparatus according to (14) above.
(16)
The predetermined device can be worn by the user.
The correction unit corrects at least one of the detectable range and the reference direction based on the information regarding the posture of the user estimated based on the information from the state detection unit.
The information processing apparatus according to (14) or (15).
(17)
The state detection unit includes at least one of an acceleration sensor, a gyro sensor, and a biosensor.
The correction unit corrects at least one of the detectable range and the reference direction based on the information from one or a plurality of sensors included in the state detection unit.
The information processing apparatus according to any one of (14) to (16).
(18)
The predetermined device is a headphone or an earphone.
The information processing apparatus according to any one of (1) to (17).
(19)
The information processing device is a device that controls the predetermined device or the predetermined device from the outside of the predetermined device via wire or wirelessly.
The information processing apparatus according to any one of (1) to (18).
(20)
Detects the user's gesture performed in the detection area of the detector that detects the object,
The detectable range of the gesture in the detection area and the reference direction for detecting the gesture based on at least one of the information about the gesture and the information about the state of the predetermined device including the detection unit. , Correct at least one of
Information processing method.
(21)
A first acquisition unit that acquires sensor information from a contact sensor that detects contact with the user's human body or object, and
A second acquisition unit that acquires sensor information from the non-contact sensor that detects the user's movement including the user's non-contact movement, and a second acquisition unit.
A gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor.
Based on the sensor information from the contact sensor and the sensor information from the non-contact sensor, a correction unit that makes a correction related to the detection of the gesture or a correction related to the operation of the device, and a correction unit.
Information processing device equipped with.
(22)
The gesture detection unit detects the gesture of the user using at least the sensor information from the contact sensor.
The correction unit determines the detectable range of the gesture based on the information of the gesture position detected based on the sensor information from the non-contact sensor, which is the gesture position indicating the position where the user has performed the gesture. to correct,
The information processing apparatus according to (21).
(23)
It is equipped with an execution unit that executes a command corresponding to the detected gesture.
The execution unit determines a command to be executed based on the gesture detected based on the sensor information from the contact sensor and the gesture position information detected based on the sensor information from the non-contact sensor. ,
The information processing apparatus according to (22) above.
(24)
The gesture detection unit detects the gesture of the user using at least the sensor information from the non-contact sensor.
The correction unit corrects the detectable range of the gesture based on the sensor information from the non-contact sensor when the user comes into contact with the human body or the object.
The information processing apparatus according to (21).
(25)
The correction unit detects the user's contact with the human body or the object based on the sensor information from the contact sensor, and the non-contact sensor when the user's contact with the human body or the object is detected. Corrects the detectable range of the gesture based on the sensor information from
The information processing apparatus according to (24).
(26)
It is provided with an output control unit that gives feedback to the user when the user's hand is positioned in the corrected detectable range after detecting the contact of the user with the human body or the object.
The information processing apparatus according to (25) above.
(27)
It is equipped with a command execution unit that executes a command corresponding to the detected gesture.
The command includes at least a command with an operation amount.
The gesture detection unit detects the gesture of the user using at least the sensor information from the contact sensor.
The command execution unit determines the operation amount based on the sensor information from the non-contact sensor.
The information processing apparatus according to (21).
(28)
The user's actions constituting one gesture include a first action that triggers the start of the gesture and a second action following the first action.
The command execution unit determines the operation amount based on the speed or movement amount of the second operation determined based on the sensor information from the non-contact sensor.
The correction unit changes a set value for associating the second operation with the operation amount based on the information of the speed of the first operation detected based on the sensor information from the non-contact sensor.
The information processing apparatus according to (27).
(29)
The correction unit corrects the detectable range of the second operation based on the information of the speed of the first operation detected based on the sensor information from the non-contact sensor.
The information processing apparatus according to (28) above.
(30)
Acquires sensor information from a contact sensor that detects contact with the user's body or object,
The sensor information from the non-contact sensor that detects the movement of the user including the non-contact movement of the user is acquired, and the sensor information is acquired.
Based on the sensor information from at least one of the contact sensor and the non-contact sensor, the user's gesture for operating the device is detected.
Based on the sensor information from the contact sensor and the sensor information from the non-contact sensor, the correction related to the detection of the gesture or the correction related to the operation of the device is performed.
Information processing method.
(31)
Computer,
Gesture detector that detects the user's gesture performed in the detection area of the detector that detects the object,
The detectable range of the gesture in the detection area and the reference direction for detecting the gesture based on at least one of the information about the gesture and the information about the state of the predetermined device including the detection unit. , A correction unit that corrects at least one of
An information processing program to function as.
(32)
Computer,
A first acquisition unit that acquires sensor information from a contact sensor that detects contact with the user's body or object,
A second acquisition unit that acquires sensor information from a non-contact sensor that detects the user's movement, including the user's non-contact movement.
A gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor.
A correction unit that corrects the detection of the gesture or the operation of the device based on the sensor information from the contact sensor and the sensor information from the non-contact sensor.
An information processing program to function as.

1

Information processing system

10, 40 Information processing device 10A Earphone 10B

Headphones

11 _{, 11 L} , 11 _R , 21, 41 Input detection unit 12, 22, 32, 42

State detection unit

13, 23, 33, 43 Output unit 14, 24 , 34, 44

Communication unit

15, 35, 45

Storage unit

16, 26, 36, 46

Control unit

20, 20A, 20B Output device 30 Terminal device 31 Input unit 161, 361

Acquisition unit

162, 362, 462

Gesture detection unit

163, 363, 463 Command execution unit 164, 364, 464

Output control unit

165, 365, 465 Guessing unit 166, 366, 466 Correction unit 461A First acquisition unit 461B Second acquisition unit AD Approach direction OR Detection area AR, A1, A2 Detectable range A3, A4 area NR Non-detectable range B1 to B4 Reference direction D1 to D4 Device direction U User H Hand O1, O2 Obstacles

Claims

A gesture detection unit that detects a user's gesture performed in the detection area of the detection unit that detects an object, and a gesture detection unit.
The detectable range of the gesture in the detection area and the reference direction for detecting the gesture based on at least one of the information about the gesture and the information about the state of the predetermined device including the detection unit. A correction unit that corrects at least one of the
Information processing device equipped with.
The correction unit corrects at least one of the detectable range and the reference direction based on the information of the user's movement regarding the gesture.
The information processing apparatus according to claim 1.
The user's action information regarding the gesture includes information on the first action that causes the user to enter the detectable range.
The correction unit corrects at least one of the detectable range and the reference direction based on the information of the first operation.
The information processing apparatus according to claim 2.
The correction unit corrects the detectable range to be narrower than at least the detection region based on the information of the first operation.
The information processing apparatus according to claim 3.
The user's actions constituting one gesture include the first action and a second action following the first action.
After the first operation is detected in the detectable range, the correction unit sets the detectable range for detecting the second operation in the detectable range when the first operation is detected. Wider than
The information processing apparatus according to claim 4.
When the gesture to be detected is a gesture with a movement width, the correction unit sets the detectable range for detecting the second motion as the detectable range when the first motion is detected. Wider than
The information processing apparatus according to claim 5.
The information of the first operation includes information indicating the approach direction of the hand entering the detectable range.
The correction unit corrects at least one of the detectable range and the reference direction based on the information indicating the approach direction of the hand.
The information processing apparatus according to claim 3.
The predetermined device can be worn by the user.
The correction unit is the predetermined device estimated based on the information regarding the state in which the predetermined device is attached to the user, which is estimated based on the information indicating the approach direction of the hand, and the information indicating the approach direction of the hand. Correct at least one of the detectable range and the reference direction based on the information regarding the mounting inclination of the
The information processing apparatus according to claim 7.
The predetermined device comprises a predetermined portion located in the user's ear when worn.
The detection unit is arranged at the predetermined portion, and the detection unit is arranged.
The correction unit corrects at least one of the detectable range and the reference direction based on the information of the swinging motion in which the user raises his / her hand toward his / her ear.
The information processing apparatus according to claim 8.
When the gesture is detected, the execution unit, which executes the function associated with the gesture, is provided.
The predetermined device comprises a pair of the predetermined parts located in both ears of the user when worn.
When the swing-up motion is detected in both of the pair of the predetermined parts, the execution unit does not execute the function associated with the gesture even if the gesture is detected.
The information processing apparatus according to claim 9.
The detectable range is arranged in the detection area so that a part of the edge of the detectable range is in contact with a part of the edge of the detection area.
The gesture detection unit does not detect the gesture when the user's hand enters from a range other than the detectable range in the detection area.
The information processing apparatus according to claim 3.
When the gesture is detected, the execution unit, which executes the function associated with the gesture, is provided.
The above-mentioned functions include at least a predetermined function accompanied by an operation amount.
A predetermined gesture with a movement width is associated with the predetermined function.
When the gesture detected by the gesture detection unit is the predetermined gesture, the execution unit determines the operation amount based on the magnitude of the movement width of the predetermined gesture with respect to the detectable range. do,
The information processing apparatus according to claim 1.
The correction unit corrects at least one of the detectable range and the reference direction based on the information from the state detection unit that detects the state of the predetermined device.
The information processing apparatus according to claim 1.
The predetermined device can be worn by the user.
The correction unit is information on the wearing state of the predetermined device to the user estimated based on the information from the state detection unit, or information on the posture of the user estimated based on the information from the state detection unit. Corrects at least one of the detectable range and the reference direction based on.
The information processing apparatus according to claim 13.
The state detection unit includes at least one of an acceleration sensor, a gyro sensor, and a biosensor.
The correction unit corrects at least one of the detectable range and the reference direction based on the information from one or a plurality of sensors included in the state detection unit.
The information processing apparatus according to claim 13.
Detects the user's gesture performed in the detection area of the detector that detects the object,
The detectable range of the gesture in the detection area and the reference direction for detecting the gesture based on at least one of the information about the gesture and the information about the state of the predetermined device including the detection unit. , Correct at least one of
Information processing method.
A first acquisition unit that acquires sensor information from a contact sensor that detects contact with the user's human body or object, and
A second acquisition unit that acquires sensor information from the non-contact sensor that detects the user's movement including the user's non-contact movement, and a second acquisition unit.
A gesture detection unit that detects a gesture of the user for operating a device based on sensor information from at least one of the contact sensor and the non-contact sensor.
Based on the sensor information from the contact sensor and the sensor information from the non-contact sensor, a correction unit that makes a correction related to the detection of the gesture or a correction related to the operation of the device, and a correction unit.
Information processing device equipped with.
The gesture detection unit detects the gesture of the user using at least the sensor information from the contact sensor.
The correction unit determines the detectable range of the gesture based on the information of the gesture position detected based on the sensor information from the non-contact sensor, which is the gesture position indicating the position where the user has performed the gesture. to correct,
The information processing apparatus according to claim 17.
It is equipped with an execution unit that executes a command corresponding to the detected gesture.
The execution unit determines a command to be executed based on the gesture detected based on the sensor information from the contact sensor and the gesture position information detected based on the sensor information from the non-contact sensor. ,
The information processing apparatus according to claim 18.
The gesture detection unit detects the gesture of the user using at least the sensor information from the non-contact sensor.
The correction unit corrects the detectable range of the gesture based on the sensor information from the non-contact sensor when the user comes into contact with the human body or the object.
The information processing apparatus according to claim 17.
The correction unit detects the user's contact with the human body or the object based on the sensor information from the contact sensor, and the non-contact sensor when the user's contact with the human body or the object is detected. Corrects the detectable range of the gesture based on the sensor information from
The information processing apparatus according to claim 20.
It is provided with an output control unit that gives feedback to the user when the user's hand is positioned in the corrected detectable range after detecting the contact of the user with the human body or the object.
The information processing apparatus according to claim 21.
It is equipped with a command execution unit that executes a command corresponding to the detected gesture.
The command includes at least a command with an operation amount.
The gesture detection unit detects the gesture of the user using at least the sensor information from the contact sensor.
The command execution unit determines the operation amount based on the sensor information from the non-contact sensor.
The information processing apparatus according to claim 17.
The user's actions constituting one gesture include a first action that triggers the start of the gesture and a second action following the first action.
The command execution unit determines the operation amount based on the speed or movement amount of the second operation determined based on the sensor information from the non-contact sensor.
The correction unit changes a set value for associating the second operation with the operation amount based on the information of the speed of the first operation detected based on the sensor information from the non-contact sensor.
The information processing apparatus according to claim 23.
The correction unit corrects the detectable range of the second operation based on the information of the speed of the first operation detected based on the sensor information from the non-contact sensor.
The information processing apparatus according to claim 24.
Acquires sensor information from a contact sensor that detects contact with the user's body or object,
The sensor information from the non-contact sensor that detects the movement of the user including the non-contact movement of the user is acquired, and the sensor information is acquired.
Based on the sensor information from at least one of the contact sensor and the non-contact sensor, the user's gesture for operating the device is detected.
Based on the sensor information from the contact sensor and the sensor information from the non-contact sensor, the correction related to the detection of the gesture or the correction related to the operation of the device is performed.
Information processing method.