JP2020119042A - Gesture recognition device, gesture recognition system, gesture recognition method, and computer program - Google Patents

Abstract

To prevent a malfunction of other devices while suppressing a load on a user in a gesture operation.SOLUTION: A gesture recognition device 15 of a disclosure includes a processor 20 which determines whether a motion direction V2 of a gesture of a user is a prescribed direction with an operation object device 60 as a reference for at least one or more operation object devices 60, and operates so as to recognize the operation object device 60 in which the motion direction V2 of the gesture is determined to be the prescribed direction as a gesture directed operation object device.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a gesture recognition device, a gesture recognition system, a gesture recognition method, and a computer program.

Patent Document 1 discloses a gesture-based operation input device.

JP, 2009-104297, A

In a device that receives a command by a gesture, it is desired to suppress unintended malfunction. In particular, it is desired to prevent a gesture directed to one device from causing another device to malfunction. In order to prevent erroneous operation of other devices, it is conceivable to make them different for each gesture device that gives a command. However, in this case, the user needs to learn the gesture for each device, which imposes a heavy burden on the user. Therefore, in gesture operation, it is desired to prevent malfunction of other devices without increasing the burden on the user.

Here, Patent Document 1 discloses a technique of determining the operation of a hand and displaying a menu selected based on the determination operation in order to deal with the case where the number of operation targets by the gesture increases. However, the technique of Patent Document 1 requires the display of the menu. Therefore, it is desired to prevent malfunction of other devices while reducing the burden on the user without taking the approach of the technique of Patent Document 1.

One aspect of the present disclosure is a gesture recognition device. The disclosed gesture recognition device determines, for at least one or more operated devices, whether or not the motion direction of the user's gesture is a predetermined direction with respect to the operated device, and the motion direction of the gesture is the predetermined direction. And a processor operable to recognize the operated device determined to be the target operated device to which the gesture is directed.

Another aspect of the disclosure is a gesture recognition system. A gesture recognition system disclosed discloses a sensor for detecting a gesture motion of a user and a motion direction of the gesture based on detection data of the sensor, and the motion direction of the gesture is a predetermined direction based on the operated device. And a processor that operates to recognize the operated device as the target operated device to which the gesture is directed when it is determined that the operation direction of the gesture is the predetermined direction. ..

The disclosed gesture recognition system can also include a sensor that detects a gesture motion of a user, a plurality of operated devices, and a processor. The processor obtains the motion direction of the gesture based on the detection data of the sensor and determines whether the motion direction of the gesture is a predetermined direction with respect to the operated device, for the plurality of operated devices. The operation device is operated so as to recognize the operated device that is determined to have the movement direction of the gesture as the predetermined direction, as the target operated device to which the gesture is directed.

Another aspect of the present disclosure is a gesture recognition method. In the disclosed method, the processor determines, for at least one or more operated devices, whether the motion direction of the user's gesture is a predetermined direction with respect to the operated device, and the motion direction of the gesture is the predetermined direction. Recognizing the operated device determined to be the target operated device to which the gesture is directed.

Another aspect of the disclosure is a computer program. The disclosed computer program determines, for at least one or more operated devices, whether the motion direction of the user's gesture is a predetermined direction with respect to the operated device, and the motion direction of the gesture is the predetermined direction. The processor is caused to execute the operation of recognizing the operated device determined to be the target operated device to which the gesture is directed.

According to the present disclosure, it is possible to prevent a malfunction of another operated device while reducing the burden on the user.

FIG. 1 is a hardware configuration diagram of a gesture recognition system. FIG. 2 is a functional block diagram of the gesture recognition system. FIG. 3 is a flowchart of the recognition process. FIG. 4 is an explanatory diagram of pre-gesture recognition. FIG. 5 is an explanatory diagram of command gesture recognition. FIG. 6 is a hardware configuration diagram of a gesture recognition system according to another embodiment. FIG. 7 is a functional block diagram of a gesture recognition system according to another embodiment. FIG. 8 is a flowchart of the recognition process according to another embodiment. FIG. 9 is an explanatory diagram of pre-gesture recognition in another embodiment. FIG. 10 is an explanatory diagram of the setting process.

[Description of Embodiments of the Present Disclosure]

(1) The gesture recognition device according to the embodiment includes a processor. The processor operates to determine, for at least one or more operated devices, whether or not the operation direction of the gesture of the user is a predetermined direction based on the operated device. The processor operates so as to recognize the operated device that is determined to have the operation direction of the gesture as the predetermined direction, as the target operated device to which the gesture is directed. Even if the same gesture is used, the movement directions of the gestures are different when the operated devices installed at different positions are used as references. Therefore, depending on whether or not the gesture operation direction is the predetermined direction with respect to the operated device, it is possible to identify which operated device the gesture is directed to, and the gesture is not directed. A malfunction of the operated device can be prevented. Moreover, the gestures for the plurality of operated devices may be the same, which can reduce the burden on the user.

(2) The processor may further operate to recognize a gesture performed after recognizing the operated device as the target operated device as a command gesture for the target operated device. In this case, only the target operated device recognizes the command gesture.

(3) Based on the motion of the command gesture, the processor does not determine whether the motion direction of the command gesture is a predetermined direction with the target operated device as a reference, and the command indicated by the command gesture. Can be recognized. In this case, it is not necessary to determine the motion direction of the command gesture, and the processing load is reduced.

(4) The processor may further operate to obtain the motion direction based on detection data of a sensor that detects the motion of the gesture. In this case, the operation method is obtained based on the detection data of the sensor.

(5) The sensor may be installed separately from the operated device. The processor obtains the pre-conversion motion direction of the gesture based on the detection data, and converts the motion direction by converting the pre-conversion motion direction based on the positional relationship between the sensor and the operated device. You can ask. Here, the pre-conversion operation direction is a direction based on the sensor. The operation direction is a direction with reference to the operated device. In this case, even if the sensor and the operated device are separated from each other, the operation direction based on the operated device can be obtained based on the detection data of the sensor.

(6) It is preferable that the operation direction is obtained based on a positional relationship between the sensor and each of the plurality of operated devices, as a direction with respect to each of the plurality of operated devices. In this case, the operation direction of each of the plurality of operated devices can be obtained.

(7) The predetermined direction is preferably a direction based on a first direction from the position of the operated device to the position where the gesture is performed. In this case, it is possible to determine whether or not the operation direction is the predetermined direction by comparing the operation direction and the first direction.

(8) The processor may further operate to obtain the first direction based on detection data of a sensor that detects a position where the gesture is performed. In this case, the first direction is obtained based on the detection data of the sensor.

(9) The sensor may be installed separately from the operated device. In this case, the processor can determine the pre-conversion first direction from the sensor to the position where the gesture is performed, based on the detection data. Further, the processor can obtain the first direction by converting the pre-conversion first direction based on the positional relationship between the sensor and the operated device. In this case, even if the sensor and the operated device are apart from each other, the first direction can be obtained from the detection data of the sensor.

(10) The first direction is obtained as a direction from each of the plurality of operated devices to the position where the gesture is performed, based on the positional relationship between the sensor and each of the plurality of operated devices. preferable. In this case, the first direction is obtained for each of the plurality of operated devices.

(11) The processor preferably further operates to notify the user of the operated device recognized as the target operated device. In this case, the user can easily recognize the target operated device.

(12) The processor may be further operable to execute the positional relationship setting process. In this case, the positional relationship can be set.

(13) The gesture recognition system according to the embodiment includes a sensor that detects a gesture motion of a user and a processor. The processor obtains the motion direction of the gesture based on the detection data of the sensor, determines whether the motion direction of the gesture is a predetermined direction based on the operated device, and the motion direction of the gesture is the predetermined direction. When it is determined to be the direction, the operated device is operated so as to be recognized as the target operated device to which the gesture is directed.

(14) The gesture recognition system according to the embodiment may include a sensor that detects a gesture motion of a user, a plurality of operated devices, and a processor. The processor obtains the motion direction of the gesture based on the detection data of the sensor and determines whether the motion direction of the gesture is a predetermined direction with respect to the operated device, for the plurality of operated devices. It is possible to operate so that the operated device that is determined to have the movement direction of the gesture in the predetermined direction is recognized as the target operated device to which the gesture is directed.

(15) In the gesture recognition method according to the embodiment, the processor determines, for at least one or more operated devices, whether the operation direction of the user's gesture is a predetermined direction based on the operated device, and the gesture is used. Recognizing the operated device determined to be in the predetermined direction as the target operated device to which the gesture is directed.

(16) The computer program according to the embodiment determines, for at least one or more operated devices, whether or not the operation direction of the user's gesture is a predetermined direction based on the operated device, and the operation direction of the gesture. Causes the processor to execute an operation of recognizing the operated device determined to be in the predetermined direction as the target operated device to which the gesture is directed. The computer program is stored in a computer-readable non-transitory storage medium.

[Details of the embodiment of the present disclosure]

FIG. 1 shows a gesture recognition system 10 according to the embodiment. The gesture recognition system 10 of the embodiment includes a sensor 50 and a recognition device 15 connected to the sensor 50. The recognition device 15 of the embodiment recognizes the gesture detected by the sensor 50 and causes the operated device 60 to execute a command according to the recognized gesture. The operated device 60 is a device that can be operated by a gesture, and is, for example, a smart home appliance, an in-vehicle device, or another device. The operated device 60 is connected to the recognition device 15 by wire or wirelessly.

The sensor 50 detects a gesture motion. The sensor 50 of the embodiment outputs the relative distance from the sensor 50 to the detection target, the position (direction) of the detection target, and the speed of the detection target as detection data. The sensor 50 is, for example, a millimeter wave sensor. The millimeter wave sensor is preferably a frequency modulated continuous wave (FMCW) millimeter wave sensor. The FMCW millimeter wave sensor can detect the relative distance to the detection target, the position (direction) of the detection target, and the speed of the detection target.

The sensor 50 of the embodiment detects a motion of the user's hand as a gesture motion. The motion of the hand may be a motion of one hand or a motion of both hands. The gesture motion is not limited to the motion of the hand, and may be the motion of another body part of the user.

The recognition device 15 of the embodiment monitors the distance and position (orientation) and speed of the hand making the gesture and tracks the movement of the hand based on the detection data output from the sensor 50. By recognizing the movement of the hand, the recognition device 15 can determine what position and what movement the hand is performing.

Whether the gesture is directed to the specific operated device 60 (or the specific sensor 50) or the other operated device 60 (or the other sensor 50) is provided in the recognition device 15 of the embodiment. It has a function to discriminate. By having this function, the recognition device 15 can identify the target operated device 60 (target sensor 50) to which the gesture is directed even if the gesture operation content is the same. Here, as an example, it is assumed that the operated device 60 and the sensor 50 have a one-to-one relationship, and the two 50, 60 are very close to each other and are present at the same place. Therefore, the gesture directed to the specific operated device 60 can be regarded as being directed to the sensor 50 provided in the specific operated device 60.

In the embodiment, a pre-gesture (first gesture) and a command gesture (second gesture) are used as gestures. The pre-gesture is a gesture for causing the recognition device 15 to recognize that the specific operated device 60 is the target operated device that is the target of the operation by the gesture. The operation content of the pre-gesture may be common to the plurality of operated devices 60. The pre-gesture is, for example, an action of shaking the hand from side to side. The pre-gesture is performed toward the target operated device 60 that is the target of the gesture operation. The recognition device 15 identifies the target operated device 60 to which the gesture is directed, based on the motion direction of the gesture.

Here, as an example, as shown in FIG. 1, it is assumed that only one operated device 60 is connected to one operated device 60 and one recognition device 15. Therefore, the recognition device 15 determines whether or not the pre-gesture is directed to the operated device 60 connected to the own device 15, that is, the operated device 60 connected to the own device 15 is the target operated device. Identify whether it is a device.

When the recognition device 15 identifies that the operated device 60 connected to the own device 15 is the target operated device by the pre-gesture, the recognition device 15 enters a mode in which it is possible to accept a command by the command gesture for the target operated device 60. The command gesture is a gesture that gives a command to the target operated device 60. Even if the pre-gesture is performed, if the pre-gesture is not directed to the operated device 60, the recognition device 15 does not enter the mode for receiving the command gesture.

Here, when there are a plurality of operated devices 60, it is conceivable to distinguish the operated device 60 to which the command gesture is directed by changing the operation content of the pre-gesture for each of the plurality of operated devices 60. .. In this case, the user needs to learn the operation content of the different pre-gestures for each of the plurality of operated devices 60, which imposes a heavy burden on the user. On the other hand, in the present embodiment, even if there are a plurality of operated devices 60, the action content itself of the pre-gesture may be common. Since the pre-gesture is performed toward the operated device 60 to which the command gesture is to be given, the recognition device 15 can identify the operated device 60 to which the command gesture is to be given.

The operation content of the pre-gesture is not particularly limited. The operation content of the pre-gesture includes, in addition to the operation of shaking the hand to the left and right toward the sensor 50, for example, the operation of shaking the hand up and down toward the sensor 50, the operation of repeating pointing the finger toward the sensor 50 a plurality of times, The operation may be an operation of spreading both hands toward the sensor 50, an operation of turning the hand toward the sensor 50, or an operation of shaking the hand back and forth toward the sensor 50.

The recognition device 15 of the embodiment is configured by a computer. As shown in FIG. 1, the computer includes a processor 20 and a memory 30 connected to the processor 20. A computer program 35 that causes a computer to function as the recognition device 15 is stored in the memory 30. The processor 20 reads and executes the computer program 35 stored in the memory 30. The computer program 35 causes the processor 20 to execute the recognition process 21 (described later) shown in FIG. The memory 30 of the embodiment includes a primary storage device and a secondary storage device. The memory 30 may include a tertiary storage device.

The recognition device 15 receives the detection data output from the sensor 50. The memory 30 stores the received detection data 31. During the recognition process 21, the processor 20 reads the detection data 31 stored in the memory 30 and identifies whether the operated device 60 connected to the self device 15 is the target operated device.

FIG. 2 is a block diagram showing functions performed by the computer program 35 being executed by the processor 20. As shown in FIG. 2, the recognition device 15 includes, as functional blocks, a first gesture monitoring unit 151 that is a pre-gesture and a second gesture monitoring unit 155 that is a command gesture. The first gesture monitoring unit 151 and the second gesture monitoring unit 155 detect the motion of the user's hand, which is a gesture motion, from the detection data 31 of the sensor 50.

The first gesture monitoring unit 151 corresponds to steps S101 and S102 in FIG. 3 described later. The second gesture monitoring unit 155 corresponds to steps S107 and S108 in the same FIG.

The recognition device 15 includes a determination unit 152 as a functional block. The determination unit 152 determines whether or not the first gesture (preliminary gesture) is a gesture for the own device 15. The determination unit 152 makes a determination based on the motion direction of the gesture, as described below. When the determination unit 152 determines that the pre-gesture is a gesture for the own device 15, the second gesture monitoring unit 155 becomes valid. If the determination unit 152 does not determine that the pre-gesture is a gesture for the own device 15, the second gesture monitoring unit 155 is invalid. The determination unit 152 corresponds to steps S103 to S105 in FIG. 3 described later.

The recognition device 15 includes a notification unit 153 that performs notification processing for the notification device 70 as a functional block. The notification unit 153 performs processing for notifying the user that the operated device 60 has been recognized as the target operated device by the advance gesture. The notification device 70 notifies the user by sound, light, screen display, vibration, or the like. The notification device 70 may be included in the operated device 60. The notification unit 153 corresponds to step S106 in FIG. 3 described later.

The recognition device 15 includes an instruction execution unit 156 as a functional block. The command execution unit 156 identifies the operation content of the command gesture detected by the second gesture monitoring unit 155, and gives the operated device 60 the command indicated by the operation content. The instruction execution unit 156 corresponds to step S109 in FIG. 3 described later.

FIG. 3 shows a procedure of the recognition process 21 executed in the recognition device 15 according to the computer program 35. In step S101, the processor 20 detects the movement of the user's hand based on the detection data 31 of the sensor 50 and monitors the pre-gesture. Pre-gesture monitoring continues until a pre-gesture is made. Note that the command gesture is not monitored while the pre-gesture is being monitored.

When the user performs the action set as the pre-gesture, the processor 20 detects that the pre-gesture has been performed (Yes in step S102). The detection that the pre-gesture has been performed can be performed using, for example, a machine-learned model for recognizing the pre-gesture operation content.

The processor 20 determines whether the detected pre-gesture is directed to the operated device 60 (or the sensor 50) connected to the self device 15 (step S103 to step S105). The determination is performed by identifying whether or not the motion direction of the pre-gesture (for example, the direction of waving the hand) is the predetermined direction based on the operated device 60. Here, as an example, as shown in FIG. 4, the first vector (first direction) V1 heading from the operated device 60 to the position where the gesture is performed is the second vector V2 in the waving direction (movement direction). It is determined by whether or not The predetermined direction is not limited to the direction orthogonal to the first vector V1, and may be, for example, a direction intersecting the first vector V1 at a predetermined angle other than orthogonal, or may be parallel to the first vector V1. It may be a direction.

In determining whether they are orthogonal to each other, the processor 20 moves from the operated device 60 (sensor 50) to the first gesture position from the detection data indicating the position (first gesture position) at which the first gesture (previous gesture) is performed. The first vector V1 to head is calculated (step S103). Here, the operated device 60 is considered to be in the same position as the sensor 50.

The processor 20 calculates the second vector V2 indicating the movement direction of the hand shaking motion from the detection data tracking the hand shaking motion (step S104). Then, the processor 20 determines whether or not the first vector V1 and the second vector V2 are orthogonal to each other (step S105).

If they are orthogonal (Yes in step S105), the processor 20 determines that the pre-gesture is directed to the operated device 60 connected to the self device 15. That is, the processor 20 recognizes the operated device 60 connected to the own device 15 as the target operated device to which the gesture is directed. In this case, the processor 20 enables the function as the second gesture monitoring unit 155, and can accept a gesture performed thereafter as a command gesture directed to the operated device 60 connected to the own device 15. Further, the processor 20 notifies the user that the operated device 60 connected to the own device 15 is recognized as the target operated device to which the gesture is directed (step S106). The notification is performed by outputting sound, light, or the like from the notification device 70. The notification allows the user to recognize that the operated device 60 is in a state in which a command gesture can be commanded.

If they are not orthogonal to each other (No in step S105), the processor 20 determines that the pre-gesture is not directed to the operated device 60 connected to the own device 15, and returns to step S101. In this case, the function as the second gesture monitoring unit 155 remains disabled, and even if a command gesture is performed after that, the processor 20 does not operate for the command gesture.

For example, as shown in FIG. 4, it is assumed that the first gesture recognition system 10A and the second gesture recognition system 10B exist. In the initial state, the recognition devices 15 of both systems 10A and 10B are in the first gesture monitoring mode M1. In the first gesture monitoring mode M1, the pre-gesture that is the first gesture is monitored, and the command gesture that is the second gesture is not monitored. That is, in the first gesture monitoring mode M1, the function of the first gesture monitoring unit 151 is valid and the function of the second gesture monitoring unit 155 is invalid.

In this state, it is assumed that the user 100 has performed a pre-gesture toward (directly facing) the operated device (light bulb) 60 or the sensor 50 of the first gesture recognition system 10A. In this case, the first vector V1 from the sensor 50 (operated device 60) of the first gesture recognition system 10A toward the first gesture position is orthogonal to the second vector V2 in the waving direction. Therefore, the recognition device 15 of the first gesture recognition system 10A recognizes the operated device 60 connected to the own device 15 as the target operated device to which the first gesture is directed. In addition, the recognition device 15 of the first gesture recognition system 10A operates the notification device 70 connected to the own device 15 to perform the notification of step S106.

On the other hand, the first vector V1 from the sensor 50 (operated device 60) of the second gesture recognition system 10B toward the first gesture position is not orthogonal to the second vector V2 in the direction of waving the hand. Therefore, the recognition device 15 of the second gesture recognition system 10B does not recognize the operated device 60 connected to the own device 15 as the target operated device to which the first gesture is directed.

Therefore, as shown in FIG. 5, after the pre-gesture is performed, the recognition device 15 of the second gesture recognition system 10B remains in the first gesture monitoring mode M1 while the recognition device of the first gesture recognition system 10A. 15 becomes the second gesture monitoring mode M2. In the second gesture monitoring mode M2, the command gesture that is the second gesture is monitored. That is, in the second gesture monitoring mode M2, the function of the second gesture monitoring unit 155 is valid and the function of the first gesture monitoring unit 151 is invalid. Since the recognition device 15 of the second gesture recognition system 10B remains in the first gesture monitoring mode M1, the malfunction caused by the command gesture performed toward the first gesture recognition system 10A is prevented.

The recognition device 15 of the first gesture recognition system 10A in the second gesture monitoring mode M2 monitors the second gesture (command gesture) performed near the first gesture position (step S107 in FIG. 3 ). As shown in FIG. 5, the predetermined range near the first gesture position is the executable range 200 of the command gesture. The executable range 200 is set based on the detection data 31 indicating the position where the pre-gesture is performed. The executable range 200 is, for example, a region having a predetermined radius centered on the first gesture position. The processor 20 monitors only gestures performed within the feasible range 200, and does not recognize gestures performed outside the feasible range 200 as command gestures.

When the user performs the operation set as the command gesture in the executable range 200, the processor 20 in the second gesture monitoring mode M2 detects that the command gesture has been performed (Yes in step S108). ). In this case, the processor 20 recognizes the command content according to the motion content of the command gesture and gives the command to the operated device 60 (step S109). The command is, for example, a command to light the operated device 60, which is a light bulb. When the lighting command is given, the operated device 60, which is a light bulb, is turned on. After executing the instruction, the processor 20 returns to the first gesture monitoring mode M1.

In the embodiment, the processor 20 only detects the motion of the command gesture, and does not determine the motion direction with respect to the pre-vector such as that performed in steps S103 to S105. The motion of the command gesture is detected by using, for example, a machine-learned model for recognizing the content of the gesture motion. When a command gesture is detected using the machine-learned model, if the command gesture is not sufficiently directed, the gesture may not be directed to the target operated device 60, that is, the operation direction may not be the predetermined direction. A command gesture can be detected. Therefore, the user does not have to face the target operated device 60 when performing the command gesture, and the degree of freedom of the user increases.

When the processor 20 does not detect the command gesture within the predetermined time after entering the second gesture monitoring mode M2, the processor 20 ends the second gesture monitoring mode M2 and returns to the first gesture monitoring mode M1 (step S110).

FIG. 6 shows a gesture recognition system 10C according to another embodiment. Note that, in the other embodiments, the description based on FIGS. 1 to 5 is applied to the points that are not particularly described.

In FIG. 6, the operated devices 61 and 62 are separated from the sensor 50. More specifically, a plurality of operated devices 61 and 62 are connected to one recognition device 15. The plurality of operated devices 61 and 62 are installed at positions separated from each other. Further, the sensor 50 connected to the recognition device 15 is installed at a position where a gesture directed to the operated device 61 and a gesture directed to the operated device 62 can be detected. The relative positional relationship between the sensor 50 and each of the plurality of operated devices 61 and 62 is known to the recognition device 15.

Since the sensor 50 is installed separately from the operated devices 61 and 62, the recognition device 15 illustrated in FIG. 6 performs conversion based on the positional relationship between the sensor 50 and the operated devices 61 and 62 to perform advance conversion. It is determined which of the plurality of operated devices 61 and 62 the gesture is directed to. Thereby, it becomes possible to process the gesture operation with respect to the plurality of operated devices 61 and 62 by the single recognition device 15. As shown in FIG. 6, the memory 30 stores conversion reference data 33 that is referred to in order to perform conversion based on the positional relationship between the sensor 50 and the operated devices 61 and 62. Further, the processor 20 can execute a setting process 23 (described later) for setting the conversion reference data 33 according to the computer program 35.

FIG. 7 is a block diagram showing functions exhibited by the computer program 35 being executed by the processor 20 in the recognition device 15 according to another embodiment shown in FIG. In FIG. 7, a coordinate conversion unit 157 is provided in addition to the functional blocks shown in FIG. Further, in FIG. 7, a first processing unit 161 for gesture recognition with respect to the first operated device 61 and a second processing unit 162 for gesture recognition with respect to the second operated device 62 are provided.

The first processing unit 161 includes a coordinate conversion unit 157 in addition to the functional blocks shown in FIG. 2, and executes the recognition process 21 for the first operated device 61. The second processing unit 162 has the same configuration as the first processing unit 161, and executes the recognition process 21 for the second operated device 62.

FIG. 8 shows the recognition process 21 executed by each of the first processing unit 161 and the second processing unit 162. However, for ease of understanding, the recognition process 21 executed in the first processing unit 161 will be described below as an example. In step S101, the processor 20 detects the movement of the user's hand based on the detection data 31 of the sensor 50 and monitors the pre-gesture.

When the user performs the action set as the pre-gesture, the processor 20 detects that the pre-gesture has been performed (Yes in step S102).

The processor 20 determines whether the detected pre-gesture is directed to the operated device 60 (or the sensor 50) connected to the self device 15 (step S103 to step S105). Specifically, the processor 20 calculates the pre-transformation first vector V11 (step S103). The pre-conversion first vector V11 is calculated from the detection data indicating the position (first gesture position) at which the first gesture (previous gesture) is performed. As shown in FIG. 9, the pre-conversion first vector V11 is a vector from the sensor 50 toward the first gesture position. The pre-transformation first vector V11 is represented, for example, as a vector in the sensor coordinate system having the position of the sensor 50 as the origin.

The processor 20 calculates the pre-transform second vector V22 (step S104). The pre-conversion second vector V22 is calculated from the detection data obtained by tracking the waving motion. As shown in FIG. 9, the pre-conversion second vector V22 indicates the motion direction of the waving motion with respect to the sensor 50 (pre-conversion motion direction). The pre-conversion second vector V22 is represented, for example, as a vector in the sensor coordinate system whose origin is the position of the sensor 50.

The processor 20 converts the pre-conversion first vector V11 and the pre-conversion second vector V22 using the conversion reference data 33 to obtain the first vector V1 and the second vector V2 (steps S104-1, S104-2). ).

In step S104-1, conversion from the sensor coordinate system to the operated device coordinate system is performed based on the relative positional relationship between the sensor 50 and the operated device 61. The operated device coordinate system is a coordinate system whose origin is the position of the operated device 61.

In step S104-2, the pre-conversion first vector V11 and the pre-conversion second vector V22 of the coordinate system of the operated device are rotated based on the relative positional relationship between the sensor 50 and the operated device 61, and the first vector V1 and the Two vectors V2 are obtained. The first vector V1 is a vector from the operated device 61 toward the first gesture position. The second vector V2 indicates the motion direction of the motion of shaking the hand when the operated device 61 is used as a reference.

Then, the processor 20 determines whether or not the first vector V1 and the second vector V2 are orthogonal to each other (step S105).

If they are orthogonal (Yes in step S105), the processor 20 determines that the pre-gesture is directed to the operated device 60 connected to the self device 15. Further, the processor 20 notifies the user that the operated device 60 connected to the own device 15 is recognized as the target operated device to which the gesture is directed (step S106).

If they are not orthogonal to each other (No in step S105), the processor 20 determines that the pre-gesture is not directed to the operated device 60 connected to the own device 15, and returns to step S101.

Although not shown in FIG. 8, the process after step S106 is the same as that in FIG. That is, after the step S106 shown in FIG. 8, the processing after the step S107 shown in FIG. 3 is executed.

FIG. 10 is an explanatory diagram of the implementation of the setting process 23 of the conversion reference data 33 used for vector conversion. Here, the setting of the conversion reference data 33 for the television as the first operated device 61 will be described. Because of the setting process 23, the processor 20 causes the screen of the television 61 or the like to display the user 100 instructing an action that is a pre-gesture. In FIG. 10, as an example, a message “Please shake your hand in front of the TV” is displayed. When the user 100 shakes his/her hand in front of the television 61 according to the operation instruction, the vector from the television 61 to the gesture position and the direction of the hand should be orthogonal. When the user 100 is waving his/her hand, the recognition device 15 determines the learning first vector V111 heading from the sensor 50 to the gesture position and the waving direction based on the sensor 50 based on the detection data of the sensor 50. The learning second vector V222 shown is calculated. The processor 20 calculates the conversion reference data 33 from the learning first vector V111 and the learning second vector V222, and stores it in the memory 30. The stored conversion reference data 33 is used for recognition of a pre-gesture.

[Appendix]

It should be understood that the embodiments disclosed this time are exemplifications in all points and not restrictive. The scope of the present invention is shown not by the above meaning but by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.

10: Gesture recognition system 10A: First gesture recognition system 10B: Second gesture recognition system 10C: Gesture recognition system 15: Recognition device 20: Processor 21: Recognition process 23: Setting process 30: Memory 31: Detection data 33: Conversion Reference data 35: Computer program 50: Sensor 60: Operated device 61: First operated device 62: Second operated device 70: Notification device 100: User 151: First gesture monitoring unit 152: Judgment unit 153: Notification unit 155: Second gesture monitoring unit 156: Instruction execution unit 157: Coordinate conversion unit 161: First processing unit 162: Second processing unit 200: Executable range M1: First gesture monitoring mode M2: Second gesture monitoring mode

Claims (16)

For at least one or more operated devices, it is determined whether the operation direction of the user's gesture is a predetermined direction based on the operated device, and the operation direction of the gesture is determined to be the predetermined direction. A gesture recognition device comprising a processor that operates to recognize an operation device as a target operated device to which the gesture is directed. The gesture recognition device according to claim 1, wherein the processor further operates to recognize a gesture performed after recognizing the operated device as the target operated device as a command gesture for the target operated device. The processor recognizes the command indicated by the command gesture based on the motion of the command gesture without determining whether the motion direction of the command gesture is a predetermined direction with respect to the target operated device. The gesture recognition device according to claim 2. The gesture recognition device according to claim 1, wherein the processor further operates to obtain the movement direction based on detection data of a sensor that detects the movement of the gesture. The sensor is installed separately from the operated device,
The processor is
Based on the detection data, the pre-conversion motion direction of the gesture is obtained,
By converting the pre-conversion operation direction based on the positional relationship between the sensor and the operated device, the operation direction is obtained,
The pre-conversion operation direction is a direction with respect to the sensor,
The gesture recognition device according to claim 4, wherein the operation direction is a direction with respect to the operated device as a reference. The gesture recognition device according to claim 5, wherein the movement direction is obtained as a direction with respect to each of the plurality of operated devices, based on a positional relationship between the sensor and each of the plurality of operated devices. The gesture recognition device according to claim 1, wherein the predetermined direction is a direction based on a first direction from a position of the operated device to a position where the gesture is performed. The gesture recognition device according to claim 7, wherein the processor further operates to obtain the first direction based on detection data of a sensor that detects a position where the gesture is performed. The sensor is installed separately from the operated device,
The processor is
Based on the detection data, a pre-conversion first direction from the sensor to the position where the gesture is performed is obtained,
The gesture recognition device according to claim 8, wherein the first direction is obtained by converting the pre-conversion first direction based on a positional relationship between the sensor and the operated device. The said 1st direction is calculated|required as a direction which goes to the position where the said gesture was performed from each of these to-be-operated devices based on the positional relationship of the said sensor and each of these to-be-operated devices. Gesture recognizer. The gesture recognition device according to claim 1, wherein the processor further operates to notify a user of the operated device recognized as the target operated device. The gesture recognition device according to any one of claims 5, 6, 9, and 10, wherein the processor further operates to execute the setting process of the positional relationship. A sensor that detects the motion of the user's gesture,
The motion direction of the gesture is obtained based on the detection data of the sensor, it is determined whether the motion direction of the gesture is a predetermined direction with reference to the operated device, and the motion direction of the gesture is the predetermined direction. When it is determined that the operated device, a processor that operates to recognize the operated device as a target operated device to which the gesture is directed,
Gesture recognition system with. A sensor that detects the motion of the user's gesture,
A plurality of operated devices,
A processor,
Equipped with
The processor is
Obtaining the motion direction of the gesture based on the detection data of the sensor,
For the plurality of operated devices, it is determined whether the movement direction of the gesture is a predetermined direction based on the operated device,
A gesture recognition system that operates to recognize the operated device that is determined to have the movement direction of the gesture as the predetermined direction, as a target operated device to which the gesture is directed. The processor determines, for at least one or more operated devices, whether the operation direction of the user's gesture is a predetermined direction with respect to the operated device, and the operation direction of the gesture is determined to be the predetermined direction. A gesture recognition method, comprising recognizing the operated device as a target operated device to which the gesture is directed. It is determined for at least one or more operated devices whether the operation direction of the gesture of the user is a predetermined direction with reference to the operated device, and the operation direction of the gesture is determined to be the predetermined direction. A computer program for causing a processor to perform an operation of recognizing an operated device as a target operated device to which the gesture is directed.

Images