JP6551941B2 - Gesture determination device - Google Patents

Description

  The present invention relates to a gesture determination device.

  As conventional techniques, a gesture acquisition unit that acquires a gesture performed by an operator, a gaze tracking unit that acquires a movement of a gaze of the operator, and a gaze pattern determination that determines whether the movement of the gaze satisfies a predetermined pattern Means, a gesture selection means for determining an allowance gesture for accepting an input when the movement of the sight line satisfies a predetermined pattern, and an instruction corresponding to the gesture is generated only when the acquired gesture is an allowance gesture There is known a gesture recognition device including an instruction generation unit (see, for example, Patent Document 1).

  This gesture recognition device prevents an operation other than a gesture from being erroneously recognized as a gesture.

JP2015-153195A

  However, in the conventional gesture recognition apparatus, the operator has to perform the movement of the sight line of a predetermined pattern, so the operation load is large.

  Accordingly, an object of the present invention is to provide a gesture determination device that suppresses erroneous determination as a gesture that pays back an operation without increasing an operation load.

  According to one aspect of the present invention, an imaging unit that periodically images an operator's hand, and a movement amount when the hand moves in one direction calculated based on a plurality of captured images are based on a movement amount threshold value. If it is large and the moving speed is greater than the first speed threshold value, it is determined that a hand-paying gesture has been performed, operation information corresponding to the hand-paying gesture is output, and after the determination is made, the first Setting a second velocity threshold larger than the velocity threshold and determining a gesture in which the first velocity threshold is switched to the second velocity threshold for a predetermined period of time and the hand is used And a gesture determination apparatus including the unit.

  According to the present invention, it is possible to suppress an erroneous determination as a gesture in which an operation of returning the hand is performed without increasing the operation load.

Fig.1 (a) is a schematic diagram which shows an example of the inside of the vehicle by which the gesture determination apparatus which concerns on embodiment is mounted, FIG.1 (b) is a block diagram which shows an example of a gesture determination apparatus, 1 (c) is a block diagram illustrating an example of a vehicle communication system to which a gesture determination device is connected. FIG. 2A and FIG. 2B are schematic diagrams for explaining an example of an operation of putting a hand and an operation of returning it. Fig.3 (a) is a graph which shows an example of determination of the payment gesture which used one speed threshold value which concerns on a comparative example, FIG.3 (b) is the 1st of the gesture determination apparatus which concerns on embodiment. It is a graph which shows an example of determination of the gesture to pay using the speed threshold value and 2nd speed threshold value. FIG. 4 is a flowchart showing an example of an operation related to the determination of a payment gesture of the gesture determination device according to the embodiment.

(Summary of the embodiment)
The gesture determination device according to the embodiment includes an imaging unit that periodically images an operator's hand, and a movement amount when the hand moves in one direction calculated based on a plurality of captured images. If the movement speed is larger than the threshold value and the movement speed is larger than the first speed threshold, it is determined that a gesture to perform a hand is performed, and operation information corresponding to the gesture to perform a hand is output, and determination is performed. , A second speed threshold value larger than the first speed threshold value, and a gesture for paying a hand by switching the first speed threshold value to the second speed threshold value for a predetermined period of time. And a determination unit configured to be determined.

  Since the gesture determination device determines the gesture to pay the hand after determining the gesture to move the hand by the second velocity threshold larger than the first velocity threshold, the hand returning operation is easy. It is not determined to be a gesture to be paid, and erroneous determination as a gesture to pay a gesture of returning the hand without increasing the operation load can be suppressed.

Embodiment
(Outline of Gesture Determination Device 1)
Fig.1 (a) is a schematic diagram which shows an example of the inside of the vehicle by which the gesture determination apparatus which concerns on embodiment is mounted, FIG.1 (b) is a block diagram which shows an example of a gesture determination apparatus, 1 (c) is a block diagram illustrating an example of a vehicle communication system to which a gesture determination device is connected. FIG. 2A and FIG. 2B are schematic diagrams for explaining an example of an operation of putting a hand and an operation of returning it. Note that, in each drawing according to the embodiment described below, the ratio between figures may be different from the actual ratio. In FIGS. 1B and 1C, the flow of main signals and information is indicated by arrows.

  The gesture determination device 1 enables, for example, the operation of the electronic device mounted on the vehicle 8 based on the made gesture. The operator can operate the function of the electronic device corresponding to the gesture by performing a predetermined gesture. This electronic device is, for example, the navigation device 84 and the music and video playback device 85 shown in FIG. The gesture determination device 1 and the electronic device, as one example, constitute a vehicle communication system 81, as shown in FIG. 1 (c). In the present embodiment, the operator's hand 9 is a detection target.

As shown in FIGS. 1A to 1C, 2A, and 2B, the gesture determination device 1 is a camera as an imaging unit that periodically images the hand 9 of the operator. 3 and the movement amount X when the hand 9 moves in one direction calculated based on the plurality of captured images 35 is larger than the movement amount threshold value 50, and the movement speed V is the first speed threshold value. 51 is greater than, it is determined that the gesture to pay hand 9 is performed to output the operation information S ₂ corresponding to the gesture to pay hand 9, after the determination is made greater than the first speed threshold 51 As a determination unit that sets a second speed threshold value 52 and determines a gesture for paying hand 9 by switching the first speed threshold value 51 to the second speed threshold value 52 for a predetermined period of time. And a controller 5.

(Configuration of camera 3)
The camera 3 is an imaging device using a semiconductor element such as a CCD image sensor (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

  The camera 3 is, for example, disposed on a display device 87 disposed on a center console 80 of the vehicle 8 as shown in FIG. 1A. Therefore, the imaging region 30 is a region in front of the display device 87. For example, the camera 3 captures an image of a gesture performed in front of the display device 87. In addition, the camera 3 is configured to irradiate near infrared rays in the direction of the imaging region 30, for example.

Camera 3, periodically captures an area 30, and outputs to the control unit 5 generates the image information S ₁ as the information of the captured image. The image obtained by imaging the imaging region 30 is an image 35 shown in FIGS. 2A and 2B as an example.

(Configuration of control unit 5)
Fig.3 (a) is a graph which shows an example of determination of the payment gesture which used one speed threshold value which concerns on a comparative example, FIG.3 (b) is the 1st of the gesture determination apparatus which concerns on embodiment. It is a graph which shows an example of determination of the gesture to pay using the speed threshold value and 2nd speed threshold value. The diagonal lines in FIG. 3A and FIG. 3B indicate the amount of movement X of the hand 9 moved by the determined payment gesture.

  The control unit 5 includes, for example, a CPU (Central Processing Unit) that performs operations and processes on acquired data according to a stored program, a RAM (Random Access Memory) that is a semiconductor memory, a ROM (Read Only Memory), and the like. Microcomputer. In this ROM, for example, a program for operating the control unit 5 is stored. For example, the RAM is used as a storage area for temporarily storing calculation results and the like.

  For example, as illustrated in FIG. 1B, the control unit 5 includes a movement amount threshold value 50, a first speed threshold value 51, a second speed threshold value 52, gesture information 53, have.

  The movement amount threshold value 50 is a threshold value of the movement amount X of the hand 9 used for determining a gesture to be paid. This movement amount X corresponds to the hatched area in FIGS. 2 (a) and 2 (b). That is, the movement amount X is a distance moved at a speed exceeding the first speed threshold value 51 and the second speed threshold value 52.

  The first speed threshold value 51 is a threshold value related to the moving speed V of the hand 9. When the moving speed V of the hand 9 exceeds the first speed threshold value 51 and the moving amount X of the hand 9 is larger than the moving amount threshold value 50, the control unit 5 determines that a gesture to be paid has been performed.

  The gesture to be paid here is, as shown in FIGS. 2A and 2B, after moving the hand 9 in one direction, a gesture in which the hand 9 is again paid in the opposite direction to one direction for the purpose of correction etc. In some cases, there is a case where the hand 9 is moved in one direction and then returned to the original position. Below, after performing a payment gesture by a comparative example and this embodiment, the difference of the judgment at the time of performing return operation is explained. The graphs in FIGS. 3A and 3B are the same.

When the speed threshold value is one as in the comparative example shown in FIG. 3A, the moving speed V is larger than the speed threshold value 100 during the time t ₁ to the time t ₄ when the paying operation is performed. Is detected in the period from time t ₂ to time t ₃ , and if the movement amount X in this period is larger than the movement amount threshold value 50, it is determined that the gesture to be paid has been performed.

Also, in the operation of returning the hand 9 at time t ₅ to time t ₈ , movement is performed during the period of time t ₆ to time t ₇ because the velocity threshold 100 is constant despite the return more slowly than the operation of paying If the speed V is greater than the speed threshold value 100 and the movement amount X during this period is greater than the movement amount threshold value 50, it is determined that the gesture is to be paid in the opposite direction, and an erroneous determination not intended by the operator is made. Ru.

On the other hand, as shown in FIG. 3B, the control unit 5 of the present embodiment has a moving speed V greater than the first speed threshold value 51 during the time t ₁₀ to the time t ₁₃ when the paying operation is performed. is detected to the time t _{11 ~} time t _12, the moving amount X of this period is greater than the movement amount threshold 50, it is determined that the gesture is made to pay.

Control unit 5, the moving velocity V in the operation at least pay is after the time t ₁₂ to be less than the first speed threshold 51, calculates a second speed threshold 52.

Then, the control unit 5, so determining the operation of returning with the second speed threshold 52 greater than the first speed threshold 51, as shown in FIG. 3 (b), the time t _{14 ~} Time moving speed V of the operation of returning made to t ₁₅ does not exceed the second speed threshold 52, it is not determined that the gesture to pay. However, the second velocity threshold 52 is, for example, larger than the first velocity threshold 52 and smaller than the maximum movement velocity V _max of the immediately preceding operation, since the gesture for the return operation is not performed. It is preferable to be a value.

Therefore, the control unit 5 calculates the second speed threshold value 52 by multiplying the maximum value of the moving speed V (maximum moving speed V _max ) by a constant α as shown in the following formula (1).
Second velocity threshold 52 = maximum movement velocity V _max × α (1)
The constant α is determined such that the second velocity threshold 52 is larger than the first velocity threshold 51 and smaller than the maximum value (maximum moving velocity V _max ).
Therefore, it is preferable that the constant α satisfy the following formula (2).
First speed threshold value 51 <maximum movement speed V _max × α <maximum movement speed V _max (2)
That is, the constant α is preferably in the range of the following formula (3).
First velocity threshold 51 / maximum movement velocity V _max <α <1 (3)

The is preferably a second speed threshold 52 is smaller than the maximum moving speed V _max, when the second speed threshold 52 is at a maximum moving speed V _max or more, be determined as a gesture to pay operation of returning Because it becomes difficult. This is similar to ignoring the return action after a paying gesture. If the returning operation is ignored, the gesture to be paid after the operation to be performed intentionally by the operator is not determined against the intention and the operability is not good.

  In addition, when the second velocity threshold 52 becomes equal to or less than the first velocity threshold 51, it becomes easy to determine that the return operation after the payment gesture is a gesture to be performed, and the operability is not good either.

Therefore, the control unit 5 of the present embodiment, as shown in the above equation (2), has a second speed threshold value 52 that is greater than the first speed threshold value 51 and is the maximum for the gesture to be paid immediately before. The second velocity threshold 52 is set to be a threshold smaller than the moving velocity V _max .

The control unit 5 calculates the second velocity threshold 52 by the equation (1) which multiplies the maximum movement velocity V _max in the payment gesture by a predetermined constant α when the payment gesture is performed. Check if equation (2) holds. When the expression (2) is established, the control unit 5 determines whether or not the action of returning using the second speed threshold value 52 calculated by the expression (1) is a gesture.

Moreover, the control part 5 calculates constant (alpha) so that Formula (3) may be satisfy | filled when Formula (2) is not materialized. The constant α satisfying the expression (3) is, for example, a half of the difference between the value of the first speed threshold value 51 / maximum moving speed V _max that is the left side of the expression (3) and 1 that is the right side. The value of

As a modified example, the control unit 5 adds a value calculated based on the maximum value of the movement speed V of the gesture to be paid (maximum movement speed V _max ) to the first speed threshold value 51 to add a second speed threshold. The value 52 may be calculated. The calculation of the second speed threshold value 52 is performed by the following formula (4) as an example.
Second velocity threshold 52 = first velocity threshold 51+ {maximum travel velocity V _max − (maximum travel velocity V _max −first velocity threshold 51) × α} (4)
Here, α is a number larger than 0 and smaller than 1.

  The control unit 5 uses the second speed threshold value 52 only for a predetermined period, and switches from the second speed threshold value 52 to the first speed threshold value 51 after the predetermined period has elapsed. . As an example, this predetermined period is 2 seconds.

The gesture information 53 is, for example, information on a gesture performed by the hand 9. The gestures performed by the hand 9 are, for example, a gesture of wiping the hand 9 and a gesture of drawing a circle by the hand 9. The control unit 5 determines, for example, due to a hand 9 based on a plurality of image information S ₁ and gesture information 53, it determines that operation is performed to pay, a gesture to pay.

The gesture for paying the hand 9 is, for example, a gesture for scrolling the display image displayed on the display device 87 by the electronic device to be operated in the moving direction. Operation target electronic device, for example, to scroll the display image in the moving direction of the hand 9 based on the operation information S _2.

(Configuration of vehicle communication system 81)
The vehicle communication system 81 is a communication system using a standard such as CAN (Controller Area Network) as an example. The vehicle communication system 81 communicates with, for example, the gesture determination device 1 and the navigation device 84 via a vehicle LAN (Local Area Network) 82.

  As shown in FIG. 1C, the vehicle communication system 81 is, for example, a gesture determination device 1, a vehicle control unit 83, a navigation device 84, a music and video reproduction device 85, an air conditioner 86, a display device 87, etc. It is configured to communicate. The vehicle control unit 83 is, for example, a microcomputer that controls the vehicle communication system 81.

When the navigation device 84 is operated with the gesture determination device 1 as an operation target, the operation information S ₂ output from the gesture determination device 1 is input to the navigation device 84 via the vehicle LAN 82. The navigation device 84 performs an operation based on the operation information S ₂ input.

  Below, an example of operation | movement regarding determination of the gesture for which the gesture determination apparatus 1 of this Embodiment performs is demonstrated according to the flowchart of FIG.

(Operation)
When the power of the vehicle 8 is turned on, the camera 3 of the gesture determination device 1 periodically outputs image information S ₁ that is information of the image 35 of the captured image area 30 to the control unit 5. Control unit 5, in chronological stores image information S ₁ acquired.

Control unit 5, the movement of the hand 9 monitors whether or not detected based on the image information S ₁ and gesture information 53 acquired. When “Yes” in Step 1 is established, that is, when the movement of the hand 9 is detected (Step 1: Yes), the control unit 5 calculates the moving speed V and the moving amount X of the paying motion (Step 2).

The control unit 5 compares the calculated movement speed V with the first speed threshold value 51, and compares the calculated movement amount X with the movement amount threshold value 50 to determine whether or not a gesture for paying has been performed. Determine Control unit 5 determines that the gesture to pay is performed (Step3: Yes), it outputs the operation information _{S 2} based on the gesture, calculating a second speed threshold 52 (Step4).

The control unit 5 uses the calculated second velocity threshold 52 to determine whether the returning operation is a payment gesture. Control unit 5 calculates the amount of movement X and the moving velocity V of the operation of returning, based on the plurality of image information _{S 1} (Step5).

  Whether the control unit 5 compares the calculated movement speed V with the second speed threshold value 52 and compares the calculated movement amount X with the movement amount threshold value 50 to return the gesture. It is determined whether or not (Step 6).

Control unit 5 determines that a gesture operation for returning pay (Step7: Yes), outputs the operation information _{S 2} based on the gesture to pay (Step8), the process proceeds to step 1. The control unit 5 switches the second speed threshold 52 to the first speed threshold 51 after a predetermined period has elapsed.

  Here, when the control unit 5 determines in step 3 that the payment gesture is not performed (step 3: No), the process proceeds to step 1.

  If the control unit 5 determines that the return operation is not a payment gesture in Step 7 (Step 7: No), the process proceeds to Step 1. The control unit 5 switches the second speed threshold 52 to the first speed threshold 51 after a predetermined period has elapsed.

(Effect of the embodiment)
The gesture determination apparatus 1 according to the present embodiment can suppress erroneous determination as a gesture that pays for the action of returning the hand 9 performed after the gesture of paying the hand 9 without increasing the operation load. Specifically, the gesture determination device 1 determines the returning operation after determining the gesture for paying the hand 9 by the second speed threshold value 52 that is larger than the first speed threshold value 51. Therefore, the gesture determination device 1 does not determine that the action of returning the hand 9 is not a gesture of easily moving the hand 9 and erroneously determines that the gesture of returning the hand 9 is not performed without increasing the operation load. it can.

The gesture determination device 1 is set so that the second speed threshold value 52 is larger than the first speed threshold value 51 and smaller than the maximum movement speed V _{max of the} gesture to be paid immediately before. When it is performed as a gesture to pay, it is suppressed that it is not determined unintentionally.

  Since the erroneous determination is suppressed without increasing the operation load, the gesture determination device 1 suppresses the determination not intended by the operator and improves the operability.

  Although the above-described gesture determination device 1 is configured to detect a gesture on the front of the display device 87, the present invention is not limited to this, and is configured to detect a gesture performed in the vicinity of steering, a floor console, etc. Alternatively, it may be configured to detect a gesture made in the vicinity of an input device such as a touch pad.

  The gesture determination device 1 according to the above-described embodiment and modification includes, for example, a part of the program executed by a computer, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and the like according to the application. It may be realized by.

  As mentioned above, although some embodiment and modification of this invention were demonstrated, these embodiment and modification are only examples, and do not limit the invention based on a claim. These novel embodiments and modifications can be implemented in other various forms, and various omissions, replacements, changes and the like can be made without departing from the scope of the present invention. Further, all combinations of the features described in the embodiments and the modifications are not necessarily essential to the means for solving the problems of the invention. Furthermore, these embodiments and modifications are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

DESCRIPTION OF SYMBOLS 1 ... Gesture determination apparatus, 3 ... Camera, 5 ... Control part, 8 ... Vehicle, 9 ... Hand, 30 ... Imaging area, 35 ... Image, 50 ... Movement amount threshold value, 51 ... 1st speed threshold value, 52: second speed threshold value 53: gesture information 80: center console 81: vehicle communication system 82: vehicle LAN 83: vehicle control unit 84 navigation device 85 music and video reproduction device 86 ... Air conditioner, 87 ... Display device, 100 ... Speed threshold

Claims (3)

An imaging unit that periodically images an operator's hand;
Gesture to pay a hand when the movement amount when the hand moves in one direction calculated based on the plurality of captured images is larger than the movement amount threshold value and the movement speed is larger than the first speed threshold value The operation information corresponding to the gesture of paying the hand is output, and after the determination is made, a second speed threshold value greater than the first speed threshold value is set, and A determination unit configured to switch the first speed threshold value to the second speed threshold value for a predetermined period and determine a gesture to pay the hand;
Gesture determination device equipped with The determination unit calculates the second speed threshold by multiplying a maximum value of the moving speed by a constant,
The constant is determined such that the second speed threshold is greater than the first speed threshold and less than the maximum value.
The gesture determination device according to claim 1. The determination unit calculates the second speed threshold by adding a value calculated based on the maximum value of the moving speed to the first speed threshold.
The gesture determination device according to claim 1.

Images