WO2018061603A1 - Gestural manipulation system, gestural manipulation method, and program - Google Patents

Abstract

The present invention provides a gestural manipulation system, etc., which allows an operator to discriminate between a right hand and a left hand and change the respective corresponding object to be manipulated, and more easily and accurately give a manipulation indication to a plurality of various objects to be manipulated. The gestural manipulation system includes: an imaging unit for imaging the hands of an operator who manipulates the steering gear of a vehicle; a determination unit for detecting a hand present in a detection area that is set between the steering gear and the operator on the basis of the imaging data of the imaged operator's hands before an acceptance period for gesture input by the operator, and determining whether the hand is a right hand or a left hand; a selection unit for selecting, before the acceptance period and in accordance with a pattern for the determined hand, any object to be manipulated from among a plurality of different objects to be manipulated that are associated with the vehicle; and an indication unit for giving, in the acceptance period, a manipulation indication to the object to be manipulated on the basis of gesture input by the hand.

Description

Gesture operation system, gesture operation method and program

The present invention relates to a gesture operation system and a gesture operation method for instructing operation on an operation target in response to a gesture input by an operator.

Generally, it is known to use a switch, a touch panel, or the like as a method for operating an in-vehicle device such as a head-up display or a car navigation system. However, the number of switches that can be installed may be limited due to space limitations, and the driver may pay attention to the touch panel, and may not be safe due to circumstances. For this reason, there has been proposed a device for operating an in-vehicle device by a gesture input by a driver instead of a switch operation or a touch panel operation (Patent Documents 1 to 5).

JP 2010-184600 A JP 2016-018223 A Japanese Unexamined Patent Publication No. 2015-0588804 JP 2014-021748 A JP 2014-109994 A

As described above, there is a device for operating an in-vehicle device by a gesture input by the driver, but there is no device for determining whether the driver's right hand or left hand and changing the operation target according to each, There has been a need for such a device.

In recent years, the number of devices installed in the vehicle has increased, and the applications used during driving have also increased. For these various operation targets, operation instructions can be easily performed by gesture input by the operator. In addition, it has been demanded to carry out accurately.

The present invention has been made under the above-described circumstances, and determines whether it is a right hand or a left hand, changes the operation target according to each, and makes it easier and more accurate for an operator to issue operation instructions to a plurality of various operation targets. It is an object of the present invention to provide a gesture operation system, a gesture operation method, and a program that can be performed in a simple manner.

In order to achieve the above object, a gesture operation system according to the present invention includes an imaging unit that images an operator's hand that operates a vehicle steering device, and the captured image before the gesture input reception period by the operator. A discriminating unit that detects a hand existing in a detection region set between the steering device and the operator based on imaging data of the operator's hand and discriminates either the right hand or the left hand And before the reception period, according to the determined hand pattern, a selection unit that selects any one of a plurality of different operation objects related to the vehicle, and the reception period, And an instruction unit for giving an operation instruction to the operation target based on a gesture input by a hand.

Here, the gesture operation system according to the present invention changes the shape of the operation area for receiving the gesture input according to the determined right hand or left hand, and the operation area is A setting unit that is set between an apparatus and the operator may be further included, and the instruction unit may perform the operation instruction based on a gesture input by the hand in the operation area.

The shape of the operation area may be set so that the determined hand is asymmetrical with respect to the operator so that the operation of the steering device is not hindered, and swipeable up and down and left and right. Good.

The setting unit may arrange the operation area on the basis of the position of the hand held over the operator before the reception period.

When the hand pattern and the content of the operation instruction for the operation target are associated with each other, the instruction unit further includes the content of the operation instruction associated with the determined state of the right hand or the left hand. An operation instruction for the operation target based on the above may be performed.

The gesture operation system according to the present invention further includes a generation unit that generates an operation region for receiving the gesture input between the steering device and the operator, and the instruction unit is configured to include the operation region in the operation region. The operation instruction may be performed based on hand gesture input.

The instruction unit may feed back the result of the operation instruction.

In order to achieve the above object, a gesture operating method according to the present invention includes a step of imaging an operator's hand operating a vehicle steering apparatus, and the imaging operation is performed before a period for accepting a gesture input by the operator. Detecting a hand existing in a detection region set between the steering device and the operator based on imaging data of an operator's hand, and determining either a right hand or a left hand; Before the acceptance period, according to the determined hand pattern, selecting any operation target from a plurality of different operation targets related to the vehicle;
Performing an operation instruction for the operation target based on a gesture input by the hand during the reception period.

In order to achieve the above object, a program according to the present invention causes a computer to execute the above gesture operation method.

According to the present invention, it is possible to distinguish between the right hand and the left hand, change the operation target according to each, and perform an operation instruction to a plurality of various operation objects more easily and accurately.

It is a block diagram which shows the structural example of the gesture operation system which concerns on embodiment of this invention. It is a figure which shows an example in the vehicle carrying the gesture operation system of FIG. It is a figure for demonstrating the outline of the gesture input implement | achieved by the gesture operation system which concerns on embodiment in time series. It is a figure for demonstrating the example of a setting of a detection area. It is a figure for demonstrating the example of a setting of an operation area | region. It is a flowchart which shows the example of a whole process of a gesture operation system. It is a flowchart which shows the detail of the preliminary | backup operation | movement detection process of FIG. It is a flowchart which shows the detail of this operation process of FIG. It is a figure which shows an example of the correspondence of a hand state and an operation target. It is a figure which shows an example of the correspondence of a hand state and an operation target. It is a figure which shows the state of various hands. It is a figure which shows the 1st correspondence of an operation target and operation instruction content. It is a figure which shows the 2nd correspondence of an operation target and operation instruction content. It is a figure which shows the 3rd correspondence of an operation target and operation instruction content. It is a figure for demonstrating the example of a setting of the operation area | region made to produce | generate on the basis of the position of the hand detected at the time of a preliminary | backup movement detection process. It is a figure for demonstrating the example of a setting of the operation area | region made to produce | generate asymmetrical left and right. It is a figure for demonstrating the discrimination | determination process performed in order to detect the preliminary | backup motion of a hand at the time of preliminary motion detection. It is a figure for demonstrating the example of arrangement | positioning of a sensor.

Hereinafter, a gesture operation system (computer) according to an embodiment of the present invention will be described. The gesture operation system 1 gives an operation instruction to an operation target device related to a vehicle according to a gesture input of an operator who is a driver of the vehicle.

[Configuration of Gesture Operation System 1]
FIG. 1 is a block diagram illustrating a configuration example of a gesture operation system 1 according to an embodiment of the present invention.

1, the gesture operation system 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and a sensor 31, and these components 11, 12. , 13 are connected to the bus 40. The operation target 20 is also connected to the bus 40.

The operation target 20 includes, for example, a HUD (Head-Up Display) 21, a car navigation device 22 including a display unit 221, a room or a headlight 23, and the like.

The navigation device 22 will be described as a single navigation device, but can be integrated into the CPU 11, ROM 12 and RAM 13.

The CPU 11 is connected to each of the constituent elements 11 to 13, 20, and 31 through the bus 40 to perform control signal and data transfer processing, and execute and calculate various programs for realizing the entire operation of the gesture operation system 1. Processing, processing related to gesture input, and the like are performed.

The various functions of the CPU 11 will be described as appropriate in the operation description of the gesture operation system 1 described later.

The ROM 12 stores an operating system program and various data necessary for operation control of the entire gesture operation system 1, and the program is read into the RAM 13 and started to be executed by the CPU 11. The program is stored in a recording medium such as a DVD-ROM or HDD.

The RAM 13 is provided with a recording area for temporarily storing data and programs necessary for gesture input, and holds the data and programs.

The sensor 31 is an imaging device for detecting a gesture input performed by an operator during driving. The sensor output is processed by the CPU 11 and an operation is instructed according to the gesture input. As will be described later, in the gesture operation system 1 of this embodiment, a driver who operates the steering (steering device) 5 performs a gesture by hand. If the gesture can be detected, the configuration of the sensor 31 is as follows. Any type of sensor can be used. For example, the sensor 31 may be an RGB color sensor capable of imaging a hand, an infrared distance sensor, a laser distance sensor, an ultrasonic distance sensor, a distance image sensor, an electrolytic sensor, an image sensor, or the like.

[State in the car]
FIG. 2 is a diagram for explaining a state in the vehicle on which the gesture operation system 1 is mounted.

As shown in FIG. 2, a steering wheel 5 and an instrument panel (instrument panel) 26 are provided in the vehicle. Moreover, HUD21 is provided in the windshield surface which an operator can visually recognize.

The sensor 31 is provided almost at the center of the instrument panel 26. The sensor 31 detects a gesture by the operator's hand.

The display unit 221 is provided in the car navigation device 22.

[Outline of gesture input]
Next, an outline of gesture input realized by the gesture operation system 1 will be described with reference to FIG. FIG. 3A to FIG. 3E are diagrams for explaining the outline in time series.

When the operator 100 is operating the steering wheel 5 in the vehicle (FIG. 3 (a)), when the user holds the right hand 101 in the detection area t1 (FIG. 3 (b)), the CPU 11 moves the hand to the detection area t1. 101 is determined to exist, and an operation region s1 for accepting a gesture input by the hand 101 is generated between the steering 5 and the operator 100 (FIG. 3C).

Then, within a predetermined period after the operation area s1 is generated, for example, as shown in FIGS. 3D and 3E, the operator 100 moves the right hand 101 in the x direction to input a gesture. In this case, the CPU 11 detects the gesture input and executes an operation instruction to the operation target 20 such as the car navigation device 22.

As described above, in the gesture operation system 1 of the present embodiment, the operation region s1 is generated after the hand is detected in the detection region t1. Furthermore, in the operation area s1, when the gesture input by the hand of the operator 100 is accepted, an operation instruction to the operation target 20 is performed.

In the following description of the embodiment, a predetermined period after the operation area s1 is generated is referred to as a “reception period” of gesture input. In the gesture operation system 1 of this embodiment, the detection area t1 is set “before the reception period”.

The detection area t1 before the gesture input reception period (FIG. 3B) and the operation area s1 during the reception period (FIGS. 3C to 3E) are respectively shown in FIGS. The detailed arrangement | positioning aspect is shown.

FIG. 4 is a diagram for explaining (a) a top view of the detection region t1 and (b) a side view of the detection region t1. FIG. 5 is a diagram for explaining (a) a top view of the operation area s1 and (b) a side view of the operation area s1. 4 and 5 merely illustrate the arrangement and shape of the regions t1 and s1.

[Operation of Gesture Operation System 1]
Next, the entire process executed to realize this gesture input will be described with reference to FIGS. 1, 2, and 4 to 6. FIG. FIG. 6 is a flowchart showing the entire gesture operation control process.

In FIG. 6, in the gesture operation system 1, the preliminary operation detection process is performed before the gesture input reception period (step S10), and then the main operation process is performed (step S11). The processes in steps S10 to S11 are shown in detail in FIGS. 7 to 8 described later.

FIG. 7 is a flowchart of processing (preliminary motion detection processing) sequentially executed before the gesture input acceptance period. FIG. 8 is a flowchart of processing (this operation processing) sequentially executed during the gesture input acceptance period.

[Preliminary motion detection processing]
In FIG. 7, the CPU 11 acquires one frame of image data (imaging data) captured by the sensor 31 (step S101), and determines whether the hand of the operator 100 exists in the image data (step S101). S102). In step S102, the CPU 11 extracts only a region having a certain distance (for example, a three-dimensional component) from the image data from the sensor 31, sets the region as the detection region t1 (FIG. 4), and detects the detection region t1. It is determined whether or not the hand of the operator 100 exists inside.

In this gesture operation system 1, it is determined that a hand is present from the analysis results of various elements such as the movement amount and shape of an object (such as a hand) obtained from the acquired image data and the barycentric coordinates. For example, in the example of FIGS. 4A and 4B, the CPU 11 binarizes the image data from the sensor 31 based on a preset threshold value, and detects the contour of the object included in the image data. To do. Then, the CPU 11 extracts a contour having a preset area from the plurality of detected contours, and recognizes the contour as the hand 101 when the center-of-gravity coordinates of the contour are included in the detection region t1. If it is determined that the hand is present.

If it is determined in step S102 that the hand is present, the CPU 11 determines whether the preliminary operation condition is satisfied (step S103).

In this gesture operation system 1, acceptance of gesture input, which will be described later, becomes effective when the preliminary operation for validating the gesture input satisfies a certain condition.

As the preliminary movement, for example, there is an action of stopping the hand with the palm extended. In this case, in the detection area t1, the CPU 11 determines that the preliminary operation condition is satisfied when the hand is stationary for a predetermined time with the palm extended.

The above-described determination as to whether or not the palm has been expanded can be made based on hand unevenness information, for example. In addition, the determination of whether or not the hand is stationary can be made based on the amount of movement of the hand barycentric coordinates. For example, it is determined that the hand is stationary when the movement amount of the center-of-gravity coordinates of the hand within a certain frame is equal to or less than a threshold value.

The preliminary operation and the preliminary operation condition described above are not limited to the above examples, and can be freely set.

The detection region t1 described above may be any region that can detect the hand of the operator 100, and can be implemented by various other alternative positions, sizes, shapes, and the like. For example, the detection region t1 may be moved without being fixedly arranged at the same position at the time of detection in step S103. By such movement, for example, the position of the detection region t1 at the time of determination in step S103 may be different from the position of the detection region t1 at the time of determination in step S102 (when determining whether a hand is present). it can.

When it is determined in step S103 that the preliminary operation condition is satisfied, the CPU 11 performs initial setting of the operation target 20 (step S104).

In this embodiment, as an initial setting, a state of a hand satisfying the preliminary operation condition of the above-described step S102 is selected from the operation targets of hardware (HUD 21, car navigation device 22, etc.) and software (music, map, etc.). It is conceivable to determine a corresponding operation target according to (pattern).

Here, a description will be given with reference to FIGS. 9A and 9B in relation to the determination of the operation target 20 described above. FIG. 9A is a diagram illustrating an example of data d1 including a hand state when hardware is determined. FIG. 9B is a table showing an example of data d2 including a hand state when software is determined (called).

For example, according to FIG. 9A, when it is determined that the left hand is in the par state, the HUD 21 is determined as the operation target 20, or when it is determined that the right hand is in the goo state, It is exemplified that air conditioning is determined as the operation target 20.

In this gesture operation system 1, either data d1 or d2 shown in FIG. 9A or 9B is recorded in the ROM 12. In the process of step S104 in FIG. 7, the CPU 11 reads the data d1, d2 from the ROM 12, and determines the operation target 20 corresponding to the state of the right hand or the left hand.

That is, the CPU 11 (determination unit) determines whether the operator's hand determined in step S102 is the right hand or the left hand before the gesture input reception period (step S104). For example, as such a determination method, a right hand or a left hand is determined based on whether or not the center of gravity position of the hand is relatively on the right side or the left side (preset range) on the horizontal plane in the optical axis direction of the sensor 31. It is conceivable.

And CPU11 (selection part) selects the operation target 20 matched with the state of any hand from several various operation target.

The hand state described above is not limited to the example shown in FIG. 9A or 9B, and various states may be included.

FIGS. 10A to 10O illustrate various hand states. 10 (i) shows the goo state in FIGS. 9A and 9B, FIG. 10 (j) shows the choke state in FIGS. 9A and 9B, and FIG. 10 (k) shows FIGS. 9A and 9B. The state of par in is shown. In FIG. 10, various other hand states are shown. Thereby, the correspondence with the various various operation target 20 can be set.

After step S104 in FIG. 7, the CPU 11 (generating unit) generates the operation area s1 (step S105). For example, in FIGS. 5A and 5B, the CPU 11 detects the hand 101 held by the operator 100 based on the image data from the sensor 31, and between the steering wheel 5 and the operator 100. The operation area s1 is generated around the hand 101.

The operation area s1 described above is not limited to the shape shown in FIGS. 5A and 5B, and can be set freely.

[This processing]
In FIG. 8, the CPU 11 acquires one frame of image data captured by the sensor 31 (step S111), and determines whether there is a gesture input operation by the operator 100 in the image data (step S112). In step S112, the CPU 11 analyzes the movement of the hand of the operator 100 in the operation area s1 from the image data from the sensor 31, and determines whether the operation is a gesture input operation. In this gesture operation system 1, it is determined as a gesture input operation from the analysis results of various elements such as the amount and direction of movement of the hand and the change in the state of the hand.

For example, when the state of the right hand or the left hand based on the image data matches the pre-registered hand state (FIG. 9A or 9B), the CPU 11 (instruction unit) performs an operation corresponding to the gesture input. An operation instruction is given to the object 20 (step S113).

In this gesture operation system 1, the content of the operation instruction to the operation target 20 associated with the state of the right hand or the left hand is recorded in the ROM 12 as data. During the processing in step S113, the data is read and an operation instruction is issued by the CPU 11.

11 to 13 are diagrams showing data d11 to d13 of correspondence between the operation target 20 and the operation instruction content 25. FIG. The ROM 12 stores the data d11 to d13. According to the process of step S113, the CPU 11 reads these data from the ROM 12, and issues an operation instruction to the operation target 20 corresponding to the gesture input along the operation instruction content 25. For example, in FIG. 11, a map scale or the like is given as the operation target 20, and operation instruction content 25 related to operations such as wide area / details of the map is shown. In FIG. 12, an engine or the like is cited as the operation target 20, and an operation instruction content 25 related to an operation such as on / off of the engine is shown. In FIG. 13, a camera video or the like is cited as the operation target 20, and an operation instruction content 25 related to an operation such as switching of the back camera video / panoramic video is shown.

Next, the CPU 11 (instruction unit) feeds back the result of the operation instruction in step S113 (step S114). As an example of feedback, for example, changing the display on the display unit 221 or the instrument panel 26 of the HUD 21, the car navigation device 22, or changing the operation of the operation target 20 (for example, ON / OFF of an air conditioner, etc.) ), Notification by outputting sound from a speaker, and the like. This feedback plays a role of visually or audibly confirming an operation instruction for an appropriate operation target of the operator 100.

Further, the CPU 11 (instruction unit) determines whether the operation has been completed, that is, whether the operation has been completed (step S115). For example, when the hand has moved out of the detection area t1, or when the hand has moved out of the operation area s1, the CPU 11 determines that the operation has been completed. According to the flowchart of FIG. 8, even if the hand state does not match the pre-registered hand pattern (FIG. 9A, FIG. 9B) in the determination process of step S <b> 112, Judged as a state.

As described above, according to the gesture operation system 1 of the present embodiment, a hand existing in the detection region t1 is detected before the gesture input reception period, and whether the hand is either the right hand or the left hand. The operation target is selected according to the state of the hand. Then, during the gesture input acceptance period, an operation instruction for the operation target is performed based on the gesture input by the hand of the operator 100. Here, since the state of the hand includes various states of the right hand or the left hand, it is possible to set the correspondence between the various hand states and a plurality of various operation objects (including hardware and software) 20. This means that the operator can be instructed to operate the various types of operation objects 20. For example, operations of a plurality of different application functions (such as map display and music playback) can be instructed to the same hardware (such as a car navigation device).

In addition, since the operation target 20 described above is selected before the gesture input reception period, even if the operation target is selected by mistake, the correct operation target 20 can be obtained by re-selecting the operation target before the reception period. Can be selected.

And, it is configured to accept a gesture input after selection of the operation target 20, and is configured to be able to easily perform an operation instruction while driving the vehicle.

Next, some modified examples of the gesture operation system 1 of the present embodiment will be described.

(Modification 1)
Although the position where the operation area s1 is arranged is not described above, the operation area s1 may be arranged based on the position of the hand immediately after the preliminary movement.

FIG. 14 shows an arrangement example of the operation area s1. In the arrangement examples of FIGS. 14A and 14B, the operation area s1 is arranged at a position corresponding to the position of the hand 101 held by the operator 100 in the detection area t1 immediately after the preliminary movement. Therefore, in this gesture operation system 1, it is possible to perform gesture input while moving the hand at a position desired by the operator.

(Modification 2)
The shape of the operation area s1 can be freely set. FIG. 15 shows a shape that is changed depending on whether it is the right hand or the left hand, unlike those shown in FIGS.

15 (a) and 15 (b), the CPU 11 generates an operation area s1 in which the right hand 101 can be swiped while moving the right elbow to the left.

On the other hand, in FIG. 15 (c) and FIG. 15 (d), the operation area s1 in which the left hand 102 can be swiped while the left elbow is moved to the right is set by the CPU 11 (setting unit).

As a result, the operator 100 can perform a swipe operation up and down and left and right while making the corresponding hand asymmetrical when viewed from the operator 100 while driving so as not to hinder the operation of the steering 5. The operability of gesture input is improved.

(Modification 3)
In the gesture operation system 1, when it is determined whether the preliminary operation condition is satisfied (step S103 in FIG. 7), a plurality of detection areas may be used. For example, FIG. 16 illustrates a mode in which a preliminary operation is detected when two detection regions t1 and t11 are used. In FIG. 16, in addition to the presence of the hand 101 in the detection area t1, in another detection area t11 smaller than the detection area t1, the center of gravity position p of the hand 101 (for example, the x-axis, y-axis, and z in the world coordinate system). Each axis value) and the hand 101 is stationary for a certain time. Accordingly, it is possible to determine whether the preliminary operation condition is satisfied while determining the presence of the hand described in step S103.

(Modification 4)
In step S103 shown in FIG. 7, the center of gravity position p of the hand of the operator 100 is analyzed based on the image data from the sensor 31, and the state of the hand having the center of gravity position p and the hand registered in advance are analyzed. When the state matches, it can be determined that the motion of the hand satisfies the preliminary operation condition as the preliminary operation.

(Modification 5)
The sensor 31 can be changed as long as it can capture an image of the movements of the hands 101 and 102 of the operator 100 who operates the steering wheel 5. For example, FIG. 17 shows that (a) the sensor 31 is installed on the ceiling of the vehicle, (b) the sensor 31 is installed near the map lamp, and (c) the sensor 31 is near the car navigation device 22. The example of arrangement | positioning which shows the case where it installs and the case where (d) sensor 31 is installed in a dashboard is shown.

(Modification 6)
In the gesture operation system 1 of the above embodiment, the case where the operation area s1 is generated in the process of step S105 has been described. Separately from this, the operation area s1 may not be generated each time, but may be set in advance for the operator 100.

(Modification 7)
In the above description, aspects of passengers other than the driver (passenger seat, rear seat) have not been mentioned, but it is also possible for these passengers to input gestures as the operator 100 in the same manner as the driver. Good. For example, a sensor 31 capable of imaging a passenger is provided, and the gesture operation system 1 according to the above-described embodiment performs processing for the passenger's gesture input based on the image data from the sensor 31 (steps S101 to S105). Steps S111 to S115) may be performed.

The above variations can be implemented in combination with the gesture operation system 1 of the above embodiment.

1 Gesture Operation System 5 Steering 11 CPU
12 ROM
13 RAM
20 Operation target 21 Head-up display (HUD)
22 Navigation device (example of operation target)
23 Light 25 Operation instruction content 26 Instrument panel 31 Sensor 100 Operator 101, 102 Hand 221 Display unit t1, t11 Detection area s1 Operation area

Claims (10)

1. An imaging unit for imaging an operator's hand operating a vehicle steering device;
   Detecting a hand existing in a detection region set between the steering device and the operator based on the imaged data of the operator's hand before the period for accepting the gesture input by the operator And a discriminator for discriminating either the right hand or the left hand,
   Prior to the acceptance period, according to the determined hand pattern, a selection unit that selects any operation target from a plurality of different operation targets related to the vehicle;
   An instruction unit that issues an operation instruction to the operation object based on a gesture input by the hand during the reception period.
2. A setting for changing the shape of the operation area for accepting the gesture input and setting the operation area between the steering device and the operator according to the determined right hand or left hand. Further comprising
   The gesture operation system according to claim 1, wherein the instruction unit performs the operation instruction based on a gesture input by the hand in the operation area.
3. The shape of the operation region is set so that the determined hand is asymmetrical with respect to the operator so that the operation of the steering device is not hindered, and can be swiped up and down and left and right. The gesture operation system according to claim 2.
4. 4. The gesture according to claim 2, wherein the setting unit arranges the operation area with reference to a position of a hand held over the operator before the reception period. 5. Operation system.
5. When the hand pattern and the content of the operation instruction for the operation target are associated with each other, the instruction unit further includes the content of the operation instruction associated with the determined state of the right hand or the left hand. The gesture operation system according to any one of claims 1 to 4, wherein an operation instruction is performed on the operation target based on the function.
6. A generator that generates an operation region for receiving the gesture input between the steering device and the operator;
   The gesture operation system according to claim 1, wherein the instruction unit performs the operation instruction based on a gesture input by the hand in the operation area.
7. The gesture operation system according to any one of claims 1 to 6, wherein the instruction unit feeds back a result of the operation instruction.
8. The gesture operation system according to any one of claims 1 to 7, wherein the instruction unit determines whether or not the gesture input operation has been completed.
9. Imaging an operator's hand operating a vehicle steering device;
   Detecting a hand existing in a detection region set between the steering device and the operator based on the imaged data of the operator's hand before the period for accepting the gesture input by the operator And determining the right hand or left hand,
   Before the acceptance period, according to the determined hand pattern, selecting any operation target from a plurality of different operation targets related to the vehicle;
   A step of giving an operation instruction to the operation target based on a gesture input by the hand during the reception period.
10. A program for causing a computer to execute the gesture operation method according to claim 9.

Images