JP2020149228A - Control device and control method - Google Patents

Abstract

To improve usability.SOLUTION: A control device is provided, comprising a recognition unit and a setting unit. The recognition unit recognizes first gesture manipulation pointing at a target device to be controlled. The setting unit sets a reference point based on the first gesture manipulation recognized by the recognition unit. The recognition unit recognizes second gesture manipulation based on the reference point set by the setting unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to a control device and a control method.

Conventionally, there is an in-vehicle device that accepts a gesture operation for an in-vehicle device in a vehicle (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-121386

However, in the prior art, for example, it is not easy to finely adjust the set temperature and volume of the air conditioner, audio, etc. by gesture operation, and there is room for improvement in improving usability.

The present invention has been made in view of the above, and an object of the present invention is to provide a control device and a control method capable of improving usability.

In order to solve the above-mentioned problems and achieve the object, the control device according to the present invention includes a recognition unit and a setting unit. The recognition unit recognizes the first gesture operation pointing to the target device to be controlled. The setting unit sets a reference point based on the first gesture operation recognized by the recognition unit. The recognition unit recognizes the second gesture operation based on the reference point set by the setting unit.

According to the present invention, usability can be improved.

FIG. 1A is a diagram showing a mounting example of a control device. FIG. 1B is a diagram showing an outline of a control method. FIG. 2 is a block diagram of the control device. FIG. 3 is a diagram showing a specific example of the invalid region. FIG. 4A is a diagram (No. 1) showing a specific example of the second gesture operation. FIG. 4B is a diagram (No. 2) showing a specific example of the second gesture operation. FIG. 4C is a diagram (No. 3) showing a specific example of the second gesture operation. FIG. 4D is a diagram (No. 4) showing a specific example of the second gesture operation. FIG. 5 is a flowchart executed by the control device.

Hereinafter, embodiments of the control device and control method disclosed in the present application will be described in detail with reference to the accompanying drawings. The present invention is not limited to the embodiments shown below.

First, the outline of the control device and the control method according to the embodiment will be described with reference to FIGS. 1A and 1B. FIG. 1A is a diagram showing a mounting example of the control device according to the embodiment. FIG. 1B is a diagram showing an outline of a control method. Further, in the present embodiment, the case where the target device is the in-vehicle device 50 will be described as an example.

As shown in FIG. 1A, the control device 1 according to the embodiment is mounted on the vehicle C and functions as a user interface of various in-vehicle devices 50. Specifically, the control device 1 recognizes the occupant's gesture operation based on the measurement result of the 3D sensor 10 that detects the three-dimensional information in the vehicle, and outputs a control command corresponding to the recognized gesture operation to the in-vehicle device 50. To do.

For example, the in-vehicle device 50 controls the opening degree of the power window (hereinafter, referred to as a window) of the vehicle C based on the control command transmitted from the control device 1.

Here, for example, it is difficult for a user seated in the passenger seat of the vehicle C to open and close the window of the rear seat. Further, if the user opens and closes the window by, for example, a gesture operation of moving the hand up and down, the action of raising the hand and the action of lowering the hand may be recognized as separate gestures of the window. It becomes difficult to finely adjust the opening.

Therefore, in the control method according to the embodiment, it is decided to set a reference point and recognize the gesture operation based on the set reference point. That is, in the control method according to the embodiment, it is possible to adjust the opening degree of the window according to the amount of movement of the hand from the reference point.

Specifically, as shown in FIG. 1B, first, in the control method according to the embodiment, the first gesture operation is recognized (step S1). Here, the first gesture operation is an operation of pointing to a target device to be controlled.

For example, in the control method according to the embodiment, when the window frame W of the window which is the target device is on the extension line pointed to by the first gesture operation, it is recognized as the first gesture operation. Further, in the control method according to the embodiment, the device that the user desires to operate can be detected by recognizing the first gesture operation.

Subsequently, in the control method according to the embodiment, the reference point P is set based on the first gesture operation (step S2). For example, in the control method according to the embodiment, the position of the finger in the first gesture operation can be set as the reference point P.

After that, in the control method according to the embodiment, the second gesture operation following the first gesture operation is recognized based on the set reference point P (step S3). Specifically, in the control method according to the embodiment, in the recognition process of the second gesture operation, the movement of the hand is recognized and the movement vector V from the reference point P is calculated.

Then, in the control method according to the embodiment, the movement vector V is converted into a control amount, and a control command corresponding to the control amount is output to the in-vehicle device 50. In this case, if the movement vector V is upward, the window can be closed, and if the movement vector V is downward, the window can be opened.

Further, the user can increase the movement vector V and increase the control amount by moving the hand greatly, and can decrease the movement vector V and decrease the control amount by moving the hand small. it can.

As described above, in the control method according to the embodiment, the reference point P is set based on the first gesture operation, and the second gesture operation following the first gesture operation is recognized with the reference point P as a reference.

As a result, according to the control method according to the embodiment, the user can arbitrarily adjust the control amount, so that usability can be improved.

Next, a configuration example of the control device 1 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram of the control device 1. Note that FIG. 2 also shows the 3D sensor 10 and the in-vehicle device 50.

The 3D sensor 10 is a sensor that detects three-dimensional information in the vehicle. The 3D sensor 10 detects three-dimensional information in the vehicle by using, for example, an active stereo method or a ToF (Time of Flight) method. The 3D sensor 10 is not limited to the above example, and may have other configurations as long as it can calculate three-dimensional information in the vehicle.

Further, in addition to the 3D sensor 10, a camera that images the inside of the vehicle may be provided. That is, the control device 1 may recognize the gesture operation by combining the image data and the three-dimensional information.

The in-vehicle device 50 is a control device that controls the opening degree of the window in response to a control command input from the control device 1.

As shown in FIG. 2, the control device 1 according to the embodiment includes a control unit 2 and a storage unit 3. The control unit 2 includes an acquisition unit 21, a recognition unit 22, a setting unit 23, and an output unit 24. The storage unit 3 stores the coordinate information 31, the reference point information 32, and the command information 33.

The control device 1 includes, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a data flash, an input / output port, and various circuits.

The CPU of the computer functions as the acquisition unit 21, the recognition unit 22, the setting unit 23, and the output unit 24 of the control unit 2, for example, by reading and executing the program stored in the ROM.

Further, at least one or all of the acquisition unit 21, the recognition unit 22, the setting unit 23, and the output unit 24 of the control unit 2 are used by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). It can also be configured.

Further, the storage unit 3 corresponds to, for example, a RAM or a data flash. The RAM or data flash can store coordinate information 31, reference point information 32, command information 33, information on various programs, and the like. The control device 1 may acquire the above-mentioned program and various information via another computer or a portable recording medium connected by a wired or wireless network.

The coordinate information 31 is information regarding the three-dimensional coordinates of the window frame W in the vehicle interior space, the three-dimensional coordinates regarding the installation position of the 3D sensor 10, and the mounting angle of the 3D sensor 10. The reference point information 32 is information about the reference point set based on the first gesture operation. The command information 33 is information related to the control command associated with each second gesture operation.

The control unit 2 recognizes the first gesture operation, sets the reference point P, and recognizes the second gesture operation based on the reference point.

The acquisition unit 21 acquires the three-dimensional information input from the 3D sensor 10, and passes the acquired three-dimensional information to the recognition unit 22 each time it is acquired.

The recognition unit 22 recognizes the first gesture operation pointing to the device to be controlled based on the three-dimensional information. Specifically, first, the recognition unit 22 recognizes a specific part (arm, hand, etc.) of the user to be operated by the first gesture operation.

Subsequently, the recognition unit 22 calculates the direction pointed to by the specific portion, and determines whether or not the window frame W exists in the direction pointed by the specific portion. When the window frame W exists in the direction indicated by the specific direction, the recognition unit 22 recognizes it as the first gesture operation.

At this time, the recognition unit 22 can also recognize only the first gesture operation that has been continuously recognized for a predetermined time (for example, 3 seconds) as the first gesture operation. In other words, the recognition unit 22 recognizes the operation in which the user points the specific portion toward the window frame W and stands still for a predetermined time as the first gesture operation.

As a result, it is possible to suppress erroneous recognition of recognizing an unintended operation of the user as a first gesture operation. If the recognition unit 22 does not satisfy the above recognition conditions, the recognition unit 22 does not recognize it as the first gesture.

Further, the recognition unit 22 can recognize the second gesture operation based on the reference point set by the setting unit 23, and this point will be described later with reference to FIGS. 4A to 4D.

The setting unit 23 sets the reference point P based on the first gesture operation recognized by the recognition unit 22. Specifically, the setting unit 23 sets the position of the specific portion in the first gesture operation as the reference point P. When the reference point is set, the setting unit 23 stores the information about the reference point as the reference point information 32 in the storage unit 3. The recognition unit 22 can recognize the second gesture operation based on the reference point information 32.

Further, for example, when the recognition unit 22 recognizes the operation in the vertical direction as the second gesture operation, the setting unit 23 can set an invalid area for the second gesture operation. FIG. 3 is a diagram showing a specific example of the invalid region.

As shown in FIG. 3, the setting unit 23 sets the reference line Pl having the same height as the reference point P. An invalid area la is provided in the width direction (vertical direction) of the reference line Pl with reference to the reference line Pl. This invalid area la is an area that does not recognize the second gesture operation.

As a result, it is possible to suppress erroneous recognition of the second gesture operation based on the shaking of the vehicle C. That is, the action of the user unintentionally moving the hand after the first gesture operation can be excluded from the recognition target as the second gesture operation. The width of the invalid area la in the vertical direction is set in advance based on an experiment or the like in consideration of vibration of the vehicle C or the like.

Although the case where the invalid region la is a band-shaped region is shown here, the invalid region la may be a region centered on the reference point P and may be circular or elliptical.

Returning to the description of FIG. 2, the output unit 24 will be described. The output unit 24 generates a control command corresponding to the second gesture operation recognized by the recognition unit 22 and outputs the control command to the in-vehicle device 50. 4A to 4D are diagrams showing a specific example of the second gesture operation.

As shown in FIG. 4A, the recognition unit 22 recognizes the gesture operation using five fingers as the second gesture operation. In this case, the recognition unit 22 fully opens the window when the five fingers move below the reference line Pl, and fully closes the window when the five fingers move above the reference line Pl. It can be recognized as a gesture operation.

That is, as shown in FIG. 1B, when the second gesture operation using one finger is recognized, a control command in which the control amount of the window is adjusted based on the movement vector V is output. On the other hand, when the second gesture operation using five fingers is recognized, either the fully open or fully closed control command is output.

That is, the recognition unit 22 can facilitate the operation by the user by adjusting the strength of the control command according to the number of fingers used for the second gesture operation. Although the number of fingers recognized by the recognition unit 22 as the second gesture operation is 1 or 5, it may be 2 to 4, and the fingers of both hands are used. It may be 6 or more.

Further, the recognition unit 22 may recognize the gesture operation in which the finger moves left and right along the reference line P1 as the second gesture operation. For example, when one finger moves left and right along the reference line P1, the recognition unit 22 can recognize this as a gesture operation indicating the height position of the window. For example, when one finger moves left and right along the reference line P1 while the window is open from the height position of the reference line P1, the window closes to the height position of the reference line P1 and the reference line is closed. When one finger moves left and right along the reference line P1 while the window is closed from the height position of P1, the window may be opened to the height position of the reference line P1. That is, although the same gesture operation is performed, different controls (window opening control or window closing control) are performed depending on the state at the time of operation (window opening / closing state). This allows the user to perform intuitive operations according to the current state without having to remember multiple gestures.

Further, the recognition unit 22 makes a gesture of fully closing the window when the window is fully opened or opened from the height position of the reference line P1 and five fingers move left and right along the reference line P1. Recognize as an operation, and conversely, if the window is fully closed or closed from the height position of the reference line P1 and five fingers move left and right along the reference line P1, the window is fully opened. It may be recognized as a gesture operation. In this case, the user does not need to change the gesture operation according to the control content, so that the troublesomeness can be reduced.

Further, as shown in FIG. 4B, the recognition unit 22 moves the tip of the index finger and the tip of the thumb up and down from the reference point P in a state where the tip of the index finger and the tip of the thumb are close to each other with the reference point P as a reference. As a second gesture, it is possible to recognize an action such as moving the finger to the reference point P or moving it closer to the reference point P. For example, in this case, the output unit 24 controls a control amount based on the sum of the distance between the tip of the thumb and the reference point P after movement and the distance between the tip of the index finger and the reference point P, for example, the control amount for opening and closing the window. To adjust. The control amount may be adjusted based on the distance between the tip of the thumb and the tip of the index finger.

Further, as shown in FIG. 4C, the recognition unit 22 can also recognize the operation of following the arcuate locus starting from the reference point P as the second gesture operation. In this case, the recognition unit 22 recognizes as a gesture to open the window when taking a clockwise locus Vr from the reference point P, and recognizes as a gesture to close the window when taking a counterclockwise locus Vl from the reference point P.

Further, in this case, the output unit 24 outputs a control command whose control amount is adjusted according to the length of the orbits of the loci Vr and Vl and the size of the circle to the in-vehicle device 50.

Further, as shown in FIG. 4D, the recognition unit 22 may recognize the operation of stopping at the reference point P for a predetermined time (for example, 3 seconds) as the second gesture operation after setting the reference point P by the first gesture operation. it can.

In this case, the output unit 24 outputs a control command associated with the reference point P. In the present embodiment, the control command associated with the reference point P is a control command in which the opening degree of the window is set to the height Ta.

In this case, for example, the recognition unit 22 calculates the position T and the height Ta of the window frame W pointed to by the user. Then, the output unit 24 outputs a control command for setting the opening degree of the window to the height Ta to the in-vehicle device 50.

That is, in this case, it is possible to adjust to the opening degree specified by the user regardless of the current opening degree of the window. As a result, the height can be adjusted to an arbitrary height specified by the user, so that usability can be improved.

Here, the case where the control command associated with the reference point P is a control command in which the opening degree of the window is the height Ta is shown, but the present invention is not limited to this, and the control command is arbitrarily set. You may decide.

Next, the processing procedure executed by the control device 1 according to the embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing a processing procedure executed by the control device 1. The processing procedure is repeatedly executed by the control unit 2 of the control device 1.

As shown in FIG. 5, when the control device 1 first starts acquiring the three-dimensional information (step S101), it determines whether or not the first gesture operation has started recognizing (step S102).

When the control device 1 starts recognizing the first gesture operation (step S102, Yes), it determines whether or not a predetermined time has elapsed after recognizing the first gesture operation (step S103).

When the control device 1 determines in the process of step S103 that a predetermined time has elapsed (steps S103, Yes), the control device 1 determines whether or not the first gesture operation is continuing (step S104). The process of step S104 is performed by determining whether or not a specific portion to be operated by the gesture operation, for example, the movement of the finger has stopped (or more than a predetermined time).

When the control device 1 determines in the process of step S104 that the first gesture operation is continuing (steps S104, Yes), the control device 1 sets the reference point P based on the first gesture operation (step S105).

Subsequently, the control device 1 recognizes the second gesture operation following the first gesture operation (step S106) based on the reference point P, and generates a control command corresponding to the second gesture operation (step S107).

Then, the control device 1 outputs a control command to the in-vehicle device 50 (step S108), and ends the process. Further, when the control device 1 has not started recognizing the first gesture operation in the determination process of step S102 (steps S102, No), or the first gesture operation is not continued in the determination process of step S104. In the case (step S104, No), the process proceeds to step S102.

Further, in the process of step S103, if the predetermined time has not elapsed (steps S103, No), the control device 1 continues the process of step S103.

As described above, the control device 1 according to the embodiment includes a recognition unit 22 and a setting unit 23. The recognition unit 22 recognizes the first gesture operation pointing to the target device to be controlled. The setting unit 23 sets the reference point P based on the first gesture operation recognized by the recognition unit 22.

The recognition unit 22 recognizes the second gesture operation based on the reference point P set by the setting unit 23. Therefore, according to the control device 1 according to the embodiment, usability can be improved.

By the way, in the above-described embodiment, the case where the control device 1 is mounted on the vehicle C and the target device is a window has been described, but the present invention is not limited to this. That is, the control device 1 may be mounted on a vehicle other than the vehicle, or may control a target device other than the window.

Further, in the above-described embodiment, the case where the reference point P is set based on the first gesture operation and the second gesture operation following the first gesture operation is recognized based on the reference point P has been described, but the present invention is limited to this. It is not something that is done.

That is, the second gesture operation may be recognized based on the reference point P set in the past. In other words, the reference point P once set may be reused and used.

Further effects and variations can be easily derived by those skilled in the art. For this reason, the broader aspects of the invention are not limited to the particular details and representative embodiments described and described above. Therefore, various changes are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and their equivalents.

1 Control device 10 3D sensor 21 Acquisition unit 22 Recognition unit 23 Setting unit 24 Output unit P Reference point Pl Reference line la Invalid area

Claims (6)

A recognition unit that recognizes the first gesture operation that points to the target device to be controlled,
It is provided with a setting unit that sets a reference point based on the first gesture operation recognized by the recognition unit.
The recognition unit
A control device characterized by recognizing a second gesture operation based on a reference point set by the setting unit. The setting unit
The control device according to claim 1, wherein an invalid area for the second gesture operation is set in a predetermined range based on a reference point set based on the first gesture operation. The recognition unit
The control device according to claim 1 or 2, wherein the first gesture operation continued for a predetermined time is recognized. An output unit that outputs a control command corresponding to the second gesture operation recognized by the recognition unit to the target device is further provided.
The recognition unit
Recognize the number of fingers used for the second gesture operation
The output unit
The control device according to claim 1, 2 or 3, wherein the strength of the control command is changed according to the number of fingers recognized by the recognition unit. The recognition unit
Recognizes the second gesture operation that stops at the reference point set by the setting unit,
The output unit
The control device according to claim 4, wherein a control command corresponding to the reference point is output. A recognition process that recognizes the first gesture operation that points to the target device to be controlled,
Including a setting step of setting a reference point based on the first gesture operation recognized by the recognition step.
The recognition step is
A control method comprising recognizing a second gesture operation based on a reference point set by the setting step.

Images