JP3941786B2 - Vehicle operation input device and method - Google Patents

Description

  The present invention relates to a vehicle operation input device and method for vehicle-mounted device operation by a driver.

Conventionally, as an operation input method for controlling various devices mounted on a vehicle, for example, an audio, an air conditioner, a navigation system, or the like, a contact type switch or a voice recognition device is conventionally used.
However, in the case of a contact type switch, the driver needs to confirm the switch to be operated and whether the operated switch is a switch for the intended function, and during the confirmation, pay attention to the road on which it is traveling. There is a risk of losing. In the case of using the voice recognition device, the rate at which the driver's voice is correctly recognized tends to decrease due to road noise and other noise inside and outside the vehicle.

Therefore, for example, in Japanese Patent Laid-Open No. 7-303289, a driver's hand movement and shape put out into the vehicle interior center console space are recognized by an infrared array sensor, and an operation input of an in-vehicle device is performed. A non-contact type device has been proposed.
JP 7-303289 A

In this non-contact type device, it is possible to reduce the risk of distraction of the road to confirm the position of the hand by widening the area where the driver puts his hand.
However, in order to detect a driver's hand from a widely set recognition area and recognize its movement and shape, there is a problem that a recognition device for that purpose becomes complicated and expensive.
On the other hand, if the recognition area is narrowed, the attention to the road may be distracted as described above, and the driver's hand that has been put out in the recognition area will be easily removed from the area, making it difficult to confirm the operation input.

In order to simplify the recognition device, it is not preferable to recognize the complicated shape of the hand such as counting the number of fingers, so it is better to identify the operation at the position where the hand is put out. Therefore, it is not easy to determine the operation input.
The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a vehicle operation input device and method in which attention to the road is not distracted and the operation input can be easily determined while using the shape and movement of the hand. And

Thus the present invention captures the steering wheel as the imaging field of view with a camera grip position on the steering wheel on the basis of imaging data, shape or by detecting the state of the driver's hand, such as movement of the hand steering wheel The operation mode for the in-vehicle device is selected according to the grip position above .

Since the driver's hand state is detected by the grip position on the steering wheel, it is possible to position his / her hand at the intended position with high accuracy without being distracted from the road, unlike in free space. Since it can be easily detected, the operation input can be confirmed easily.

Next, embodiments of the invention will be described in detail.
FIG. 1 is a block diagram illustrating a configuration of an operation input device 10 according to the embodiment.
An operation input setting unit 12 is connected to the infrared camera 11, and a head-up display (HUD) 13 and an operation input execution unit 14 are connected to the operation input setting unit.
FIG. 2 shows an arrangement layout of the infrared camera 11. The infrared camera 11 is a space in front of the driver's seat 2 in the vehicle 1 and captures an area (hatched area) including the steering wheel S from the front of the driver D seated in the driver's seat 2 to the instrument panel 3. Installed on the ceiling 4 of the passenger compartment.

When the meter surface of the instrument panel 3 is located substantially below the front end of the ceiling 4 and the B pillar 6 constituting the rear edge of the opening of the passenger compartment door 5 is located at the seat back 2a of the driver's seat 2, Since the imaging field of view corresponds to a range ahead of about a half position between the front end of the ceiling 4 and the B pillar 6, the infrared camera 11 is ahead of a half position between the front end of the ceiling and the B pillar. Therefore, it is preferable that the vehicle 1 be disposed closer to the driver's seat 2 than the central axis C extending in the front-rear direction.
On the head-up display 13, guide information at each stage of operation selection and parameter change described later is displayed in the front view of the driver D on the windshield 7.

Returning to FIG. 1, the operation input setting unit 12 receives an imaging signal from the infrared camera 11, and detects the position, shape, movement, etc. of the driver's hand on the steering wheel from the image data. And an operation selection unit 22 for selecting an operation mode (including in-vehicle devices) based on the detection result of the detection unit 21, and an operation parameter is changed based on the detection results of the detection unit 21 and the operation selection unit 22. And a parameter changing unit 23 for the purpose. The outputs of the operation selection unit 22 and the parameter change unit 23 are input to the operation input execution unit 14.
The operation input execution unit 14 executes control of in-vehicle devices (not shown) based on the outputs of the operation selection unit 22 and the parameter change unit 23.

The operation input setting unit 12 is further connected with a left / right changeover switch 15 that can be operated by the driver so as to switch the display of selection guide information and change guide information (to be described later) and judgment of operation input according to the dominant hand of the driver. It has become.
The operation input setting unit 12 receives a steering signal from the steering device, and is set so as not to function while the steering wheel is rotationally steered and when a predetermined steering force is applied. Yes.

Next, an operation input procedure in the operation input device 10 will be described. Here, it is assumed that the left / right changeover switch 15 is set to the right hand, that is, the right handed side.
The operation input device 10 is activated when an ignition switch of the vehicle is turned on or a switch (not shown) provided for exclusive use is turned on.
The operation input is performed by photographing the state of the hand along the steering wheel with the infrared camera 11, discriminating the driver's intention based on the captured image data, and presenting a corresponding selection screen correspondingly.

FIG. 3 shows an example of the state of a hand photographed by the infrared camera 11, and FIG. 3A shows a state in which the left and right hands (left hand TL, right hand TR) hold each lateral part of the steering wheel S in a “goo” shape. , (B) is a state where the left hand TL is gripping the lateral portion and the right hand TR is gripping the upper portion of the steering wheel S, and (c) is a state where the left hand TL is gripping the lateral portion of the steering wheel S and the right hand TR is Each of the opened states is shown.
Note that the left hand TL and the right hand TR are hereinafter referred to as “both hands” when indicating both, and are also simply referred to as “hands” when not distinguished from each other.

The detection unit 21 of the operation input setting unit 12 recognizes the imaging data from the infrared camera 11 and detects the position and shape of the driver's hands TL and TR as shown in FIG. 3 and the change (movement) thereof. Then, the detection result is output to the operation selection unit 22 and the parameter change unit 23.
In the activated state, the detection unit 21 captures image data from the infrared camera 11 at a time interval sufficiently shorter than the hand movement, and monitors the state of the driver's hand.

4 and 5 are flowcharts mainly showing the flow of operation input control in the operation input setting unit 12.
In step 101, the operation selection unit 22 checks whether or not the driver's hands TL and TR are gripping the left and right lateral portions of the steering wheel S, as shown in FIG. To do. In this flow, the hand state check and determination are all performed based on the detection result of the detection unit 21.
Step 101 is repeated until the left and right sides are held, and then in step 102, it is checked whether or not the driver has opened the right hand TR in a “par” shape as shown in FIG.

When one right hand TR is opened from the state where both hands TL and TR grip the side of the steering wheel S, the operation selection unit 22 determines that the operation input has started, and proceeds to step 104.
If the right hand TR is not open in the check in step 102, the process proceeds to step 103 to check whether the position of any of the hands TL, TR has changed.
When holding each side of the steering wheel S, the process returns to step 102, and when any hand moves to another position, the process returns to step 101.

In step 104, the operation selection unit 22 displays the selection guide information G on the head-up display 13, and enters operation mode selection.
FIG. 7 shows an example of selection guide information, in which three sets of operation target in-vehicle devices and hand states are associated as operation modes. (Ga) corresponds to the navigation operation (navigation) when the left hand grasps the side of the steering wheel and the right hand grasps the upper part, and (gb) indicates that the state where both hands grasp the side of the steering wheel operates the air conditioner ( (Gc) indicates that a state in which the left hand grasps the lateral part of the steering wheel and the right hand grasps the lower part corresponds to an audio operation (audio).
The selection guide information G is displayed on the head-up display 13 in parallel with these (ga) to (gc) being adjacent to each other.

Here, the in-vehicle device can be selected depending on where the steering wheel S is gripped by the right hand TR opened in a “par” shape.
Therefore, the driver refers to the selection guide information G and moves the right hand TR to the upper part (m1) or the lower part (m3) of the steering wheel S as shown in FIGS. With the part (m2) still held, it is gripped again to select navigation operation, audio operation, or air conditioner operation.
In step 105, the background color of the selection guide information ga, gb or gc related to the operation mode selected by the driver among the selection guide information G displayed on the head-up display 13 is changed. As a result, the driver can check whether or not the operation mode he / she has selected at the grip position of the steering wheel S is correctly selected.
In addition, when the selected operation mode is different from his intention, the driver can change the grip position on the steering wheel S by sliding the steering wheel S while holding the steering wheel S.

After confirming the background color change of the selection guide information G, as shown in FIG. 8 (c), the driver confirms that the right hand TR holding the steering wheel S is in a “par” shape again at that position. Open to.
In step 106, the operation selection unit 22 checks whether or not this confirmation operation has been performed, that is, whether or not the right hand TR has been opened at the operation mode selection position.
Step 106 is repeated until the right hand TR is opened, and then the routine proceeds to step 107.

In step 107, when the operation mode is hierarchical, it is checked whether the selection and confirmation operation at the lowest layer has been completed.
If the operation mode is not the lowest layer, the process returns to step 104 and the above and the above is repeated.
For example, when there are further modes such as cooling, heating, and ventilation under the air conditioner operation, the process returns to step 104, and the operation selection unit 22 determines whether the operation mode and the hand state are as illustrated in FIG. The associated selection guide information G1 is displayed on the head-up display 13.
(Gd) in the selection guide information G1 corresponds to cooling when the left hand grips the side of the steering wheel and the right hand grips the top, and (ge) indicates that the state where both hands grip the side of the steering wheel is heating. (Gf) indicates that the state in which the left hand grasps the lateral portion of the steering wheel and the right hand grasps the lower portion corresponds to air blowing.

When the confirmation operation is completed for the lowermost operation mode in the check in step 107, the operation selection unit 22 outputs the selection result to the operation input execution unit 14 in step 108. As a result, the operation input execution unit 14 sets the air conditioner to perform cooling control of the selected operation mode, for example, the air conditioner device.
Similarly, if the audio operation is selected and confirmed, the operation input execution unit 14 targets the audio device, and if the navigation operation is selected and confirmed, the operation input execution unit 14 targets the navigation device.

Subsequently, in step 109, the parameter changing unit 23 checks whether or not a changeable parameter is set in the operation mode determined in step 106. If there is no parameter setting, the current flow ends and the process returns to step 101.
If there is a parameter setting, the process proceeds to step 110, where the parameter changing unit 23 displays the change guide information H in which the changeable parameter and the hand state are associated with each other on the head-up display 13.
The changeable parameter (value) includes, for example, a radio station and volume adjustment when audio operation is selected as the operation mode.

First, FIG. 9 shows an example of the change guide information H regarding the setting of the radio station. Here, (ha) corresponds to radio station A when the left hand grips the side of the steering wheel and the right hand grips the top, and (hb) indicates the state where both hands grip the side of the steering wheel. Corresponding to B, (hc) indicates that the state in which the left hand grasps the lateral portion of the steering wheel and the right hand grasps the lower portion corresponds to the radio station C.
Among the change guide information H, information corresponding to the currently set radio station when the audio operation is selected can be distinguished from other information by changing the background color.

When the driver wants to change the radio station, the change guide information is referred to, and the right hand TR opened in a “par” shape by the operation mode selection confirmation operation, for example, as shown in FIGS. Select any radio station by moving it to the top, side or bottom of the steering wheel and grasping again.
In step 111, the background color of the change guide information ha, hb or hc concerning the new specific radio station selected as the parameter value by the driver among the change guide information H displayed on the head-up display 13 is changed. . As a result, the driver can check whether or not the radio station that he / she has selected at the grip position of the steering wheel S is correctly selected.

The driver who has confirmed the change in the background color of the change guide information then opens the right hand TR holding the steering wheel S in a “par” shape at that position as shown in FIG.
In step 112, the parameter changing unit 23 checks whether or not this confirmation operation has been performed, that is, whether or not the right hand TR is opened at the selected position of the radio station.
After step 112 is repeated until the right hand TR is opened, the process proceeds to step 113.

In step 113, the parameter change unit 23 outputs the selection result to the operation input execution unit 14. As a result, the operation input execution unit 14 sets the audio device to have the selected parameter value, that is, to change the reception destination to the selected radio station.
In step 114, the parameter changing unit 23 checks whether all the changeable parameters have been set.
For example, if the volume adjustment remains for the audio, the process returns to step 110.

FIG. 10 shows an example of the change guide information H1 displayed in step 110 for volume adjustment. Here, the state in which the left hand holds the side of the steering wheel and the right hand that is also holding the steering wheel is shifted upward or downward along the steering wheel as shown by the arrow increases the volume (VOL). Or it shows that it corresponds to reduction.
When the driver wants to increase the volume with reference to the change guide information H1, the right hand TR is moved from the state in which the side of the steering wheel S is held with both hands TL and TR as shown in FIG. The right hand TR holding the steering wheel S is opened in a “par” shape at the position as shown in (b) as a finalizing operation after sliding and moving along the steering wheel S.
On the other hand, when it is desired to reduce the volume, the right hand TR is slid downward along the steering wheel S from the state of grasping the lateral portion of the steering wheel S with both hands TL and TR as shown in FIG. The right hand TR holding the steering wheel S as shown in (b) is opened in a “par” shape at that position as a finalizing operation after moving.

The confirmation operation is confirmed in step 112, and the result is output to the operation input execution unit 14 in step 113, so that the operation input execution unit 14 increases or decreases the volume of the audio device.
When all the parameters are set in the check in step 114, the current flow is terminated and the process returns to step 101, and the next operation mode can be selected.

The above has described the case where the left / right changeover switch 15 is set to the right-handed side. However, a left-handed driver can change the left / right changeover switch 15 to move the left hand.
In this case, the right and left of the display and check contents at each step are switched as follows.
First, in step 102, it is checked whether the left hand TL is opened in a “par” shape as shown in FIG. 6C instead of FIG. 6B.

Next, as shown in FIG. 13 instead of FIG. 7, the selection guide information G displayed in step 104 corresponds to the navigation operation when the right hand grips the side of the steering wheel and the left hand grips the top. (Ga '), the state where both hands grasped the side of the steering wheel was made to correspond to the air conditioner operation (gb'), the state where the right hand grasped the side of the steering wheel and the left hand grasped the lower part for audio operation It becomes the selection guide information G2 consisting of the corresponding (gc ′).
Similarly, the selection guide information G1 corresponds to cooling when the right hand grips the side of the steering wheel and the left hand grips the top (gd '), and when both hands grip the side of the steering wheel. The selected guide information G3 includes (gf ′) in which the right hand grasps the lateral portion of the steering wheel and the left hand grasps the lower portion (gf ′).

Although not particularly illustrated, similarly, the change guide information shown in FIG. 9 displayed in step 110 and the change guide information shown in FIG. It is switched to the one replaced with the left hand.
In addition, the confirmation operation in step 112 also checks whether the left hand has been opened, as shown in FIG.

In the present embodiment, the infrared camera 11 corresponds to the camera in the invention, and the head-up display 13 corresponds to the display unit.
Further, the detection unit 21 constitutes a hand state detection means, and the selection guide information G, G1, G2, etc. and the change guide information H, H1, etc., respectively constitute guide information.
Steps 101 to 114 in the flowcharts of FIGS. 4 and 5 constitute input setting means, in particular, steps 104 to 108 constitute operation mode selection means, and steps 109 to 114 constitute parameter change means.

  The present embodiment is configured as described above, and the infrared camera 11 images the steering wheel as an imaging field of view, detects the state of the driver's hand TL, TR on the steering wheel S based on the imaging data, Since the operation mode selection for the in-vehicle device and the operation input for changing the parameter are set based on the hand state, the position of the detection target hand is limited on the steering wheel S. Unlike in the case of, you do not need to worry about the correct position of the hand, you can position your hand at the intended position with high accuracy without distracting from the road, operation is easy because it is easy to detect It becomes easy to confirm the input.

  Regarding the operation mode selection, it is possible to easily select a plurality of juxtaposed operation modes by changing the grip position on the steering wheel S of the hand. In addition to the grip position, the parameter change can be performed on the steering wheel S. Since the direction of hand movement is also used, continuous and stepless parameter adjustment such as audio volume is easy.

The operation mode selection is activated based on the detected hand state, for example, based on a specific shape change that opens in a “par” shape from the gripped state, so that it is easy to activate and an operation that is not intended by mistake by normal steering operation. It is prevented that the mode is selected.
On the other hand, since the selection of the operation mode is also performed based on the hand shape after selection, it is easily determined by a series of hand changes on the steering wheel S.
The same applies to parameter changes.

Further, when selecting the operation mode or changing the parameters, the selection guide information G, G1,... Corresponding to the hand grip position, shape or movement on the steering wheel S is used as the change guide information H, H1,. Since the information is displayed on the up-display 13, the driver can accurately select and change with reference to the guide information.
The guide information changes the position, shape, or movement of the dominant hand, and the other hand indicates a state of holding a predetermined position on the steering wheel S. The hand firmly grips the steering wheel S to ensure stable vehicle travel.
By switching the left / right changeover switch 15, the guide information can be displayed in which the grip position or the like is changed according to the dominant hand, so that the ease of operation is improved in accordance with the physical characteristics of the driver.

  Further, when the steering wheel S is rotationally steered or when a steering force exceeding a predetermined value is applied to the steering wheel S, the function of selecting the operation mode or changing the parameter does not operate. There is no need to select an operation mode or change parameters.

  In the embodiment, the change guide information is displayed on the head-up display 13 after the selection and determination of the operation mode, and the process shifts to the parameter change of the selected operation mode. Without limitation, when a predetermined operation mode is already selected, control of parameter change after step 109 is activated by a predetermined position and shape of the hand on the steering wheel S as in steps 101 and 102. You may make it do.

Further, the operation mode selection and parameter change targets are not limited to those shown in the embodiment, and the operation modes and parameters related to various in-vehicle devices other than those illustrated can be targeted.
As an example of parameters, three radio stations are shown in the change guide information in FIG. 9, but this is not limited to three stations, and positions on the steering wheel S may be set corresponding to a larger number of stations. .

  In the embodiment, the “goo” -shaped grip is used for selecting the operation mode and the like as the hand shape, and the “par” -shaped opening is used for the confirming operation. The combination of the shapes may be arbitrarily set.

It is a block diagram which shows the structure of embodiment of this invention. It is explanatory drawing which shows the arrangement layout of an infrared camera. It is a figure which shows the example of the state of the hand in an imaging screen. It is a flowchart which shows the flow of operation input control. It is a flowchart which shows the flow of operation input control. It is a figure which shows the state of the hand for operation input start. It is a figure which shows the example of selection guide information. It is explanatory drawing which shows the operation point of operation mode selection and parameter change. It is a figure which shows the example of change guide information. It is a figure which shows the example of change guide information. It is explanatory drawing which shows the other operation point of parameter change. It is explanatory drawing which shows the other operation point of parameter change. It is a figure which shows the example of selection guide information for left-handed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Driver's seat 2a Seat back 3 Instrument panel 4 Ceiling 5 Car compartment door 6 B pillar 7 Front glass 10 Operation input device 11 Infrared camera 12 Operation input setting part 13 Head up display 14 Operation input execution part 15 Left-right switch 21 Detection unit 22 Operation selection unit 23 Parameter change unit C Central shaft D Driver G, G1, G2, G3 Selection guide information H, H1 Change guide information S Steering wheel TL Left hand TR Right hand

Claims (17)

A camera that images the state of the driver's hand on the steering wheel;
Hand state detecting means for detecting the state of the hand on the steering wheel based on data captured by the camera;
Based on the state of the hand detected by該手state detecting means, have a input setting means for setting an operation input to the vehicle equipment,
The vehicle operation input device , wherein the input setting means includes an operation mode selection means for selecting an operation mode according to a grip position on the steering wheel of the hand detected by the hand state detection means . The vehicle operation input device according to claim 1, wherein the input setting unit includes a parameter changing unit that changes a parameter in a predetermined operation mode. 3. The vehicle operation input according to claim 2, wherein the parameter changing means changes the parameter according to a grip position or a movement direction on the steering wheel of the hand detected by the hand state detecting means. apparatus. 4. The vehicle operation input according to claim 1, wherein the operation mode selection unit or the parameter change unit is activated based on a hand state detected by the hand state detection unit. 5. apparatus. 4. The vehicle operation input according to claim 1, wherein the operation mode selection unit or the parameter change unit is activated based on a hand shape detected by the hand state detection unit. 5. apparatus. The said operation mode selection means determines the said selected operation mode based on the hand state detected by the said hand state detection means after that, The any one of Claim 1 to 5 characterized by the above-mentioned. Vehicle operation input device. The said operation mode selection means determines the said selected operation mode based on the shape of the hand detected by the said hand state detection means after that, The any one of Claim 1 to 5 characterized by the above-mentioned. Vehicle operation input device. 8. The vehicle according to claim 2, wherein the parameter changing unit determines the changed parameter based on a hand state detected by the hand state detecting unit thereafter. Operation input device. 8. The vehicle according to claim 2, wherein the parameter changing unit determines the changed parameter based on a hand shape detected by the hand state detecting unit thereafter. Operation input device. With a display,
The vehicle according to any one of claims 1 to 9, wherein the input setting means displays guide information associated with a hand grip position, shape, or movement on a steering wheel on the display unit. Operation input device. The vehicle operation according to claim 10, wherein the guide information changes a position, shape, or movement of a dominant hand, and indicates that the other hand holds a predetermined position on the steering wheel. Input device. The vehicle operation input device according to claim 11, wherein the guide information can be switched between a right hand and a left hand according to a dominant hand. 13. The input setting unit according to claim 1, wherein the input setting unit does not operate while the steering wheel is rotationally steered and when a steering force exceeding a predetermined value is applied to the steering wheel. The vehicle operation input device described. 14. The camera according to claim 1, wherein the camera is installed on the driver's seat side from the center axis of the vehicle and in front of a half position between the front end of the ceiling of the passenger compartment and the B pillar. The vehicle operation input device according to claim 1. The vehicle operation input device according to claim 1, wherein the camera is an infrared camera. Use the camera to image the steering wheel as the field of view,
Based on the image data, the driver's hand state on the steering wheel is detected,
An operation input method for a vehicle , wherein an operation mode for an in-vehicle device is selected according to a detected grip position on a steering wheel of a hand. 17. The vehicle according to claim 16, wherein guide information associated with a hand grip position, shape, or movement on a steering wheel is presented to a driver to guide operation mode selection or parameter change operation. Operation input method.

Images