JP2004171476A - Hand pattern switching unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a hand pattern switching unit by which operability is enhanced by grouping switches for devices targeted for control and by gathering the switches in each group having a similar operation and adjustment pattern for association. <P>SOLUTION: The hand pattern switching unit recognizes the operation intention of a driver (switch operator) against switching systems by referring to the geometry of fingers recognized from the input picture of a camera 3; then selects one control object out of a plurality of the control objects grouped by recognizing the succeeding geometry and motion of the fingers by means of a switch control object setting means 14; and outputs specified switch operation information. The plurality of switches in each control object are gathered by the similar operation/adjustment configuration for association, resulting in easy learning. Thus, the operability is enhanced. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a hand pattern switch device in which the operations of a multi-function switch integrating various functions of a vehicle-mounted controlled device are grouped, and a plurality of switches having similar functions are aggregated to improve operability. .
[0002]
[Prior art]
Vehicles include devices such as turn signals, wipers, lights, and mirrors that are directly necessary for traveling, and devices such as audio and air-conditioning devices are installed as comfortable equipment to provide comfortable space for passengers. I have. Since each of these devices is provided with an operation switch for each function, the number of devices is required for each function, and the total number is several tens. In-vehicle devices tend to increase, and accordingly, the number of operation switches increases. In addition, since these switches are basically arranged in a position near the driver's seat of an instrument panel (hereinafter referred to as "instrument panel") for operation by an operator, it is difficult to install the switches with an increase in the number of switches. Become.
[0003]
2. Description of the Related Art In recent years, as a technique for operating the operation of a vehicle-mounted device such as an air-conditioning device or an audio device without touching an operation panel of the vehicle-mounted device, a camera is used to image a part of a driver's body (for example, a left hand). It has been proposed to recognize the image pattern and obtain operation information for the on-vehicle device (for example, see Patent Document 1). It has also been proposed to detect a gesture indicated as the shape or movement of a driver's finger to obtain operation information for a vehicle-mounted device (for example, see Patent Document 2).
[0004]
This type of technology is realized by, for example, a pattern recognition process for recognizing the shape of a finger from an image of a finger or a motion detection process for detecting a motion by tracking a change in the position of the recognized palm. , A hand pattern switch or the like. Therefore, it is conceivable to integrate the switches of each function of each device into a multi-function switch by applying the hand pattern switch.
[0005]
[Patent Document 1]
JP-A-11-134090 [Patent Document 2]
JP 2001-216069 A
[Problems to be solved by the invention]
However, when the switches for each function of the above devices are replaced with a hand switch and integrated into a multi-function switch, various functions can be provided, but the operation becomes extremely complicated with an increase in the number of functions, and the operability is rather deteriorated. Problem arises.
[0007]
The present invention has been made in view of the above points, and groups switches of controlled devices such as traveling devices and comfortable equipment, and groups of switches of similar operation adjustment form are grouped to associate and operate. It is an object of the present invention to provide a hand pattern switching device that improves the performance.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, a hand pattern switching device according to the present invention captures a finger placed in a predetermined imaging region, recognizes a finger shape from an input image thereof, and detects movement of a palm in the imaging region. A switch control target setting for detecting and obtaining predetermined switch operation information, detecting a recognized finger shape or a preset hand movement pattern and selecting one from a plurality of switch control targets. Means, wherein the plurality of switches are similar in operation adjustment form.
[0009]
That is, the hand pattern switch device according to the present invention recognizes that the operator (switch operator) intends to operate the switch device based on the shape of the finger recognized from the input image, and then determines the shape of the subsequent finger or the like. One control target is selected from a plurality of control targets grouped by recognizing the movement, and predetermined switch operation information is output. The multiple switches in each controlled object are integrated with functions that have similar operation and adjustment forms by integrating functions, so that operators can easily associate them with each other and improve operability as multi-function switches. ing.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a hand pattern switch device according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows an example of a layout in which switches of devices mounted on a vehicle are systematically grouped (classified). As shown in FIG. 1, the system 1 is classified into “traveling devices” necessary for traveling as a control target in the system 1 and “comfort equipment” for comforting the occupant's environment. , Lights and door mirrors. Similarly, the comfort equipment is classified into audio equipment and air conditioning equipment (air conditioner) in the system 2, and the audio equipment is further classified into radio, CD, tape, and the like in the system 3.
[0011]
Then, the switches of the control target devices classified in the system 2 or the system 3 are further classified in the system 4. The number of switches of each device is 2 to 5 so that the operator can easily store the switches. Furthermore, the switch of each device integrates functions, that is, collects those having similar operation and adjustment forms so that the operator can easily associate and remember them and improve the operability as a multifunctional switch. I have.
[0012]
For example, "Radio" volume, channel, band switching, frequency adjustment, "CD" fast forward / rewind, track forward / reverse, repeat, eject, and "tape" reverse, fast forward / rewind, The operation of adjusting and adjusting the temperature of the truck, the air flow, the wind direction, and the switching between the inside and outside air are similar (see FIG. 6), and the similar ones are assembled. The same applies to the switches of “direction indicator”, “wiper”, “light”, and “door mirror” in the traveling device. As described above, since they are grouped and laid out in a large classification such as switches for lights, audios, and air conditioners, they can be operated relatively without hesitation.
[0013]
Next, an operation of selecting one of a plurality of switch control targets in the system 4 by applying the hand pattern switch device to the switches of the group of “comfort equipment” control target devices will be described.
FIG. 2 is a diagram showing a schematic configuration of a main part of the apparatus of the embodiment, showing a state of a driver's seat in a vehicle and functions of the apparatus realized by a microcomputer or the like. The driver's seat is provided with a steering wheel (handle) 1 and a combination switch (not shown) that are steered by an operator (driver), and the console panel is provided with an operation unit 2 such as audio equipment and air conditioning equipment. Have been. On the ceiling above the driver's seat, a video camera 3 is provided as an image pickup device for picking up an operator's finger extending his arm toward the image pickup area, with the side of the steering wheel 1 as an image pickup area. I have. The camera 3 is a small camera such as a CCD camera.
[0014]
Incidentally, the imaging area A of the camera 3 is set at a position on the side of the steering wheel 1 at least 50 mm away from the outer peripheral portion of the steering wheel 1 as shown in FIG. . The imaging area A is set as a substantially rectangular area having a size of about 600 mm in the fingertip direction of the palm of the driver who extends the arm to the side of the steering wheel 1 and about 350 mm in the width direction of the palm. ing.
[0015]
The imaging area A set in this way is an area where the operator does not image the fingers of the operator who grips the steering wheel 1 or operates a combination switch (not shown) provided on the column shaft. . In particular, this is a position where the driver can extend his / her arm without losing the driving posture while putting his / her arm on an armrest (armrest) 5 provided on the side of the driver's seat. It is a position where fingers do not touch.
[0016]
In the case where a gear shift lever (not shown) is arranged in the imaging area A set in this way, whether the finger extended to the imaging area A is operating the gear shift lever. Alternatively, it is preferable to provide a confirmation function for determining whether or not a switch operation is being performed. As this confirmation function, for example, the grip of the gear shift lever by the operator may be detected via a pressure-sensitive sensor provided on the gear shift lever. By taking such measures, it is possible to easily determine whether the finger extended to the imaging area A is operating the gear shift lever or the state of operating the hand pattern switch device. Thus, it is possible to reliably prevent the driving operation from being erroneously detected as the switch operation information.
[0017]
Now, the hand pattern switch device basically recognizes the shape and movement of the operator's finger imaged by the camera 3 from the input image and obtains predetermined switch operation information based on the recognition result. And plays the role of giving switch operation information to the above-mentioned audio equipment, air conditioning equipment and the like in place of the operation unit 2.
[0018]
Specifically, as shown in FIG. 2, the hand pattern switch device binarizes the input image captured by the camera 3 to remove its background image component, and extracts only a finger image. A center-of-gravity detecting unit 12 for obtaining the center of gravity of the palm from the extracted finger image; And
[0019]
Further, the hand pattern switch device is a switch control object that recognizes the switch operation indicated by the shape and movement of the operator's finger according to the recognition result by the shape recognition unit 13 and the position of the center of gravity of the palm detected by the center of gravity detection unit 12. An operation instruction recognition unit 14 is provided as a means. The operation instruction recognizing unit 14 roughly determines (identifies) the finger shape based on the information recognized by the shape recognizing unit 13 and tracks the movement of the center of gravity of the palm. The system includes a displacement detector 17 for detecting displacement of a finger and a timer 18 for monitoring the movement of the palm as time elapses. The operation instruction recognizing unit 14 obtains predetermined switch operation information specified by the shape and movement of the operator's finger based on the results of these determinations and monitoring. It is configured to output to devices and the like.
[0020]
The operation instruction recognizing unit 14 is provided with a guide unit 19 that provides a predetermined guidance (announcement) to the operator according to the above-described determination result and the like. This guidance includes, for example, an audio device or an air conditioner (operation target device), a voice message for specifying the volume / channel setting, the air volume / temperature, etc. (operation target function), and a “pip” or “ The driver is notified via the speaker 20 as a confirmation sound such as "P".
[0021]
In the above-described hand pattern switch device, when the position of the finger is stopped for a predetermined time in a predetermined finger shape for a predetermined time, this state is recognized as an operation intention of the switch device by an operator (switch operator) ( (Operation start recognition means), and thereafter, a process of detecting a switch operation based on the shape and movement of the finger is executed.
[0022]
Next, the switch operation will be described. In this hand pattern switch device, for example, a finger pattern as shown in FIGS. 5A to 5D is used as a finger shape for designating a switch operation. Specifically, a finger pattern [hand shape 1] in the form of a grip fist as shown in FIG. 5 (a), a finger pattern [hand shape 2] in the form of a pointing finger with an index finger as shown in FIG. 5 (b), and FIG. A finger-shaped pattern with a thumb raised as shown in FIG. 5 (c) [hand shape 3] and a finger-shaped pattern with a thumb and index finger raised as shown in FIG. 5 (d) [hand shape 4] are used. Can be
[0023]
By the way, the finger pattern in the shape of the pointing finger and the finger pattern in which the thumb and the index finger stand up express the image of pressing the switch. The finger pattern in the form of a grip fist represents an image in which a lever-like switch is gripped and a shift operation is performed forward and backward, and the finger shape in which a thumb is raised represents an image representing a standby instruction.
[0024]
Now, when the driver is steering the vehicle by gripping the steering wheel 1 as shown in FIG. 4, the operator's finger is out of the imaging area A as shown in the operating state A1, and at that time, Since the input image is only an image component to be excluded as a background in the vehicle cabin, the hand pattern switch device does not operate.
On the other hand, when the operator's finger comes off the steering wheel 1 and enters the imaging area A as shown in the operation state A2, the recognition process for the finger is thereby started. In this case, for example, when the finger forms a fist-shaped [hand shape 1] and the palm does not move (stops) for a predetermined time (for example, 1 second or more), the operator changes the switch operation to this state. It is desirable to recognize the intention and instruct the start of switch operation information input.
[0025]
Thereafter, it is determined whether or not the shape of the finger has been changed to a finger-shaped pattern [hand shape 4] for pressing the switch. When a finger-shaped pattern is detected, the movement is detected assuming that a selection switch is repeatedly pressed (pushed) to select and designate a predetermined device as shown in an operation state A3. Specifically, the movement of the finger having the pointing finger back and forth (displacement) is detected from the change (movement) of the center of gravity of the palm, and the push operation is detected. In this case, every time one push operation is detected, for example, as shown in FIG. 6A, the control target device is cyclically set as “air conditioner”, “radio”, “tape”, “CD”, or the like. The control algorithm may be set so as to select and specify. In particular, at this time, it is also useful to output an operation confirmation sound such as "beep" every time one push operation is detected, or to notify the selected control target as a voice message. With such guidance, the operator can recognize the device selected and specified without visually confirming the operation state, so that the operator can concentrate on the driving operation. Then, after the desired control target device is selected, when the finger shape changes, this state is selected when the control target device selection is completed and the operator recognizes that the next switch operation is intended. good.
[0026]
Specifically, it is determined whether or not the pattern is changed to the pointing hand pattern [hand shape 2] as shown in the operation state A5. When the finger-shaped pattern [hand shape 2] moves back and forth, the movement is detected by repeatedly pressing the selection switch (push) to select and designate a predetermined control target. Specifically, in this case, the movement (displacement) of the finger having the pointing shape moving back and forth is detected from the change (movement) of the position of the center of gravity of the finger, and the push operation is detected. Each time one push operation is detected, for example, when “air conditioner” is selected as the control target device as shown in FIG. 6A, the control target is set to “temperature” and “air volume”. , "Wind direction", "inside / outside air", and the like. When "radio" is selected as the control target device, the control target is cyclically selected and designated as "volume", "channel", "FM / AM", "frequency adjustment" or the like.
[0027]
Similarly, when "tape" is selected as the control target device, the control target is cyclically selected and designated as "reverse", "fast forward", "cue", "take out", and the like, and If "CD" is selected as the control target device, the control target may be cyclically selected and designated as "fast forward", "search for", "repeat", "take out", and the like. .
[0028]
When the desired control target is selected as described above, for example, as shown in the operation state A6, the finger shape is changed to the thumb-up pattern [hand shape 3]. , To start inputting switch operation information to be executed next. Also in this instruction, when the finger shape is a thumb shape [hand shape 3] and the palm does not move (stops) for a predetermined time (for example, 1 second or more), the operator changes this state to the next state. It is sufficient to recognize that the switch operation is intended. In this case, it is preferable to emit a sound such as “Pee” different from the above-mentioned confirmation sound as the confirmation sound.
[0029]
Thereafter, as shown in the operation state A7, the finger shape is changed to the fist shape [hand shape 1], and the switch operation information is input by moving the finger back and forth. At this time, the movement of the finger in the form of a fist is detected from the displacement amount of the center of gravity and the like, and switch operation information corresponding to the movement is output. Specifically, in the input of the switch operation information, the information content to be detected is changed according to the control target selected and designated by the push operation described above.
[0030]
That is, depending on the control target, there are a case where the control amount (switch operation information) is to be changed in multiple steps (stepwise) with a large change amount, and a case where the control amount (switch operation information) is to be continuously and finely changed with a small change amount. Therefore, in accordance with the selected operation target, for example, a distance mode (operation state A8a) in which the movement amount (displacement amount) of the finger is used as the switch operation information, and a time for obtaining the switch operation information from the stop time of the finger displaced by a predetermined distance. A mode (operational state A8b) or a distance / time mode (operational state A8c) using these modes is selectively set.
[0031]
Incidentally, the distance mode is a mode in which switch operation information is output in multiple stages according to the amount of movement of the hand from the reference position, and is suitable for selecting and setting the radio broadcast channel selection and the air volume and direction in the air conditioning equipment. is there. The time mode is a mode for outputting switch operation information according to the stop time of the palm displaced from the reference position, and is suitable for, for example, finely adjusting the volume of an audio device, the temperature of an air conditioner, and the like. Further, in the distance / time mode, when the palm is slightly moved, switch operation information corresponding to the amount of movement is output, and when the palm has moved a predetermined distance or more, it is determined according to the stop time of the palm at the moved position. In this mode, switch operation information is output. This distance / time mode is suitable for a case where the control target is roughly adjusted and then finely adjusted.
[0032]
The above-described operation instruction recognition unit 14 is configured to control the hand in the imaging region A according to the input mode (distance mode / time mode / distance / time mode) that is selectively set according to the control target as described above. The movement is tracked, and the movement amount of the hand (the displacement amount and the displacement direction) and the stop time of the hand whose movement has stopped are measured. Then, switch operation information corresponding to these measurement results is output.
[0033]
Note that the correspondence between various switch control targets and the above-described modes and the content (detection distance and time) detected as a switch operation amount may be fixedly set in advance. However, the operation instruction recognizing unit 14 may be provided with a control screen 14a shown in FIG. 7, for example, which can be changed as appropriate according to the preference of the switch operator (operator). Specifically, for each of a plurality of switch control targets, the input mode in which the switch operation information is input is selectively designated, and the amount of displacement (movement distance) of the palm, which is the unit of detection, and the measurement time are specified. What is necessary is just to be able to set each variably.
[0034]
In the above embodiment, the case where the hand pattern device as a multifunctional switch is applied to the switches of the radio, the CD, the tape, and the air conditioner in the comfortable equipment as the control target equipment has been described. However, the present invention is not limited to this. It may be applied to a traveling device. Further, according to a user's request, the hand switch device can be modified to be applied to a device such as a wiper or a door mirror instead of a CD or a tape as a device to be controlled by the hand switch device.
[0035]
Furthermore, the invention is not limited to the embodiments described above. Although the embodiment has been described on the assumption that the vehicle is a right-hand drive vehicle, it is needless to say that the present invention can be similarly applied to a left-hand drive vehicle. is there.
Further, in a simple system, as shown in FIG. 6 (b), the control target is selected as “volume”, “channel”, “temperature”, “air volume” or the like in a lump in a lump with the device. It is also possible to configure so that Needless to say, the shape of the finger used for inputting information is not limited to the above-described example. In addition, the present invention can be variously modified and implemented without departing from the gist thereof.
[0036]
【The invention's effect】
As described above, according to the present invention, the hand pattern switch device recognizes that the operator (switch operator) intends to operate the switch device based on the shape of the finger recognized from the input image, and then performs the subsequent operation. One control target is selected from a plurality of control targets that are grouped by recognizing the shape and movement of the finger of the user, and predetermined switch operation information is output. A plurality of switches in each control target have a function. By integrating those having similar operation / adjustment forms and associating them, the operation can be performed with relatively little hesitation, and the operability as a multi-function switch can be improved easily and easily.
[0037]
Also, only when the operator wants to operate the switch device, it is only necessary to shape his / her finger and stop the hand for a predetermined time in the imaging area, so that the operator can easily and reliably operate without any extra operation burden. The switch device can be operated. In addition, the hand pattern switch device can easily and reliably determine the intention of the operator and reliably recognize only the intended shape and movement of the finger, and can switch information for various control targets. It can output properly.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an example of a layout in which control target devices to which a hand pattern switching device according to the present invention is applied are grouped and a plurality of switches having similar operation adjustment modes are collected.
FIG. 2 is a diagram showing a schematic configuration of a hand pattern switch device according to the present invention.
FIG. 3 is a diagram showing an imaging region of a finger in the hand pattern switch device shown in FIG. 2;
FIG. 4 is a diagram showing a schematic procedure of an operation of inputting an instruction for switch operation information in the hand pattern switch device.
FIG. 5 is a diagram showing an example of a finger shape for giving a switch operation instruction to the hand pattern switch device.
FIG. 6 is a diagram illustrating an example of selection of a control target according to a push operation by a finger.
FIG. 7 is a diagram illustrating an example of an input mode setting screen associated with a control target.
[Explanation of symbols]
Reference Signs List 1 steering wheel 2 operation unit such as audio equipment 3 camera (imaging device)
14. Operation instruction recognition unit (switch control object setting means)
19 Guide

Claims (1)

A hand pattern switch device for capturing a finger placed in a predetermined imaging region, recognizing a finger shape from the input image, and detecting hand movement in the imaging region to obtain predetermined switch operation information. hand,
A switch control target setting unit that detects a recognized finger shape or a preset hand movement pattern and selects one from a plurality of switch control targets, wherein the plurality of switches have an operation adjustment mode. A hand pattern switch device, wherein similar devices are assembled.

Images