JP2006285370A - Hand pattern switch device and hand pattern operation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize an input operation based on accurate hand pattern recognition in a simple configuration, for a hand pattern switch device and a hand pattern operation method. <P>SOLUTION: This hand pattern switch device is provided with an object detection sensor 5 for grasping an object existing in a detection region space from a detection base point, and for detecting the direction and distance of the respective parts of the object from the detection base point, a hand pattern extracting means 2 for extracting a hand pattern relating to the instructing operation of an operator based on the detected direction and distance, an instruction operation discriminating means 3 for discriminating the instructing operation of the operator based on the extracted hand pattern and an operation commanding means 4 for commanding a predetermined operation to a command object corresponding to the discriminated instructing operation. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a hand pattern switch device and a hand pattern operation method for performing operation input to an in-vehicle device using a gesture.

  2. Description of the Related Art Conventionally, a gesture input operation system that realizes an input operation using a gesture has been studied as a human interface technique for an input operation on an electronic device. This system enables intuitive input operations using gestures such as gestures and hand gestures as a new input operation means to replace conventional input operation devices that require physical operation such as various switches, keyboards, and touch panels. Many methods for providing a virtual operation device have been proposed.

  In recent years, as an application example of the above-described technology in the field of the automobile industry, for example, a hand shape or movement (hand pattern) indicated by an occupant is detected as an input operation, and mounted on a vehicle according to the input operation. Technologies for controlling various in-vehicle devices such as audio devices, car navigation devices, communication devices, and air conditioners have been developed. According to such a technique, various in-vehicle devices can be operated without taking the eyes from the front while the driver is driving.

  For example, Patent Document 1 shows a shape gesture represented by the shape of a hand and a direction in which the hand moves by using an image sensor and a camera that detect object reflected light as an image in an operation input device of an in-vehicle device. A direction gesture is detected, and an operation menu of the in-vehicle device is selected according to the type of shape gesture, and a change parameter change direction (increase direction, decrease direction, etc.) on the operation menu based on the direction gesture is given. And a control configuration for instructing a change amount). With such a control configuration, it is possible to provide an operating device that is intuitively easy for the user (operator such as a driver) to perform natural operations.

Further, Patent Document 2 is a non-contact type information input device for in-vehicle devices, and has a configuration in which a far-infrared sensor that captures a predetermined area including a driver's hand is provided on the key body head of an ignition key. Are listed. With such a configuration, it is possible to accurately extract the driver's hand pattern from the captured image using the position of the steering wheel as a reference position, and to prevent erroneous detection of the hand operation related to normal driving operation. ing.
JP 2001-216069 A JP-A-2005-47412

By the way, in conventional gesture operation input devices for general in-vehicle devices such as these, an imaging device such as a CCD camera or an infrared camera is used as a detection means for detecting a gesture of operation input. A method of detecting and recognizing an operator's hand pattern by performing various arithmetic processes on image data captured by the apparatus is employed.
In such image recognition, it is necessary to identify the hand portion and the background portion in the image data in order to recognize the hand shape and motion that are the hand pattern of the operator. However, in the conventional operation input devices illustrated in Patent Documents 1 and 2, since a planar image captured by an optical imaging device is used, depending on the illuminance conditions and the state of the background part, Pattern identification may be difficult.

  In particular, the imaging environment in the passenger compartment is easily affected by disturbance due to changes in external light and the like, and the illuminance condition and the state of the background portion cannot be generalized. Therefore, for example, when direct sunlight is only on a part of the hand performing the hand pattern operation, or when an object with a color similar to the color of the skin is reflected in the background part, the hand pattern It is difficult to improve the recognition accuracy.

  In order to solve such a problem, a method of three-dimensionally extracting a hand pattern based on a plurality of captured image data by providing a plurality of imaging devices has been proposed. That is, this is a method of detecting the spatial positional relationship of objects in the imaging range based on difference information between a plurality of image data and extracting only a hand pattern from them. However, in this case, it is necessary to grasp the shape and movement of the hand three-dimensionally, which not only complicates the arithmetic logic, but also significantly increases the processing load and the costs associated therewith.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a hand pattern switch device and a hand pattern operation method capable of realizing an input operation based on accurate hand pattern recognition with a simple configuration. .

  In order to achieve the above-mentioned goal, the hand pattern switching device of the present invention (Claim 1) captures an object existing in the detection area space from the detection base point, the direction of each part of the target object from the detection base point, and An object detection sensor for detecting a distance to each part; and a hand pattern extraction means for extracting a hand pattern related to an instruction operation by an operator based on the direction and distance detected by the object detection sensor; An instruction operation determination means for determining the instruction operation of the operator based on the hand pattern extracted by the hand pattern extraction means; and a command target corresponding to the instruction operation determined by the instruction operation determination means. An operation command means for commanding a predetermined operation is provided.

The direction of each part of the object detected by the object detection sensor is a direction defined by information excluding distance information among the three-dimensional position information of the object in the detection region space, and is a two-dimensional Location information.
The hand pattern extracting means preferably selects the direction of each part of the object constituting the hand pattern based on the distance and extracts each part as the hand pattern. 2).

That is, in this case, among the three-dimensional position information of the object in the detection area space, the two-dimensional position information is selected based on the distance information, and the result is extracted as a hand pattern.
Preferably, the detection area space is a part of a vehicle interior space of the vehicle, and the command target is a device mounted on the vehicle.

In addition, the object detection sensor captures an image of the detection area space, detects the shape of the object on the imaging surface as an image, and detects a distance to the object corresponding to each part of the image. Preferably, the hand pattern extraction means extracts the hand pattern based on the image and the distance detected by the object detection sensor.
The hand pattern extraction means extracts the hand pattern image from the image detected by the object detection sensor based on the distance detected by the object detection sensor and determines the instruction operation determination. Preferably, the means discriminates an instruction operation by the hand pattern based on the hand pattern image extracted by the hand pattern extraction means.

Further, it is preferable that the hand pattern extracting means extracts only each part of the object existing within a predetermined distance range set in advance as the hand pattern.
Further, the hand pattern extraction means includes pattern type determination means for determining a plurality of types of the hand pattern set in advance according to the magnitude of the distance of the extracted hand pattern to the detection base point, Preferably, the instruction operation determining means determines that the instruction operation by the hand pattern is different according to the type of the hand pattern determined by the pattern type determining means.

In this case, the pattern type discriminating means belongs to which division distance range of the plurality of division distance ranges obtained by dividing the extracted hand pattern within the predetermined distance range in the distance direction. More preferably, the type of the hand pattern is discriminated.
In addition, it is preferable that the instruction operation determination unit includes a shape operation recognition unit that recognizes an instruction operation based on the operation of the hand pattern based on a temporal change in the shape of the hand pattern extracted by the hand pattern extraction unit ( Claim 9).

The instruction operation determining means recognizes an instruction operation by the operation of the hand pattern based on a change in distance over time to the object corresponding to each part of the image detected by the object detection sensor. It is preferable to have motion recognition means (claim 10).
The object detection sensor detects the distance to the object based on the phase difference between the irradiation wave on the object and the reflected wave, and determines the incident angle of the reflected wave with respect to the object detection sensor. Accordingly, a three-dimensional image sensor that detects the shape of the object as a planar image on the imaging surface is preferable.

  Further, in this case, the hand pattern extracting means converts the plane image of the object existing within a predetermined distance range from the plane image detected by the three-dimensional image sensor to the image of the hand pattern. The instruction operation determination means is converted by the binarization conversion means for converting the image extracted by the hand pattern extraction means into a binarized image by threshold processing, and converted by the binarization conversion means. And a finger posture specifying means for specifying the posture of the palm and finger as the instruction operation of the operator in the binarized image, and the posture corresponding to the posture of the palm and finger specified by the finger posture specifying means More preferably, the instruction operation is discriminated (claim 12).

The object detection sensor is fixed to a ceiling surface side in the vehicle interior space, and is an area above the ceiling surface and below a knob of a shift lever disposed below the vehicle interior. It is preferable to detect an object in (Claim 13).
In addition, the instruction operation determination means assumes that the operator's hand related to the hand pattern is placed on the shift lever, from the hand pattern in a predetermined area above the knob of the shift lever. It is preferable to determine the instruction operation of the operator.

  Further, the hand pattern operation method of the present invention (Claim 15) is mounted on a vehicle by detecting an operator's gesture as an instruction operation using an object detection sensor fixed in the vehicle interior of the vehicle. A hand pattern operation method for performing a predetermined operation on a device, wherein the object detection sensor detects the shape of the object in a predetermined detection area space and the distance to the object, and the distance to the object Based on the object, the hand pattern related to the instruction operation of the operator is extracted, and then the instruction operation by the hand pattern extracted based on the shape of the object is determined to determine the predetermined operation. It is characterized by carrying out.

  According to the hand pattern switch device of the present invention (Claim 1), it is possible to accurately grasp the object and its respective parts based on the direction information and distance information of the object existing in the detection region space. Further, it is possible to extract a hand pattern from the entire detected object, discriminate the instruction operation of the operator, and to instruct a predetermined operation corresponding to the instruction operation to the instruction target.

  Further, according to the hand pattern switch device of the present invention (claim 2), the object can be grasped by the separation based on the distance by referring to the distance information of each part of the object. Further, by extracting each part of the cut object as a hand pattern, it is possible to easily and accurately determine the instruction operation of the operator. Further, it is possible to reduce the processing load of the calculation related to hand pattern extraction, and to extract a hand pattern easily and accurately.

Further, according to the hand pattern switch device of the present invention (Claim 3), in-vehicle device control by the hand pattern can be accurately performed in the vehicle compartment space of the vehicle having many disturbance factors related to the detection of the object. .
Further, according to the hand pattern switch device of the present invention (claim 4), since the hand pattern is extracted from the object detected by the object detection sensor based on only the distance to the object, the hand pattern extraction is performed. Therefore, it is possible to reduce the processing load of the calculation, and it is possible to easily and accurately extract the hand pattern. In addition, since the hand pattern instruction operation is recognized based on the captured image of the object, the control logic and know-how in the prior art using an optical sensor can be used. Further, these effects can be realized with a simple configuration at low cost.

Further, according to the hand pattern switch device of the present invention (Claim 5), when recognizing an instruction operation by a hand pattern, only the extracted hand pattern is referred to, so that the processing load can be further reduced.
According to the hand pattern switch device of the present invention (claim 6), the hand pattern extraction accuracy can be improved with a simple configuration.

Moreover, according to the hand pattern switch device of the present invention (claim 7), various hand pattern variations (types) can be set. For example, it is possible to assign different operation instruction operations corresponding to the distance to the object detection sensor to the operation of one hand pattern.
Further, according to the hand pattern switch device of the present invention (Claim 8), the types of hand patterns can be classified by the division distance range. That is, when the variation (type) of the hand pattern is associated with the distance to the object detection sensor, the distance range in which the same variation is associated can be set as the divided distance range, which is useful for the operator. It is possible to provide an operation range that is easy to understand intuitively. A plurality of division distance ranges having such a width can be provided.

Further, according to the hand pattern switch device of the present invention (Claim 9), it is possible to recognize a change in the shape of the hand pattern over time with a simple configuration and to move in the imaging plane direction of the object detection sensor. The movement of the hand pattern can be recognized.
According to the hand pattern switch device of the present invention (claim 10), a change in hand pattern distance with time can be recognized with a simple configuration, and the direction perpendicular to the imaging surface of the object detection sensor. It is possible to recognize the movement of the hand pattern moving in the direction of distance (that is, in the distance direction).

In addition, according to the hand pattern switch device of the present invention (claim 11), the plane image on the imaging surface and the distance to the object can be easily detected using the three-dimensional image sensor. Thereby, the control logic and know-how in the conventional technology using the optical sensor can be effectively utilized.
Further, according to the hand pattern switch device of the present invention (claim 12), it is possible to easily recognize the instruction operation of the hand pattern from the extracted hand pattern image.

Further, according to the hand pattern switch device of the present invention (claim 13), an area where the gesture operation is easy for the operator can be detected by the object detection sensor, and usability can be improved. In addition, the operation of the shift lever and the gesture operation can be easily discriminated, and the gesture recognition accuracy can be improved.
Moreover, according to the hand pattern switch device of the present invention (claim 14), the operator's hand placed on the shift lever can be determined as a hand pattern instruction operation. In addition, the operator can perform a hand pattern instruction operation with a hand placed on the shift lever. In other words, it is not necessary to move the hand or arm greatly for the instruction operation, and the in-vehicle devices can be operated in a natural posture with the hand on the shift lever. Therefore, it is possible to provide a hand pattern switch device that is extremely easy to use.
Further, such an effect can be easily obtained with a simple configuration in which the area in which the instruction operation determination unit determines the instruction operation from the hand pattern is set to a predetermined area above the knob of the shift lever.

  Further, according to the hand pattern operation method of the present invention (claim 15), it is possible to reduce the processing load of calculation related to hand pattern extraction, and to extract a hand pattern accurately and easily. This also makes it possible to accurately determine the operator's instruction operation, and to accurately instruct an operation corresponding to the instruction operation to the instruction target, thereby improving controllability. .

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
1 to 9 show a hand pattern switch device as an embodiment of the present invention. FIG. 1 is a schematic configuration diagram showing the overall configuration of the device. FIG. 2 is a vehicle of a vehicle to which the device is applied. 3 is a schematic overhead view showing the imaging region of the object detection sensor in the room, FIG. 3 is a schematic diagram for explaining the characteristics of the distance image imaged by the object detection sensor of this apparatus, and (a) is the distance image. (B) is a schematic diagram for explaining a detection area and a distance band obtained by capturing a distance image, (c) is an image obtained by extracting a distance band a portion from the distance image, and (d) is a distance image. An image obtained by extracting the distance band b portion, (e) an image obtained by extracting the distance band c portion from the distance image, and FIG. 4 depicting the hand pattern shape and distance extracted by the apparatus as a pseudo three-dimensional shape. FIG. 5 is an oblique projection, and FIG. FIG. 6 and FIG. 7 are control flow charts showing the contents of control in this apparatus, and FIG. 8 is a finger for giving an operation instruction to this apparatus. It is a figure which shows the example of a shape, (a) is a finger shape which raised the index finger and thumb, (b) is a finger shape with the operation | movement which puts in and out of the thumb, (c) is rightward in the state which raised the index finger. FIG. 9 shows a finger shape with a movement that moves, and FIG. 9D shows a finger shape with a movement that moves to the left with the index finger raised, and FIG. 9 shows an operation instruction status by an operator using this apparatus. It is a typical vehicle interior perspective view.

[overall structure]
FIG. 1 shows the interior of a vehicle to which the present hand pattern switch device is applied and the functional configuration of a control device in the present hand pattern switch device. First, as shown in FIG. 1, a steering wheel (handle) that is steered by an operator (driver) is provided in front of the driver seat in the passenger compartment 7, and a speed change is performed on the side closer to the passenger seat than the driver seat. A shift lever 8 is provided for controlling the machine.

  An instrument panel is arranged in front of the driver's seat, and in-vehicle devices and various instruments mounted on the vehicle are arranged in the interior. In addition, in this instrument panel, for example, the operation panel 6 of the in-vehicle audio device or the air conditioner can be easily operated from both the driver's seat side and the passenger seat side. Is placed in front of. In the vehicle to which the hand pattern switch device is applied, the in-vehicle audio device and the air conditioner can be directly operated using the operation panel 6, and the gesture (generally, “hand” It is also referred to as “pattern, hand shape, palm finger shape”, etc., and the operation of these “hand pattern, hand shape, palm finger shape” is also included).

  This hand pattern switch device controls a vehicle-mounted audio device and an air conditioner based on information detected by the three-dimensional image sensor 5 and a three-dimensional image sensor 5 that captures a hand gesture shape and its operation (hand pattern) as an input operation. An ECU (control unit) 1 serving as a control device and a speaker 11 that emits a sound, a signal sound, or the like by an output signal from the ECU 1 are configured.

  As shown in FIG. 1, the three-dimensional image sensor (object detection sensor) 5 is fixed on the ceiling surface side above the shift lever 8 so that the imaging range is oriented downward. The three-dimensional image sensor (hereinafter also simply referred to as a sensor) 5 is an object in a predetermined detection region (detection region space, region surrounded by a one-dot chain line in FIG. 1) 10 in the vehicle compartment 7 of the vehicle. Is a sensor that detects the shape of the object on the imaging surface and the distance D from the detection base point to the object in the detection area 10.

Note that the imaging surface here refers to a virtual two-dimensional plane on which an object detected as an image is projected, which will be described later. The object here refers to a general object to be imaged that exists in the detection region 10, and for example, not only a hand gesture as an input operation, but also a shift lever 8 or the like existing in the background portion is also an object. include.
Further, the detection area 10 where the sensor 5 detects the object is a space between the driver's seat and the passenger seat (intermediate in the vehicle width direction), and in a plan view as shown by a one-dot chain line in FIG. This is the entire internal space of a substantially quadrangular pyramid space region (indicated by a one-dot chain line in FIG. 1) sandwiched between the rectangular region and the sensor 5. As shown in FIG. 2, the position and width of the detection region 10 in a plan view are positions where the driver can naturally reach in a posture where the driver is seated, and at least the shape of the driver's hand. For example, in this embodiment, the area including the shift lever 8 is set to the left side area of the driver's seat.

  The three-dimensional image sensor 5 in the present embodiment irradiates an object existing in the detection region 10 with infrared rays using a laser device or an LED, detects a reflected wave reflected from the object, and detects the irradiated wave and the reflected wave. The distance D to the object is detected on the basis of the phase difference. In addition, the direction in which the object is present is specified according to the incident angle of the reflected wave reflected from the object with respect to the sensor 5, and the shape of the object is detected as a planar image on the imaging surface.

  That is, the three-dimensional image sensor 5 is defined by information excluding distance information from the shape of the object on the imaging surface (that is, not the three-dimensional shape, but the three-dimensional position information of the object in the detection area space). The presence of the object is detected by two types of detection information, that is, the direction and two-dimensional position information) and the distance D to the object. The shape of the object detected here and the distance D to the object are input to the ECU 1.

  In addition, when the presence of the object detected in the sensor 5 is virtually illustrated as an image projected on the above-described imaging surface, an image as illustrated in FIG. In FIG. 3A, as shown in FIG. 3B, an object detected when the operator makes a gesture within the detection range 10 is projected. Further, the color shading in FIG. 3 corresponds to the distance D of each part of the object, and the closer to the three-dimensional sensor 5 in the detection region 10, the thinner (here, the coarse hatch display) and the farther it is. It is expressed darkly (in this case, a dense hatch display).

  That is, the object detected by the sensor 5 is perpendicular to the plane with respect to each of the unit pixels constituting the image in a plane image that does not include optical information such as illuminance and luminance (only indicates the presence of the object). A pseudo three-dimensional shape is formed in which only one piece of position information in a specific direction (that is, distance information with respect to the sensor 5) is given. A schematic representation of this pseudo three-dimensional shape is shown in FIG. Here, a pseudo three-dimensional shape is shown assuming that only the hand is detected by the sensor 5.

  In FIG. 4, the planar shape of the object as a set of points where the object exists is first plotted on the xy plane. That is, the figure when this figure is viewed from the z-axis direction matches the shape detected by the sensor 5. A distance D to the object is given in the z-axis direction to each point constituting the figure, and a grid-like virtual line connecting the points in the x-axis direction and the y-axis direction is used as a grid line. It is drawn. As a result, a pseudo three-dimensional shape is drawn in which the uneven shape of the hand (here, the back side of the hand) viewed from the sensor 5 side can be three-dimensionally captured.

  As described above, only one distance D of the object, that is, a value in the z-axis direction is given to each point (each part of the object) where the object exists. The three-dimensional shape shown here is only the shape on the front side of the hand, and as shown in FIG. 4, the shape in which the back surface (palm surface) of the hand is not clear, that is, as if it is on the sensor 5 side. It becomes a three-dimensional shape formed as a curved surface as if a table cloth was hung from the direction.

  Further, if attention is paid to the distance D information of each unit pixel constituting the image shown in FIG. 3A and only the objects existing in substantially the same distance band are extracted and drawn, FIG. The image is as shown in (e). The image in the distance band a shown in FIG. 3B has a shape as shown in FIG. 3C in which only the shape of the operator's hand exists, and the image in the distance band b slightly below the distance band a is FIG. 3D shows the shape of the shift knob at the tip of the shift lever 8 as a background that has no meaning as a gesture, and the image in the distance band c further below the distance band b is manipulated. 3 (e) in which the shape of the person's foot is drawn.

[ECU]
Next, functions of the ECU 1 will be described. The ECU 1 includes a hand pattern extraction unit (hand pattern extraction unit) 2 that extracts a hand pattern related to an instruction operation by an operator from an object, and an instruction operation determination unit (instruction operation determination) that recognizes an instruction operation by the hand pattern. Means) 3, the shape and distance of the hand pattern recognized by the operation command unit (operation command unit) 4 for performing a predetermined operation on the vehicle-mounted device, the hand pattern extraction unit 2 and the instruction operation determination unit 3 over time. And a memory 9 for storing the data.

Based on the distance D detected by the sensor 5, the hand pattern extraction unit 2 extracts a hand pattern related to the operator's instruction operation from the target object, that is, a gesture made to operate the in-vehicle device. It is like that. Specifically, only the objects existing within a predetermined distance range among the detected objects are extracted as hand patterns. The predetermined distance range in the present embodiment is a position above the shift knob of a shift lever 8, the distance from the sensor 5 is set to the range of the first distance D less than _1.

  Here, an object (hand pattern) existing within a predetermined distance range is extracted as an image. The hand pattern as an image extracted here is input to the instruction operation determination unit 3. For example, an image of the hand shape of the operator as shown in FIG. 3C is extracted as the hand pattern image. Information on the distance D of the hand pattern extracted here is stored in the memory 9 as needed.

  That is, the hand pattern extraction unit 2 extracts only the image shown in FIG. 3C where the hand shape exists from the distance image shown in FIG. 3A obtained from the sensor 5, and FIG. , (E), the background image such as the shift knob or the operator's foot is deleted. Such image sorting is performed according to the distance D. Therefore, in the present embodiment, among the three-dimensional position information of the target object in the detection area space, the two-dimensional position information is selected based on the distance information, and the result is a hand pattern (two-dimensional position regarding the target object). (Collection of information).

  The hand pattern extraction unit 2 includes a pattern type setting unit (pattern type determination unit) 2 a that determines a plurality of types of hand patterns set in advance according to the size of the distance D. The pattern type setting unit 2a is a functional unit for setting the type of the hand pattern extracted by the hand pattern extraction unit 2, and among the plurality of divided distance ranges in which the hand pattern is divided within the predetermined distance range in the distance direction. Which division distance range belongs to, the type of hand pattern is determined.

For example, in the pattern type setting unit 2a, as shown in FIG. 5, the first distance D ₁ , the second distance D _2, and the third distance D ₃ as threshold values satisfy D ₁ > D ₂ > D ₃ ≧ 0. It is set in advance. Then, the type of the hand pattern is determined depending on which range the distance D of the object is divided by these thresholds. In the example shown in FIG. 5, a distance range where D ₁ > D ≧ D ₂ is set as the first divided distance range, and a distance range where D ₂ > D ≧ D ₃ is set as the second divided distance range. , D ₃ > D ≧ 0 is set as the third divided distance range.

  The type of hand pattern means a type classification for distinguishing the contents of operation instructions associated with the shape or action of the hand pattern. For example, in the present embodiment, when the hand pattern extracted by the hand pattern extraction unit 2 moves between different division distance ranges, the hand pattern is recognized as the hand pattern related to the voice announcement. Yes. That is, in FIG. 3B, when the hand pattern by the operator moves in the vertical direction (separation direction with respect to the sensor 5), the function of the vehicle-mounted device to be operated is selected (this) The function name of the selected in-vehicle device is announced by the speaker 11 as a voice.

  That is, in the present embodiment, by specifying the distance range in which the hand pattern exists, for example, it is possible to associate a plurality of different operation instruction operations with the shape and operation of the same hand pattern, or there is a hand pattern. A function corresponding to a change in the distance range can be set in advance. The type of hand pattern determined here is stored in the memory 9 as needed.

  The instruction operation determination unit 3 recognizes an instruction operation by the hand pattern extracted by the hand pattern extraction unit 2 based on the shape of the object on the imaging surface detected by the sensor 5. In this embodiment, based on the shape corresponding to the hand pattern extracted by the hand pattern extraction unit 2 among the shapes of the objects detected by the sensor 5, the instruction operation by the hand pattern is recognized. . Note that the instruction operation determination unit 3 according to the present embodiment can recognize an instruction operation based on a hand pattern by recognizing a change with time in the shape of the hand pattern, that is, an operation.

In order to function in this manner, the instruction operation determination unit 3 includes a binarization conversion unit (binarization conversion unit) 3a, a finger posture identification unit (hand posture identification unit) 3b, and an action recognition unit 3c. It is configured.
First, the binarization conversion unit 3a functions to convert the image extracted by the hand pattern extraction unit 2 into a binarized image by threshold processing. The binarization processing of the image in the binarization conversion unit 3a is, for example, the case where each density of unit pixels constituting the image (in this case, corresponding to the distance D) is set to a preset threshold density. A general binarization process that converts two values according to the magnitude relationship. That is, here, a monochrome image related to the distance information of the hand pattern is generated.

  The finger posture specifying unit (hand posture specifying unit) 3b functions to specify the posture of the palm and the finger as an instruction operation of the operator in the binarized image converted by the binarization conversion unit 3a. Here, conventional posture identification is performed by determining the size and orientation of the palm square that contains the palm from the binarized image, and determining the finger position of the little finger, thumb, etc. based on the shape of the outline of the palm The method is used. By specifying the postures of the palm and fingers, it is possible to grasp the shape characteristics of the hand pattern, for example, which finger the operator extends in which direction. The palm and finger postures extracted here (that is, the shape of the hand pattern) are stored in the memory 9 as needed.

  On the other hand, the motion recognition unit 3c is a functional unit for grasping the motion of the hand pattern, and a shape motion recognition unit (recognizing a motion instruction operation based on a time-dependent shape change (variation with the passage of time of the hand pattern shape) ( (Shape motion recognition means) 3d and a distance motion recognition unit (distance motion recognition means) 3e for recognizing a motion instruction operation based on a time-dependent distance change (variation of the hand pattern distance D with time). Has been.

The motion recognition unit 3 c recognizes the motion of the hand pattern based on the shape of the hand pattern stored in the memory 9 and the history of the change in the distance D over time. For example, the shape motion recognition unit 3d recognizes a motion that changes the posture of a palm and a finger, and the distance motion recognition unit 3e recognizes a motion that moves in the distance direction of the hand pattern. .
Based on the shape and type of the hand pattern identified and recognized by each functional unit described above and its operation, the instruction operation determination unit 3 determines the instruction operation of the hand pattern operation by the operator. The instruction operation determined here is input to the operation command unit 4.

In the present embodiment, the specific instruction operation determined by the instruction operation determination unit 3 is set for the shape of the hand pattern as shown in FIGS.
First, as shown in FIG. 8A, when a hand-shaped hand pattern shape (hand pattern shape A) with an index finger and a thumb raised is detected, the instruction operation determination unit 3 determines that the operator It is determined that the instruction operation by is desired. That is, the instruction operation determination unit 3 enters a standby state for waiting for an instruction operation by the hand pattern that follows. Therefore, when this hand pattern shape is not detected, even if each hand pattern shape and hand pattern operation described below is detected, the corresponding instruction operation is not recognized. With such a configuration, erroneous recognition of a hand pattern and malfunction of the present apparatus can be prevented.

  In addition, as shown in FIG. 8B, when the finger shape in a state where no finger is raised and a state where only the thumb is raised is sequentially detected (hand pattern operation B is detected), the instruction operation determination is performed. The unit 3 selects the function of the in-vehicle device to be operated according to the number of repetitions of these finger shapes. In other words, one of the functions of the in-vehicle device to be operated can be selected from among the plurality of in-vehicle devices by the operation of inserting and removing the thumb by the operator.

  For example, the air volume operation function of the air conditioner is selected when the thumb is taken in and out once, the temperature operation function of the air conditioner is selected when the thumb is performed twice, and the volume operation function of the in-vehicle audio device is selected when the thumb is performed three times. It has come to be. When the thumb is taken in and out four times, the air volume operation function of the air conditioner is selected again, and these functions can be selected repeatedly in order.

  In addition, after each function is selected by such an operation, as shown in FIGS. 8C and 8D, the movement of the finger shape with the index finger raised to the left or right is posted. The instruction operation determination unit 3 is configured to set the operation amount for each function selected according to the movement direction of the hand pattern. For example, in the state in which the air volume operation function of the air conditioner is selected, as shown in FIG. 8C, when a finger shape (hand pattern operation C) with a movement of the index finger moving in the right direction is detected, It is determined that the hand pattern is an instruction operation for increasing the air volume (air volume up). For example, as shown in FIG. 8D, when a finger shape (hand pattern operation D) with a movement of the index finger moving to the left is detected, the hand pattern reduces the air volume (air volume down). It is determined that the operation is an instruction operation.

In the present embodiment, an operation instruction of the hand pattern by the operator, as being implemented in a state of hanging the hand shift knob of a shift lever 8, the distance is the first distance D ₁ less than the range from the sensor 5, In addition, the hand pattern existing in the range of the second predetermined distance D ₂ or more is determined as a predetermined instruction operation in the instruction operation determination unit 3. That is, here, except for the hand pattern extracted within the predetermined region D ₂ ≦ D <D ₁ above the shift lever 8, the hand pattern as shown in FIGS. Even so, it is determined that it is not an instruction operation for operating the in-vehicle device.

  As described above, the hand pattern instruction operation determined by the instruction operation determination unit 3 is input to the operation command unit 4, and the operation command unit 4 is determined by the instruction operation determination unit 3. A predetermined operation is commanded to a command target corresponding to the instruction operation. In this way, a predetermined operation on the device mounted on the vehicle is performed by the operator's hand pattern operation.

[flowchart]
The hand pattern switch device according to an embodiment of the present invention is configured as described above, and performs control according to a control flow described below. The control flow shown in FIG. 6 and FIG. 7 is repeatedly executed as appropriate at a predetermined cycle inside the ECU 4. Note that the control flow shown in FIG. 6 is a main flow in this apparatus, and the control flow shown in FIG. 7 is a flow for recognizing a hand pattern instruction operation.

First, in step A10 in the main flow, the object in the predetermined detection area 10 is detected as an image by the sensor 5. And the shape of the target object detected here and the distance D to the target object are inputted into ECU1, and it progresses to Step A20.
In step A20, the hand pattern extraction unit 2, whether the magnitude of the distance D to the detected object is a first distance D less than ₁ is determined. Here, in the case of D <D ₁ proceeds to step A30, in the case of D ≧ D ₁ as it is and this flow is finished.

Note that, in step A20, the first distance D ₁ is set to be the distance to the predetermined position above the shift knob of a shift lever 8 from the sensor 5. Therefore, when an object is detected below the shift knob of the shift lever 8, this flow ends. Therefore, unlike the prior art, it is not necessary to consider the luminance, illuminance difference, and the like between the hand pattern and the periphery (background portion), and the background portion can be easily deleted from the detection data.

In step A30, the distance D is the object in the first distance D ₁ range, is extracted as the hand pattern. The hand pattern is extracted as an image, and information on the distance D of the hand pattern is stored in the memory 9 and then input to the instruction operation determination unit 3, and the process proceeds to step A40.
In step A40, the input hand pattern image is binarized and converted in the binarization conversion unit 2a. In the subsequent step A50, the shape of the hand and the finger is converted from the binarized image converted in the previous step. The posture is specified and stored in the memory 9.

In the subsequent step A60, an instruction content recognition flow for recognizing an instruction operation using the hand pattern is executed based on the information on the distance D of the hand pattern stored in the memory 9 and the shape of the hand pattern. In subsequent step B70, it is determined whether or not the voice flag Fv set in the instruction content recognition flow is Fv = 1.
The voice flag Fv is a flag for announcing the function of the in-vehicle device currently selected in the hand pattern switch (that is, the air volume of the air conditioner, the temperature of the air conditioner, and the volume of the in-vehicle audio device) to the operator by voice It is. Such an announcement is performed when this flag is set to Fv = 1, and no announcement is performed when this flag is set to Fv = 0.

  Here, if Fv = 1, the function of the currently selected in-vehicle device, that is, any of the air volume of the air conditioner, the temperature of the air conditioner, and the volume of the in-vehicle audio device is announced by voice, and the process proceeds to step A90. Thus, Fv = 0 is returned, and the process proceeds to Step B100. If Fv = 0, the process proceeds directly to step A100. The voice flag will be described later. Thereafter, in step A100, in accordance with the instruction operation of the hand pattern recognized in the instruction content recognition flow, a predetermined operation is performed on the in-vehicle device, and this flow is finished.

Next, the instruction content recognition flow according to step A60 will be described with reference to FIG.
First, in step B10 of the instruction content recognition flow, the pattern type determination unit 2a determines whether or not the hand pattern extracted by the hand pattern extraction unit 2 has moved between different division distance ranges. That is, here, it is determined whether or not the hand pattern by the operator has moved in the vertical direction (direction of contact with the sensor 5). If it is determined that the hand pattern has moved between different division distance ranges, the process proceeds to step B20, the voice flag Fv is set to Fv = 1, and the process proceeds to step B30. On the other hand, if it is not determined that the hand pattern has moved between different division distance ranges, the process proceeds to step B30 with Fv = 0.

In step B30, the movement recognition unit 3c recognizes the movement of the hand pattern based on the shape of the hand pattern stored in the memory 9 and the history of the distance, and in step B40, the movement recognition unit 3c determines the shape and type of the hand pattern and the movement thereof. Based on this, the instruction operation determination unit 3 recognizes the instruction operation of the hand pattern operation by the operator, and the flow ends.
In the recognition of the instruction operation in step B40, the instruction operation determination unit 3 assumes that the operator's hand is placed on the shift lever 8, a predetermined area above the shift knob of the shift lever 8, that is, An instruction operation for operating the in-vehicle device of the operator is determined from the hand pattern in the region of D ₂ ≦ D <D ₁ . Accordingly, it is possible to recognize a hand pattern operation in a state in which the hand is put on the shift lever 8, and the operator can perform a hand pattern instruction operation with a natural posture with the hand placed on the shift lever. It will be possible.

[effect]
As described above, according to the hand pattern switch device of the present invention, when extracting a hand pattern from the object detected by the three-dimensional image sensor 5, the hand pattern and other information are extracted using information on the distance D to the object. Therefore, it is possible to improve the accuracy of hand pattern extraction, reduce the processing load of calculation related to hand pattern extraction, and easily and accurately extract a hand pattern. become.

Further, in the present embodiment, the hand pattern is extracted and recognized using parameters that are not affected by the illumination conditions such as the shape and distance of the target object, so that control that is resistant to disturbance is possible.
At this time, as shown in FIG. 9, even when the operator tries to perform an operation using the hand pattern with the hand held on the shift lever 8, the detection area of the sensor 5 is the shift knob of the shift lever 8. because it is set in a predetermined area D <D ₁ above than, as in the prior art, the brightness of the hand pattern and the peripheral (background portion), it is not necessary to consider the photometric like, easily instruction operation It is possible to discriminate the hand pattern relating to.

In addition, in the instruction operation determination unit 3, it is assumed that a hand pattern operation instruction by the operator is applied to the shift knob of the shift lever 8, and the region where the hand pattern exists is a predetermined region D ₂ ≦ D < whether the D ₁ is determined. That is, since an object existing in the predetermined region D ₂ ≦ D <D ₁ is determined as a hand pattern for the instruction operation, the operator performs an instruction operation for the hand pattern with the hand placed on the shift lever. It can be carried out. Therefore, it is not necessary to move the hand or arm greatly for the instruction operation, and the operation of the in-vehicle devices can be performed with a natural posture with the hand placed on the shift lever. Thus, according to the hand pattern switch device of the present embodiment, it is possible to reduce the burden on the operator's hand and arm and to provide a hand pattern switch device that is extremely easy to use.

In addition, according to the present apparatus, for the hand pattern in the predetermined area D ₂ ≦ D <D ₁ , the instruction operation content of the operator is determined according to the shape and operation, and thus, for example, the hand pattern is placed on the shift knob. It is possible to discriminate whether the operator's finger shape and its movement are for mere shift lever operation or for instruction operation of this device, and operation of the shift lever 8 (shift change operation) The movement of the finger that performs is not erroneously recognized as an instruction operation.

Even if the same hand pattern shape and action as the hand pattern for the instruction operation are extracted, if the area where the hand pattern exists is not within the predetermined area D ₂ ≦ D <D ₁ , it is indicated. Since it is not determined that the operation is an operation, an operation performed without placing a hand on the shift lever 8 is not erroneously recognized as an instruction operation, and the recognition accuracy can be further improved.

  In addition, since the extracted hand pattern can be processed as a planar image, for example, the control logic and know-how in hand pattern recognition technology using a CCD camera or the like that has been cultivated in the past can be utilized, and two-dimensional A method of identifying a hand pattern by an arithmetic process on data can be used as it is. Accordingly, accurate hand pattern recognition can be realized with a simple configuration at low cost.

  In the embodiment described above, the hand pattern type determination unit 2a can set the type of hand pattern according to the size of the distance D to the object. That is, even if the shape and operation of the same hand pattern is used, setting can be made such that the operation instruction operation is different if the distance D to the object detection sensor is different. Thereby, the shape of the hand pattern for operation and the number of operations can be reduced, and the operation input can be simplified.

  Even if it is a gesture operating device capable of recognizing a large number of gesture operations, if the operator cannot input an operation without memorizing all the gesture operations, it is an operation device that is easy to use for the operator. I can't say that. However, according to the hand pattern switch device of the present embodiment, since the number of gesture operations that the operator has to remember in advance can be reduced, an operation device with extremely high usability can be provided. is there.

  In addition, the device recognizes not only static gestures such as the shape and distance of hand patterns, but also gesture changes over time, that is, gesture movements (ie, dynamic gestures) with a simple configuration. can do. At that time, not only can the gesture operation in the imaging surface direction of the 3D image sensor 5 (operation not including movement in the distance direction) be recognized, but also the gesture operation in the direction perpendicular to the imaging surface. Can be recognized.

  In addition, since the three-dimensional image sensor 5 in the present embodiment is an image sensor that detects the distance D to the object and detects the shape of the object as a planar image, a process of extracting a hand pattern as a planar image. Is simple, and an instruction operation of the hand pattern can be easily recognized from the extracted hand pattern image.

[Others]
Although the embodiments of the present invention have been described above, the present invention is not limited to such embodiments, and various modifications can be made without departing from the spirit of the present invention.

  For example, in the above-described embodiment, the three-dimensional image sensor 5 is used as a means for detecting an object in the vehicle interior space of the vehicle, but for example, an object in the vehicle interior space using ultrasonic waves or the like is used. A kind of sonar sensor that detects the distance and direction to the object detection sensor may be used. The object detection sensor in this case cannot be said to image a target object in a predetermined area in the vehicle interior, but internally detects an image of the target object based on the direction in which the target object exists. Therefore, it can be regarded that the planar shape of the object is substantially captured. Therefore, the same effect as the above-described embodiment can be expected. Similarly, a laser radar or the like may be used as the object detection sensor.

Moreover, the detection area 10 of the object imaged by the three-dimensional image sensor 5 can be changed as appropriate according to the vehicle type, the internal shape of the passenger compartment 7, the main use of the apparatus, and the like.
In the above-described embodiment, as shown in FIG. 5, there are three types of division distance ranges, and when the hand pattern moves between different division distance ranges, the function of the selected in-vehicle device is Although it is notified by voice announcement, it can be configured such that different associations are made in each divided distance range regarding association between the hand pattern and the instruction operation. For example, the hand pattern of the first divided distance range is determined as the instruction operation of the air conditioner and the in-vehicle audio device as described above, and the hand pattern of the second divided distance range and the third divided distance range is used. Alternatively, it may be determined as an instruction operation of another in-vehicle device (a car navigation device, a room light, etc.).

For example, in this case, control is performed according to the flow shown in FIG. 10 instead of the instruction content recognition flow shown in FIG. 7 in the above-described embodiment.
First, in the flow shown in FIG. 10, in step C10, in the pattern type determination unit 2a, the magnitude of the hand pattern distance D stored in the memory 9 is not less than the second distance D ₂ and less than the first distance D _1. It is determined whether or not. That is, here, it is determined whether or not the type of the hand pattern extracted by the hand pattern extraction unit 2 belongs to the first division distance range. Here, if D ₂ ≦ D <D ₁ , the process proceeds to step C30, where the hand pattern is classified as belonging to the first division distance range (for example, the type related to the car navigation device). Is determined, and the process proceeds to Step C60. If D ₂ ≦ D <D ₁ is not satisfied (that is, at least D <D ₂ ), the process proceeds to step C20.

The hand pattern distance D determined here refers to the hand pattern distance D extracted immediately before in the main flow. For example, when a plurality of distance information is stored in the memory 9. The latest of them is being used.
In step C20, the size of the distance D of the hand pattern, a third distance whether there is D ₃ than is a second distance D less than ₂ is judged. That is, here, it is determined whether or not the type of the hand pattern extracted by the hand pattern extraction unit 2 belongs to the second division distance range. Here, if D ₃ ≦ D <D ₂ , the process proceeds to step C 40, and the hand pattern type (for example, the type related to the room light) is assumed as the hand pattern belonging to the second division distance range. Determination is made and the process proceeds to step C60. If D ₃ ≦ D <D ₂ , since D <D ₃ , the process proceeds to step C50, where the hand pattern belongs to the third division distance range (for example, in-vehicle communication). The device type is determined), and the process proceeds to Step C60.

  In step C60, the movement recognition unit 3c recognizes the movement of the hand pattern based on the shape of the hand pattern stored in the memory 9 and the history of the distance. In the subsequent step C70, based on the shape and type of the hand pattern and its operation, the instruction operation determination unit 3 recognizes the instruction operation of the hand pattern operation by the operator, and the flow is finished.

According to such a configuration, the switch device can be made multifunctional by discriminating the instruction operation according to the operation position of the hand pattern (distance to the sensor 5) in multiple stages.
Further, the division distance range for classifying the hand pattern type can be set to the width of the operation range that is easily understood by the operator. For example, the distance width between D _{1 and} D ₂ and the distance width between D _{2 and} D ₃ can be arbitrarily set, and these distance widths are set in advance as in the present embodiment. Alternatively, it may be adjusted as appropriate by the operator. The number of the division distance ranges is arbitrary, and the number of types of hand patterns can be increased as the number of divisions is increased.

Further, in the above-described embodiment, it is assumed that the operation instruction of the hand pattern by the operator is performed in the state where the operator holds the hand on the shift knob of the shift lever 8 in the instruction operation determination unit 3. The instruction operation is determined from the hand pattern in the predetermined area D ₂ ≦ D <D _1, but the range for determining such an instruction operation can be set as appropriate.

  Further, in the above-described embodiment, as shown in FIGS. 8C and 8D, the operation amount (the air volume operation amount or the temperature operation amount of the air conditioner, the volume operation of the in-vehicle audio device) in each function of the selected in-vehicle device. However, instead of such a configuration, movement of the hand in the vertical direction (separation direction with respect to the sensor 5) is set. The operation amount in each function may be set by For example, when in-vehicle audio is selected, it is conceivable that the volume is set large by moving the hand upward, and the volume is set small by moving the hand downward. Thereby, it becomes possible to operate an in-vehicle device more intuitively.

It is a typical lineblock diagram showing the whole hand pattern switch device composition concerning one embodiment of the present invention. 1 is a schematic overhead view showing an imaging region of an object detection sensor in a vehicle compartment of a vehicle to which a hand pattern switch device according to an embodiment of the present invention is applied. It is a schematic diagram for demonstrating the characteristic of the distance image imaged with the target object detection sensor of the hand pattern switch apparatus concerning one Embodiment of this invention, (a) is a schematic diagram which shows a distance image, (b) is. Schematic diagram for explaining a detection area and a distance band obtained by capturing a distance image, (c) an image obtained by extracting a distance band a part from the distance image, (d) an image obtained by extracting a distance band b part from the distance image, (E) is an image obtained by extracting the distance zone c from the distance image. It is the oblique projection figure which drawn the shape and distance of the hand pattern extracted with the hand pattern switch apparatus concerning one Embodiment of this invention as a pseudo three-dimensional shape. It is a schematic diagram which shows the division | segmentation distance range set according to the distance from the target object detection sensor of the hand pattern switch apparatus concerning one Embodiment of this invention. It is a control flowchart which shows the control content in the hand pattern switch apparatus concerning one Embodiment of this invention. It is a control flowchart which shows the control content in the hand pattern switch apparatus concerning one Embodiment of this invention. It is a figure which shows the example of the finger shape for giving an operation instruction with respect to the hand pattern switch apparatus concerning one Embodiment of this invention, (a) is the finger shape which raised the index finger and the thumb, (b) is the thumb. Finger shape with moving in and out, (c) finger shape with movement moving right with index finger up, (d) with movement moving left with index finger up Each finger shape is shown. It is a typical vehicle interior perspective view which shows the operation instruction | indication state by the operator using the hand pattern switch apparatus concerning one Embodiment of this invention. It is a control flowchart which shows the control content in the hand pattern switch apparatus concerning the modification of this invention.

Explanation of symbols

1 ECU (control unit)
2 Hand pattern extraction unit (hand pattern extraction means)
2a Pattern type determination unit (pattern type determination means)
3. Instruction operation determination unit (instruction operation determination means)
3a Binarization conversion unit (binarization conversion means)
3b finger posture specifying unit (hand posture specifying means)
3c motion recognition unit 3d shape motion recognition unit (shape motion recognition means)
3e Distance motion recognition unit (distance motion recognition means)
4 Operation command section (Operation command means)
5 Three-dimensional image sensor (object detection sensor)
6 Operation panel 7 Car compartment 8 Shift lever 9 Memory 10 Detection area (predetermined area)
11 Speaker

Claims (15)

An object detection sensor for detecting an object existing in a detection area space from a detection base point, and detecting a direction of each part of the object from the detection base point and a distance to the part;
Hand pattern extraction means for extracting a hand pattern related to an instruction operation by the operator based on the direction and distance detected by the object detection sensor;
Instruction operation determining means for determining the instruction operation of the operator based on the hand pattern extracted by the hand pattern extracting means;
A hand pattern switch device comprising: an operation command unit that commands a predetermined operation to a command target corresponding to the instruction operation determined by the instruction operation determination unit. The hand pattern extracting means selects the direction of each part of the object constituting the hand pattern based on the distance, and extracts each part as the hand pattern. The hand pattern switch device according to 1. The detection area space is a part of the vehicle interior space of the vehicle,
The hand pattern switch device according to claim 1, wherein the command target is a device mounted on the vehicle. The object detection sensor images the inside of the detection area space, detects the shape of the object on the imaging surface as an image, and detects the distance to the object corresponding to each part of the image,
The hand according to any one of claims 1 to 3, wherein the hand pattern extraction means extracts the hand pattern based on the image and the distance detected by the object detection sensor. Pattern switch device. The hand pattern extraction means extracts an image of the hand pattern from the image detected by the object detection sensor based on the distance detected by the object detection sensor,
5. The hand pattern switch device according to claim 4, wherein the instruction operation determining means determines an instruction operation by the hand pattern based on the hand pattern image extracted by the hand pattern extracting means. The said hand pattern extraction means extracts only each part of the said target object which exists in the predetermined distance range set beforehand as said hand pattern, The any one of Claims 1-5 characterized by the above-mentioned. Hand pattern switch device. The hand pattern extraction means has pattern type determination means for determining a plurality of types of the hand pattern set in advance according to the magnitude of the distance of the extracted hand pattern to the detection base point,
7. The instruction operation determining means determines that the instruction operation by the hand pattern is different according to the type of the hand pattern determined by the pattern type determining means. The hand pattern switch device according to claim 1. The pattern type discriminating unit discriminates which divided distance range of a plurality of divided distance ranges obtained by dividing the extracted hand pattern within a predetermined distance range in the distance direction. The hand pattern switching device according to claim 7, wherein the type of the hand pattern is discriminated. The instruction operation determination means includes shape motion recognition means for recognizing an instruction operation due to an operation of the hand pattern based on a temporal change in shape of the hand pattern extracted by the hand pattern extraction means. The hand pattern switch device according to any one of claims 4 to 8. The pointing operation determination means recognizes the pointing operation based on the movement of the hand pattern based on a change in distance over time to the target corresponding to each part of the image detected by the target detection sensor. It has a means, The hand pattern switch apparatus of any one of Claims 4-9 characterized by the above-mentioned. The object detection sensor detects a distance to the object based on a phase difference between an irradiation wave to the object and a reflected wave thereof, and according to an incident angle of the reflected wave with respect to the object detection sensor. The hand pattern switch device according to any one of claims 4 to 10, wherein the hand pattern switch device is a three-dimensional image sensor that detects the shape of the object as a planar image on the imaging surface. The hand pattern extraction means extracts a planar image of the object existing within a predetermined distance range as a hand pattern image from the planar image detected by the three-dimensional image sensor. ,
The instruction operation determination unit includes a binarization conversion unit that converts the image extracted by the hand pattern extraction unit into a binarized image by threshold processing, and the binarization converted by the binarization conversion unit. A finger posture specifying unit that specifies a posture of a palm and a finger as an instruction operation of the operator in the image, and determining the instruction operation according to the posture of the palm and the finger specified by the finger posture specifying unit The hand pattern switch device according to claim 11, wherein: The object detection sensor is fixed to a ceiling surface side in the vehicle interior space, and is an object in a region above a knob of a shift lever disposed below the ceiling surface and below the vehicle interior. An object is detected, The hand pattern switch apparatus of any one of Claims 3-12 characterized by the above-mentioned. The instruction operation determining means assumes that the operator's hand related to the hand pattern is placed on the shift lever, and performs the operation from the hand pattern in a predetermined area above the knob of the shift lever. The hand pattern switch device according to claim 13, wherein the instruction operation of the person is discriminated. A hand pattern operation method for performing a predetermined operation on a device mounted on a vehicle by detecting an operator's gesture as an instruction operation using an object detection sensor fixed in a vehicle interior of the vehicle,
The object detection sensor detects the shape of the object in a predetermined detection area space and the distance to the object,
After extracting the hand pattern related to the instruction operation of the operator from the object based on the distance to the object,
A method of operating a hand pattern, comprising: discriminating an instruction operation by the hand pattern extracted based on the shape of the object and performing the predetermined operation.

Images