JP3933139B2 - Command input device - Google Patents

Description

  The present invention relates to a command input device that receives a command input to a vehicle-mounted device and is expressed by a movement of a part of a body such as an occupant's hand, and in particular, displays a virtual image operation switch. The present invention relates to a command input device using a virtual switch in which air is blown out.

  There is a selection device that generates a virtual switch of a three-dimensional stereoscopic virtual image of an operation switch panel using a hologram and receives switch selection (see Patent Document 1). According to this selection device, the overhang of the center console can be suppressed and the living space in the vehicle can be expanded.

However, since an input operation is performed on the virtual image operation switch, the user cannot obtain physical contact with the operation switch and cannot feel the actual operation of the input operation. For this reason, it is difficult to perceive which part should be operated, and there is a disadvantage that the operability at the time of command input is inferior.
JP-A-9-190278

An object of this invention is to provide the command input device with high operativity.
According to the present invention, a command input device that is mounted on a vehicle and receives an input of a command expressed by an occupant, the imaging unit that images a predetermined detection area in the vehicle, and the image captured by the imaging unit , Detecting means for detecting any one or more of the position, shape or movement of the detection target in the detection area, and displaying a virtual image operation switch in a space in the vehicle, and toward the virtual image of the displayed operation switch The command is recognized from a virtual switch generating unit that generates a virtual switch from which air is blown, an operation switch image displayed by the virtual switch generating unit, and an image of the detection target detected by the detecting unit. Command input means comprising: command recognition means for executing; and execution means for executing a command recognized by the command recognition means Location is provided.
Thereby, since the user can recognize which part should be operated by visual recognition with respect to the operation switch of a virtual image and physical recognition by air, a command input device with high operability can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The command input device 100 of this embodiment receives an input of a command expressed by an occupant of a vehicle, specifically, a driving driver. The command input device 100 is connected to these in-vehicle devices so as to be able to exchange information via the in-vehicle LAN. That is, the command received by the command input device 100 is sent to each device for command execution via the in-vehicle LAN. The operation of the command input means 100 is not limited to the driver, and it is also possible to accept an input of an occupant's hand command that can reach the command input device 100. The command of the present embodiment is a command given by a passenger, for example, a driver to a computer that controls the in-vehicle device, and is a control command for the navigation device 200, the audio device 300, the air conditioner control device 400, and the communication device 500 mounted on the vehicle. . The command is expressed by the form of the occupant's arm, occupant's elbow, occupant's hand or occupant's finger, or the movement of the occupant's arm, occupant's elbow, occupant's hand, or occupant's finger.

  A block configuration of the command input device 100 of the present embodiment is shown in FIG. As illustrated in FIG. 1, the command input device 100 includes an imaging unit 1, a detection unit 2, a command recognition unit 3, a virtual switch generation unit 4, and an execution unit 5.

  The “imaging means 1” images a predetermined detection area. In the present embodiment, an infrared camera 11 is used as the imaging unit 1. The infrared camera 11 captures infrared rays emitted from the detection target and outputs an infrared image as a luminance value distribution. The installation position of the infrared camera 11 is shown in FIG. As shown in FIG. 2, the infrared camera 11 of this embodiment is provided in the center console of a vehicle. The infrared camera 11 images a “predetermined detection area 11A” including the vehicle steering shown in FIG. That is, the infrared camera 11 is installed so as to include the passenger's hand holding the steering wheel in the imaging region. The infrared camera 11 is suitable for an in-vehicle command input device because it does not require illumination means that hinders an occupant driving during imaging. Of course, the imaging means 1 is not limited to the infrared camera 11, and a CCD camera 12 that can acquire an image to be detected may be used.

  The “detecting means 2” detects any one or more of the position, shape or movement of the detection target in the detection area from the image picked up by the image pickup means 1. The detection unit 2 includes a position detection unit 21 that detects the position of the detection target, a shape detection unit 22 that detects the shape of the detection target, and a motion detection unit 23 that detects the movement of the detection target.

  The position detection unit 21 obtains a two-dimensional position (a position on the xy coordinates) of the detection target along the detection area surface from the two-dimensional image captured by the imaging unit 1. When the imaging unit 1 is the infrared camera 11, the three-dimensional position (the position in the z-axis direction) of the detection target in the direction perpendicular to the detection area plane is obtained based on the luminance value of the detection target in the infrared image.

  The function of the position detection unit 21 of the search means 2 when the image pickup means 1 (infrared camera 11) picks up the hand of the passenger gripping the steering will be described with reference to FIG. As shown in FIG. 3, the detection area 11 </ b> A includes a part of the steering center side of the vehicle. 11B is a spatial coordinate of the detection area 11A. The infrared camera 11 images the hand 53 of the occupant 50 who holds the steering wheel 40 before performing the command input process. Since the position of the steering wheel 40 is fixed, in this embodiment, the position of the hand 53 of the occupant 50 holding the steering wheel 40 is set as a reference position, and the luminance value of infrared rays emitted from the hand 53 at this time is set as the reference luminance value. A method for detecting the position of the hand when the occupant moves the hand 53 in the detection area 11A will be described. First, the xy coordinate position of the hand 53 (hand moved from the steering) to be detected is acquired from the captured infrared image. Then, by comparing the brightness value of the hand 53 to be detected (the hand moved from the steering wheel) with the reference brightness value (the brightness value of the hand 53 when the occupant 50 holds the steering wheel 40), the reference position is obtained. The distance of the hand 53 with respect to the z coordinate position (direction perpendicular to the detection area plane) can be obtained. Since the position of the detection target can be determined based on the reference position and the reference luminance, an accurate position can be detected.

  Further, the position detection unit 21 can obtain the positional relationship between the occupant's arm and the occupant's hand based on the detected position of the arm, elbow or hand, and can obtain the direction of the occupant's hand therefrom.

  The shape detection unit 22 detects the shape of the detection target from the shape of a pixel having a predetermined or predetermined range of luminance values. Here, the shape of the detection target includes the shape of an occupant's arm, elbow, hand, or finger, and the shape of a hand such as “Par”, “Goo”, or “Cho” of Janken, or any one or more of five fingers. This includes various shapes that are expressed by the hand, such as a shape with the fingers extended and a circle with the index finger and thumb. The shape of the detection target may be determined by general pattern matching. The motion detection unit 23 detects the motion of the detection target based on the temporal change in the shape of the detection target detected by the shape detection unit 22.

  The “virtual switch generation means 4” has an operation switch display unit 41 and an air blowing unit 42, displays a virtual image operation switch in a space in the vehicle, and air blows out toward the virtual image of the displayed operation switch. Virtual switch to be generated. The virtual switch functions as an interface for each in-vehicle device 200-500.

  The operation switch display unit 41 displays a virtual image operation switch in a space in the vehicle. The operation switch display unit 41 of the present embodiment generates a three-dimensional virtual image operation switch in the vehicle interior space using a hologram. As shown in FIG. 2, the operation switch display unit 41 is provided in the center console of the vehicle, and generates a virtual image 4 </ b> A of the operation switch toward the passenger. An example of the virtual image of this operation switch is shown in FIG. The virtual image 4A of the operation switch shown in FIG. 4 includes an operation switch that raises and lowers “TEMP” that adjusts the temperature of the air conditioner control device 400, an operation switch that raises and lowers “Volume” that adjusts the volume of the audio device 300, and an audio device. And an operation switch for transitioning “Select” for selecting 300 channels or selecting playback contents.

  The operation switch display unit 41 includes a position determination unit 411 and a display control unit 412. The position determination unit 411 determines the display position of the virtual image of the operation switch according to the position of the detection target detected by the detection unit 2. Specifically, the positional relationship between the occupant's arm and the occupant's hand is detected, the direction of the operation switch is determined according to the direction of the occupant's hand, and the spatial position of the operation switch is determined according to the position of the occupant's hand. .

  FIG. 5 shows a display example of the operation switch. FIG. 5 is a top view of the driver 50 sitting in the driver's seat of the vehicle. The position determination unit 411 obtains the positional relationship between the detected arm 51, elbow 52, and hand 53 (detection target) of the driver 50, and the virtual image plane of the operation switch is transferred from the elbow 52 of the driver 50 to the hand 53. A virtual image of the operation switch is displayed at a predetermined angle with respect to the direction X. In the present embodiment, a virtual image of the operation switch is displayed so as to be substantially perpendicular to the direction of the hand of the driver 50 (direction X from the elbow 52 to the hand 53). The display control unit 412 displays the operation switch at the position determined by the position determination unit 411. The determined position is specified in a coordinate system as shown in FIG.

  The display control unit 412 starts or ends the display of the operation switch according to the shape or movement of the detection target detected by the detection unit 2. The display control unit 412 according to the present embodiment operates with a detection object (such as an occupant's arm, elbow, hand, or finger) having a predetermined shape or a detection object (such as an occupant's arm, elbow, hand, or finger) that exhibits a predetermined movement. Corresponding to the display start or display end of the switch, and when the hand shape or the like is detected as a predetermined shape or movement detection target, the display of the operation switch is started or ended. Specifically, the display control unit 412 starts the display of the operation switch when the “Janken” “par” is detected, and ends the display of the operation switch of the display control unit 412 when the “goo” of the Janken is detected. Of course, the instruction to start or end the display of the operation switch can be issued by hand movement instead of the hand shape. For example, the operation switch display may be started when the user presses the button with the index finger, and the display of the operation switch 4A may be terminated when the displayed operation switch 4A is moved away. . In this case, the gesture image pattern and the motion image pattern are acquired in advance, and the gesture and motion are determined by comparison with the reference pattern.

  In this embodiment, not only the driver 50 but also an operation by a passenger on the passenger seat is possible. In this case, the display position (direction, position) of the operation switch is determined according to the position of the passenger's hand. In order to detect the position of the passenger's passenger's hand, for example, an activation button of the virtual switch generating means 4 is provided on the armrest, and the assistant is based on the luminance value of the passenger's passenger's hand pressing the activation button. It is preferable to detect the position of the hand of the seat occupant. By pressing this button, the virtual image 4A of the operation switch can be displayed, and a command can be input from the passenger seat direction.

  The “air blowing part 42” blows air toward the virtual image of the operation switch displayed by the operation switch display part 41. The air outlet 421 is provided in the center console as shown in FIG. The air blowing unit 42 includes a blowing port 421 that blows out air and an air control unit 422. A virtual image of the operation switch is displayed in the interior space of the vehicle, and the occupant cannot obtain a physical perception even if the operation switch is operated. For this reason, even if the occupant operated the operation switch, there was no actual feeling and the operability was low. In this embodiment, the air blowing part 42 which blows air toward the operation switch of the displayed virtual image is provided. The air blown out from the air blowing portion 42 is perceived by the sense of touch of the driver's hand. The driver can confirm the input to the operation switch based on the physical perception of the air blown toward the operation switch. The air supplied from the air blowing section 42 has directivity and applies a predetermined pressure to the occupant's hand, so that the occupant can obtain a physical tactile sensation in the hand. Although not particularly limited, the air blowing section 42 of the present embodiment supplies air so that the pressure when the air collides with a detection target (driver's hand) is 0.1 MPa or less. The driver can operate the operation switch using the air supplied from the air blowing means as a clue.

  The air blowing unit 42 of the present embodiment takes in air used for air conditioning in the vehicle interior and blows out air toward the operation switch. Moreover, the air blowing part 42 has a blowing port 421 for blowing out air. The outlet 421 is provided at the end of the air flow path for taking in air used for air conditioning. The air flow path is narrowed as it approaches the blowout port 421, and air constricted from the blowout port 421 is blown out. The form of the outlet 421 is not particularly limited, and is appropriately designed according to the position of the operation switch. Preferably, the outlet 421 is narrowed so that air flows in a region narrower than a region indicating each operation switch.

  FIG. 6 shows a virtual image 4A of the operation switch in which the switches 1 to 3 generated by the virtual switch generating unit 4 are shown, and the air blowing portion 42. In this example, three air blowing portions 42 are provided. Air is blown out from the three outlets 421A to 421C of the air outlet 42, respectively. The air blowing port 421A blows air 42A1 toward the switch 1, the air blowing port 421B blows air 42A2 toward the switch 2, and the air blowing port 421C blows air 42A3 toward the switch 3. The occupant can accurately identify the switches 1 to 3 by perceiving three directional air flows.

  The air control unit 422 controls the timing of blowing air and the direction of air. The air control unit 422 acquires information on the display position of the operation switch determined by the position determination unit 411, and adjusts the blowing direction of the blowing port 421 so that air is blown toward the displayed operation switch. The air control unit 422 controls air blowing on / off in accordance with on / off of the operation switch display by the display control unit 412. That is, when the operation switch display is started, the air blowing is started, and when the operation switch display is ended, the air blowing is stopped. As a result, air can be blown out only when necessary, so that energy can be saved.

  The “command recognition unit 3” recognizes a command from the image of the operation switch displayed by the virtual switch generation unit 4 and the image of the detection target (occupant's arm, elbow, hand or finger) detected by the detection unit 2. To do. The command recognition unit 3 includes an image comparison unit 31 that compares the image of the operation switch and the image to be detected. The image comparison unit 31 compares the image of the operation switch with the image of the detection target, and the operation switch display unit 41 displays the position of the detection target (occupant's hand, finger, etc.) detected by the detection means 2. It is determined whether or not the position of the image of the operation switch coincides. In the present embodiment, whether or not the occupant has operated the operation switch is determined based on whether or not the occupant's hand (detection target) matches the position of the operation switch. That is, when the position of the occupant's hand (image) substantially coincides with the position of the operation switch M, it is determined that the occupant has operated the operation switch M, and it is determined that a command corresponding to the operation switch M has been input. In this embodiment, the operation switch is operated according to the position of the passenger's hand, but a command from the passenger may be recognized at the position of the tip of the passenger's finger.

  When the command recognition unit 3 recognizes the occupant's command, it is preferable to send a recognition completion signal to the virtual switch generation unit 4. The operation switch display unit 41 that has received the recognition completion signal changes the color of the operation switch, or the air blowing unit 42 that has received the recognition completion signal changes the air flow rate or pauses the air, and issues a command. Notify the occupant of the recognition.

  The “execution unit 5” sends the command recognized by the command recognition unit 3 to the corresponding external device (200 to 500) and executes the command.

  A control procedure of the command input device 100 of the present embodiment configured as described above will be described with reference to FIG. FIG. 7 is a flowchart at the time of command input of the command input device 100. In step S1, the infrared camera 11 starts imaging the detection area 11A. The detection area 11A includes steering, and images of arms, elbows, hands, fingers, and the like from the shoulders of the occupants who hold the steering to the fingertips are imaged (see FIGS. 2 and 3).

  In step S2, the detection means 2 detects a detection target. In this example, the position, brightness, shape or movement of the occupant's arm, elbow, hand or finger, or the direction of the arm or hand, or the state of the other occupant's arm or hand are detected as detection targets. The detection process of the detection target uses a generally used method.

  In step S3, the detection means 2 determines whether or not the passenger's hand is holding the steering wheel. If it is detected that the occupant is holding the steering wheel, the process proceeds to step S4. If not detected, the process returns to step S2.

  In step S4, the detection means 2 stores the reference position of the hand and the reference brightness value of the hand when the passenger's hand is holding the steering wheel. By performing these steps S2 to S4, the reference position and the reference brightness can be updated as needed, and the influence of the reference brightness shifting due to the gradual change of the hand temperature can be avoided.

  In step S5, the detection means 2 determines whether or not the detection target has a predetermined shape associated with the start command. In this example, it is determined whether or not the detected shape of the hand as the detection target is “par” of janken, and if it is detected that the shape of the hand is “par”, the process proceeds to step S6. On the other hand, if the start command is not expressed by the passenger, the process returns to step S2.

  In step S6, the detection means 2 obtains the two-dimensional position of the detection target along the detection area plane from the acquired infrared image. The detection means 2 detects the three-dimensional position in the direction perpendicular to the detection area surface by comparing the luminance value of the acquired infrared image with the reference luminance value acquired in step S4. Then, the detection means 2 detects the direction of the hand from the detected positional relationship (two-dimensional position, three-dimensional position) between the occupant's arm and the occupant's hand.

  In step S7, the virtual switch generation unit 4 determines the display position of the virtual image of the operation switch according to the position of the detection target detected by the detection unit. Specifically, the position determination unit 411 of the operation switch display unit 41 determines the display position of the virtual image of the operation switch based on the detected positional relationship between the passenger's arm and the passenger's hand. The display position includes the direction of the operation switch surface. Further, the air control unit 422 blows out air from the outlet 421 toward the operation switch displayed at the determined display position.

  In step S <b> 8, the display control unit 412 displays an operation switch at the position determined by the position determination unit 411.

  In step S9, the air control unit 422 determines the air blowing direction. The air blowing direction is a direction toward the operation switch displayed at the position determined in step S7.

  In step S10, the air control unit 422 starts air blowing in the direction determined in step S9.

  Thus, the virtual image operation switch and the virtual switch from which air is blown toward the operation switch are presented to the occupant (S11).

  In the subsequent step S12, the command recognition means 3 determines whether the position of the occupant's finger (image) detected by the detection means 2 substantially coincides with the position of any of the operation switches (images). It is determined whether or not the command has been recognized. On the other hand, if the command is not recognized (No in S12) and it is detected in step S18 that a fixed time has elapsed, the process automatically proceeds to step S16, the presentation of the virtual switch is terminated, and the process returns to step S1.

  In step 13, the command recognition means 3 notifies the occupant that the command has been recognized. Further, in step 14, the execution means 5 inputs the recognized command to the various in-vehicle devices 200 to 500.

  When the shape of the hand “goo” corresponding to the “end command” is detected, the virtual switch generation unit 4 ends the presentation of the virtual switch (display of the operation switch and air balloon) (step S16), and step S1. Return to.

  Since the command input device 100 of the present embodiment is configured and functions as described above, the following effects can be obtained.

  In the present embodiment, a detection region is imaged by the imaging means 1, a detection target (for example, any one or more of an occupant's arm, elbow, hand, or finger) is detected, an operation switch is displayed, and the displayed operation A virtual switch is generated in which air is blown toward the virtual image of the switch. For this reason, the blown air gives a physical perception to the occupant, and the occupant's hand can be guided to the operation switch. That is, the occupant can visually recognize the position of the operation switch with the virtual image operation switch and can also tactilely recognize the position of the operation switch with air. Thereby, a command input device with high operability for the passenger can be provided.

  According to this embodiment, since the display position of the virtual image of the operation switch is determined according to the position of the detection target, the virtual switch can be generated at a position (including the direction) that is easy for the occupant to operate. A high command input device 100 can be provided.

  Since the presentation of the virtual switch starts or ends when a detection object of a predetermined shape or a detection object that performs a predetermined movement is detected, the virtual switch can be presented only when a command is input, and the space inside the vehicle can be visually Can be extended to

  By using an infrared camera that does not require illumination means as the imaging means, it is possible to prevent the occupant's vision from being affected.

  In the present embodiment, the two-dimensional position of the detection target along the detection area plane is obtained from the infrared image acquired by the infrared camera, and the three-dimensional position of the detection target in the direction perpendicular to the detection area plane from the luminance value of the detection target. , The spatial position of the detection target (for example, an occupant's arm, elbow, hand, or finger) can be obtained.

  In this embodiment, since the position of the hand of the occupant holding the steering wheel is acquired as the reference position and the luminance of the hand of the occupant holding the steering wheel is acquired as the reference luminance, the detection target (for example, the arm, elbow, hand or finger of the passenger) An accurate position can be obtained.

  Based on the reference position and the reference luminance, the position (three-dimensional position) in the direction perpendicular to the imaging area plane can be acquired from the luminance value of the passenger's hand to be detected, and the accurate space of the passenger's hand can be obtained. The position can be acquired.

  Since the operation switch is displayed based on the three-dimensional position of the detection target (for example, an occupant's arm, elbow, hand or finger) obtained in this way, the operation switch can be displayed at a position where the operation is easy. Further, since the command can be recognized based on the three-dimensional position of the detection target (for example, an occupant's arm, elbow, hand, or finger), the command can be accurately recognized.

  Since the imaging means and / or the virtual switch generation means 4 are provided in the center console, the living space in the vehicle can be expanded.

  The embodiment described above is described in order to facilitate understanding of the present invention, and is not described in order to limit the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

It is a block block diagram of the command input device of this embodiment. It is a figure which shows the example of arrangement | positioning of an imaging means and a virtual switch production | generation means. It is a figure for demonstrating the process of a detection means. It is a figure which shows an example of an operation switch. It is a figure for demonstrating the determination process of the display position of an operation switch. It is a figure for demonstrating an air blowing process. It is a flowchart figure which shows the control procedure of this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Command input device 1 ... Imaging means 11 ... Infrared camera 2 ... Detection means 21 ... Position detection part 22 ... Shape detection part 23 ... Motion detection part 3 ... Command recognition means 31 ... Image comparison part 4 ... Virtual switch production | generation means 41 ... Operation switch display unit 411 ... position determination unit 412 ... display control unit 42 ... air blowing unit 421 ... blowing port 422 ... air control unit 5 ... execution means 200 ... navigation device 300 ... audio device 400 ... air conditioner control Device 500 ... Communication device

Claims (9)

A command input device that is mounted on a vehicle and receives an input of a command expressed by an occupant,
An infrared camera for imaging a predetermined detection area in the vehicle;
From the infrared image captured by the infrared camera, one or more of the position, shape, or movement of the detection target in the detection region is detected, and the detection region surface of the detection target based on the infrared image Detecting means for detecting a z coordinate position in a direction perpendicular to the detection area surface of the detection target based on a luminance value of the detection target ;
A virtual image operation switch is displayed at a display position in the interior space determined according to the xyz coordinate position of the detection target detected by the detection means, and air is blown toward the displayed virtual image operation switch. Virtual switch generation means for generating a virtual switch;
Command recognition means for recognizing the command from the image of the operation switch displayed by the virtual switch generation means and the image of the detection target detected by the detection means;
A command input device having execution means for executing the command recognized by the command recognition means;   The command input device according to claim 1, wherein the detection target is at least one of an occupant's arm, elbow, hand, and finger. The detection means detects a positional relationship between an occupant's arm, elbow, hand or finger included in the detection region;
3. The command input according to claim 1, wherein the virtual switch generation unit determines a display position of a virtual image of the operation switch based on a positional relationship between an occupant's arm and an occupant's hand detected by the detection unit. apparatus. The detection means detects the direction from the occupant's elbow to the hand based on the positional relationship between the occupant's arm and the occupant's hand,
The virtual switch generation unit forms a virtual image of the operation switch so that the virtual image plane is displayed substantially perpendicular to the direction from the elbow to the hand of the occupant detected by the detection unit. The command input device described.   The command input device according to claim 1, wherein the virtual switch generation unit starts or ends the generation of the virtual switch when the detection unit detects a detection target having a predetermined shape.   The command input device according to claim 1, wherein the virtual switch generation unit starts or ends generation of the virtual switch when the detection unit detects a detection target that performs a predetermined movement. The infrared camera captures an area including the steering of the vehicle, command input device according to claim 1 to obtain the luminance value of the occupant's hand gripping the steering as a reference luminance value. The command input device according to claim 7 , wherein the detection unit obtains the three-dimensional position of the occupant's hand from the luminance value of the occupant's hand to be detected based on the reference luminance value.   The command input device according to claim 1, wherein the imaging unit and / or the virtual switch generation unit is provided in a center console of the vehicle.

Images