JP2008210359A - Operation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an operation device capable of securing operability substantially same to that in reflecting type configuration, while adopting prompter type configuration. <P>SOLUTION: The operation device of the present invention is provided with an operation part 20 arranged with an operation switch, a display part 40 allowing naked-eye stereoscopic view, imaging means 50, 500 for imaging a hand of a user for operating the operation part, from a plurality of directions, and a stereoscopic view image generating means 76 for generating a stereoscopic view image of the hand, based on parallax images obtained from the imaging means, and combines an operation menu image for expressing an arrangement position of the operation switch in the operation part and a function of the operation switch, with the stereoscopic view image of the hand generated by the stereoscopic view image generating means, to be displayed on the display. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an operation device that superimposes and displays an image of a user's hand operating an operation unit on an operation menu image.

  Conventionally, a multi-function switch operation plate that is arranged at a suitable position in the vehicle and is arranged by integrating a plurality of types of switches for operating various on-vehicle devices, and a suitable position in the vehicle in a visual field area that can be viewed forward by the operator And a display for displaying an array image of the plurality of types of switches on the screen, and when the operator operates a target of the switch with a fingertip, the detection unit detects the target switch based on the detection signal. A controller that controls the operator to recognize the arrangement position and function of the image of the target switch displayed on the screen on the display, and the operator is displayed on the display in a forward view state. A vehicle characterized in that a target switch on the multi-function switch operation plate can be operated in real time while visually recognizing an image of the target switch. Device switch safe operation system is known (e.g., see Patent Document 1).

In addition, a touch panel type display in which a switch touched by the user is provided in the display screen, a first mirror means arranged with the reflecting surface facing the user, and a display image of the touch panel type display There is known an operation device including a reflection optical system having a second mirror unit that reflects the reflection surface of the first mirror unit as a reflected image (see, for example, Patent Document 2). In this operation device, the display screen of the touch panel display that is touch-operated by the user is displayed to the user via the reflective optical system by the first and second mirror means.
JP 2000-6687 A JP 2005-335510 A

  By the way, in the structure of the prompter system as disclosed in the above-mentioned Patent Document 1, the user's hand operating the operation unit is imaged with a camera, and the image of the hand is superimposed on the operation menu image displayed on the display unit. Therefore, so-called blind operation of the in-vehicle device is facilitated.

  However, in the configuration of the prompter system as disclosed in Patent Document 1 described above, since the hand image displayed on the display unit is generated based on the image data from one camera, only two-dimensional display can be performed. . For this reason, since the user operates the operation switch of the operation unit while viewing the hand image that looks planar like the operation menu image, it is difficult to grasp the sense of distance between the operation switch and the hand, and the operability is not good. There is.

  On the other hand, in the reflection type configuration disclosed in the above-described Patent Document 2 (a configuration in which the user's hand operating the operation unit is shown to the user using a mirror), the user's eyes are three-dimensionally. Since it is reflected, it is easy to grasp the sense of distance between the operation switch and the hand, and the operability is very good.

  In view of the above, an object of the present invention is to provide an operating device that can ensure operability substantially equal to that of a reflective configuration while adopting a prompter configuration.

In order to achieve the above object, an operating device according to a first aspect of the present invention includes an operation unit in which an operation switch is disposed,
An indicator capable of autostereoscopic viewing,
Imaging means for photographing a user's hand operating the operation unit from a plurality of directions;
A stereoscopic image generation unit that generates a stereoscopic image of the hand based on a parallax image obtained from the imaging unit;
On the display unit, the operation menu image representing the position of the operation switch and the function of the operation switch in the operation unit and the stereoscopic image of the hand generated by the stereoscopic image generation unit are combined and displayed. It is characterized by.

According to a second aspect of the present invention, in the operating device according to the first aspect,
The operation device according to claim 1, wherein the operation unit and the indicator are arranged at a position between the driver seat and the passenger seat in the vehicle width direction in the vehicle interior.
The imaging means includes at least two imaging means for looking down the operation unit obliquely downward from the driver seat side in the vehicle width direction,
The operation device, wherein the stereoscopic image generation unit generates the stereoscopic image of the hand in a line-of-sight direction from the driver's seat based on parallax images obtained from the two imaging units.

According to a third aspect of the present invention, in the operating device according to the first aspect,
The operation unit and the indicator are arranged at a position between the driver seat and the passenger seat in the vehicle width direction in the vehicle interior,
An operator discriminating means for discriminating whether the operation unit is operated from the driver's seat side or the passenger seat side;
The stereoscopic image generation means generates the stereoscopic image of the hand in the line-of-sight direction from the driver seat side when the operator determination means determines that the operation unit is operated from the driver seat side. When the operator determining unit determines that the operation unit is operated from the passenger seat side, the stereoscopic image of the hand is generated in the line-of-sight direction from the passenger seat side. .

4th invention is the operating device which concerns on 3rd invention,
The imaging means includes three or more imaging means spaced apart from each other in the vehicle width direction,
The stereoscopic image generation means is based on parallax images obtained from two imaging means on the driver seat side when the operator determination means determines that the operation unit is operated from the driver seat side. When the operator discriminating unit determines that the operation unit is operated from the passenger seat side, the stereoscopic image of the hand is generated and obtained from the two imaging units on the passenger seat side. The stereoscopic image of the hand is generated based on the parallax image.

According to a fifth aspect of the present invention, in the operating device according to the third aspect,
The imaging means includes two imaging means movable in the vehicle width direction,
The stereoscopic image generation means is a parallax image obtained from two imaging means moved to the driver seat side when the operator determination means determines that the operation unit is operated from the driver seat side. The stereoscopic image of the hand is generated on the basis of the two images, and when the operation determining unit determines that the operation unit is operated from the passenger seat side, the two images moved to the passenger seat side are generated. The hand stereoscopic image is generated based on a parallax image obtained from the means.

A sixth aspect of the present invention is the operating device according to the first aspect,
On the display device, an image portion representing the operation switch in the operation menu image is displayed in a stereoscopic view, and a surface obtained by offsetting the stereoscopic image of the hand by a predetermined height from the display surface of the display device has a height of zero. It is displayed as a reference plane.

7th invention is the operating device which concerns on 6th invention,
An image portion representing an operation switch in the operation menu image is displayed in a stereoscopic view with a display surface of the display as a reference surface having a height of zero,
The predetermined height to be offset is substantially equal to the stereoscopic height of the image portion representing the operation switch.

An eighth invention is the operating device according to the sixth or seventh invention,
The height of the finger from the display surface of the display in the stereoscopic image of the hand displayed when the finger of the hand touches the operation unit is the stereoscopic view of the image portion representing the operation switch in the operation menu image. It is characterized by substantially coincident with the height of.

  According to the present invention, it is possible to obtain an operating device that can ensure operability substantially equal to that of a reflective configuration while adopting a prompter configuration.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a perspective view showing Example 1 of the operating device 10 according to the present invention mounted on a vehicle. FIG. 2 is a schematic cross-sectional view of the operating device 10 shown in FIG. 1 as viewed from the side, and also shows functional blocks. As shown in FIG. 1, the operating device 10 of the present embodiment includes an operating unit 20, a display unit 40, and a camera 50, and the operating unit 20 and the display unit 40 are separately located at physically separated positions in the vehicle interior. Be placed.

  The operation unit 20 is configured by a touch panel. The operation unit 20 is disposed at a position where the driver can easily operate by reaching out. For example, the operation unit 20 is arranged in a console box as shown in FIG. Or the operation part 20 may be arrange | positioned in the near side rather than the display part 40 in an instrument panel. The operation unit 20 is arranged such that its substantially flat operation surface is substantially horizontal.

  The operation unit 20 may have a normal touch panel configuration, but preferably includes a transparent touch panel capable of emitting light. The transparent touch panel has, for example, a laminated structure in which a thin film transparent electrode is opposed to each other through a dot spacer with a glass substrate provided with a thin film transparent electrode facing down and a film substrate provided with a thin film transparent electrode facing up. An operation signal (operation position signal) on the transparent touch panel is supplied to the display control unit 80 (see FIG. 2) via an FPC (flexible printed circuit) bonded to the side of the transparent touch panel. The display control unit 80 realizes various functions (switching of operation menu images described later and control of various in-vehicle devices 30) according to an operation signal (operation position signal) on the transparent touch panel. The light emission of the transparent touch panel may be realized by, for example, an LED that causes the transparent touch panel to emit blue light. Note that it is not necessary to provide a display such as a touch panel display on the back side of the transparent touch panel of the operation unit 20 (that is, it is not necessary to display the operation switch in the operation unit 20 on the operation unit 20).

  The display unit 40 is configured by a display capable of autostereoscopic viewing. The display unit 40 capable of autostereoscopic viewing may be realized by using a parallax division method (binocular convergence method) of the line of sight such as a parallax barrier (parallax barrier) method, a lenticular method, or the like. Depth Fused 3D) method may be used.

  In the case of the parallax barrier method, as shown in the principle diagram shown in FIG. 3A, the display unit 40 alternates left-eye pixels in the horizontal direction of a pixel column (for example, a pixel column composed of TFT liquid crystals). The region 41L and the right-eye pixel region 41R are set, and the light from the left-eye pixel region 41L reaches only the left eye and the light from the right-eye pixel region 41R is caused by the parallax barrier (vertical slit) 42. Configured to reach only the right eye. The parallax barrier 42 may be configured by a switch liquid crystal that can be electronically controlled. Further, the parallax barrier 42 may be set behind the pixel row and on the near side of the light source.

  In the case of the lenticular method, the display unit 40 has left-eye pixel areas 41L and right-eye pixel areas 41R alternately set in the horizontal direction of the screen as shown in the principle diagram of FIG. The kamaboko-shaped lenticular lens 44 is configured such that light from the left-eye pixel region 41L reaches only the left eye and light from the right-eye pixel region 41R reaches only the right eye. Thus, in the parallax division method, it is possible to give a sense of depth by projecting different images to the left and right eyes of a human.

  In the case of the DFD method, the display unit 40 is configured by superimposing two transparent TFT liquid crystals on the front and rear sides at an appropriate interval. The same image with different brightness is displayed on two transparent TFT LCDs. As described above, in the DFD method, a continuous sense of depth can be obtained between the two surfaces by changing the luminance ratio of the same image overlapped in the front and rear.

  The display unit 40 is arranged at a position that is easy for the user to see, preferably at a position where the driver can see without greatly changing the field of view during driving. For example, the display part 40 is arrange | positioned in the center part of the upper surface of an instrument panel, as shown in FIG. Or the display part 40 may be arrange | positioned in a meter. In addition, the display unit 40 may be installed with an angle in the driver direction so that a stereoscopic effect can be easily obtained.

  As shown in FIG. 1, the camera 50 (imaging means) is a stereo camera (two-lens camera) including two cameras 51 and 52 that are arranged apart from each other in the vehicle width direction. Hereinafter, if necessary, the camera 51 on the passenger seat side (left side in the case of a right-hand drive vehicle) in the vehicle width direction is also referred to as a “left-eye camera 51”, and the driver seat side (right handle) in the vehicle width direction. The right camera 52 in the case of a car is also referred to as a “right-eye camera 52”. Each of the cameras 51 and 52 is a small color camera using, for example, a CCD or a CMOS as an image sensor, and is arranged so that the operation unit 20 (and the user's hand operating it) can be seen from different directions. In the illustrated example, the left-eye camera 51 is disposed on the passenger seat side with respect to the center of the operation unit 20 in the vehicle width direction, and is disposed so as to capture the entire operation unit 20 obliquely downward from the passenger seat side. The right-eye camera 52 is disposed closer to the driver's seat than the center of the operation unit 20 in the vehicle width direction, and is disposed so as to capture the entire operation unit 20 obliquely downward from the driver's seat side. Thus, each camera 51 and 52 is arrange | positioned so that the operation part 20 may be imaged with the parallax and convergence angle corresponding to the parallax and convergence angle when a user looks at the display part 40. FIG. As a result, the stereoscopic information of the hand of the user who operates the operation unit 20 can be acquired by the camera 50.

  When the user operates the operation unit 20, the user's hand enters the imaging area of each camera 51, 52 in order to operate the operation unit 20, and the hand operating the operation unit 20 is imaged by each camera 51, 52. The camera image including the hand image is supplied to the display image generation circuit 70 (see FIG. 2).

  The display image generation circuit 70 generates an image suitable for display on the display unit 40 capable of autostereoscopic viewing under the control of the display control unit 80.

  As shown in FIG. 2, the display image generation circuit 70 includes a background image generation circuit 71, a left eye image synthesis circuit 72, a right eye image synthesis circuit 74, and a stereoscopic image generation circuit 76.

  The background image generation circuit 71 generates a background image displayed on the display unit 40. Here, the background image is an operation menu image as shown in FIG. In addition, the background image may include a map image of the navigation system. The operation menu image represents the position of the operation switch (touch switch) of the operation unit 20 and its function. The operation menu image serves to notify the user of various functions realized by operations on the operation switches and to notify the user of the position of each operation switch to be operated to realize the various functions.

In the example illustrated in FIG. 4, the operation menu image includes graphic images F <b> 1 to F <b> 8 representing eight operation switches in the operation unit 20. The user who sees the operation menu image knows that the operation switches are arranged in two rows in the front and rear, and also has a function that can be realized by operating each operation switch by looking at the characters in the graphic images F1 to F8. I can know. For example, if a user wants to email, the user can enter “email”
It can be understood that the operation switch located at the position corresponding to the graphic image F3 including the character, that is, the third operation switch from the left in the front row may be pressed.

  Various operation menu images may be prepared, and may be switched as appropriate according to the user's operation status with respect to the operation switch of the operation unit 20. In this case, the position and function of the operation switch of the operation unit 20 are changed according to the switching of the operation menu image. With such a configuration, the operation switches of many in-vehicle devices 30 can be realized in the operation unit 20, and the operation switches can be efficiently integrated. For example, although not shown in FIG. 4, the operation unit 20 includes various operations for various operations of an air conditioner and an audio device as well as operations of in-vehicle devices for information communication systems such as e-mails, telephones, and peripheral facility guidance. An operation switch may be realized.

  The left-eye image synthesis circuit 72 generates a left-eye image based on the operation menu image from the background image generation circuit 71 and the camera image from the left-eye camera 51. Specifically, the left-eye image composition circuit 72 first extracts only the hand image (hereinafter referred to as “hand image”) from the camera image from the left-eye camera 51. The extraction of the hand image may be realized using a portion other than the hand (operation unit 20) included in the camera image and a chromaticity (saturation) difference or contrast difference of the hand. Typically, a chroma key synthesis technique that is widely known per se is used. This is a technique in which an image is shot against a background of a specific color (key color), and another image is superimposed on the key color for composition. A color generally used as a key color is blue. This is because blue is the most opposite to human skin color and is suitable for processing to extract human hands. Accordingly, when the operation unit 20 is colored with a key color (for example, blue) or light is emitted with the key color, and the blue portion is extracted with the chroma key in the camera image, only the hand image remains. It should be noted that other methods for hand image extraction, such as luminance synthesis technology (designating a luminance signal in a luminance signal (Luminance including information on luminance and contrast of an image), and extracting only the signal below or above that signal are extracted. In the following, the case where the chroma key composition technique is mainly used will be described.

  When the left-eye image composition circuit 72 extracts the hand image, the left-eye image composition circuit 72 composes the extracted hand image with the operation menu image from the background image generation circuit 71 to generate the left-eye image. The left-eye image generated in this way is supplied to the stereoscopic image generation circuit 76.

  Similarly, the right-eye image synthesis circuit 74 generates a right-eye image based on the operation menu image from the background image generation circuit 71 and the camera image from the right-eye camera 52. The right eye image composition circuit 74 generates an operation menu image common to the left eye image composition circuit 72. The right-eye image generated in this way is supplied to the stereoscopic image generation circuit 76.

  The stereoscopic image generation circuit 76 is a display unit 40 capable of autostereoscopic viewing based on the left-eye image from the left-eye image synthesis circuit 72 and the right-eye image from the right-eye image synthesis circuit 74. A stereoscopic image suitable for the above is generated. At this time, the stereoscopic image generation circuit 76 generates a stereoscopic image so that the operation menu image is visible to the user's eye on a two-dimensional plane and the hand image is visible to the user's eye on a three-dimensional solid. . That is, the stereoscopic image generation circuit 76 uses the display surface of the display unit 40 as a reference plane having a height of zero, and the height and direction of each finger of the hand image with respect to the operation menu image corresponds to the operation at each position of the actual hand. A stereoscopic image is generated so that a stereoscopic view corresponding to the height and direction with respect to the unit 20 is obtained. The stereoscopic image generated by the stereoscopic image generation circuit 76 is displayed on the display unit 40.

  When the display unit 40 uses the parallax division method, the stereoscopic image generation circuit 76 divides the left-eye image from the left-eye image synthesis circuit 72 at a predetermined interval, and displays the divided image pieces as the display unit. The right-eye image from the right-eye image synthesis circuit 74 is divided at predetermined intervals, and the divided image pieces are assigned to the respective right parts of the display unit 40. This is assigned to the eye image pixel area 41R. In this way, the stereoscopic image generation circuit 76 makes the left eye image appear only in the user's left eye (the light in the left eye image pixel region on the display unit 40 reaches only the user's left eye. So that the right-eye image appears only to the right eye of the user (so that the light in the right-eye image pixel area on the display unit 40 reaches only the right eye of the user). A visual image is generated. As a result, only the hand image obtained by the left-eye camera 51 is shown in the user's left eye, and only the hand image obtained by the right-eye camera 52 is shown in the user's right eye. By dividing the line of sight by the parallax of two hand images, the image displayed on the display unit 40 (the hand portion in the stereoscopic image) can be shown to the user as if it were stereoscopic.

  When the display unit 40 is a DFD method, the stereoscopic image generation circuit 76 converts the left eye image from the left eye image synthesis circuit 72 and the right eye image from the right eye image synthesis circuit 74. Based on this, a display hand image to be displayed on the two front and rear displays is generated, and information regarding the height of each position of the hand relative to the operation unit 20 in the display hand image is acquired. Next, the stereoscopic image generation circuit 76 determines the luminance of each pixel of the display hand image based on the height information. In this way, the stereoscopic image generation circuit 76 generates front and rear display hand images (stereoscopic images) having a luminance difference corresponding to the height of the respective positions of the hand with respect to the operation unit 20. Thereby, it is possible to make the user's eyes appear as if the image displayed on the display unit 40 (the hand portion in the stereoscopic image) is stereoscopic.

  FIG. 5 shows a plurality of examples of stereoscopic images. Hereinafter, the image of the hand portion in the stereoscopic image is referred to as a “stereoscopic hand image”. In the drawing, since it is impossible to show the stereoscopic hand image stereoscopically, a planar image is shown. However, actually, the stereoscopic hand image displayed on the display unit 40 is It looks three-dimensional as described above.

  Note that the composition position of the stereoscopic hand image, that is, the position in the operation menu image, is determined by the relative positional relationship between the actual hand position in the operation unit 20 and the position of each operation switch. The position of the hand of the hand image in the image) and the relative positional relationship between the graphic images F1 to F8 are determined so as to be accurately reproduced. This composite position may be calculated based on the relative relationship (coordinate conversion formula) between the coordinate system in the camera image and the coordinate system in the operation menu image.

  In the example shown in FIG. 5, the tip of the index finger of the stereoscopic hand image is shown at the position where the graphic image F3 is operated. In this case, the user can know that the switch function corresponding to the graphic image F3, that is, the mail function is realized by performing the switch operation with the index finger at the current hand position. In this way, the user can operate a desired operation switch in the operation unit 20 while looking at the display unit 40 without directly viewing the operation unit 20. Therefore, when the user is a driver, a switch operation (so-called blind touch operation) can be performed with the operation unit 20 near the hand while looking at the display unit 40 in front of the field of view without greatly changing the driving posture or line of sight. Therefore, it is possible to realize a safe switch operation that does not interfere with driving.

  In particular, in the present embodiment, the hand image superimposed on the operation menu image is seen as a three-dimensional image as described above, so that the operability is greatly improved as compared with the case where the hand image looks planar. To do. That is, in the conventional prompter method using a single camera shown in FIG. 6A, the hand image superimposed on the operation menu image is 3D information of the hand (stereo information on the hand and fingertip). Since it is lost and becomes a composite image on a plane, the information obtained from the image of the hand is, for example, a reflection-type configuration as shown in FIG. In other words, the operability is reduced by that amount. On the other hand, in this embodiment, by generating a stereoscopic hand image that is substantially equivalent to the reflected image of the hand that can be seen by the reflection method described above, the hand lost by the conventional prompter method is adopted while adopting the prompter method. 3D information (stereo information at hand and fingertip) can be recovered. For this reason, according to the present embodiment, it is possible to realize high operability equivalent to the reflection type while adopting the prompter method. Further, in the present embodiment, a hand image is generated as compared with the conventional prompter method in which a planar hand image is superimposed by generating a stereoscopic hand image that is substantially equivalent to a reflection image of a hand that is seen as a reflection type as described above. As a result, the area of the operation menu image that is concealed is reduced, so that the positional relationship between the operation menu image and the hand can be easily grasped, and the operability is improved.

  In this embodiment, as shown in FIG. 2, the angle β formed by the operation unit 20 with respect to the line-of-sight direction of the camera 50 is set to be the same or substantially the same as the angle α formed by the display unit 40 with respect to the user's line-of-sight direction. Therefore, in a side view, the line-of-sight direction with respect to the hand of the camera 50 substantially corresponds to the line-of-sight direction with respect to the display unit 40 of the user (as a result, the stereoscopic direction of the stereoscopic hand image on the display unit 40). To improve. In addition, according to such an angular relationship, a stereoscopic hand that is substantially equivalent to the reflected image of the hand that is seen by the above-described reflection type without requiring special processing such as a line-of-sight conversion process when generating the stereoscopic hand image. An image can be generated.

  Here, the stereoscopic hand image is preferably generated by directly using the raw image of the hand as shown in FIG. 5A, but the freshness of the hand as shown in FIG. 5B. This may be a silhouette-type stereoscopic hand image that eliminates the above, or a semi-transparent silhouette-type stereoscopic hand image that allows the operation menu image to be seen behind to be seen as shown in FIG. Alternatively, as shown in FIG. 5D, a stereoscopic hand image obtained by processing a raw image of a hand to be translucent may be used. According to the stereoscopic hand image shown in FIGS. 5B and 5C, it is possible to prevent a physiological discomfort that can be received by the user due to realistic display of hand details. In addition, according to the stereoscopic hand image shown in FIGS. 5C and 5D, the area of the operation menu image hidden by the hand image is further reduced to grasp the positional relationship between the operation menu image and the hand. Can be made easier. However, in FIGS. 5A and 5B, the area for concealing the menu image due to the stereoscopic parallax in the stereo camera 50 is reduced as compared with the conventional prompter system that displays the two-dimensional hand image. This improves the visibility.

  The following modifications can be considered for the first embodiment described above.

  For example, in the above-described first embodiment, as shown in FIG. 5A, a stereoscopic hand image is generated in a direct view, so that either the driver or the passenger on the passenger seat can operate the operation menu. Although it is easy to grasp the positional relationship between the image and the height direction of the hand (straight direction of the screen), for example, emphasizing that the operability of the driver is particularly important, the two cameras 51 and 52 are either Alternatively, it may be arranged on the driver's seat side to generate a stereoscopic hand image (for example, see FIG. 9 described later) viewed from the driver's seat side.

  FIG. 7 is a perspective view showing Example 2 of the operating device 100 according to the present invention mounted in a vehicle. FIG. 8 is a functional block diagram showing main functions of the operating device 100 shown in FIG. In the second embodiment, components that may be the same as those in the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  The second embodiment is mainly characterized in that the camera image used for generating the stereoscopic hand image is switched depending on whether the operation unit 20 is operated from the driver seat side or the passenger seat side. Have. In the following, the configuration unique to the second embodiment will be described mainly.

  As shown in FIG. 7, the camera 500 is a three-lens camera including three cameras 501, 502, and 503 that are spaced apart from each other along the vehicle width direction. Accordingly, each of the cameras 501, 502, and 503 is a small color camera using, for example, a CCD or CMOS as an image sensor so that the operation unit 20 (and the user's hand operating it) can be seen from different directions. Be placed. In the illustrated example, the camera 501 is disposed closer to the passenger seat side than the center of the operation unit 20 in the vehicle width direction, the camera 502 is disposed near the center of the operation unit 20 in the vehicle width direction, and the camera 503 is It is arranged closer to the driver's seat than the center of the portion 20 in the vehicle width direction. Each of the cameras 501, 502, and 503 is disposed so as to capture the entire operation unit 20 with a line of sight obliquely downward from the front side of the vehicle. In this way, the pair of cameras 501 and 502 are arranged so as to capture the operation unit 20 with the parallax and the convergence angle corresponding to the parallax and the convergence angle when the passenger on the passenger side looks at the display unit 40. The cameras 502 and 503 are arranged so as to capture the operation unit 20 with a parallax and a convergence angle corresponding to the parallax and the convergence angle when the driver views the display unit 40.

  The operating device 100 includes an operator determination unit 90 and a camera switching unit 92 as additional components to the configuration in the first embodiment.

  The operator determination unit 90 determines whether the operation unit 20 is operated from the driver seat side or the passenger seat side. There are various types of determination methods of this type, and any appropriate method may be adopted. For example, as a simple method, in the case of a right-hand drive vehicle, the operator determination unit 90 operates the operation unit 20 from the driver seat side when the hand in the camera image captured by the camera 500 is the left hand. If the hand in the camera image captured by the camera 500 is the right hand, the operation unit 20 is operated from the assistant side (ie, the operator). May be a passenger in the passenger seat). Alternatively, infrared sensors are installed on both sides of the operation unit 20 in the vehicle width direction (passenger seat side and driver seat side), and it is determined by which side infrared sensor the presence of the hand extending to the operation unit 20 is detected. May be. Alternatively, for example, a switch that is turned on when operating from the passenger seat side may be provided in the passenger compartment, and the operator may be determined based on the on / off state of the switch, or an image by the in-vehicle camera The operator may be determined based on the recognition result, the change in capacitance between the passenger seat and driver seat, and the operation unit 20.

  The camera switching unit 92 switches the connection state of the cameras 501, 502, and 503 with respect to the left-eye image composition circuit 72 and the right-eye image composition circuit 74 according to the determination result from the operator determination unit 90. Specifically, when the operation unit 20 is operated from the driver's seat side, the camera switching unit 92 sends the camera image from the camera 502 to the left-eye image synthesis circuit 72 as shown by a solid line in FIG. And the camera image from the camera 503 is input to the image synthesis circuit 74 for the right eye. On the other hand, when the operation unit 20 is operated from the assistant side, the camera switching unit 92 inputs the camera image from the camera 501 to the image synthesis circuit 72 for the left eye, as indicated by a dotted line in FIG. In addition, the camera image from the camera 502 is input to the image synthesis circuit 74 for the right eye.

  FIG. 9 is a diagram illustrating a stereoscopic image output when the operation unit 20 is operated from the driver seat side as an example.

  When the operation unit 20 is operated from the driver's seat side, as described above, the stereoscopic hand image is based on the camera images of the two cameras 502 and 503 that look down on the operation unit 20 obliquely downward from the driver's seat side. Therefore, the stereoscopic hand image becomes a stereoscopic view of the hand viewed from the driver's seat side, as shown in FIG. Similarly, although not shown, a stereoscopic hand image generated and output when the operation unit 20 is operated from the passenger side is two cameras 501 and 502 that look down the operation unit 20 obliquely downward from the passenger seat side. Therefore, the stereoscopic view of the hand viewed from the passenger seat side is obtained. Thus, according to the second embodiment, since the stereoscopic hand image is generated by the stereoscopic view viewed from the operator side, the above-described stereoscopic hand image (see FIG. 5A and the like) is generated by the direct view. Compared to the first embodiment, it becomes easier to grasp the up / down state of the finger with respect to the operation unit 20 (and consequently the up / down state of the finger with respect to the operation menu image), and the operability is further improved. That is, according to the second embodiment, the stereoscopic hand image is displayed in a stereoscopic view corresponding to the direction of the line of sight from the operator side, so that the operator's hand extends to the display unit 40 in the eyes of the operator. The operability is further improved.

  In the second embodiment, a stereoscopic hand image may be generated in another form as shown in FIGS. 5B to 5D in relation to the first embodiment.

  The following modifications and improvements can be considered for the second embodiment described above.

  For example, in the above-described second embodiment, three cameras 501, 502, and 503 are used, but two cameras may be set on each of the driver seat side and the passenger seat side. In this case, when the operation unit 20 is operated from the driver's seat side, a stereoscopic image is generated using two cameras on the driver's seat side (two cameras separated in the vehicle width direction), and the operation unit 20 Is operated from the passenger seat side, the stereoscopic image may be generated using two cameras on the passenger seat side (two cameras separated in the vehicle width direction).

  In the second embodiment, the three cameras 501, 502, and 503 are used. As shown in FIGS. 10A and 10B, a stereo camera 600 (movable in the vehicle width direction) ( Cameras 602 and 603) may be set. In this case, the stereo camera 600 may be movably provided on a slide rail 608 along the vehicle width direction, for example, and may be driven using an appropriate actuator 610 such as a motor as a drive source. In such a configuration, when the operation unit 20 is operated from the driver's seat side, as shown in FIG. 10A, a stereoscopic image is generated using the stereo camera 600 moved to the driver's seat side, When the operation unit 20 is operated from the passenger seat side, a stereoscopic image may be generated using the stereo camera 600 moved to the passenger seat side as shown in FIG.

  In addition, when the display unit 40 is configured by a liquid crystal parallax barrier 3D display, normally, as shown in FIG. 11A, a centrally divided dual view state is formed, and different images are displayed on the left and right screens. (Driver seat image and passenger seat image) may be displayed, and when an operation is performed on the operation unit 20, as shown in FIG. 11B, it is viewed from the operator (in this case, the driver) side. A stereoscopic image (left-eye image + right-eye image) may be generated and output.

  The third embodiment is mainly different from the first and second embodiments described above in that the graphic images F1 to F8 representing the operation switches in the operation menu image are also displayed in a stereoscopic view. Constituent elements that may be the same as those in the first embodiment are given the same reference numerals, and descriptions thereof are omitted.

  FIG. 12 is a schematic cross-sectional view of the operating device 200 according to the third embodiment as viewed from the side.

  The background image generation circuit 710 generates a background image (in this case, an operation menu image) displayed on the display unit 40, and sends the generated background image to the left-eye image synthesis circuit 72 and the right-eye image synthesis circuit 74. Supply. In the present embodiment, the background image generation circuit 710 generates graphic images F1 to F8 representing operation switches in the operation menu image as stereoscopic parallax display images as shown in FIG.

  The stereoscopic image generation circuit 760 is a display unit 40 capable of autostereoscopic viewing based on the left-eye image from the left-eye image synthesis circuit 72 and the right-eye image from the right-eye image synthesis circuit 74. A stereoscopic image suitable for the above is generated. At this time, in the stereoscopic image generation circuit 760, a portion of the operation menu image excluding the graphic images F1 to F8 is visible to the user on a two-dimensional plane, and the hand images and the graphic images F1 to F8 of the operation menu image are displayed. A stereoscopic image is generated so as to be seen by the user in a three-dimensional solid. In the present embodiment, the stereoscopic image generation circuit 760 uses each surface of the hand image of the hand image with respect to the reference surface as a reference surface having a height of zero (A) that is offset from the display surface of the display unit 40 by a predetermined height A [mm]. A portion of the hand image of the stereoscopic image is generated such that the height of the stereoscopic view corresponds to the height of the actual hand position relative to the operation unit 20 at each position. In addition, the stereoscopic image generation circuit 760 uses the display surface of the display unit 40 as a reference surface having a height of zero, and the upper surface 140 (see FIG. 13) of the graphic images F1 to F8 has a predetermined height B [ mm], the graphic image portions F1 to F8 of the stereoscopic image are generated. The stereoscopic image generated by the stereoscopic image generation circuit 760 is displayed on the display unit 40.

  FIG. 13 is a diagram illustrating an example of graphic image portions F1 to F8 of the operation menu image displayed on the display unit 40. As shown in FIG. 13, the graphic images F1 to F8 of the operation menu image are stereoscopically displayed (convex) as if the upper surface 140 is on the near side from the display surface of the display unit 40 by a predetermined height B [mm]. Displayed). Hereinafter, the graphic images F1 to F8 of the operation menu image displayed stereoscopically in this way are referred to as “stereoscopic switch images”. Note that the graphic image (in this example, the graphic image F3) for the operation switch that cannot be operated during travel due to travel restrictions or the like may be displayed two-dimensionally as shown in FIG. Accordingly, the user can easily grasp the operation switch that cannot be operated at present.

  FIG. 14 is a diagram schematically illustrating the A view of FIG. 13, and FIG. 14A illustrates a state in which the operation switches corresponding to the stereoscopic switch images F1 and F5 in the operation unit 20 are not operated. FIG. 14A shows a state in which the operation switch corresponding to the stereoscopic switch image F1 in the operation unit 20 is operated and turned on. The stereoscopic image generation circuit 760 responds to an operation switch when the operation switch in the operation unit 20 is operated based on an operation signal (operation position signal) on the transparent touch panel, as shown in FIG. The height of the upper surface 140 of the stereoscopic switch image to be performed (stereoscopic switch image F1 in this example) with respect to the reference surface (display surface of the display unit 40) is changed to B ′ (<B) [mm]. Thus, the user can visually confirm that the operation device 200 has detected an operation on the operation switch.

  FIG. 15 is a diagram schematically illustrating a relative height relationship between the stereoscopic switch image and the stereoscopic hand image. FIG. 15A illustrates the user's hand and the operation surface (touch panel surface) in the operation unit 20. ) And FIG. 15B shows the height relationship between the stereoscopic switch image and the stereoscopic hand image in the display unit 40 corresponding to the state of FIG.

  As shown in FIG. 15A, when the fingertip of the user's hand comes to a position where it touches the operation surface (that is, when it reaches the zero position of the y coordinate in the figure), as shown in FIG. The fingertip G of the sight image is seen at a position of a predetermined height A (height along the Y axis in the figure) with respect to the display surface of the display unit 40. At this time, if the predetermined height A is set to coincide with the height B of the upper surface 140 of the stereoscopic switch image (that is, A = B), as shown in FIG. The stereoscopic display is such that the fingertip G of the hand image touches the upper surface 140 of the stereoscopic switch image. That is, the stereoscopic display is such that the fingertip G of the stereoscopic hand image is placed on the upper surface 140 of the stereoscopic switch image. Thereby, since the actual contact state of the operation unit 20 with the touch panel matches the state of the stereoscopic display on the display unit 40, the operability for the user is improved.

  When the user presses the operation surface (switch operation) from the state shown in FIG. 15A, the operation switch of the operation unit 20 is turned on, and the mechanical switch is pushed as if the stereoscopic switch image is as described above. Stereoscopic display (see alternate long and short dash lines). Thereby, the user can feel a realistic operation feeling as if he / she operated an actual hard switch.

  When the user's hand moves away from the operation surface from the state shown in FIG. 15A (for example, when the user moves away to the height h in the y coordinate), the fingertip G of the stereoscopic hand image is displayed on the display unit. It is at a position of height H (height along the Y axis in the figure) with respect to 40 display surfaces. The height H may be h + A, or may be h ′ + A depending on the scaling factor of the stereoscopic image. However, h ′ = k · h (k is a constant).

  Thus, according to the third embodiment, as described above, the stereoscopic switch image is displayed and the stereoscopic hand image is displayed with an offset corresponding to the height of the stereoscopic switch image. The stereoscopic switch image and the stereoscopic switch image can be displayed with an appropriate height relationship. This makes it easier for the user to grasp the sense of distance of the hand with respect to the operation switch of the operation unit 20, and allows the user to feel a realistic operational feeling, improving operability.

  The following modifications and improvements can be considered for the third embodiment described above.

  For example, although the case where the predetermined height A is set to coincide with the height B of the upper surface 140 of the stereoscopic switch image has been described, the predetermined height A is equal to the height B of the upper surface 140 of the stereoscopic switch image. It suffices if they substantially match. For example, as the predetermined height A, an appropriate value within the range of the height B ′ or more of the upper surface 140 of the stereoscopic switch image in the operation state and not more than the height B may be selected. For example, the predetermined height A may be A = (B−B ′) / 2 + B ′. Alternatively, the predetermined height A may be a value slightly larger than the height B of the upper surface 140 of the stereoscopic switch image.

  Further, when the display unit 40 is configured by a 3D display capable of concave display, when the user operates the operation switch, the stereoscopic switch image corresponding to the operation switch may be a concave stereoscopic display (that is, , B ′ may be a negative value).

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, the circuit configuration of the display image generation circuit 70 shown in FIGS. 2 and 8 is merely an example, and some functions of a certain circuit may be realized by another circuit. The circuit configuration of the display image generation circuit 70 shown in FIGS. 2 and 8 is merely an example, and some functions of a certain circuit may be realized by software. For example, part or all of the display image generation circuit 70 may be configured by a microcomputer or the like, and the same function may be realized by firmware. Further, a part or all of the display image generation circuit 70 shown in FIGS. 2 and 8 and / or the function of the display control unit 80 may be incorporated in the ECU of the navigation system, for example.

  Further, in the above-described embodiment, the operation of the operation unit 20 is possible, so the display of the display unit 40 does not have to be a touch panel type, but the display of the display unit 40 may be a touch panel type.

  In the above-described embodiment, the operation menu image may be a graphic image corresponding to the operation switch in the operation unit 20 superimposed on another image. For example, the operation menu image may be various types such as a TV or a DVD being played back. The graphic image of the operation switch may be superimposed on the image or the map image of the navigation system.

  In the embodiment described above, the operation devices 10 and 100 are embodied as a vehicle operation device as a preferred embodiment, but the present invention is not limited to this and is used in other applications. It may be an operating device.

It is a perspective view which shows Example 1 of the vehicle mounting state of the operating device 10 which concerns on this invention. FIG. 2 is a schematic cross-sectional view of the operating device 10 shown in FIG. 1 as viewed from the side. It is a principle figure of the display part 40 using a parallax division system. It is a figure which shows an example of an operation menu image. It is a figure which shows the some example of a stereoscopic vision image. It is a figure which shows the general structure of the conventional prompter system and a reflection type, respectively. It is a perspective view which shows Example 2 of the vehicle mounting state of the operating device 100 which concerns on this invention. It is a functional block diagram which shows the main functions of the operating device 100 shown in FIG. It is a figure which shows the stereoscopic image output when the operation part 20 is operated from the driver's seat side. It is a figure which shows the structure which provided the stereo camera 600 which can move to a vehicle width direction. It is a figure which shows the other modification schematically. FIG. 10 is a schematic cross-sectional view of a controller device 200 according to Embodiment 3 as viewed from the side. It is a figure which shows an example of the part of the graphic images F1-F8 of the operation menu image displayed on the display part. It is a figure which shows typically the A view of FIG. It is a figure which shows typically the relative height relationship of a stereoscopic vision switch image and a stereoscopic vision hand image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,100 Operation apparatus 20 Operation part 30 Various vehicle equipment 40 Display part 50,500 Camera 70 Display image generation circuit 71 Background image generation circuit 72 Left-eye image synthesis circuit 74 Right-eye image synthesis circuit 76 Stereoscopic image generation circuit 80 Display control unit 90 Operator determination unit 92 Camera switching unit

Claims (8)

An operation unit in which operation switches are arranged; and
An indicator capable of autostereoscopic viewing,
Imaging means for photographing a user's hand operating the operation unit from a plurality of directions;
A stereoscopic image generation unit that generates a stereoscopic image of the hand based on a parallax image obtained from the imaging unit;
On the display unit, the operation menu image representing the position of the operation switch and the function of the operation switch in the operation unit and the stereoscopic image of the hand generated by the stereoscopic image generation unit are combined and displayed. An operation device characterized by. The operation device according to claim 1, wherein the operation unit and the indicator are arranged at a position between the driver seat and the passenger seat in the vehicle width direction in the vehicle interior.
The imaging means includes at least two imaging means for looking down the operation unit obliquely downward from the driver seat side in the vehicle width direction,
The operation device, wherein the stereoscopic image generation unit generates the stereoscopic image of the hand in a line-of-sight direction from the driver's seat based on parallax images obtained from the two imaging units. The operation device according to claim 1, wherein the operation unit and the indicator are arranged at a position between the driver seat and the passenger seat in the vehicle width direction in the vehicle interior.
An operator discriminating means for discriminating whether the operation unit is operated from the driver's seat side or the passenger seat side;
The stereoscopic image generation means generates the stereoscopic image of the hand in the line-of-sight direction from the driver seat side when the operator determination means determines that the operation unit is operated from the driver seat side. An operation device that generates a stereoscopic image of the hand in a line-of-sight direction from the passenger seat side when the operator determination unit determines that the operation unit is operated from the passenger seat side. The imaging means includes three or more imaging means spaced apart from each other in the vehicle width direction,
The stereoscopic image generation means is based on parallax images obtained from two imaging means on the driver seat side when the operator determination means determines that the operation unit is operated from the driver seat side. When the operator discriminating means determines that the operation unit is operated from the passenger seat side, the stereoscopic image of the hand is generated and obtained from the two imaging means on the passenger seat side. The operating device according to claim 3, wherein the stereoscopic image of the hand is generated based on a parallax image. The imaging means includes two imaging means movable in the vehicle width direction,
The stereoscopic image generation means is a parallax image obtained from two imaging means moved to the driver seat side when the operator determination means determines that the operation unit is operated from the driver seat side. The stereoscopic image of the hand is generated on the basis of the two images, and when the operation determining unit determines that the operation unit is operated from the passenger seat side, the two images moved to the passenger seat side are generated. The operating device according to claim 3, wherein the stereoscopic image of the hand is generated based on a parallax image obtained from the means.   On the display device, an image portion representing the operation switch in the operation menu image is displayed in a stereoscopic view, and a surface obtained by offsetting the stereoscopic image of the hand by a predetermined height from the display surface of the display device has a height of zero. The operating device according to claim 1, wherein the operating device is displayed as a reference plane. An image portion representing an operation switch in the operation menu image is displayed in a stereoscopic view with a display surface of the display as a reference surface having a height of zero,
The operation device according to claim 6, wherein the predetermined height to be offset substantially matches a stereoscopic view height of an image portion representing the operation switch.   The height of the finger from the display surface of the display in the stereoscopic image of the hand displayed when the finger of the hand touches the operation unit is the stereoscopic view of the image portion representing the operation switch in the operation menu image. The operating device according to claim 6 or 7, which substantially matches the height of the operating device.

Images