JP2017111711A - Operation device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation device for a vehicle which hardly causes an operational error, even in a gesture operation and is excellent in operability.SOLUTION: The vehicle includes a display part (a projector 30 and a projection surface 30a) which displays operation images (a first operation image and a second operation image) which are operation targets by an occupant, a gesture recognition part (a camera 26 and a recognition part 44) which recognizes the gesture of the occupant as operation input with respect to the operation images, and a control part 46 which performs predetermined control according to the operation input with respect to the operation images. The display part is arranged on a dash board 18 on the side of a front passenger seat. The gesture recognition part recognizes the motion of the body of the occupant taken in a position between a driver seat and the front passenger seat as a gesture.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an operation device for a vehicle that recognizes an occupant's gesture as an operation input of the vehicle.

  Patent Document 1 discloses a so-called gesture operation in which a movement of a passenger's hand is detected in front of a screen of a car navigation apparatus mounted on a vehicle, and some functions of the car navigation apparatus are operated based on the movement of the hand. Is disclosed.

  Patent Document 2 discloses a device that enables keyboard input by projecting a virtual keyboard image onto a pad portion of a steering wheel and detecting a position touched by an occupant.

JP 2010-184600 A JP 2005-321948 A

  In the device of Patent Document 1, an operation target (switch operation unit) is displayed on the screen of the car navigation device. In the device of Patent Document 2, the operation target (virtual keyboard image) is displayed on the pad portion of the steering wheel. Since the screen of the car navigation device and the pad portion of the steering wheel have a small surface area, the operation area by the gesture is narrow and difficult to operate. For this reason, the occupant must concentrate on the gesture operation, and there is a problem that erroneous operation is likely to occur. Furthermore, in the apparatus of Patent Document 2, since it is necessary to perform an operation by directly touching the display position of the virtual keyboard image, there is a problem in that the operability is remarkably deteriorated if it is disposed at a position away from the driver who mainly performs the operation. is there.

  The present invention has been made in consideration of such problems, and an object of the present invention is to provide a vehicular operating device that is less likely to be erroneously operated even by a gesture operation and that has good operability.

  The present invention provides a display unit that is provided on an interior member of a vehicle and displays an operation image to be operated by an occupant, a gesture recognition unit that recognizes the occupant's gesture as an operation input for the operation image, and the operation image A control unit that performs a predetermined control in response to the operation input to the vehicle, wherein the display unit is disposed on a dashboard on a passenger seat side that is an interior member of the vehicle, The gesture recognizing unit recognizes the movement of the occupant's body performed at the position between the driver's seat and the passenger seat in the vehicle width direction of the vehicle as the gesture.

  There are no instruments on the passenger side dashboard. Even if some instruments are arranged due to expansion of the display area of the meter or the like, no other operation means necessary for driving operation is arranged. Therefore, in general, a wide area is available on the dashboard on the passenger seat side. According to the present invention, the operation image is displayed on the passenger side dashboard. For this reason, a wide display area of the operation image can be secured, and it is possible to configure a GUI (Graphical User Interface) that is unlikely to cause an erroneous operation even in a gesture operation. In addition, a gesture is recognized between the driver seat and the passenger seat. For this reason, it is possible to operate the operation image on the passenger seat side even from an occupant seated in the driver's seat, thereby ensuring operability.

  In the present invention, the operation image is a fixed image displayed at a fixed position as a selection target, and the fixed image is moved by moving in the operation image corresponding to the gesture recognized by the gesture recognition unit. A pointer image displayed in a selectable manner may be included. When the pointer image can be operated by a gesture operation, the operation image becomes easier to operate, and a GUI with fewer erroneous operations can be configured. In addition, when the operation image becomes easy to operate, the occupant will not be concerned about the gesture operation.

  In the present invention, the control unit selects the fixed image when the pointer image is in contact with the fixed image area or approaches a predetermined distance or less, and performs the predetermined control according to the function of the fixed image. You may do it. According to the present invention, since the function is executed only by selecting the fixed image from the pointer image, the operability can be improved. Also, if the operability is improved, the passenger will not be concerned about the gesture operation.

  In the present invention, the display unit includes a projector that projects the operation image, and a concave portion having a concave shape and a convex portion having a convex shape with respect to the concave portion are formed on the surface of the dashboard. When the projector projects the operation image, an image boundary line that is a boundary line that divides a predetermined area in the image constituting the operation image is a boundary line between the concave portion and the convex portion. It is also possible to match the uneven boundary line. According to the present invention, since the projection mapping is performed so that the region on the operation image corresponds to the unevenness of the dashboard, the region of the operation image can be more easily recognized by the occupant, and the operability can be improved. . Also, if the operability is improved, the passenger will not be concerned about the gesture operation.

  In the present invention, among the images constituting the operation image, a boundary line that divides the fixed image displayed at a fixed position as a selection target and a region of another image different from the fixed image is the image boundary. It may be set as a line. According to the present invention, since the section between the fixed image and the other image to be selected by the pointer image is associated with the unevenness on the dashboard, the area of the fixed image is easily recognized by the occupant. In addition, when the area of the fixed image becomes easy to recognize, the occupant does not care about the gesture operation.

  In this invention, the said recessed part is formed as a general surface of the said dashboard, the said convex part is formed so that it may protrude in a vehicle interior side with respect to the said general surface, and the said convex part is with respect to the said dashboard. Attachment and removal from the dashboard may be possible. According to the present invention, by configuring the convex portion to be detachable, versatility when a GUI change or a gesture operation is not used is enhanced.

  In the present invention, a rear seat roof console may be provided on a ceiling of the vehicle, and the projector may be provided integrally with the rear seat roof console.

  According to the present invention, even when a projection method for projecting an operation image is used, the possibility of being blocked by an occupant is low. Moreover, even when the display part is enlarged from the passenger side dashboard to the driver side instrument panel, it is possible to cope.

  In the present invention, the gesture recognizing unit determines whether the recognized gesture is by the driver's side passenger or the passenger's side passenger, and the control unit performs the determination by the gesture recognizing unit. Different control may be performed according to the result. According to the present invention, the convenience is improved by discriminating the operation from the driver seat side and the operation from the passenger seat side. For example, complicated operations can be prohibited from the driver's seat side but allowed from the passenger seat side.

  In the present invention, the gesture recognition unit includes a camera that is provided at a predetermined location in the vehicle cabin and images the hand of the occupant. The gesture recognition unit recognizes the gesture based on imaging information of the camera, and corresponds to an imaging range of the camera. You may make it perform the said determination by the approach position or approach angle of the said hand and arm. According to the present invention, it is possible to easily discriminate between an operation from the driver's seat side and an operation from the passenger seat side only by image analysis without adding a special configuration.

  In the present invention, the vehicle has a driving support in which the vehicle autonomously performs at least one of a manual operation mode in which an acceleration / deceleration operation and a steering operation are performed by an operation of the occupant and an acceleration / deceleration operation and a steering operation of the vehicle. The mode may be set to be switchable, and the display unit may enlarge the operation image displayed during the driving support mode than the operation image displayed during the manual driving mode. According to the present invention, it is possible to display appropriate content by automatically changing the display area according to the operation mode at the time of displaying the operation image, and convenience is improved.

According to the present invention, it is possible to secure a wide display area of an operation image, and it is possible to configure a GUI that is unlikely to cause an erroneous operation even with a gesture operation. In addition, even an occupant seated in the driver's seat can operate the operation image on the passenger seat side, and operability can be ensured.

FIG. 1 is a schematic diagram showing a simplified interior of a vehicle. FIG. 2A is a front view of the dashboard, and FIG. 2B is a cross-sectional view of the dashboard. FIG. 3 is a block diagram of the vehicle operating device according to the present embodiment. FIG. 4 is a schematic diagram schematically showing the first operation image. FIG. 5 is a schematic diagram schematically showing the first operation image. FIG. 6 is a schematic diagram schematically showing the first operation image. FIG. 7 is a schematic diagram schematically showing the second operation image. FIG. 8 is a schematic diagram schematically showing the second operation image. FIG. 9 is a schematic diagram schematically showing the second operation image. FIG. 10 is a flowchart of operation 1 of the vehicle operating device. FIG. 11 is a flowchart of operation 2 of the vehicle operating device. FIG. 12 is a flowchart of operation 3 of the vehicle operating device. FIG. 13 is a flowchart of operation 4 of the vehicle operating device. FIG. 14 is an external view of a vehicle related to the related technology of the present embodiment. FIG. 15 is a block diagram of a modified example of the vehicle operating device according to the present embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, a vehicle operation device 40 according to the present invention will be described in detail with reference to the accompanying drawings by giving preferred embodiments.

[1. Interior of vehicle 10]
The interior of the vehicle 10 will be described with reference to FIG. In FIG. 1, the interior of the vehicle 10 is shown deformed for ease of explanation. A driver's seat 14 and a passenger seat 16 are provided in the passenger compartment 12 of the vehicle 10 toward the front. A dashboard 18 is provided in front of the driver seat 14 and the passenger seat 16. A front window 20 is provided above the dashboard 18, and a room mirror 22 is provided in the upper center of the front window 20. A camera 26 is provided on the roof 24 in the vicinity of the room mirror 22. Further, a roof console 28 for the rear seat is provided in the roof 24 at the approximate center in the vehicle width direction and above or behind the driver seat 14 and the passenger seat 16. In addition to the monitor and the room lamp, the rear seat roof console 28 is integrally provided with a projector 30.

  The camera 26 is arranged toward a predetermined space, for example, a space between the driver seat 14 and the passenger seat 16 in the vehicle width direction of the vehicle 10 and close to the dashboard 18. Take pictures of gestures performed by the crew. The camera 26 includes a two-dimensional or three-dimensional position detection sensor. For example, a TOF (Time Of Flight) camera, a CCD camera, or the like can be used. The camera 26 may include a motion sensor. In this case, imaging can be started when the occupant performs an input operation using a gesture. The camera 26 transmits imaging information to an operation device ECU 42 described later by wire or wirelessly.

  The projector 30 is disposed mainly toward the dashboard 18 on the passenger seat 16 side. The projector 30 is controlled by an operation device ECU 42 described later, and projects an operation image (moving image or still image) for operating the vehicle 10 onto the dashboard 18. That is, the projection surface 30 a of the projector 30 is arranged on the surface of the dashboard 18. In the present embodiment, so-called projection mapping is performed in which the control unit 46 of the operation device ECU 42 controls the projection position, operation image, and the like of the projector 30.

[2. Dashboard 18]
The dashboard 18 will be described with reference to FIGS. 2A and 2B. The surface of the dashboard 18 becomes a projection surface 30a (see FIG. 1) of the projector 30. The surface of the dashboard 18 is subjected to surface processing so that the operation image is projected clearly. For example, a special paint is applied. The dashboard 18 is provided with a convex three-dimensional object in the passenger compartment 12 in order to make the three-dimensional display object in the operation image projected by the projector 30 resemble an actual three-dimensional object.

  Specifically, as shown in FIG. 2A, a first convex portion 34 having a substantially circular shape in front view is provided at a predetermined position on the surface of the dashboard 18. On both sides of the first convex portion 34 in the vehicle width direction, there are provided a second convex portion 36 and a third convex portion 38 that are substantially circular in a front view. In the present specification, a portion of the surface of the dashboard 18 other than the first convex portion 34, the second convex portion 36, and the third convex portion 38 is referred to as a general surface 32. In the embodiment shown in FIG. 2A, the general surface 32 is wider than the first surface 34, the second surface 36, the third surface 38 and the general surface 32 of the surface of the dashboard 18. The first convex portion 34, the second convex portion 36, and the third convex portion 38 have a convex shape with respect to the general surface 32. In other words, the general surface 32 is a concave portion with respect to the first convex portion 34, the second convex portion 36, and the third convex portion 38.

  The 1st convex part 34, the 2nd convex part 36, and the 3rd convex part 38 may be fixed to the dashboard 18, can be attached to the main body of the dashboard 18, and are removed from the main body of the dashboard 18. It may be possible. For example, as shown in FIG. 2B, a male part 34 a is formed on the back side of the first convex part 34, and a female part 18 a is formed on the main body surface of the dashboard 18. The first convex portion 34 is attached to the main body of the dashboard 18 by fitting the male portion 34a to the female portion 18a. The first convex portion 34 has the highest height in the central portion (height in the direction perpendicular to the surface 32) in front view, and the height gradually decreases from the central portion in the radial direction. The second convex portion 36 and the third convex portion 38 have the same shape and structure except that they have a smaller diameter than the first convex portion 34.

  In addition, the magnitude | size, the shape, and arrangement | positioning of the 1st convex part 34, the 2nd convex part 36, and the 3rd convex part 38 are not restricted to embodiment shown in FIG. 2A and FIG. 2B. If the female portion 18a is formed at a plurality of locations and convex portions having various shapes are prepared, it is possible to form uneven shapes having various patterns on the surface of the dashboard 18.

[3. Configuration of Vehicle Operation Device 40]
The configuration of the vehicle operating device 40 will be described with reference to FIG. The vehicle operation device 40 according to the present embodiment projects a camera 26 that captures the occupant's arms, an operation device ECU 42 that performs various processes, a dashboard 18 on which operation images are displayed, and an operation image projected on the dashboard 18. And a storage device 48 that stores information of operation images projected by the projector 30. The camera 26 and the projector 30 have already been described with reference to FIG. The dashboard 18 has already been described with reference to FIGS. 2A and 2B.

  The operating device ECU 42 is a control circuit that performs predetermined control based on image information captured by the camera 26. The operation device ECU 42 is a computer including a microcomputer, such as a CPU (Central Processing Unit), ROM (including EEPROM), RAM (Random Access Memory), A / D converter, D / A converter, etc. I / O device etc. The operating device ECU 42 functions as various function realizing units when the CPU reads and executes a program recorded in the ROM. In the present embodiment, the operating device ECU 42 functions as the recognition unit 44 and the control unit 46 by executing a program. The operating device ECU 42 may be divided into a plurality of parts, or may be integrated with other ECUs. Note that the operating device ECU 42 can also be realized by other hardware.

  The recognition unit 66 is configured to recognize an occupant's hand and arm from image information captured by the camera 26 and to recognize a gesture operation performed by the hand and arm. The control unit 46 reads the information of the operation image (the first operation image 62 shown in FIGS. 4 to 6 and the second operation image 80 shown in FIGS. 7 to 9), and causes the projector 30 to project the operation image. Configured as follows. Further, the control unit 46 is configured to operate the display object in the operation image in accordance with the gesture operation recognized by the recognition unit 44. Furthermore, the control unit 46 is configured to perform various types of control, for example, image transition of the projector 30 and control of other devices on the in-vehicle network according to operations performed in the operation image. The control unit 46 controls the position of the projection surface 31 of the projector 30 based on the position of each part of the dashboard 18 and the contents of the operation image.

  The storage device 48 stores one or more operation images. The storage device 48 receives an image request from the operation device ECU 42 and outputs information on an operation image corresponding to the request to the operation device ECU 42. The operation image information stored in the storage device 48 can be updated by a storage medium such as a magnetic disk, an optical disk, or a flash memory, or by communication.

  The operating device ECU 42 is connected to the bus 50. In addition, a driving support ECU 52, a meter ECU 54, an air conditioner ECU 56, an audio device 58, a car navigation device 60, and the like are connected to the bus 50. The driving support ECU 52, the operation device ECU 42, the meter ECU 54, the air conditioner ECU 56, the audio device 58, the car navigation device 60, and the like constitute an in-vehicle network (for example, CAN). The operation device ECU 42, the meter ECU 54, the air conditioner ECU 56, the audio device 58, the car navigation device 60, etc. can communicate information via the bus 50.

  The driving assistance ECU 52 controls driving assistance, braking assistance, and steering assistance when the automatic driving mode or the semi-automatic driving mode is selected by the occupant's switch operation. The meter ECU 54 controls meter display. The air conditioner ECU 56 controls the air conditioner.

[4. First operation image 62]
The first operation image 62 projected on the surface of the dashboard 18 will be described with reference to FIGS. The control unit 46 of the operation device ECU 42 reads information on the first operation image 62 as shown in FIG. 4 from the storage device 48 and outputs a display instruction to the projector 30. In response to the display instruction, the projector 30 projects the first operation image 62 on the surface of the dashboard 18. Note that the first operation image 62 shown in FIG. 4 is an initial image among the image transitions performed in the present embodiment. Of course, the first operation image 62 is not limited to the content shown in FIG.

  The first operation image 62 is a CG image, and includes a speed image 64 indicating an image of the vehicle speed, a radar image 66 indicating an image of radar detection, a fuel image 68 indicating an image of the remaining amount of fuel, and other images 70. Including.

  The speed image 64 includes a circular display object 64a that operates according to the vehicle speed. The display object 64a repeats the operation of appearing from the center of the velocity image 64, expanding in the radial direction, and disappearing at the edge. The display object 64a operates based on speed information output from the meter ECU 54 (see FIG. 3) or speed information output from a vehicle speed sensor (not shown). The projector 30 projects the speed image 64 on the surface of the first convex portion 34. At this time, the image boundary line 64b that is a boundary line that divides the velocity image 64 and the surrounding image is made to coincide with the uneven boundary line 34b that is the boundary line between the general surface (concave portion) 32 and the first convex portion 34. .

  The radar image 66 has a display object 66a indicating the detection direction of an on-vehicle radar (not shown). The display object 66a rotates in one direction around the center of the radar image 66 at a constant period. When the in-vehicle radar detects an obstacle, the position of the obstacle is indicated by a relative position centered on the vehicle 10 in the radar image 66. The projector 30 projects the radar image 66 on the surface of the second convex portion 36. At this time, the image boundary line 66 b that is a boundary line that divides the radar image 66 and the surrounding image is made to coincide with the uneven boundary line 36 b that is the boundary line between the general surface (concave portion) 32 and the second convex portion 36. .

  The fuel image 68 has a display object 68a that operates according to the increase or decrease of the fuel in the fuel tank. The display object 68a operates based on fuel information output from the meter ECU 54 (see FIG. 3) or remaining amount information output from a sensor that measures the remaining amount of fuel, such as a thermistor (not shown). The projector 30 projects the fuel image 68 on the surface of the third convex portion 38. At this time, an image boundary line 68b that is a boundary line that divides the fuel image 68 and the surrounding image is made to coincide with an uneven boundary line 38b that is a boundary line between the general surface (concave portion) 32 and the third convex portion 38. .

  Here, it is assumed that the occupant's arm is imaged by the camera 26 in a state where the first operation image 62 shown in FIG. 4 is projected on the surface of the dashboard 18. At this time, a region image 74 as shown in FIG. 5 is displayed at a predetermined position in the first operation image 62, here on the right side. Further, a pointer image 76 appears from the area image 74. The control unit 46 associates the position of the occupant's hand in the imaging space with the position of the pointer image 76 when the pointer image 76 appears. In this way, when the occupant performs a gesture operation to move the hand, the pointer image 76 moves in the first operation image 62.

  As shown in FIG. 6, it is assumed that the pointer image 76 is moved to the vicinity of the fixed position in the first operation image 62, in this case, the lower side in conjunction with the movement of the passenger's hand. At this time, the first fixed image 78 is displayed at the fixed position of the movement destination. The first fixed image 78 corresponds to a switch for making a transition to the next image. When the pointer image 76 touches the first fixed image 78 or approaches a predetermined distance or less, the function associated with the first fixed image 78 is selected. In the present embodiment, the function associated with the first fixed image 78 is a transition from the first operation image 62 to the second operation image 80 shown in FIG. The control unit 46 changes the image to be projected from the first operation image 62 to the second operation image 80.

[5. Second operation image 80]
The second operation image 80 that transitions from the first operation image 62 will be described with reference to FIGS. The control unit 46 of the operation device ECU 42 reads information on the second operation image 80 as shown in FIG. 7 from the storage device 48 and outputs a display instruction to the projector 30. In response to the display instruction, the projector 30 projects the second operation image 80 on the surface of the dashboard 18. Note that the second operation image 80 shown in FIG. 7 is an image in a layer next to the initial image in the image transition performed in the present embodiment. Similar to the first operation image 62, the second operation image 80 is not limited to the content as shown in FIG.

  The second operation image 80 is a CG image, a meter image 82 showing an image of a meter, an audio image 84 showing an image of the audio device 58, a navigation image 86 showing an image of the car navigation device 60, and an image of an air conditioner. And an air-conditioner image 88 showing. The meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 correspond to a switch for making a transition to a display image or an operation image of each device. Further, the second operation image 80 has a second fixed image 90 at the center of the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88, and has a trajectory image 92 around the second fixed image 90.

  The projector 30 projects the second fixed image 90 on the surface of the first convex portion 34. At this time, an image boundary line 90b that is a boundary line that divides the second fixed image 90 and the surrounding image is changed into an uneven boundary line 34b that is a boundary line between the general surface (concave portion) 32 and the first convex portion 34. Match.

  When the occupant performs a predetermined first gesture operation, for example, a first gesture operation for rotating the palm, as shown in FIG. 8, the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 are displayed on the trajectory 92. Rotate along. When any one of the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 is arranged in front of the second fixed image 90, the occupant performs a predetermined second gesture operation, for example, a wrist. When the second gesture operation for flipping up is performed, the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 move to the inside of the second fixed image 90 as shown in FIG. 9. Then, the function associated with the moved images 82, 84, 86, 88 is selected.

  When the occupant moves the meter image 82 into the second fixed image 90, the control unit 46 changes the image of the surface of the dashboard 18 from the second operation image 80 to an image (not shown) on the display screen of the meter panel. Transition. At this time, display value information such as the vehicle speed is input from the meter ECU 54 and displayed on the display screen image of the meter panel. Further, in response to a gesture operation performed while the display screen of the meter panel is projected on the surface of the dashboard 18, an operation instruction such as display switching is output to the meter panel.

  When the occupant moves the audio image 84 into the second fixed image 90, the control unit 46 changes the image of the surface of the dashboard 18 from the second operation image 80 to the operation screen image of the audio device 58 (not shown). Transition to. At this time, various information is input from the audio device 58 and displayed on the operation screen image of the audio device 58. Furthermore, in response to a gesture operation performed while the operation screen of the audio device 58 is projected on the surface of the dashboard 18, an operation instruction for audio operation is output to the audio device 58.

  When the occupant moves the navigation image 86 into the second fixed image 90, the control unit 46 changes the image of the surface of the dashboard 18 from the second operation image 80 to the image of the display screen of the car navigation device 60 (not shown). ). At this time, various types of information are input from the car navigation device 60 and displayed on the display screen image of the car navigation device 60. Further, in response to a gesture operation performed while the display screen of the car navigation device 60 is projected on the surface of the dashboard 18, an operation instruction for a navigation operation is output to the car navigation device 60.

  When the occupant moves the air conditioner image 88 into the second fixed image 90, the control unit 46 changes the image of the surface of the dashboard 18 from the second operation image 80 to an image of the operation screen of the air conditioner (not shown). Let At this time, information on a set value such as a set temperature is input from the air conditioner ECU 56, and an operation screen of the air conditioner is displayed on the image. Further, in response to a gesture operation performed while the operation screen of the air conditioner is projected on the dashboard 18, an operation instruction for operating the air conditioner is output to the air conditioner ECU 56.

[6. Operation of operation device 40 for vehicle]
[6-1. Operation 1]
The operation 1 of the vehicle operation device 40 will be described with reference to FIG. The following process is started at a predetermined timing, for example, when the vehicle operating device 40 is turned on. In the following process, the first operation image 62 shown in FIGS. 4 to 6 is projected on the surface of the dashboard 18.

  In step S1, the projection of the first operation image 62 by the projector 30 and the imaging of the gesture operation space by the camera 26 are performed. Further, the projector 30 that has received a projection instruction from the control unit 46 projects the first operation image 62 on the surface of the dashboard 18. Further, the camera 26 images the space between the driver's seat 14 and the passenger seat 16. The recognizing unit 44 of the operating device ECU 42 receives image information captured by the camera 26 and performs a recognition process. As described above, imaging by the camera 26 may be performed according to the detection result of the motion sensor, or may be performed constantly. Further, when the operation is not imaged for a predetermined time, the imaging by the camera 26 may be stopped.

  In step S <b> 2, it is determined whether or not a passenger's hand has been recognized from the imaging information of the camera 26. When the occupant holds his hand over the gesture operation space, the hand is imaged by the camera 26 and the occupant's hand is recognized by the recognition unit 44. When the occupant's hand is recognized by the recognition unit 44 (step S2: YES), the process proceeds to step S3. On the other hand, when the occupant's hand is not recognized by the recognition unit 44 (step S2: NO), the process of step S2 is repeatedly executed.

  In step S <b> 3, a pointer image 76 is displayed in the first operation image 62. The control unit 46 outputs a display instruction to the projector 30. The projector 30 projects the first operation image 62 according to the display instruction. At this time, as shown in FIG. 5, a region image 74 is displayed in the first operation image 62, and then a pointer image 76 appears from the region image 74.

  In step S4, the pointer image 76 is associated with the passenger's hand. The control unit 46 associates the position of the occupant's hand in the imaging space with the position of the pointer image 76 when the pointer image 76 appears. When the occupant moves the hand, the movement amount and movement direction of the hand recognized by the recognition unit 44 are converted into the movement amount and movement direction of the pointer image 76. Then, the information on the first operation image 62 and the information on the pointer image 76 that moves in the first operation image 62 are output to the projector 30. At this time, a predetermined part of the hand, for example, the position of the fingertip may be associated with the position of the pointer image 76, or the position of the palm may be associated with the position of the pointer image 76.

  In step S5, it is determined whether or not there is an instruction by the pointer image 76. The pointer image 76 moves in conjunction with the movement of the occupant's hands and arms. As shown in FIG. 6, when the pointer image 76 contacts the first fixed image 78 or approaches a predetermined distance or less (step S5: YES), the process proceeds to step S6. On the other hand, when the pointer image 76 is in contact with the first fixed image 78 or not approaching the predetermined distance or less (step S5: NO), the process proceeds to step S7.

  In step S6, the process instructed by the gesture operation is executed. The function associated with the first fixed image 78 touched by the pointer image 76 is a transition from the first operation image 62 to the second operation image 80 shown in FIG. The control unit 46 causes the image projected on the surface of the dashboard 18 to transition from the first operation image 62 to the second operation image 80.

  In step S <b> 7, it is determined whether or not the passenger's hand is recognized from the imaging information of the camera 26. When the occupant's hand is recognized by the recognition unit 44 (step S7: YES), the process proceeds to step S5. Here, the operation of the pointer image 76 by the passenger is continued. On the other hand, when the occupant's hand is not recognized by the recognition unit 44 (step S7: NO), the process proceeds to step S8.

  In step S8, the pointer image 76 is not displayed in the first operation image 62. At this time, since it is presumed that the hand of the occupant has not been captured by the camera 26, the pointer image 76 is not displayed.

[6-2. Operation 2]
The operation 2 of the vehicle operating device 40 will be described with reference to FIG. The following processing is an example of step S6 in FIG. In the following process, the second operation image 80 shown in FIGS. 7 to 9 is projected on the surface of the dashboard 18.

  In step S11, projection of the second operation image 80 by the projector 30 and imaging of the gesture operation space by the camera 26 are performed. Receiving the projection instruction from the control unit 46, the projector 30 projects the second operation image 80 on the surface of the dashboard 18. Further, the camera 26 images the space between the driver's seat 14 and the passenger seat 16. The recognizing unit 44 of the operating device ECU 42 receives image information captured by the camera 26 and performs a recognition process.

  In step S12, it is determined whether or not the passenger's hand is recognized from the imaging information of the camera 26. When the occupant holds his hand over the gesture operation space, the hand is imaged by the camera 26 and the occupant's hand is recognized by the recognition unit 44. When the occupant's hand is recognized by the recognition unit 44 (step S12: YES), the process proceeds to step S13. On the other hand, when the occupant's hand is not recognized by the recognition unit 44 (step S12: NO), the process of step S12 is repeatedly executed.

  In step S13, it is determined whether there is an instruction by the occupant's gesture operation from the imaging information of the camera 26. When the occupant performs the first gesture for rotating the palm, as shown in FIG. 8, the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 rotate along the track 92. When any one of the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 is arranged in front of the second fixed image 90, the occupant performs a predetermined second gesture operation, for example, a wrist. When the second gesture operation for flipping up is performed, any one of the meter image 82, the audio image 84, the navigation image 86, and the air conditioner image 88 moves into the second fixed image 90 as shown in FIG. 9. Then, the function associated with the moved images 82, 84, 86, 88 is selected.

  In step S14, the process instructed by the gesture operation is executed. The control unit 46 changes the image projected on the surface of the dashboard 18 from the first operation image 62 to an operation image or display image corresponding to the images 82, 84, 86, 88 moved to the second fixed image 90. Let Furthermore, an operation instruction is output to the meter panel, the air conditioner, the audio device 58, and the car navigation device 60 in accordance with a gesture operation performed in a state where the operation image or the display image is projected on the surface of the dashboard 18.

[6-3. Operation 3]
The operation | movement 3 of the operating device 40 for vehicles is demonstrated using FIG. The process of step S21 to step S24 shown in FIG. 12 is the same as the process of step S1 to step S4 shown in FIG. Moreover, the process of step S27-step S30 shown in FIG. 12 corresponds with the process of step S5-step S8 shown in FIG. That is, the operation 3 is obtained by adding the processes of steps S25 and S26 to the operation 1 described with reference to FIG. Hereinafter, the description of the same process as that of the operation 1 described with reference to FIG. 10 is omitted, and only the added process will be described.

  After the pointer image 76 and the occupant's hand are associated in step S24, the approach direction of the hand and arm is determined in step S25. The recognizing unit 44 determines whether the recognized hand and / or arm enters the imaging area from the driver seat side or the passenger seat side. The approach direction can be determined by the approach position or the approach angle of the hand and arm. When entering from the driver's seat side (step S25: driver's seat side), the process proceeds to step S26. On the other hand, when entering from the passenger seat side (step S25: passenger seat side), the process proceeds to step S27.

  In step S26, some operations are restricted. This process is performed when there is a high possibility that the occupant performing the gesture operation is a driver. The control unit 46 cannot select some operations, for example, the navigation image 86 shown in FIGS. 7 to 9 or cannot partially operate the audio device 58.

[6-4. Operation 4]
The operation 4 of the vehicle operation device 40 will be described with reference to FIG. When projecting the first operation image 62 in each process of step S1 shown in FIG. 10 and step S21 shown in FIG. 12, and / or projecting the second operation image 80 in the process of step S11 shown in FIG. Processing of steps S41 to S43 shown in FIG. 13 may be performed.

  In step S41, the operation mode is determined. The control unit 46 determines the driving mode based on the driving mode information output from the driving support ECU 52. When the vehicle 10 is in the automatic driving (including semi-automatic driving) mode (step S41: automatic driving mode), the process proceeds to step S42. On the other hand, when the vehicle 10 is in the manual operation mode (step S41: manual operation mode), the process proceeds to step S43.

  In step S42, the projector 30 projects an enlarged display range of the operation image (first operation image 62, second operation image 80). At this time, the control unit 46 instructs the projector 30 to enlarge the display area, and the projector 30 projects an operation image according to the instruction.

  In step S43, the projector 30 projects the display range of the operation images (the first operation image 62 and the second operation image 80) without enlarging them. At this time, the control unit 46 instructs the projector 30 to display a normal display area, and the projector 30 projects an operation image according to the instruction.

[6-5. Other actions]
The dashboard 18 may not be provided with the first convex portion 34, the second convex portion 36, and the third convex portion 38. In such a case, the projection plane 30a may be shifted. For example, other operations may be performed in the following embodiment.

  When the camera 26 can detect a three-dimensional position, for example, when the camera 26 is a TOF camera, the projection height of the first operation image 62 and the second operation image 80 may be changed according to the height of the hand performing the gesture operation. Is possible. When the position of the hand recognized by the recognition unit 44 is high, there is a high possibility that the occupant performing the gesture operation is tall. In this case, the first operation image 62 and the second operation image 80 are preferably projected on the upper portion of the dashboard 18. For this reason, the control unit 46 instructs the projector 30 to place the display area above the dashboard 18, and the projector 30 projects an operation image in accordance with the instruction. On the other hand, when the position of the hand recognized by the recognition unit 44 is low, there is a high possibility that the occupant performing the gesture operation is short. In this case, the first operation image 62 and the second operation image 80 are preferably projected on the lower part of the dashboard 18. Therefore, the control unit 46 instructs the projector 30 to place the display area below the dashboard 18, and the projector 30 projects an operation image in accordance with the instruction.

[7. Related technology]
The present embodiment may have the following configuration. As shown in FIG. 14, a camera 102 is provided at the side end of the roof 100 of the vehicle 10. The camera 102 is arranged toward the side of the vehicle 10. A projector 104 is provided in the passenger compartment 12. The projector 104 is disposed toward the window 108 of the rear sliding door 106. The side mirror 110 is provided with a projector 112. The projector 112 is disposed downward.

This embodiment operates as follows. When an occupant approaches the vehicle 10, the camera 102 images the occupant. An ECU (not shown) performs image recognition processing based on imaging information of the camera 102. If it is identified as an occupant, the position of the occupant is further determined. When the position of the passenger is around the rear slide door 106, the projector 104 projects a welcome image 114 onto the window 108. When the position of the occupant is around the front door 118, the projector 112 projects the welcome image 118 on the ground. The control body such as the projection of the projectors 104 and 112 is an ECU.
Instead of recognizing the occupant by image recognition, the occupant (driver) may be recognized by communication between the key carried by the driver and the vehicle 10.

[8. Modified example]
The vehicle operation device 40 shown in FIG. 3 includes the projector 30 as a display unit. However, the vehicle operation device 40 ′ illustrated in FIG. 15 may include the display 120 as a display unit. Is provided. In this case, the display 120 is embedded in the dashboard 18. Further, the display 120 may be attached to the surface of the dashboard 18. The display 120 may be a liquid crystal panel or an organic EL.

[9. Summary of this embodiment]
The vehicle operation devices 40, 40 ′ according to the present embodiment are operation images (first operation image 62, second operation image 80, hereinafter referred to as operation images) that are provided on the interior member of the vehicle 10 and are to be operated by a passenger. 62, 80)) and a display unit (projector 30, projection surface 30a, display 120, hereinafter also referred to as display units 30, 30a, 120), and an occupant's gesture as an operation input to operation images 62, 80. A gesture recognizing unit (camera 26, recognizing unit 44, hereinafter also referred to as gesture recognizing unit 26, 44), and a control unit 46 that performs predetermined control in response to an operation input on the operation images 62, 80. . The display units 30 a and 120 are arranged on the dashboard 18 on the passenger seat 16 side which is an interior member of the vehicle 10. The gesture recognition units 26 and 44 recognize the movement of the occupant's body performed as a gesture at a position between the driver's seat 14 and the passenger seat 16 in the vehicle width direction of the vehicle 10.

  No instrument or the like is arranged on the dashboard 18 on the passenger seat 16 side. Even if some instruments are arranged due to expansion of the display area of the meter or the like, no other operation means necessary for driving operation is arranged. Therefore, generally, the dashboard 18 on the passenger seat 16 side has a wide area. According to this embodiment, the operation images 62 and 80 are displayed on the dashboard 18 on the passenger seat 16 side. For this reason, it is possible to secure a wide display area for the operation images 62 and 80, and it is possible to configure a GUI that is unlikely to cause an erroneous operation even with a gesture operation. A gesture is recognized between the driver's seat 14 and the passenger seat 16. Therefore, even an occupant seated in the driver's seat 14 can operate the operation images 62 and 80 on the passenger seat 16 side, and operability can be ensured.

  The operation image 62 is operated corresponding to a fixed image (first fixed image 78, hereinafter also referred to as a fixed image 78) displayed at a fixed position as a selection target, and a gesture recognized by the gesture recognition units 26 and 44. The pointer image 76 is displayed so that the fixed image 78 can be selected by moving in the image 62. When the pointer image 76 can be operated by a gesture operation, the operation image 26 can be operated more easily, and a GUI with few erroneous operations can be configured. In addition, when the operation image becomes easy to operate, the occupant will not be concerned about the gesture operation.

  The control unit 46 selects the fixed image 78 when the pointer image 76 contacts the area of the fixed image 78 or approaches a predetermined distance or less, and performs predetermined control according to the function of the fixed image 78. In this way, since the function is executed only by selecting the fixed image 78 from the pointer image 76, the operability can be improved. Also, if the operability is improved, the passenger will not be concerned about the gesture operation.

  The display unit 30 includes a projector 30 that projects operation images 62 and 80. On the surface of the dashboard 18, a concave portion having a concave shape (general surface 32, hereinafter also referred to as a concave portion 32) and a convex portion having a convex shape with respect to the concave portion 32 (first convex portion 34, second convex portion). 36, a third convex portion 38, hereinafter also referred to as convex portions 34, 36, 38). When the projector 30 projects the operation images 62 and 80, the image boundary lines 64b, 66b, 68b, and 90b, which are boundary lines that divide a predetermined area among the images constituting the operation images 62 and 80, are formed in the concave portion 32. And the concavo-convex boundary lines 34b, 36b, and 38b, which are boundary lines between the convex portions 34, 36, and 38. In this way, projection mapping is performed so that the regions on the operation images 62 and 80 correspond to the unevenness of the dashboard 18, so that the region of the operation images 62 and 80 can be more easily recognized by the occupant, and the operability can be improved. Can do well. Also, if the operability is improved, the passenger will not be concerned about the gesture operation.

  Of the images constituting the operation images 62 and 80, a fixed image (second fixed image 90, hereinafter referred to as a fixed image 90) displayed at a fixed position as a selection target, and other images different from the fixed image 90 are displayed. A boundary line that divides the region is set as an image boundary line 90b. As described above, since the section between the fixed image 90 and other images and the unevenness on the dashboard 18 are made to correspond to each other, it is easy for the passenger to recognize the area of the fixed image 90. Further, when the area of the fixed image 90 is easily recognized, the occupant will not be distracted by the gesture operation.

  The recess 32 is formed as a general surface 32 of the dashboard 18. Further, the convex portions 34, 36, and 38 are formed so as to be convex toward the inside of the passenger compartment 12 with respect to the general surface 32. The convex portions 34, 36, and 38 can be attached to and detached from the dashboard 18. As described above, the convex portions 34, 36, and 38 are configured to be detachable, so that versatility when a GUI change or a gesture operation is not used is enhanced.

  A rear seat roof console 28 is provided on the ceiling (roof 24) of the vehicle 10. The projector 30 is provided integrally with the rear seat roof console 28. By doing in this way, even when using the projection system which projects the operation images 62 and 80, possibility that it will be obstruct | occluded by a passenger | crew is low. In addition, even when the display units 30a and 120 are enlarged from the dashboard 18 on the passenger seat 16 side to the instrument panel on the driver seat 14 side, it can be dealt with.

  The gesture recognition units 26 and 44 determine whether the recognized gesture is caused by an occupant on the driver's seat 14 side or an occupant on the passenger seat 16 side. The control unit 46 performs different control according to the determination result by the gesture recognition units 26 and 44. The convenience is improved by discriminating the operation from the driver's seat 14 side and the operation from the passenger seat 16 side. For example, a complicated operation (for example, an operation of the car navigation device 60) can be prohibited from the driver's seat 14 side but allowed from the passenger seat 16 side.

  The gesture recognition units 26 and 44 include a camera 26 that is provided at a predetermined location in the passenger compartment 12 and images the occupant's hands and arms, recognizes the gesture based on the imaging information of the camera 26, and moves the hands and arms with respect to the imaging range of the camera 26. The determination is made based on the approach position or the approach angle. By doing in this way, it is possible to easily discriminate between the operation from the driver's seat 14 side and the operation from the passenger seat 16 side only by image analysis without adding a special configuration.

  The vehicle 10 can be switched between a manual driving mode in which an acceleration / deceleration operation and a steering operation are performed by an occupant operation, and a driving support mode in which the vehicle 10 autonomously performs at least one of an acceleration / deceleration operation and a steering operation of the vehicle 10. Set to The display units 30a and 120 enlarge the operation images 62 and 80 displayed during the driving support mode than the operation images 62 and 80 displayed during the manual operation mode. By automatically changing the display area according to the operation mode at the time when the operation images 62 and 80 are displayed, appropriate content can be displayed, and convenience is improved.

DESCRIPTION OF SYMBOLS 10 ... Vehicle 12 ... Vehicle compartment 14 ... Driver's seat 16 ... Passenger seat 26 ... Camera 28 ... Rear seat roof console 30 ... Projector 30a ... Projection surface 32 ... General surface (concave part) 34 ... First convex part 36 ... Second convex part Part 38 ... Third convex parts 34b, 36b, 38b ... Concave and convex boundary lines 40, 40 '... Vehicle operation device 42 ... Operation device ECU 44 ... Recognition part 46 ... Control part 62 ... First operation images 64b, 66b, 68b, 90b ... image boundary line 78 ... first fixed image 80 ... second operation image 90 ... second fixed image 120 ... display

Claims (10)

A display unit that is provided on an interior member of the vehicle and displays an operation image to be operated by an occupant;
A gesture recognition unit for recognizing the occupant's gesture as an operation input for the operation image;
A vehicle operation device comprising: a control unit that performs predetermined control according to the operation input with respect to the operation image;
The display unit is disposed on a dashboard on the passenger seat side which is an interior member of the vehicle,
The said gesture recognition part recognizes the motion of the said passenger | crew's body performed in the position between the driver's seat and front passenger seat in the vehicle width direction of the said vehicle as said gesture. The vehicle operating device according to claim 1,
The operation image is a fixed image displayed at a fixed position as a selection target, and the fixed image can be selected by moving in the operation image corresponding to the gesture recognized by the gesture recognition unit. A vehicle operating device comprising a pointer image. The vehicle operating device according to claim 2,
The control unit selects the fixed image when the pointer image is in contact with the fixed image region or approaches a predetermined distance or less, and performs the predetermined control according to the function of the fixed image. A vehicle operating device. In the vehicle operating device according to any one of claims 1 to 3,
The display unit includes a projector that projects the operation image,
On the surface of the dashboard, a concave portion having a concave shape and a convex portion having a convex shape with respect to the concave portion are formed,
When the projector projects the operation image, the projector defines an image boundary line that is a boundary line that divides a predetermined region in the image constituting the operation image, and an unevenness that is a boundary line between the concave portion and the convex portion. An operating device for a vehicle, characterized by being matched with a boundary line. The vehicle operating device according to claim 4,
Of the images constituting the operation image, a boundary line that divides the fixed image displayed at a fixed position as a selection target and a region of another image different from the fixed image is set as the image boundary line. A vehicle operating device characterized by the above. In the vehicle operating device according to claim 4 or 5,
The recess is formed as a general surface of the dashboard;
The convex portion is formed to be convex toward the vehicle interior side with respect to the general surface,
The vehicle operating device, wherein the convex portion is attachable to and detachable from the dashboard. In the vehicle operating device according to any one of claims 4 to 6,
A roof console for the rear seat is provided on the ceiling of the vehicle,
The vehicle operating device, wherein the projector is provided integrally with the rear seat roof console. In the operating device for vehicles according to any one of claims 1 to 7,
The gesture recognizing unit determines whether the recognized gesture is by the driver's side passenger or the passenger's side passenger,
The vehicle operation device according to claim 1, wherein the control unit performs different control according to a result of the determination by the gesture recognition unit. The vehicle operating device according to claim 8, wherein
The gesture recognizing unit includes a camera that is provided at a predetermined location in the vehicle interior and images the hand and arm of the occupant, recognizes the gesture based on imaging information of the camera, and enters the hand and arm with respect to an imaging range of the camera. The vehicle operating device characterized in that the determination is performed based on a position or an approach angle. In the vehicle operating device according to any one of claims 1 to 9,
The vehicle is switched between a manual driving mode in which an acceleration / deceleration operation and a steering operation are performed by an operation of the occupant, and a driving support mode in which the vehicle autonomously performs at least one of the acceleration / deceleration operation and the steering operation of the vehicle. Set to possible,
The operation device for a vehicle, wherein the display unit enlarges the operation image displayed during the driving support mode as compared with the operation image displayed during the manual operation mode.

Images