JP2014102657A - Manipulation assistance system, manipulation assistance method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a manipulation assistance system, a manipulation assistance method, and a computer program, which allow a user to eventually select a desired manipulatable object from a number of manipulatable objects through simple manipulation.SOLUTION: A manipulation assistance system is configured such that; manipulatable objects comprising a plurality of menu icons 41 are displayed on a windshield 36; a predetermined number of menu icons 41 out of the plurality of menu icons 41 are designated as selectable candidate icons, which can be directly selected using a right-hand manipulation input device 3, based on a touch manipulation made on a left-hand manipulation input device 2 or the right-hand manipulation input device 3; and a menu icon 41 is selected from the menu icons 41 designated as the predetermined number of selectable candidate icons and operated based on a touch manipulation made on the right-hand manipulation input device 3.

Description

  The present invention relates to an operation support system, an operation support method, and a computer program that support operation of equipment in a vehicle by touch operation.

  Conventionally, when a user performs an operation on a device such as a personal computer, an ATM, a ticket vending machine, a navigation device, or a smartphone, various man-machine interfaces have been used as means for receiving the user's operation on the device. Particularly, in recent years, as one of the man-machine interfaces, an operation surface that accepts a user's touch operation is provided, and a user's touch operation on the operation surface (for example, a touch-on operation, a touch-off operation, a drag operation, a flick operation, etc.) is performed. An operation device that is detected as a user operation is often used. Examples of such an operation device include a touch pad, a touch panel, a tablet, and the like (hereinafter referred to as a touch pad), which allows a user to perform an intuitive and easy-to-understand operation.

  Here, as an operation mode when the user operates the touch pad or the like, the user may perform an operation without visually recognizing the touch pad or the like. Specifically, it is a case where an operation of a touch pad or the like is performed while performing other work, for example, an operation of a touch pad or the like may be performed while a driving operation of the vehicle is performed. For example, in US 2010/0268426, input devices (touchpads) are provided at two positions on the left and right of a vehicle handle, and a plurality of items displayed on a display by operations on the two input devices are provided. A configuration for selecting one operation object from the operation objects (icons) is described. Further, it is described that one input device and the other input device are operated in correspondence with different levels of the operation target.

US Patent Application Publication No. 2010/0268426 (FIG. 7A-7R)

  Here, as for the operation to be performed by the user who drives the vehicle, it is desirable to simplify the number of operations as much as possible. However, the technique described in Patent Document 1 has a configuration in which operation objects that can be operated with one input device and the other input device are selected in a hierarchy. In other words, one input device can only perform operations at the level associated with one input device, and the other input device can only perform operations at the level associated with the other input device. Therefore, there is a problem that the user cannot select one operation object that the user finally desires unless the corresponding operation is performed on each of the one input device and the other input device.

  The present invention has been made to solve the above-described problems, and includes a first function for selecting a plurality of operation objects and a further function for selecting an arbitrary operation object from among the plurality of operation objects. 2 functions are provided separately for one input device and the other input device, and under certain conditions, the other input device is assigned the same function as the other input device, so that only the operation of the other input device can be performed. Thus, an arbitrary operation object desired by the user can be finally selected from a large number of operation objects.

  As described above, in the present invention, the number of operations related to the user's operation is reduced as compared with the conventional one, and any operation object desired by the user from among a large number of operation objects can be obtained only by the operation on the other input device. An object of the present invention is to provide an operation support system, an operation support method, and a computer program that can finally select an item.

  In order to achieve the object, an operation support system (1) according to claim 1 of the present application is provided on a steering wheel (5) of a vehicle, and receives a user operation via an operation surface (11). 2) and a second operation input device (3), and an operation comprising a display screen (37) for displaying an operation object (41) operated by the first operation input device and the second operation input device. In the support system, the operation object display means (31) for displaying a plurality of operation objects on the display screen, and a predetermined number of the operation objects among the plurality of operation objects are selected as selection object candidates. , While accepting an operation of switching the operation object to be selected as the selection object candidate to another operation object, a drag operation or a second operation input device or the second operation input device Selection target candidate that sequentially switches the operation target object to be selected as the selection target candidate to another operation target object along the moving direction of the touch point in the drag operation or the flick operation when a lick operation is received Based on the selection means (31) and the operation received by the second operation input device, any one of the predetermined number of the operation objects selected as the selection object candidates by the selection object candidate selection means And an object selection means (31) for selecting the operation object and operating the selected operation object.

  An operation support system (1) according to claim 2 is the operation support system according to claim 1, wherein the operation object (41) to be selected as the selection object candidate is transferred to another operation object. The second operation input device (3) accepts only the operation related to the operation object in the object selection means (31) except during the time of accepting the operation for switching.

  The operation support system (1) according to claim 3 is the operation support system according to claim 1 or 2, wherein the selection target candidate is selected by the selection target candidate selection means (31). The coordinate system of the display area (60) on which the operation target (41) selected as the selection target candidate is displayed and the coordinate system on the operation surface (11) of the second operation input device (3). The object selection means (31) includes coordinate correspondence means (31) to be associated, and the object selection means (31) is selected by an operation received on the operation surface of the second operation input device and the coordinate system associated by the coordinate correspondence means. One of the operation objects is selected from candidates.

  Further, the operation support system (1) according to claim 4 is the operation support system according to any one of claims 1 to 3, wherein the operation object display means (31) is the selection object candidate. The operation object (41) selected as the selection object candidate by the selection means (31) is distinguished from other operation objects and displayed.

  An operation support system (1) according to claim 5 is the operation support system according to any one of claims 1 to 4, wherein the operation surface (11) of the second operation input device (3) is provided. A projecting means (31) for projecting a plurality of projecting portions (21) at a predetermined arrangement interval on the near side, and the operation object display means (31) is selected by the selection object candidate selecting means by the selection object candidate selecting means. The operation object (41) selected as a candidate is displayed in an arrangement form corresponding to the arrangement form of the plurality of protrusions.

  The operation support system (1) according to claim 6 is the operation support system according to claim 5, wherein the predetermined number of the protrusions (21) protruded by the protrusion means (31). It is a number.

According to a seventh aspect of the present invention, there is provided an operation support method according to a first operation input device (2) and a second operation input device (2) which are provided on a vehicle handle (5) and receive a user operation via an operation surface (11). 3) and a display screen (37) for displaying an operation object (41) operated by the first operation input device and the second operation input device. An operation object display step of displaying the operation object on the display screen, and selecting a predetermined number of the operation objects as a selection object candidate from among the plurality of operation objects, and selecting the selection object candidate While accepting an operation for switching the operation object to another operation object, the first operation input device or the second operation input device receives a drag operation or a flick operation. If digits, and the operation object to be selected to the selected candidate to the other of the operation object, and selection candidate selection step of switching sequentially along the moving direction of the touch point in the drag operation or the flick operation,
Based on the operation received by the second operation input device, one of the operation objects is selected from the predetermined number of the operation objects selected as the selection object candidates by the selection object candidate selection step. And an object selection step of operating the selected operation object.

  Furthermore, a computer program according to an eighth aspect is provided on a steering wheel (5) of a vehicle, and receives a user operation via an operation surface (11), a first operation input device (2) and a second operation input device (3 ) And a display screen (37) for displaying an operation object (41) operated by the first operation input device (2) and the second operation input device (3). In the computer program to be performed, an operation object display function for displaying a plurality of operation objects on the display screen, and selecting a predetermined number of the operation objects as a selection object candidate among the plurality of operation objects, While accepting an operation of switching the operation object to be selected as the selection object candidate to another operation object, the first operation input device or the second operation input device When the drag operation or the flick operation is received, the operation object to be selected as the selection target candidate is sequentially transferred to another operation object along the moving direction of the touch point in the drag operation or the flick operation. Based on the selection target candidate selection function to be switched and the operation received by the second operation input device, any one of the predetermined number of the operation objects selected as the selection target candidates by the selection target candidate selection function And an object selection function for operating the selected operation object is executed.

  According to the operation support system according to claim 1 having the above-described configuration, when the user performs an operation of selecting any one of the operation objects from the plurality of operation objects displayed on the display screen, the operation object Even if there are many, it is possible to select and operate the operation object desired by the user with a small number of operations, and the user can select and operate the operation object from among a large number of operation objects only by operation on one operation input device. It becomes possible to finally select any desired operation object.

  In addition, according to the operation support system of the second aspect, the first operation input device and the second operation are performed except for accepting an operation of switching the operation object to be selected as a selection object candidate to another operation object. Since different functions are given to the input device, it is possible to quickly perform a complicated operation using a plurality of operation input devices, and it is also possible to perform an operation using only one operation input device in a necessary situation.

  According to the operation support system according to claim 3, when a selection target candidate is selected, the coordinate system of the display area in which the operation target selected as the selection target candidate is displayed and the second operation input Since the coordinate system on the operation surface of the device is made to correspond, and any one operation target object is selected from the selection target candidates displayed in the display area by the operation received on the operation surface of the second operation input device, the second It becomes possible to accurately select an arbitrary selection target candidate desired by the user from among the selection target candidates displayed on the display screen by an operation on the operation surface of the operation input device.

  According to the operation support system of the fourth aspect, the operation target set as the selection target candidate is displayed by being distinguished from other operation target, so that the current operation can be performed by operating the second operation input device. The user can clearly grasp the selectable operation objects.

  According to the operation support system according to claim 5, the operation target selected as a selection target candidate by projecting a plurality of protrusions at a predetermined arrangement interval to the near side of the operation surface of the second operation input device. Since the object is displayed in an arrangement mode corresponding to the arrangement mode of the plurality of protrusions, the user can visually recognize the correspondence between the protrusion and the operation target selected as the selection target by viewing the display screen. It can be easily grasped. Therefore, it is possible to appropriately select an arbitrary operation target desired by the user by operating the operation surface.

  In addition, according to the operation support system according to claim 6, since the number of operation objects to be selected as a selection object candidate is the number of protrusions protruded by the protrusion means, the selection of the operation object on the basis of the protrusions The operation can be easily performed.

  According to the operation support method described in claim 7, when the user performs an operation for selecting any one of the operation objects displayed on the display screen, there are many operation objects. Even in some cases, it is possible to select and operate the operation object desired by the user with a small number of operations, and the user can select from a large number of operation objects only by operation on one operation input device. It becomes possible to finally select an arbitrary operation object.

  Furthermore, according to the computer program of the eighth aspect, there are many operation objects when the user performs an operation of selecting any one of the operation objects from the plurality of operation objects displayed on the display screen. Even in this case, it is possible to select and operate an operation target desired by the user with a small number of operations, and to select an arbitrary desired operation from a large number of operation targets only by an operation on one operation input device. It becomes possible to finally select the operation object.

It is the block diagram which showed schematic structure of the operation assistance system which concerns on this embodiment. It is the figure which showed the arrangement | positioning aspect with respect to the vehicle of an operation input device and a display screen. It is the perspective view which showed the left operation input device. It is arrow sectional drawing which cut | disconnected the left operation input device by the line XX of FIG. It is the figure which showed the drive mode of the pin in a pin drive mechanism. It is the figure which showed the main screen displayed on the windshield of a vehicle. It is the figure which showed the setting operation screen displayed on the windshield of a vehicle. It is a flowchart of the operation assistance processing program which concerns on this embodiment. It is the figure which showed the main screen displayed on a windshield immediately after ACC was turned on. It is the figure which showed the selection area | region set on the operation surface of a touchpad. It is the figure which showed the arrangement | positioning aspect of the selection object candidate icon in a main screen. It is a flowchart of the sub-processing program of the left drag process. It is the figure which showed the arrangement | positioning aspect of the selection object candidate icon in a main screen. FIG. 10 is a diagram showing a selection target candidate switching mode when a right drag operation is received by the left operation input device in a state where the main screen shown in FIG. 9 is displayed. FIG. 10 is a diagram showing a selection target candidate switching mode when a left drag operation is received by the left operation input device in a state where the main screen shown in FIG. 9 is displayed. It is a flowchart of the sub-processing program of the left flick process. FIG. 10 is a diagram illustrating a selection target candidate switching mode when a right flick operation is received by the left operation input device in a state where the main screen illustrated in FIG. 9 is displayed. It is a flowchart of the sub process program of a right touch process.

  Hereinafter, an embodiment of the operation support system according to the present invention will be described in detail with reference to the drawings.

  First, a schematic configuration of the operation support system 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of an operation support system 1 according to the present embodiment.

  As shown in FIG. 1, an operation support system 1 according to the present embodiment is installed in a vehicle, and a left operation input device 2 and a right operation input device 3 that receive an operation by a user, and a vehicle front that is installed on a dashboard. Head-up display (HUD) 4 that projects an image on glass, left operation input device 2 and right operation input device 3, and various vehicle-mounted devices (for example, audio and navigation devices) connected via a vehicle-mounted network such as CAN, It is comprised from the control apparatus etc. of vehicle equipment (for example, an air-conditioner, a power window, a door mirror).

  Here, the left operation input device 2 is disposed in the left spoke portion of the handle 5 of the vehicle as shown in FIG. The right operation input device 3 is disposed on the right spoke portion of the vehicle handle 5. On the front surface of the left operation input device 2 and the right operation input device 3, a touch pad (operation surface) for receiving a user's touch operation is provided as described later, and the user supports the handle 5 in a state described later. The left operation input device 2 and the right operation input device 3 can be operated by a touch operation. In the operation support system 1 according to the present embodiment, the user can execute various application programs and various types of vehicles mounted on the vehicle by the executed application programs, particularly via the left operation input device 2 and the right operation input device 3. It is configured to be able to operate a vessel and vehicle equipment. For example, Internet, mail, navigation device destination setting operation, map image scrolling operation, air conditioner power switching operation, temperature adjustment operation, audio channel operation, volume adjustment operation, window opening / closing, door locking, etc. Is possible.

Below, the structure of the left operation input device 2 and the right operation input device 3 is demonstrated in detail. Note that the left operation input device 2 and the right operation input device 3 basically have the same configuration, and in the following, the left operation input device 2 will be described as an example. Description is omitted. 3 is a perspective view showing the left operation input device 2, and FIG. 4 is a cross-sectional view taken along the line XX in FIG.
As shown in FIGS. 3 and 4, the left operation input device 2 includes a touch pad 11 that detects a touch operation from a user, a piezoelectric element 12 that vibrates the touch pad 11, and a cylindrical shape below the touch pad 11. This is basically composed of a plurality of pin drive mechanisms 13 that move the pins up and down, a base 14 that fixes the pin drive mechanism 13 at a predetermined position, and a control board 16 on which an operation support ECU 15 described later is mounted. The touch operation is various operations performed by the user touching the touch surface. For example, an operation of touching (touching) any point of the touch pad 11, an operation of releasing the touched state (touch state), There are drag operation and flick operation.

  The touch pad 11 includes a cover sheet 17 that protects the touch pad 11 in order from the upper surface side, a conductive film 18 that detects a user's touch operation based on a change in capacitance, the cover sheet 17 and the conductive film 18. 14 and a top plate 19 supported upward. And operation assistance ECU15 detects the touch coordinate which the user touched with the predetermined detection period by the coordinate system of the touchpad 11 so that it may mention later. Note that the touch coordinates are the coordinates of the position of the touch point where the user touches the operation surface of the touch pad 11 (it is assumed that the capacitance has changed in the capacitance method). Moreover, although a detection period changes with kinds of the touchpad 11, it becomes 200 Hz-2 kHz, for example. As the touch pad 11, a detection method (for example, a resistance film method) other than the electrostatic capacitance method may be used.

  On the other hand, the piezoelectric element 12 is attached to the lower surface of the top plate 19. Further, the touch pad 11 and the base portion 14 are configured so as to provide a predetermined gap between them by support portions 20 arranged at the four corners. As a result, when the piezoelectric element 12 is distorted by applying a signal voltage to the piezoelectric element 12, the touch pad 11 can be vibrated in the vertical direction by the distortion. In the operation support system 1 according to the present embodiment, when a predetermined area in the touch pad 11 is touched on as described later, or when the user presses the operation surface of the touch pad 11, a predetermined amount is applied to the operation surface. When the above deformation (bending) occurs, the touch pad 11 is vibrated. As a result, it is possible to give a tactile sensation when the user operates an existing button. In addition, the operation support ECU 15 can arbitrarily set the vibration direction and the amplitude of vibration by changing the voltage value. The vibration period varies depending on the type and installation mode of the piezoelectric element 12, but is, for example, 250 Hz to 2 kHz. The piezoelectric element 12 does not need to be disposed in direct contact with the touch pad 11 and may be disposed with respect to other members as long as vibration can be transmitted to the touch pad 11. Further, as a means for causing the touch pad 11 to vibrate, a small vibration motor or the like may be used instead of the piezoelectric element 12.

  The piezoelectric element 12 also functions as a sensor that detects deformation (deflection) generated on the operation surface of the touch pad 11. That is, when the operation surface of the touch pad 11 is deformed, the piezoelectric element 12 attached to the lower surface of the top plate 19 is deformed accordingly. Therefore, the operation support ECU 15 can detect that a deformation (deflection) has occurred on the operation surface of the touch pad 11 based on a signal from the piezoelectric element 12.

  Further, the pin driving mechanism 13 is arranged at a predetermined arrangement interval (for example, 1 cm in length and width) at a total of nine locations of 3 × 3 with respect to the base portion 14 so that the tip portion of the pin 21 faces the touch pad 11. . The pin 21 is supported so as to be movable up and down along the arrow Z direction in FIG. Further, an insertion hole 22 is formed on the bottom surface of the pin 21, and a support member 23 is inserted into the insertion hole 22. A piezoelectric element 24 is connected to the bottom surface of the support member 23. Then, the operation support ECU 15 uses the frictional force generated between the insertion hole 22 and the support member 23 and applies a signal voltage to the piezoelectric element 24 to vibrate, thereby causing the pin 21 to move in either the upward direction or the downward direction. It is possible to move in any direction. The details of the drive mechanism of the pin 21 are well known and will be omitted.

  In addition, the conductive film 18 and the top plate 19 constituting the touch pad 11 have through holes 25 formed at a total of nine positions corresponding to the pins 21, and when the pins 21 are moved upward, The tip of the pin 21 can be positioned above the upper surface of the touch pad 11. Then, the operation support ECU 15 drives each pin driving mechanism 13 so that the tip of the pin 21 as shown in FIG. 5 is located below the operation surface of the touch pad 11 in the nine pin driving mechanisms 13. It is possible to selectively switch between the “non-projecting state” and the “projecting state” where the tip of the pin 21 is above the operation surface of the touch pad 11. As shown in FIG. 5, the cover sheet 17 may be formed using a stretchable material (for example, silicon resin) or the like in order to protect the pins 21 that are projected from the touch pad 11. desirable.

  In addition, on the operation surface of the left operation input device 2 and the right operation input device 3, particularly on the operation surface of the left operation input device 2, an operation button 27 that is a hardware operation unit is provided as shown in FIG. 3. . Here, examples of the operation button 27 include a home button and a shortcut button. When the home button is pressed while the application is being operated as will be described later, the application is temporarily terminated, and the display screen displayed by the HUD 4 is also restored to the initial screen. When the shortcut button is pressed, functions associated with the shortcut button in advance (for example, activation of a web browser, map scale change operation, air conditioner set temperature change operation, etc.) are executed. The operation button 27 may be provided on the operation surface of the right operation input device 3. In addition, the operation button 27 may be a key that is virtually arranged on the operation surface instead of a hardware key that is a hardware operation unit. In this case, the operation support ECU 15 determines that the operation button 27 has been pressed when the touch coordinates are in the coordinate range associated with the operation button 27 in advance.

  An operation support ECU (Electronic Control Unit) 15 mounted on the control board 16 is an electronic control unit that controls the entire left operation input device 2, and includes a CPU 31 as a calculation device and a control device, and a CPU 31. In addition to the RAM 32 used as a working memory and a control program in performing various arithmetic processes, a ROM 33 in which an operation support processing program (see FIG. 8) described later is recorded, a program read from the ROM 33, and a menu described later An internal storage device such as a flash memory 34 for storing icon selection history and touch coordinate history is provided. The operation support ECU 15 constitutes various means as processing algorithms. For example, the operation object display means displays a plurality of operation objects (icons, etc.) on the display screen. The selection target candidate selecting means selects a predetermined number of operation objects among the plurality of operation objects as selection target candidates, and performs an operation of switching the operation object to be selected as a selection target candidate to another operation object. During the reception, when a drag operation or a flick operation is received in the left operation input device 2 or the right operation input device 3, the operation object to be selected as a selection target candidate is moved to another operation object. Are sequentially switched along the moving direction of the touch point. When the selection target candidate is selected by the selection target candidate selection unit, the coordinate correspondence unit includes a coordinate system of a display area in which the operation target selected as the selection target candidate is displayed, and an operation surface of the right operation input device 3. Match the coordinate system at. The protruding means causes the plurality of pins 21 to protrude at a predetermined arrangement distance from the operation surface of the touch pad 11 of the right operation input device 3.

  The HUD 4 is installed on the dashboard 35 as shown in FIG. 2, and is set so that the image projection direction is near the lower edge of the windshield 36 in front of the driver's seat. Various images are displayed on the display target area 37 of the windshield 36.

Here, FIGS. 6 and 7 are diagrams showing an example of an image displayed on the display target area 37 of the windshield 36 by the HUD 4.
First, in a state immediately after the ACC is turned on, the main screen 40 shown in FIG. The main screen 40 is a basic operation screen that receives operations for executing various applications that can be executed in the operation support system 1, and corresponds to a home screen, a desktop, or the like. As shown in FIG. 6, a plurality of menu icons (operation objects) 41 are arranged on the main screen 40. The menu icon 41 is an icon that is selected and operated to start an application that can be executed in the operation support system 1. When the menu icon 41 is operated by the user, an application corresponding to the operated menu icon 41 is displayed. Start is started. In addition to the web browser and mail application, the applications to be started include applications for performing operations on various vehicle-mounted devices (for example, audio and navigation devices) and vehicle devices (for example, air conditioners, power windows, door mirrors). It is. When any menu icon 41 is selected and operated on the main screen 40, the corresponding application is activated, and an operation screen corresponding to the activated application is displayed. In particular, when an application corresponding to the operation of various vehicle-mounted devices and vehicle equipment mounted on the vehicle is activated, a setting operation screen 42 (corresponding to the operation of the air conditioner in the example shown in FIG. 7) is shown in FIG. Is displayed. On the setting operation screen 42, the current setting state (parameters, etc.) of the device to be operated is displayed in the setting icons 43 to 45, and the setting state can be confirmed and changed.

  In addition, a predetermined number of menu icons 41 among the menu icons 41 displayed on the main screen 40 are displayed in an enlarged manner than the other icons, and the frame surrounding the icons is also thicker. Here, in the operation support system 1 according to the present embodiment, since the menu icon 41 is selected using the touch pad 11 having a small operation surface area, the selection target (hereinafter referred to as the selection) is directly selected by the right operation input device 3. There is a limit to the number of menu icons 41 that can be set as target candidates). For example, in this embodiment, up to nine icons, which is the number of pins 21 included in the right operation input device 3, can be selected as candidates for selection. Therefore, when there are a large number of menu icons 41 displayed on the main screen 40, it is necessary to identify and display the menu icon 41 as a selection target candidate. Is enlarged and the frame surrounding the icon becomes thicker. Furthermore, the menu icon 41 as a selection target candidate is displayed in an arrangement manner corresponding to the arrangement manner of the pins 21 arranged on the touch pad 11 of the right operation input device 3. That is, they are arranged at an equal interval of 3 × 3. Note that the menu icon 41 as a selection target candidate can be switched by a drag operation or a flick operation on the operation surface of the touch pad 11 of the left operation input device 2 or the right operation input device 3 as described later.

  When there are a particularly large number of menu icons 41, the menu icons 41 are divided into a plurality of groups depending on the type of application to be activated. The menu icons 41 belonging to the group are displayed on the main screen 40 in units of groups. For example, in the example illustrated in FIG. 6, a group of menu icons 41 for starting an application for performing operations related to the navigation device, a group of menu icons 41 for starting an application for performing operations regarding audio, a general application, It is divided into a group of menu icons 41 for starting applications for performing operations related to other vehicle devices, and the group of menu icons 41 related to other vehicle devices is a display target at the present time. In addition, the user can sequentially change the group to be displayed on the main screen 40 by performing a left / right flick operation or a drag operation on the left operation input device 2 or the right operation input device 3.

Further, the user can select and operate an arbitrary menu icon 41 from among the menu icons 41 that are candidates for selection by operating the right operation input device 3. Specifically, the position of the touch pad 11 corresponding to the position of the menu icon 41 desired to be selected (for example, when the menu icon 41 of “air conditioner” is selected in the example shown in FIG. 6, the area around the pin 21 at the upper left) Is touched on in the right operation input device 3, and the menu icon 41 is selected. It should be noted that pressing the pin 21 corresponding to the position of the menu icon 41 or causing the operation surface to bend a certain amount or more may be used as the selection condition. When one of the menu icons 41 is selected and operated, a corresponding application is activated, and an operation screen corresponding to the activated application is displayed.
The display means for displaying the main screen 40 and the setting operation screen 42 is configured to be mounted on the head of the driver of the vehicle, such as a liquid crystal display or an organic EL display installed in the vehicle, instead of the HUD 4. A head mounted display (HMD) or the like may be used. Further, a display of a navigation device may be used.

  On the other hand, the CAN (controller area network) interface 47 included in the left operation input device 2 and the right operation input device 3 is an in-vehicle network standard that performs multiplex communication between various in-vehicle devices installed in a vehicle and control devices for vehicle equipment. This is an interface for inputting / outputting data to / from a certain CAN. The operation support ECU 15 is connected to a control device (for example, the navigation device 48, the AV device 49, the air conditioner control ECU 50, etc.) of various vehicle-mounted devices and vehicle equipment via the CAN so as to be able to communicate with each other. When the operation support ECU 15 receives a user operation in the left operation input device 2 or the right operation input device 3 in a state in which a specific application is activated, the operation support ECU 15 transmits an operation signal to various vehicle-mounted devices or vehicle device control devices. Is transmitted, the various on-vehicle devices and vehicle equipment are operated via the left operation input device 2 and the right operation input device 3.

  Next, an operation support processing program executed by the operation support ECU 15 in the operation support system 1 having the above configuration will be described with reference to FIG. FIG. 8 is a flowchart of the operation support processing program according to the present embodiment. Here, the operation support processing program is a program that is executed after the ACC of the vehicle is turned on, and activates and operates various applications via the left operation input device 2 and the right operation input device 3. The programs shown in the flowcharts of FIGS. 8, 12, 16, and 18 below are stored in the RAM 32 and the ROM 33 provided in the operation support system 1, and are executed by the CPU 31.

  First, in step (hereinafter abbreviated as S) 1 in the operation support processing program, the CPU 31 executes various initialization processes related to the operation support processing program. In the initialization process, a history of touch coordinates, setting information of selection target candidate icons described later, and the like are initialized.

  Next, in S <b> 2, the CPU 31 drives the HUD 4 to display the main screen 40 on the display target area 37 of the windshield 36. On the main screen 40, as described above, a plurality of menu icons 41 to be operated and selected by operations by the left operation input device 2 and the right operation input device 3 are displayed (see FIG. 6). Further, as described above, the main screen 40 can be directly selected by the right operation input device 3, that is, the menu icon 41 (hereinafter, the selection target candidate icon) that becomes the selection target candidate by the right operation input device 3. Is set). For example, in the example illustrated in FIG. 6, nine icons from “air conditioner” to “communication” are selection target candidate icons, and the selection target candidate icons can be distinguished from other icons (for example, the size is increased and the icon is displayed). The surrounding frame is highlighted).

  Further, on the main screen 40 at the time of display (initial state), the selection history of the menu icon 41 stored in the flash memory 34 is read, and the menu icon 41 having a large number of selections is given priority as a selection target candidate icon. Set. Specifically, in the initial state, the menu icons 41 with the highest selection count are displayed in order from the upper left to the lower right in the main screen 40, and among the plurality of menu icons 41 displayed on the main screen 40, the upper left A total of nine menu icons 41 of 3 × 3 located on the side are selection target candidate icons. For example, FIG. 9 shows an example of the main screen 40 that is displayed immediately after the ACC is turned on. As the selection target candidate icons, “Destination”, “Map Scale”, “Map Scroll” are displayed in descending order of selection frequency. ..., Menu icons 41 relating to nine navigation devices “point registration” are set. Also, as shown in FIG. 9, when the menu icons 41 are divided into a plurality of groups depending on the type of application to be activated, they are displayed side by side in the selected number of times for each group. Further, in the initial state, a predetermined group (for example, a menu icon 41 for starting an application for performing an operation related to the navigation device) may be displayed on the main screen 40, or a group with a large number of selections is given priority. It is good also as displaying on.

  Further, the selection target candidate icon displayed on the main screen 40 is switched to another menu icon 41 by a drag operation or a flick operation on the operation surface of the touch pad 11 of the left operation input device 2 or the right operation input device 3 described later. However, the arrangement mode of the menu icon 41 as the selection candidate icon is fixed. That is, the right operation input device 3 is arranged at an equal interval of 3 × 3 corresponding to the arrangement mode of the pins 21 arranged on the touch pad 11.

  Subsequently, in S3, the CPU 31 selects the selection target candidate icons for the operation surface of the right operation input device 3 for each selection target candidate icon corresponding to the arrangement mode of the selection target candidate icons on the main screen 40 displayed in S2. A selection area for selecting a target candidate icon is set. Specifically, first, the center coordinate of each menu icon 41 set as the selection target candidate icon is acquired in the coordinate system of the display screen, and the acquired center coordinate is the coordinate of the operation surface of the touch pad 11 of the right operation input device 3. Convert to system. Thereafter, a peripheral area of the operation surface including the converted center coordinates (an area corresponding to the size of the menu icon 41) is set as a selection area. Further, the selection area is set so as to include a through hole 25 for projecting the pin 21 in each selection area. Further, in S3, the pin drive mechanism 13 is driven so that the pins 21 included in each set selection area are in the “projection state” projecting from the operation surface (see FIG. 5).

  For example, when the main screen 40 shown in FIG. 9 is displayed, nine selection areas 51 to 59 are set on the operation surface of the touch pad 11 of the right operation input device 3 as shown in FIG. Here, the selection area 51 is a selection area associated with the “destination” menu icon 41, and is set to an area including the through hole 25 of the pin 21 located on the left side of the upper stage. The selection area 52 is a selection area associated with the “map scale” menu icon 41 and is set to an area including the through hole 25 of the pin 21 located on the left side of the center stage. Similarly, the other selection areas 53 to 59 are set in association with each selection target candidate icon.

  Subsequently, in S <b> 4, based on the detection signal transmitted from the touch pad 11, the CPU 31 receives at least one point on the operation surface of the touch pad 11 of the left operation input device 2 or the right operation input device 3. It is determined whether or not it is in a touched state (hereinafter referred to as a touch state). For example, when the touch pad 11 is of a resistive film type or a capacitance type, the user can detect the touch pad 11 when a pressure greater than a predetermined value is detected or when a change in capacitance greater than the predetermined value is detected. It is determined that is touched.

  And when it determines with it being in the touch state which touched the operation surface of any touchpad 11 of the left operation input device 2 and the right operation input device 3 (S4: YES), it transfers to S5. On the other hand, when it is determined that neither the operation surface of the touch pad 11 of the left operation input device 2 or the right operation input device 3 is in a touched state (S4: NO), the process waits until the touch state is reached. .

  In S <b> 5, the CPU 31 determines whether or not the main screen 40 is particularly displayed on the display target area 37 of the windshield 36. Here, the main screen 40 is a screen on which a plurality of menu icons 41 to be operated and selected by operations by the left operation input device 2 and the right operation input device 3 are displayed as described above. In the state where 40 is displayed, the switching operation of the menu icon 41 set as the selection candidate icon is accepted.

  When it is determined that the main screen 40 is displayed (S5: YES), that is, when it is determined that the switching operation of the menu icon 41 set as the selection target candidate icon is accepted. To S7. On the other hand, when it is determined that the main screen 40 is not displayed (S5: NO), that is, an operation screen (FIG. 7) of an application other than the main screen 40 is displayed and set as a selection target candidate icon. When it is determined that the menu icon 41 is not in a state of accepting the switching operation, the process proceeds to S6.

  In S <b> 6, the CPU 31 performs operation processing by restricting operations other than the operation of touching on one point on the normal operation surface in the right operation input device 3. For example, icons (for example, setting icons 43 to 45) and buttons selected on the operation screen (FIG. 7) of the application are selected based on the touch-on operation on the operation surface of the touch pad 11 of the right operation input device 3. The right operation input device 3 is configured not to accept a drag operation or a flick operation. Note that not accepting a drag operation or flick operation means a configuration in which the CPU does not execute processing based on the drag operation or flick operation. Even when a screen other than the main screen 40 that can be operated by a drag operation or a flick operation (for example, a screen that can be scrolled to the left or right) is displayed, the drag operation or the flick operation to the right operation input device 3 is performed. Then, no operation is performed, and an operation (for example, a screen scroll process) is performed based only on a drag operation or a flick operation to the left operation input device 2. Thereafter, the process proceeds to S26.

  On the other hand, in S <b> 7, the CPU 31 determines whether or not the operation surface of the touch pad 11 of the left operation input device 2 is particularly touched.

  And when it determines with it being in the touch state which touched the operation surface of the touchpad 11 of the left operation input device 2, (S7: YES), it transfers to S8. On the other hand, when it is determined that the touch surface of the touch pad 11 of the right operation input device 3 is touched (S7: NO), the process proceeds to S19.

  In S <b> 8, based on the detection signal transmitted from the touch pad 11, the CPU 31 detects the touch coordinates, which are the coordinates of the point touched by the user, in the coordinate system of the operation surface of the touch pad 11. For example, when the touch pad 11 is a resistive film type or a capacitance type, the touch coordinate is detected by detecting the position where the pressure has changed or the position of the current that flows based on the change in the capacitance. To do.

  Next, in S9, the CPU 31 reads the touch coordinate detection history detected in S8 from the RAM 32 or the like, and determines whether or not a drag operation has been performed on the operation surface of the touch pad 11 of the left operation input device 2. To do. The drag operation refers to an operation for moving the touch coordinates while maintaining a touch state on the operation surface.

  When it is determined that a drag operation has been performed on the operation surface of the touch pad 11 of the left operation input device 2 (S9: YES), the process proceeds to S10. On the other hand, when it is determined that the drag operation is not performed on the operation surface of the touch pad 11 of the left operation input device 2 (S9: NO), the process proceeds to S11.

  In S10, the CPU 31 executes a left drag process (FIG. 12) described later. The left drag process is a process of switching the selection target candidate icons based on a drag operation received on the operation surface of the touch pad 11 of the left operation input device 2 as described later. Thereafter, the operation support processing program ends. Thereafter, the process proceeds to S13.

  In S <b> 11, the CPU 31 reads the touch coordinate detection history detected in S <b> 8 from the RAM 32 or the like, and determines whether or not a flick operation has been performed on the operation surface of the touch pad 11 of the left operation input device 2. Note that the flick operation refers to an operation for performing a touch-off (a transition from a state of touching the operation surface to a state of not touching) while moving the touch coordinates.

  When it is determined that a flick operation has been performed on the operation surface of the touch pad 11 of the left operation input device 2 (S11: YES), the process proceeds to S12. On the other hand, when it is determined that the flick operation is not performed on the operation surface of the touch pad 11 of the left operation input device 2 (S11: NO), the process proceeds to S14.

  In S12, the CPU 31 executes a left flick process (FIG. 16) described later. Note that the left flick process is a process of switching the selection target candidate icons based on the flick operation received on the operation surface of the touch pad 11 of the left operation input device 2 as described later.

  Next, in S <b> 13, the CPU 31 newly sets an operation area corresponding to the arrangement mode of the selection target candidate icons after being switched in S <b> 10 or S <b> 12, and further newly on the operation surface of the right operation input device 3. The pin drive mechanism 13 is driven so that the pin 21 in the set operation region is in the “projection state” in which the pin 21 projects. The details are the same as in S3, and a description thereof will be omitted. In addition, when the number of selection target candidate icons is increased or decreased by switching the selection target candidate icons in S10 or S12, the number of selection regions and the number of pins to be “projected” are also increased or decreased. Thereafter, the process proceeds to S26.

  On the other hand, in S <b> 14, the CPU 31 determines whether the home button arranged on the operation surface of the touch pad 11 of the left operation input device 2 has been operated. When the home button is composed of a hardware operation unit, it is determined whether or not the home button has been operated based on a detection signal transmitted from the operation unit. On the other hand, when the home button is composed of virtual keys arranged on the operation surface, it is determined that the home button is operated when the touch coordinates detected in S8 are in the coordinate range previously associated with the home button. To do.

  And when it determines with the home button arrange | positioned at the operation surface of the touchpad 11 of the left operation input device 2 having been operated (S14: YES), it transfers to S15. On the other hand, when it is determined that the home button arranged on the operation surface of the touch pad 11 of the left operation input device 2 is not operated (S14: NO), the process proceeds to S16.

  In S15, the CPU 31 once terminates the currently executed application, and the display screen displayed in the display target area 37 of the windshield 36 also returns to the main screen 40 (FIG. 9) in the initial state. When the home button is a virtual key, a signal voltage corresponding to a vibration waveform set in advance when touch-on is detected is applied to the piezoelectric element 12, thereby distorting the piezoelectric element 12 and the left operation input device. The second touchpad 11 is also vibrated. Then, by causing the vibration, a tactile sensation as if the actual button is pressed is given to the user who touches the operation surface of the touch pad 11. Thereafter, the process proceeds to S26. However, when the selection target candidate icon is changed by returning to the main screen 40 (FIG. 9) in the initial state, the process proceeds to S13.

  On the other hand, in S <b> 16, the CPU 31 determines whether or not a shortcut button arranged on the operation surface of the touch pad 11 of the left operation input device 2 has been operated. When the shortcut button includes a hardware operation unit, it is determined whether or not the shortcut button has been operated based on a detection signal transmitted from the operation unit. On the other hand, if the shot cut button is composed of virtual keys arranged on the operation surface, the shortcut button is operated when the touch coordinates detected in S6 are in the coordinate range previously associated with the shortcut button. judge.

  And when it determines with the shortcut button arrange | positioned at the operation surface of the touchpad 11 of the left operation input device 2 having been operated (S16: YES), it transfers to S17. On the other hand, when it is determined that the shortcut button arranged on the operation surface of the touch pad 11 of the left operation input device 2 is not operated (S16: NO), the process proceeds to S18.

  In S17, the CPU 31 temporarily terminates the currently executed application, and executes functions associated with the shortcut buttons (for example, activation of a web browser, map scale change operation, air conditioner set temperature change operation, etc.). When the shortcut button is a virtual key, a signal voltage corresponding to a vibration waveform set in advance when touch-on is detected is applied to the piezoelectric element 12, thereby distorting the piezoelectric element 12 and the left operation input device. The second touchpad 11 is also vibrated. Then, by causing the vibration, a tactile sensation as if the actual button is pressed is given to the user who touches the operation surface of the touch pad 11. Thereafter, the process proceeds to S26.

  In S18, the CPU 31 executes other processing based on the touch operation. Thereafter, the process proceeds to S26.

  On the other hand, in S <b> 19 that is executed when it is determined that the touch surface of the touch pad 11 of the right operation input device 3 is touched, the CPU 31 executes the user based on the detection signal transmitted from the touch pad 11. The touch coordinates, which are the coordinates of the point to be touched, are detected in the coordinate system of the operation surface of the touch pad 11. For example, when the touch pad 11 is a resistive film type or a capacitance type, the touch coordinate is detected by detecting the position where the pressure has changed or the position of the current that flows based on the change in the capacitance. To do.

  Next, in S20, the CPU 31 converts the touch coordinates detected in the coordinate system of the operation surface of the touch pad 11 in S19 into a coordinate system of a display area in the display screen, in particular, a display target candidate icon is displayed. . For example, when the main screen shown in FIG. 11 is displayed, a rectangular shape including nine selection target candidate icons “Destination”, “Map Scale”, “Map Scroll”,. It is converted into the coordinate system of the display area 60. Thereby, the coordinate system of the display area 60 on which the selection target candidate icon is displayed can correspond to the coordinate system on the operation surface of the touch pad 11 of the right operation input device 3. The display area 60 has a shape corresponding to the shape of the operation surface of the touch pad 11 of the right operation input device 3.

  Next, in S21, the CPU 31 reads the touch coordinate detection history detected in S19 from the RAM 32 or the like, and determines whether or not a drag operation has been performed on the operation surface of the touch pad 11 of the right operation input device 3. To do.

  When it is determined that a drag operation has been performed on the operation surface of the touch pad 11 of the right operation input device 3 (S21: YES), the process proceeds to S22. On the other hand, when it is determined that the drag operation is not performed on the operation surface of the touch pad 11 of the right operation input device 3 (S21: NO), the process proceeds to S23.

  In S22, the CPU 31 executes a right drag process. The right drag process is a process of switching the selection target candidate icons based on the drag operation received on the operation surface of the touch pad 11 of the right operation input device 3. Here, the right drag process is basically the same as the left drag process (FIG. 12) described later, except that the object for accepting the operation is not the left operation input apparatus 2 but the right operation input apparatus 3. is there. Therefore, details are omitted. Thereafter, the process proceeds to S26.

  On the other hand, in S23, the CPU 31 reads the touch coordinate detection history detected in S19 from the RAM 32 or the like, and determines whether or not a flick operation has been performed on the operation surface of the touch pad 11 of the right operation input device 3. .

  When it is determined that a flick operation has been performed on the operation surface of the touch pad 11 of the right operation input device 3 (S23: YES), the process proceeds to S24. On the other hand, when it is determined that the flick operation is not performed on the operation surface of the touch pad 11 of the right operation input device 3 (S23: NO), that is, one point on the operation surface is normally in a touch state. If the touch operation is performed, the process proceeds to S25.

  In S24, the CPU 31 performs a right flick process. Note that the right flick process is a process of switching the selection target candidate icons based on the flick operation received on the operation surface of the touch pad 11 of the right operation input device 3. Here, the right flick process is basically the same process as the left flick process (FIG. 16) described later except that the object to receive the operation is not the left operation input apparatus 2 but the right operation input apparatus 3. is there. Therefore, details are omitted. Thereafter, the process proceeds to S26.

  In S25, the CPU 31 executes right touch processing (FIG. 18) described later. As will be described later, the right touch process is performed by selecting any of the menu icons 41 set as the selection target candidate icons based on the touch operation received on the operation surface of the touch pad 11 of the right operation input device 3. This is processing for selecting the menu icon 41. Thereafter, the process proceeds to S26.

  Next, in S26, the CPU 31 determines whether or not the selection of the menu icon 41 and the operation on the selected menu icon 41 are to be ended. For example, when the ACC is turned off or when an operation with a specific operation button is accepted, it is determined that the selection of the menu icon 41 and the operation on the selected menu icon 41 are finished.

  When it is determined that the selection of the menu icon 41 and the operation on the selected menu icon 41 are to be continued (S26: NO), the process returns to S4. On the other hand, when it is determined that the selection of the menu icon 41 and the operation on the selected menu icon 41 are to be ended (S26: YES), the operation support processing program is ended. Thereafter, all the pins 21 are brought into a non-projecting state, and the display of the HUD 4 is also turned off.

  Next, the sub-process of the left drag process executed in S10 will be described with reference to FIG. FIG. 12 is a flowchart of a sub-processing program for left drag processing.

  First, in S31, the CPU 31 acquires the number of menu icons 41 to be switched as selection target candidate icons by the drag operation received on the operation surface of the touch pad 11 of the left operation input device 2. Specifically, this corresponds to the number of menu icons 41 set as selection target candidate icons immediately before accepting the drag operation. Here, the number of menu icons 41 set as the selection target candidate icons is the maximum value of the number of pins 21 arranged on the touch pad 11 of the right operation input device 3, and any number of 1 to 9 is selected. Become. For example, when a drag operation is accepted while the main screen 40 shown in FIG. 6 is displayed, the number of menu icons 41 to be switched as selection target candidate icons is “9”. On the other hand, when a drag operation is accepted while the main screen 40 shown in FIG. 13 is displayed, the number of menu icons 41 to be switched as selection target candidate icons is “7”.

  Next, in S32, the CPU 31 generates a vibration waveform that vibrates the operation surface of the touchpad 11 of the right operation input device 3 based on the number of menu icons 41 to be switched as the selection target candidate icons acquired in S31. calculate. Specifically, the greater the number of menu icons 41 acquired in S31, the greater the amplitude of the vibration waveform.

  Subsequently, in S33, the CPU 31 applies a signal voltage corresponding to the vibration waveform calculated in S32 to the piezoelectric element 12, thereby distorting the piezoelectric element 12 and vibrating the touch pad 11 of the left operation input device 2. The vibration process is continuously performed at a predetermined vibration cycle from the start of the drag operation to the end of the drag operation. Then, by generating vibration, a tactile sensation as if a finger slides on a plurality of existing buttons is given to a user who touches the operation surface of the touch pad 11.

  Next, in S <b> 34, the CPU 31 moves the touch coordinates along the drag operation received on the operation surface of the touch pad 11 of the left operation input device 2 (hereinafter referred to as the drag direction) and the movement speed of the touch coordinates associated with the drag operation ( (Hereinafter referred to as drag speed).

  Thereafter, in S35, the CPU 31 determines whether or not there is another menu icon 41 in the drag direction of the menu icon 41 set as the currently selected candidate icon among the menu icons 41 currently displayed on the main screen 40. . For example, when a left drag operation is received in a state where the main screen 40 shown in FIG. 13 is displayed, it is determined that there is another menu icon 41. On the other hand, when a rightward drag operation is received, it is determined that there is no other menu icon 41.

  When it is determined that there is another menu icon 41 in the drag direction of the menu icon 41 set as the selection target candidate icon (S35: YES), that is, the selection target candidate icon is in the drag direction on the main screen 40. If it is not located at the end, the process proceeds to S36. On the other hand, when it is determined that there is no other menu icon 41 in the drag direction of the menu icon 41 set as the selection target candidate icon (S35: NO), that is, the selection target candidate icon is in the drag direction on the main screen 40. If it is located at the end, the process proceeds to S38.

  In S <b> 36, the CPU 31 does not change the menu icon 41 displayed on the main screen 40, and performs a process of switching the menu icon 41 set as the selection target candidate icon to another menu icon 41 in the drag direction at a drag speed.

  Here, FIG. 14 is a diagram illustrating a switching mode of selection target candidate icons when a right drag operation is received in a state where the main screen 40 illustrated in FIG. 9 is displayed. In the example illustrated in FIG. 14, when the user performs a drag operation to the right with respect to the operation surface of the touch pad 11 of the left operation input device 2, the menu icons 41 set as selection target candidate icons are sequentially displayed on the right side. Switch to the menu icon 41 at Then, as the new menu icon 41 is set as the selection target candidate icon, the menu icon 41 set as the leftmost selection target candidate icon is sequentially removed from the selection target candidate icon.

  For example, in the example shown in FIG. 14, nine menu icons 41 “Destination”,... “Point registration” are set as selection target candidate icons immediately before accepting the drag operation. Thereafter, when a drag operation is performed at a distance corresponding to the arrangement interval of the menu icons 41 in the right direction, the selection target candidate icons are displayed in the nine menu icons 41 of “present location”,. Switch. Thereafter, when a drag operation of a distance corresponding to the arrangement interval of the menu icons 41 is further performed, the selection target candidate icons are changed to nine menu icons 41 of “point search (TEL)”,. Switches. It should be noted that the switching speed of the selection target candidate icon is proportional to the drag speed.

  In S37, the CPU 31 changes the display mode of the menu icon 41 on the main screen 40 corresponding to the selection target candidate icon after being switched in the process of S36. Specifically, the menu icon 41 newly set as the selection target candidate icon is highlighted (enlarged in size and thick in the frame), and the highlighting is canceled for the menu icon 41 outside the selection target candidate icon ( Restore size and frame thickness). As a result, the menu icon 41 as the selection target candidate icon can be changed by the drag operation on the operation surface of the touch pad 11 of the left operation input device 2, and the menu icon 41 currently set as the selection target candidate icon can be easily changed. It becomes possible to identify.

  On the other hand, in S <b> 38, the CPU 31 changes the menu icon 41 displayed on the main screen 40 based on the drag operation received on the operation surface of the touch pad 11 of the left operation input device 2. As a result, the menu icon 41 set as the selection target candidate icon is also switched.

  Here, FIG. 15 is a diagram showing a switching mode of selection target candidate icons when a leftward drag operation is received in a state where the main screen 40 shown in FIG. 9 is displayed. In the example illustrated in FIG. 15, when the user performs a drag operation to the left with respect to the operation surface of the touch pad 11 of the left operation input device 2, the group of menu icons 41 to be displayed on the main screen 40 is It changes from the group of the menu icon 41 regarding a navigation apparatus to the group of the menu icon 41 regarding another vehicle apparatus. As a result, the menu icon 41 set as the selection target candidate icon is also set for the newly displayed menu icon 41, and the menu icon set as the selection target candidate icon before accepting the drag operation. All 41 are excluded from the selection target candidate icons. Then, among the menu icons 41 newly displayed on the main screen 40, a predetermined number (up to nine) of menu icons 41 located on the rightmost side are newly set as selection target candidate icons.

  For example, in the example shown in FIG. 15, nine menu icons 41 “Destination”,... “Point registration” are set as selection target candidate icons immediately before accepting the drag operation. After that, when a drag operation is performed at a distance corresponding to the arrangement interval of the menu icons 41 in the left direction, the selection target candidate icons are cut into nine menu icons 41 of “door”,. Change. Thereafter, if the drag operation in the left direction is further continued, the processes based on S36 and S37 are executed, and the selection target candidate icons are sequentially switched in the left direction.

  In S39, the CPU 31 changes the display mode of the menu icon 41 on the main screen 40 corresponding to the selection target candidate icon after being switched in the process of S38. Specifically, among the menu icons 41 newly displayed on the main screen 40, the menu icon 41 set as the selection candidate icon is highlighted (enlarged size and thickened frame). As a result, the menu icon 41 as the selection target candidate icon can be changed by the drag operation on the operation surface of the touch pad 11 of the left operation input device 2, and the menu icon 41 currently set as the selection target candidate icon can be easily changed. It becomes possible to identify.

  Next, the sub-process of the left flick process executed in S12 will be described with reference to FIG. FIG. 16 is a flowchart of the sub-processing program for the left flick process.

  First, in S41, the CPU 31 acquires the number of menu icons 41 to be switched as selection target candidate icons by a drag operation received on the operation surface of the touch pad 11 of the left operation input device 2. The details are the same as S31 described above, and a description thereof will be omitted.

  Next, in S42, the CPU 31 generates a vibration waveform that vibrates the operation surface of the touch pad 11 of the right operation input device 3 based on the number of menu icons 41 to be switched as the selection target candidate icons acquired in S41. calculate. The details are the same as S32 described above, and a description thereof will be omitted.

  Subsequently, in S43, the CPU 31 applies a signal voltage corresponding to the vibration waveform calculated in S42 to the piezoelectric element 12, thereby distorting the piezoelectric element 12 and vibrating the touch pad 11 of the left operation input device 2. The vibration process is continuously performed at a predetermined vibration period from the start of the flick operation to the end of the flick operation. Then, by generating vibration, a tactile sensation as if a finger slides on a plurality of existing buttons is given to a user who touches the operation surface of the touch pad 11.

  Next, in S <b> 44, the CPU 31 moves the touch coordinates along the flick operation received on the operation surface of the touch pad 11 of the left operation input device 2 (hereinafter referred to as the flick direction) and the movement speed of the touch coordinates associated with the flick operation ( (Hereinafter referred to as flick speed).

  Subsequently, in S <b> 45, the CPU 31 changes the menu icon 41 displayed on the main screen 40 based on the flick operation received on the operation surface of the touch pad 11 of the left operation input device 2. As a result, the menu icon 41 set as the selection target candidate icon is also switched.

  Here, FIG. 17 is a diagram showing a switching mode of the selection target candidate icons when the right flick operation is received in a state where the main screen 40 shown in FIG. 9 is displayed. In the example illustrated in FIG. 17, when the user performs a flick operation to the right with respect to the operation surface of the touch pad 11 of the left operation input device 2, the group of menu icons 41 to be displayed on the main screen 40 can The group of menu icons 41 related to the device is changed to the group of menu icons 41 related to the audio. As a result, the menu icon 41 set as the selection target candidate icon is also set for the newly displayed menu icon 41, and the menu icon set as the selection target candidate icon before accepting the drag operation. All 41 are excluded from the selection target candidate icons. Then, among the menu icons 41 newly displayed on the main screen 40, a predetermined number (up to nine) of menu icons 41 at the same positions as the selection target candidate icons before the flick operation are newly set as selection target candidate icons. Is done.

  For example, in the example shown in FIG. 17, nine menu icons 41 “Destination”,... “Point registration” are set as selection target candidate icons immediately before accepting the drag operation. After that, when a right flick operation is performed, the selection target candidate icons are switched to nine menu icons 41 of “HD”,. If the flick operation is further performed in the right direction thereafter, the selection target candidate icon is switched to the group of menu icons 41 related to other vehicle devices.

  In S46, the CPU 31 changes the display mode of the menu icon 41 on the main screen 40 corresponding to the selection target candidate icon after being switched in the process of S45. Specifically, among the menu icons 41 newly displayed on the main screen 40, the menu icon 41 set as the selection candidate icon is highlighted (enlarged size and thickened frame). As a result, the menu icon 41 as the selection target candidate icon can be changed by flicking the operation surface of the touch pad 11 of the left operation input device 2, and the menu icon 41 currently set as the selection target candidate icon can be easily changed. It becomes possible to identify.

  Next, the right touch process sub-process executed in S25 will be described with reference to FIG. FIG. 18 is a flowchart of a sub-processing program for right touch processing.

  First, in S61, the CPU 31 compares the touch coordinates detected in S19 with the selection area set on the operation surface in S3 and S13, and determines whether the touch coordinates are included in any selection area. To determine. In addition, you may add to the determination conditions of said S61 that the pin 21 in a selection area | region was pressed down, or the bending more than predetermined amount with respect to the operation surface produced.

  If it is determined that the touch coordinates are included within any selected area (S61: YES), the process proceeds to S62. On the other hand, when it is determined that the touch coordinates are not included in any selection region (S61: NO), the process proceeds to S23 without selecting the menu icon 41.

  In S <b> 62, the CPU 31 applies a signal voltage corresponding to a preset vibration waveform to the piezoelectric element 12 to distort the piezoelectric element 12 and vibrate the touch pad 11 of the right operation input device 3. Moreover, it is preferable that the amplitude of the vibration waveform is larger than the amplitude when the touch pad 11 of the right operation input device 3 is vibrated in S33 and S43. Note that the vibration processing is basically performed only for a predetermined number of times (for example, once) or a period when the touch-on is detected. Then, by causing vibration, a tactile sensation is provided in which the user touches the operation surface of the touch pad 11 and presses an existing button with a finger.

  Subsequently, in S63, the CPU 31 specifies the menu icon 41 corresponding to the selection area determined to include touch coordinates in S61 as the menu icon 41 selected by the user.

  Thereafter, in S64, the CPU 31 executes various control processes based on the selection and operation of the menu icon 41 specified in S63. Specifically, the application corresponding to the specified menu icon 41 is started, and the application started based on the user operation received thereafter is executed (S6). In particular, when an application corresponding to the operation of various on-vehicle devices and vehicle equipment mounted on the vehicle is activated, the setting operation screen 42 (FIG. 7) is displayed, and further, based on the user operation received thereafter. An operation signal is transmitted to various vehicle-mounted devices and vehicle equipment control devices. On the other hand, the vehicle-mounted device or vehicle device that has received the operation signal performs control based on the received operation signal (for example, a map scale change operation, an air conditioner set temperature change operation, etc.). As a result, it is possible to operate various vehicle-mounted devices and vehicle equipment via the left operation input device 2 and the right operation input device 3.

As described above in detail, according to the operation support system 1, the operation support method by the operation support system 1, and the computer program executed by the operation support system 1 according to the present embodiment, the plurality of menu icons 41 that are the operation objects. Is displayed on the windshield 36 (S1), and based on the touch operation received by the left operation input device 2 or the right operation input device 3, a predetermined number of menu icons 41 among the plurality of menu icons 41 are displayed on the right operation input device. 3 is set to a selection target candidate icon that can be directly selected as a selection target (S36 to S39), and a predetermined number of selection target candidate icons are set based on the touch operation received by the right operation input device 3. One of the menu icons 41 is selected and operated from the menu icons 41 (S63, S 4), when the user performs an operation of selecting any one of the menu icons 41 from the plurality of menu icons 41 displayed on the display screen, even when there are many menu icons 41, the number of operations is small. The menu icon 41 desired by the user can be selected and operated, and an arbitrary menu icon 41 desired by the user can be finally selected from a large number of menu icons 41 only by an operation on the right operation input device 3. It becomes possible.
Further, since the left operation input device 2 and the right operation input device 3 are provided with different functions except during an operation of switching the menu icon 41 to be selected as the selection target candidate icon to another menu icon 41, a plurality of functions are provided. It is possible to quickly perform a complicated operation using the operation input device, and it is also possible to perform an operation with only one operation input device in a necessary situation.
Further, when a selection target candidate icon is selected, the coordinate system of the display area 60 where the selection target candidate icon is displayed corresponds to the coordinate system on the operation surface of the touch pad 11 of the right operation input device 3, and the right operation is performed. Since one of the selection target candidate icons is selected from the selection target candidate icons displayed in the display area 60 by the operation received on the operation surface of the touch pad 11 of the input device 3, the touch pad of the right operation input device 3 is selected. It becomes possible to accurately select an arbitrary selection target candidate icon desired by the user from among the selection target candidate icons displayed on the display screen by the operation on the 11 operation surfaces.
In addition, since the menu icon 41 set as the selection target candidate icon is distinguished from the other menu icons 41 and displayed, the user can clearly select the menu icon 41 that can be currently selected by touching the right operation input device 3. It becomes possible to grasp.
Further, a plurality of pins 21 are projected at a predetermined arrangement interval to the front side of the operation surface of the touch pad 11 of the right operation input device 3, and the menu icon 41 set as the selection target candidate icon is projected to the projected pin 21. Therefore, when the user visually recognizes the display screen, the user can easily understand the correspondence between the projected pin 21 and the selection target candidate icon. Therefore, the menu icon 41 intended by the user can be appropriately selected by a touch operation on the operation surface.
In addition, since the number of menu icons 41 set as selection target candidate icons is the same as the number of protruding pins 21, the selection operation of the menu icon 41 can be easily performed by the pins 21 protruding on the operation surface. It becomes possible.

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in the present embodiment, the case of performing a touch operation with a touch pad is described, but a touch panel other than the touch pad, a tablet, or the like may be used as a means for receiving the touch operation. Moreover, when using a touch panel, the display which displays the menu icon 41 is arrange | positioned so that it may overlap with a touch panel.

  In this embodiment, the pin drive mechanism 13 is provided in both the left operation input device 2 and the right operation input device 3, but may be provided only in the right operation input device 3. Further, if the processing for driving the pins is omitted, the right operation input device 3 can be configured not to include the pin driving mechanism 13.

  In the present embodiment, the left operation input device 2 and the right operation input device 3 have a configuration in which a total of nine pins 21 of 3 × 3 are arranged, but for example, a total of four 2 × 2 or 4 × 4 pins 21 are used. A total of 16 pieces may be arranged. Further, it is necessary to change the definition and number of icons to be operated by the left operation input device 2 and the right operation input device 3 in accordance with the arrangement mode of the pins 21.

  In the present embodiment, the process based on the drag operation or the flick operation received by the right operation input device 3 is executed only while the main screen 40 is displayed. Similarly, on the screen for selecting a predetermined number of selection target candidates from among the operation target objects, a process based on a drag operation or a flick operation may be executed.

  Moreover, it is good also as a structure which ECU of a navigation apparatus performs part or all of the process which operation support ECU15 mentioned above performs. In addition, a display screen that displays an icon that is an operation target may be displayed on a display of the navigation device.

1 Operation Support System 2 Left Operation Input Device 3 Right Operation Input Device 4 HUD
11 Touchpad 12 Piezoelectric Element 13 Pin Drive Mechanism 15 Operation Support ECU
21 pin 31 CPU
32 RAM
33 ROM
34 Flash memory 41 Menu icon 51-59 Selected area

Claims (8)

A first operation input device and a second operation input device that are provided on a steering wheel of a vehicle and receive a user's operation via an operation surface, and an operation target that is operated by the first operation input device and the second operation input device. In an operation support system comprising a display screen for displaying
Operation object display means for displaying a plurality of operation objects on the display screen;
While accepting an operation of selecting a predetermined number of the operation objects from among the plurality of operation objects as selection object candidates and switching the operation object to be selected as the selection object candidate to another operation object When the drag operation or flick operation is accepted in the first operation input device or the second operation input device, the operation object to be selected as the selection object candidate is moved to the other operation object. Selection target candidate selection means for sequentially switching along the moving direction of the touch point in the operation or the flick operation;
Based on the operation received by the second operation input device, one of the operation objects is selected from a predetermined number of the operation objects selected as the selection object candidates by the selection object candidate selecting unit. And an object selection means for operating the selected operation object.   The second operation input device performs only operations related to the operation object in the object selection means, except during accepting an operation for switching the operation object to be selected as the selection object candidate to another operation object. The operation support system according to claim 1, wherein the operation support system is received. When the selection target candidate is selected by the selection target candidate selection unit, the coordinate system of the display area in which the operation target selected as the selection target candidate is displayed and the operation surface of the second operation input device A coordinate correspondence means for corresponding the coordinate system;
The target object selecting means selects any one of the operation target objects from the selection target candidates by an operation received on the coordinate system matched by the coordinate handling means and the operation surface of the second operation input device. The operation support system according to claim 1 or claim 2, wherein   2. The operation object display unit displays the operation object selected as the selection object candidate by the selection object candidate selection unit so as to be distinguished from other operation objects. The operation support system according to claim 3. Protruding means for projecting a plurality of projecting portions at a predetermined arrangement interval on the front side of the operation surface of the second operation input device;
The operation object display means displays the operation object selected as the selection object candidate by the selection object candidate selection means in an arrangement form corresponding to the arrangement form of the plurality of protrusions. The operation support system according to any one of claims 1 to 4.   The operation support system according to claim 5, wherein the predetermined number is the number of the protruding portions protruded by the protruding means. A first operation input device and a second operation input device that are provided on a steering wheel of a vehicle and receive a user's operation via an operation surface, and an operation target that is operated by the first operation input device and the second operation input device. In an operation support method by an operation support system comprising a display screen for displaying
An operation object display step for displaying a plurality of operation objects on the display screen;
While accepting an operation of selecting a predetermined number of the operation objects from among the plurality of operation objects as selection object candidates and switching the operation object to be selected as the selection object candidate to another operation object When the drag operation or flick operation is accepted in the first operation input device or the second operation input device, the operation object to be selected as the selection object candidate is moved to the other operation object. A selection candidate selection step of sequentially switching along the moving direction of the touch point in the operation or the flick operation;
Based on the operation received by the second operation input device, one of the operation objects is selected from the predetermined number of the operation objects selected as the selection object candidates by the selection object candidate selection step. And an object selection step of operating the selected operation object. A first operation input device and a second operation input device that are provided on a steering wheel of a vehicle and receive a user's operation via an operation surface, and an operation target that is operated by the first operation input device and the second operation input device. In a computer program executed in an operation support system comprising a display screen for displaying
An operation object display function for displaying a plurality of operation objects on the display screen;
While accepting an operation of selecting a predetermined number of the operation objects from among the plurality of operation objects as selection object candidates and switching the operation object to be selected as the selection object candidate to another operation object When the drag operation or flick operation is accepted in the first operation input device or the second operation input device, the operation object to be selected as the selection object candidate is moved to the other operation object. A selection candidate selection function that sequentially switches along the moving direction of the touch point in the operation or the flick operation;
Based on the operation received by the second operation input device, one of the operation objects is selected from the predetermined number of the operation objects selected as the selection object candidates by the selection object candidate selection function. And an object selection function for operating the selected operation object,
A computer program for executing

Images