JP5910468B2 - Operation support system, operation support method, and computer program - Google Patents

Description

  The present invention relates to an operation support system, an operation support method, and a computer program that support operation of a device by touch operation.

  Conventionally, when a user performs an operation on a device such as a personal computer, an ATM, a ticket vending machine, or a smartphone, various man-machine interfaces have been used as means for receiving the user's operation on the device. Particularly, in recent years, as one of the man-machine interfaces, an operation surface that accepts a user's touch operation is provided, and a user's touch operation on the operation surface (for example, a touch-on operation, a touch-off operation, a drag operation, a flick operation, etc.) is performed. An operation device that is detected as a user operation is often used. Examples of such an operation device include a touch pad, a touch panel, a tablet, and the like (hereinafter referred to as a touch pad), which allows a user to perform an intuitive and easy-to-understand operation.

  Here, as an operation mode when the user operates the touch pad or the like, the user may perform an operation without visually recognizing the touch pad or the like. Specifically, it is a case where an operation of a touch pad or the like is performed while performing other work, for example, an operation of a touch pad or the like may be performed while a driving operation of the vehicle is performed. However, in such an operation mode, it is not possible to grasp the arrangement mode and shape of the operation target such as buttons and levers that are virtually arranged on the touch pad, so that the operation takes time or an incorrect operation is performed. There was a fear. Therefore, for example, in Japanese Patent Application Laid-Open No. 2010-9321, when an operation button area is set around a virtual button on the touch panel and the operation button area is touched by the user, vibration corresponding to the touched operation button area is set. A technique for recognizing a virtual button without causing the touch panel to be visually recognized by vibrating the touch panel with a pattern is described.

Japanese Patent Laying-Open No. 2010-9321 (page 9 to page 10, FIG. 7 to FIG. 10)

  According to the technique described in Patent Document 1, the user can determine whether there is a virtual button around the touched position. However, in order for the user to perform an accurate touch operation without visually recognizing the touch panel, it is necessary to grasp what kind of virtual button is arranged on what kind of arrangement on the touch panel.

  Here, in general, when the user grasps the arrangement and shape of the button without visually recognizing the actual button, after the user touches the area around the button, the touch position is slid and moved depending on the difference in tactile sensation. Know the button layout and shape. However, although the technique of Patent Document 1 describes changing the vibration pattern for each virtual button, the touch position is always the same in a stationary state or a moving state when the periphery of the same virtual button is touched. Since vibration is performed, if the moving speed of the touch point touched by the user on the touch panel is faster than the touch point detection cycle on the touch panel, the virtual button is recognized by tactile sensation at the position on the operation surface of the touch panel where the virtual button exists. For this reason, it may not be possible to properly generate vibration. Therefore, it was not possible to grasp the arrangement mode and shape of the virtual buttons.

  The present invention has been made to solve the above-described conventional problems, and is a case where the moving speed of the touch point where the user touches the operation surface to be operated is faster than the detection cycle of the touch point such as a touch pad. Even if there is, an operation support system and operation support that enables the user to accurately grasp the arrangement mode and shape of the operation object virtually arranged on the operation surface and to perform an accurate operation without any error It is an object to provide a method and a computer program.

In order to achieve the object, an operation support system (1) according to claim 1 of the present application includes an operation surface (7) that receives a user operation, and a vibration mechanism (8) that vibrates the operation surface at a predetermined vibration cycle. Is set according to the arrangement mode and shape of the operation objects (21 to 23) virtually arranged on the operation surface, and corresponds to the coordinate system of the operation surface and the vibration reference value. The vibration reference information acquisition means (11) for acquiring the attached vibration reference information (15) and the position of the touch point where the user touches the operation surface is detected every predetermined detection cycle longer than the vibration cycle. Based on the touch point detection means (11) and the history of the position of the touch point detected by the touch point detection means, the touch point predictor for predicting the position of the future touch point for each vibration cycle. (11) and, based on the position of the touch point predicted by the touch point prediction means and the vibration reference information, the vibration reference value corresponding to the position of the touch point associated with the future movement of the touch point. Displacement mode prediction means (11) for predicting the displacement mode, and operation surface vibration means (for vibrating the operation surface by the vibration mechanism based on the displacement mode of the vibration reference value predicted by the displacement mode prediction means) And 11).
In addition to the case where the user actually touches the operation surface, “touching the operation surface” means that when the capacitance changes even if the user does not touch the operation surface in the capacitance method. Is considered touched.

  Further, the operation support system (1) according to claim 2 is the operation support system according to claim 1, wherein the touch point prediction means (11) is a position of the touch point at a start time of the vibration cycle. The displacement mode prediction means (11) specifies the vibration reference value corresponding to the position of the touch point at the start time of the vibration cycle, and the vibration reference based on the specified vibration reference value. It is characterized by predicting the displacement mode of the value.

  An operation support system (1) according to claim 3 is the operation support system according to claim 1 or 2, wherein the detection cycle is at least twice the vibration cycle. To do.

  An operation support system (1) according to claim 4 is the operation support system according to any one of claims 1 to 3, wherein the operation surface vibration means (11) is the displacement mode prediction means. The operation surface is continuously vibrated while the vibration reference value predicted by is displaced.

  Further, the operation support system (1) according to claim 5 is the operation support system according to claim 4, wherein the operation surface vibration means (11) is predicted by the displacement mode prediction means (11). The larger the displacement amount per unit time of the vibration reference value, the larger the amplitude of vibration that vibrates the operation surface (7).

  Further, the operation support system (1) according to claim 6 is the operation support system according to claim 4 or 5, wherein the operation surface vibration means (11) has a high moving speed of the touch point. As a result, the amplitude of the vibration for vibrating the operation surface (7) is increased.

  An operation support system (1) according to claim 7 is the operation support system according to any one of claims 1 to 6, wherein the operation surface vibration means (11) is the displacement mode prediction means. The operation surface is vibrated based on the increase or decrease at the timing when the displacement per unit time of the vibration reference value predicted by (11) increases or decreases.

  An operation support method according to an eighth aspect of the present invention provides an operation device (2) including an operation surface (7) that receives a user operation and a vibration mechanism (8) that vibrates the operation surface at a predetermined vibration cycle. In the operation support method, the vibration is set according to the arrangement mode and shape of the operation objects (21 to 23) virtually arranged on the operation surface, and the coordinate system of the operation surface is associated with the vibration reference value. A vibration reference information acquisition step for acquiring reference information (15); a touch point detection step for detecting the position of the touch point where the user touches the operation surface for each predetermined detection cycle longer than the vibration cycle; Based on the history of the position of the touch point detected by the touch point detection step, the touch point prediction step of predicting the position of the touch point in the future for each vibration cycle; and the touch point Displacement mode prediction for predicting the displacement mode of the vibration reference value corresponding to the position of the touch point accompanying the future movement of the touch point based on the position of the touch point predicted by the measurement step and the vibration reference information And an operation surface vibration step for vibrating the operation surface by the vibration mechanism based on the displacement mode of the vibration reference value predicted by the displacement mode prediction step.

  Furthermore, the computer program according to claim 9 operates an operating device (2) including an operating surface (7) for accepting a user operation and a vibration mechanism (8) for vibrating the operating surface at a predetermined vibration cycle. In the computer program, the vibration is set according to the arrangement mode and shape of the operation target (21 to 23) virtually arranged on the operation surface, and associates the coordinate system of the operation surface with the vibration reference value. A vibration reference information acquisition function for acquiring reference information (15); a touch point detection function for detecting a position of a touch point touched by the user on the operation surface for each predetermined detection cycle longer than the vibration cycle; A touch point prediction function for predicting a future position of the touch point for each vibration period based on a history of the position of the touch point detected by the touch point detection function; Based on the position of the touch point predicted by the touch point prediction function and the vibration reference information, the displacement for predicting the displacement mode of the vibration reference value corresponding to the position of the touch point accompanying the future movement of the touch point An aspect prediction function and an operation surface vibration function that vibrates the operation surface by the vibration mechanism based on a displacement aspect of the vibration reference value predicted by the displacement aspect prediction function are executed. .

  According to the operation support system of claim 1 having the above-described configuration, the vibration reference value is set for the coordinate system of the operation surface based on the arrangement mode and shape of the operation object virtually arranged on the operation surface. Since the vibration of the operation surface is performed based on the displacement mode of the vibration reference value accompanying the future movement of the touch point, the moving speed of the touch point where the user touches the operation surface to be operated is a touch point such as a touch pad. Even if it is faster than the detection cycle, it is possible for the user to accurately grasp the arrangement mode and shape of the operation object virtually arranged on the operation surface, and to perform an accurate operation without error It becomes. In particular, even when the detection cycle of the touch point is longer than the vibration cycle of the operation surface, it is possible to accurately predict the displacement mode of the vibration reference value by predicting the position of the touch point for each vibration cycle. It becomes possible.

  According to the operation support system of claim 2, the vibration reference value corresponding to the position of the touch point at the start of the vibration cycle is specified, and the displacement mode of the vibration reference value is based on the specified vibration reference value. Therefore, even if the detection cycle of the position of the touch point is longer than the vibration cycle of the operation surface, the vibration reference value at that time can be accurately specified for each start point of the vibration cycle. The operation surface can be vibrated in an appropriate vibration mode for each vibration cycle.

  According to the operation support system of the third aspect, since the detection cycle of the position of the touch point is twice or more than the vibration cycle, the detection cycle of the position of the touch point is longer than the vibration cycle of the operation surface. Even in this case, it is possible to accurately predict the displacement mode of the vibration reference value by predicting the position of the touch point for each vibration period included in the detection period.

  According to the operation support system of the fourth aspect, since the operation surface is continuously vibrated while the vibration reference value is displaced, the periphery of the operation object virtually arranged on the operation surface is When the touch point moves, it is possible to give the user a tactile sensation equivalent to moving the slope or level difference of the actual operation target. In addition, it is possible for the user to grasp that the touch point has moved onto the operation target object or has deviated from the operation target object.

  In addition, according to the operation support system of the fifth aspect, the larger the displacement per unit time of the vibration reference value, the larger the amplitude of vibration that vibrates the operation surface. In consideration of the shape of the object, it is possible to more accurately give the user a tactile sensation equivalent to moving the slope or step of the actual operation target.

  In addition, according to the operation support system of the sixth aspect, the higher the moving speed of the touch point, the larger the amplitude of vibration that vibrates the operation surface. Therefore, the actual operation in consideration of the moving speed of the touch point. It is possible to give the user a tactile sensation equivalent to moving the slope or step of the object more accurately.

  According to the operation support system according to claim 7, the operation surface is vibrated based on the increase or decrease at the timing when the displacement per unit time of the vibration reference value increases or decreases. When the touch point moves on the edge or corner of the object, it is possible to give the user a tactile sensation equivalent to crossing the edge or corner of the actual operation object.

  Further, according to the operation support method of claim 8, the vibration reference value is set for the coordinate system of the operation surface based on the arrangement mode and shape of the operation object virtually arranged on the operation surface, Since the vibration of the operation surface is performed based on the displacement mode of the vibration reference value accompanying the future movement of the touch point, the movement speed of the touch point where the user touches the operation surface to be operated is detected as a touch point such as a touch pad. Even when it is faster than the cycle, it is possible to allow the user to accurately grasp the arrangement mode and shape of the operation object virtually arranged on the operation surface, and to perform an accurate operation without error. . In particular, even when the detection cycle of the touch point is longer than the vibration cycle of the operation surface, it is possible to accurately predict the displacement mode of the vibration reference value by predicting the position of the touch point for each vibration cycle. It becomes possible.

  Furthermore, according to the computer program according to claim 9, the vibration reference value is set for the coordinate system of the operation surface based on the arrangement mode and shape of the operation object virtually arranged on the operation surface. Because the vibration of the operation surface is performed based on the displacement mode of the vibration reference value accompanying the movement of the touch point, the movement speed of the touch point where the user touched the operation surface to be operated is detected as a touch point such as a touch pad. Even when it is faster than the cycle, it is possible to allow the user to accurately grasp the arrangement mode and shape of the operation object virtually arranged on the operation surface, and to perform an accurate operation without error. . In particular, even when the detection cycle of the touch point is longer than the vibration cycle of the operation surface, it is possible to accurately predict the displacement mode of the vibration reference value by predicting the position of the touch point for each vibration cycle. It becomes possible.

It is the block diagram which showed schematic structure of the operation assistance system 1 which concerns on this embodiment. It is the figure which showed the arrangement | positioning aspect with respect to the vehicle of an operation input device. It is the figure which showed an example of the display screen displayed on the display of an operation input device. As vibration reference information, it is the figure which showed especially the displacement of the vibration reference value along the x-axis positive direction of AA line, and the displacement of the vibration reference value along the y-axis positive direction of BB line. It is a flowchart of the operation assistance processing program which concerns on this embodiment. It is the figure which showed the displacement aspect of the vibration reference value of the touch coordinate with respect to time passage. It is a flowchart of the sub process program of a 1st vibration process. It is a flowchart of the sub process program of a 1st vibration waveform calculation process. It is the figure which showed the displacement aspect of the vibration reference value of the touch coordinate with respect to time passage. It is the figure shown about the vibration direction of a touchpad. It is a figure explaining the vibration aspect in case a touch coordinate moves to the direction which goes up the inclination of an operation target object, when a vibration reference value rises with time passage. It is a figure explaining the vibration aspect in case a touch coordinate moves to the direction which goes up the inclination of an operation target object, when a vibration reference value rises with time passage. A description will be given of a vibration mode when the vibration reference value decreases with time, that is, when the touch coordinates move in the direction of decreasing the inclination of the operation target. A description will be given of a vibration mode when the vibration reference value decreases with time, that is, when the touch coordinates move in the direction of decreasing the inclination of the operation target. It is a flowchart of the sub process program of a 2nd vibration process. It is a figure explaining the 1st method of estimating the position of future touch coordinates. It is a figure explaining the 2nd method of estimating the position of future touch coordinates. It is a figure explaining the 3rd method of estimating the position of future touch coordinates. It is the figure explaining the method of estimating the position of the touch coordinate for every vibration period. It is a figure explaining the vibration mode determined based on the vibration reference value estimated for every vibration period. It is a flowchart of the sub process program of a waveform specific process. It is the figure which showed the waveform pattern used as the candidate of a vibration waveform. It is a flowchart of the sub process program of a 2nd vibration waveform calculation process. It is a figure explaining the prediction method of the timing which a vibration reference value starts a displacement. It is a figure explaining the vibration aspect when it is estimated that a touch coordinate will move on the operation target object from the surface without an operation target object in the future. It is a figure explaining the vibration aspect when it is estimated that a touch coordinate will move to the surface without an operation target object from the operation target object in the future. It is a figure explaining the vibration aspect when it is estimated that a touch coordinate will move on the other operation target object from the present operation target object in the future.

  DETAILED DESCRIPTION Hereinafter, an operation support system according to the present invention will be described in detail with reference to the drawings based on a specific embodiment. First, a schematic configuration of the operation support system 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of an operation support system 1 according to the present embodiment.

  As shown in FIG. 1, an operation support system 1 according to the present embodiment is installed in a vehicle, and is connected to an operation input device 2 that receives an operation by a user, and the operation input device 2 via an in-vehicle network such as CAN. It is composed of various on-vehicle devices and a vehicle control ECU.

  Here, the operation input apparatus 2 is arrange | positioned at the spoke part of the steering wheel 3 of a vehicle, as shown in FIG. As will be described later, a touch pad (operation surface) that accepts a user's touch operation is provided on the front surface of the operation input device 2, and the user supports the operation input device 2 by a touch operation described later while supporting the handle 3. Is configured to be operable. The user can operate various vehicle-mounted devices and systems mounted on the vehicle via the operation input device 2. For example, a navigation device destination setting operation, map image scrolling operation, air conditioner power supply switching operation, temperature adjustment operation, audio channel operation, volume adjustment operation, and the like can be performed. In FIG. 2, the operation input device 2 is disposed on the right spoke portion of the handle 3, but may be disposed on the left spoke portion. Further, they may be arranged on both sides. The touch operation is various operations performed by the user touching the touch surface. For example, an operation of touching (touching) any point of the touch pad 7, an operation of releasing the touched state (touch state), There are drag operation and flick operation.

  As shown in FIG. 1, the operation input device 2 according to the present embodiment includes an operation support ECU 5 that performs various arithmetic processes, a touch pad 7 that receives operations from a user, and a vibrator for causing the touch pad 7 to vibrate. It comprises a piezoelectric element 8 and a CAN interface 9.

The operation input device 2 is connected to a display 6 for displaying an operation object such as a button or a lever that is an operation object by the user. The display 6 is installed at a location different from the operation surface of the touch pad 7, for example, a dashboard. The display content of the display 6 varies depending on the vehicle-mounted device or system to be operated by the operation input device 2. For example, when the navigation device is operated, an operation object such as a cursor key or a character input key is displayed. When an air conditioner is operated, operation objects such as a power switch button and a temperature adjustment button are displayed. The display contents are switched when a flick operation is received on the display 6, for example.
Further, the display 6 may be constituted by a head-up display (HUD). In that case, the projection direction of the image is set to be the windshield in front of the driver's seat. Further, a head mounted display (HMD) or the like may be used.

Below, each component which comprises the operation input apparatus 2 is demonstrated in order.
First, an operation support ECU (Electronic Control Unit) 5 is an electronic control unit that controls the entire operation input device 2. The CPU 11 as an arithmetic device and a control device, and the CPU 11 performs various arithmetic processes. In addition to the RAM 12 used as a working memory, a control program, a ROM 13 in which an operation support processing program (see FIG. 5) described later is recorded, a program read from the ROM 13, a history of touch coordinates, and a vibration reference information 15 described later. An internal storage device such as a flash memory 14 or the like. The operation support ECU 5 constitutes various means as processing algorithms. For example, the vibration reference information acquisition unit is set according to the arrangement mode and shape of the operation object virtually arranged on the touch pad 7, and the vibration reference in which the coordinate system of the touch pad 7 is associated with the vibration reference value. Information 15 is acquired. The touch point detecting means detects the position of the touch point where the user touches the touch pad 7 at every predetermined detection cycle longer than the vibration cycle. The touch point predicting unit predicts the position of the future touch point for each vibration cycle based on the history of the position of the touch point detected by the touch point detecting unit. The displacement mode predicting unit predicts the displacement mode of the vibration reference value corresponding to the position of the touch point accompanying the future movement of the touch point based on the touch point predicted by the touch point predicting unit and the vibration reference information. The operation surface vibration unit vibrates the touch pad 7 by the piezoelectric element 8 based on the displacement mode of the vibration reference value predicted by the displacement mode prediction unit.

  The touch pad 7 is an operation means for operating an operation target such as a button or a lever displayed on the display 6. As the touch pad 7, a resistive film type or a capacitance type is used, and the operation support ECU 5 detects the touch coordinates touched by the user at a predetermined detection cycle in the coordinate system of the touch pad 7. Note that the touch coordinates are the coordinates of the position of the touch point where the user touches the operation surface of the touch pad 7 (in the electrostatic capacity method, it is considered that the electrostatic capacity has changed). The detection cycle varies depending on the type of the touch pad 7, but is, for example, 200 Hz to 2 kHz. Further, when the touch coordinates move as described later (that is, when the user slides while touching the touch pad), the operation support ECU 5 is based on the displacement of the vibration reference position corresponding to the touch coordinates. The touch pad 7 is vibrated. Thereby, even when the user does not visually recognize the display 6, it becomes possible to grasp the shape and arrangement of the operation object arranged on the touch pad 7.

  FIG. 3 is a diagram showing a display screen displayed on the display 6 in association with the touch pad 7. In particular, the example shown in FIG. 3 shows an air conditioner operation screen 20 displayed on the display 6 when operating the air conditioner of the vehicle via the operation input device 2. Specifically, on the air conditioner operation screen 20, a power button 21 for switching the power supply of the air conditioner, a temperature increase button 22 for increasing the set temperature, and a temperature decrease button 23 for decreasing the set temperature are displayed as operation objects. The If the user touches the area where the power button 21 is displayed, the power supply of the air conditioner can be switched. If the user touches the area where the temperature increase button 22 or the temperature decrease button 23 is displayed, the set temperature of the air conditioner is displayed. Can be adjusted. That is, the area on the operation surface of the touch pad 7 in which the power button 21 is virtually arranged becomes an area for operating the power button 21, and the area on the operation surface of the touch pad 7 in which the temperature increase button 22 is virtually arranged is The area for operating the temperature increase button 22 is an area on the operation surface of the touch pad 7 in which the temperature decrease button 23 is virtually arranged, and the area for operating the temperature decrease button 23.

  Further, the piezoelectric elements 8 are arranged in a pair in the vicinity of the edge of the touch pad 7 in contact with the touch pad 7 as shown in FIG. Then, the operation support ECU 5 applies a signal voltage to the piezoelectric element 8 to distort the piezoelectric element 8 and vibrate the touch pad 7. At that time, the operation support ECU 5 can arbitrarily set the vibration direction and the amplitude of vibration by changing the voltage value. Moreover, although a vibration period changes with kinds and installation modes of the piezoelectric element 8, it becomes 250 Hz-2 kHz, for example. The piezoelectric element 8 does not have to be disposed in direct contact with the touch pad 7 and may be disposed with respect to another member as long as vibration can be transmitted to the touch pad 7. Further, as a means for generating vibration on the touch pad 7, a small vibration motor or the like may be used instead of the piezoelectric element 8.

  The CAN (controller area network) interface 9 is an interface for inputting / outputting data to / from CAN which is an in-vehicle network standard for performing multiplex communication between various in-vehicle devices installed in a vehicle and a control ECU of the vehicle. is there. The operation support ECU 5 is connected to various vehicle-mounted devices and vehicle control ECUs (for example, the navigation device 31, the AV device 32, the air conditioner control ECU 33, and the like) via the CAN so as to be able to communicate with each other. When the operation input device 2 accepts a user operation, the operation support ECU 5 transmits various operation signals to various vehicle-mounted devices and the control ECU of the vehicle. Perform system operations.

Next, the vibration reference information 15 stored in the flash memory 14 will be described.
Here, the vibration reference information 15 is set according to the arrangement mode and shape of the operation object (that is, the operation region) virtually arranged on the touch pad 7, and the coordinate system of the touch pad 7 and the vibration reference value are determined. The associated information. Further, the vibration reference value determines whether or not to vibrate the touch pad 7 in a state where the user performs a touch operation on the touch pad 7 and in what manner the vibration is to be performed. This is a value that serves as a reference for the setting, and is set based on the height information of the operation object virtually placed on the touch pad 7. Accordingly, the vibration reference information is determined based on the arrangement mode such as the type and position of the operation object virtually arranged with respect to the touch pad 7. Different vibration reference information (for example, vibration reference information corresponding to the arrangement pattern of the operation target when operating the air conditioner, vibration reference information corresponding to the arrangement mode of the operation target when operating the navigation device) Remembered.

  Here, FIG. 4 shows the vibration reference information set on the touch pad 7 when the air conditioner operation screen 20 shown in FIG. 3 is displayed on the display 6, particularly along the positive direction of the x-axis of the AA line. It is the figure which showed the displacement of the vibration reference value, and the displacement of the vibration reference value along the y-axis positive direction of the BB line. In FIG. 4 and subsequent figures, the display screen originally displayed on the display 6 in another area is displayed in correspondence with the touch pad 7 for the sake of explanation. As shown in FIG. 4, the vibration reference value is set based on the arrangement mode and shape of the power button 21, the temperature increase button 22, and the temperature decrease button 23 included in the air conditioner operation screen 20, and the power button 21 is displayed on the touch pad 7. This corresponds to the height information when it is assumed that the temperature increase button 22 and the temperature decrease button 23 are arranged as mechanical buttons. Accordingly, the vibration reference value along the M direction is 0 to the point A, rises from the point A to the point B at the slope angle of the outer edge of the power button 21, and from the point B to the point C, the power button 21 It changes at the height h1 at the top, and then descends from the point C to the point D at the slope angle of the outer edge of the power button 21, and becomes zero again from the point D to the point E where no button exists. Similarly, from point E to point F, the temperature rises at the slope angle of the outer edge of the temperature rise button 22, and from point F to point G changes at the height h <b> 2 at the top of the temperature rise button 22. From point H to point H with a slope angle at the outer edge of the temperature rise button 22, from point H to point I rise at the slope angle of the outer edge of the temperature drop button 23, and from point I to point J, the top of the temperature drop button 23 It transitions at the height h2 of the part and then descends from the point J to the point K at the slope angle of the outer edge of the temperature lowering button 23. Similarly, the vibration reference value along the N direction is also displaced as shown in FIG.

  Then, the operation support ECU 5 uses the displacement of the touch coordinates of the user with respect to the touch pad 7 and the vibration reference information 15 shown in FIG. Determines how to vibrate. More specifically, it vibrates so as to give the user a tactile sensation as if touching an actual operation target (for example, a tactile sensation in which a finger is placed on the corner of the button or a slope of the button is moved up or down). Determine aspects. As a result, it is possible to allow the user to recognize the arrangement mode and shape of the operation object virtually arranged on the touch pad 7 without making the display 6 visible.

  Next, an operation support processing program executed by the operation support ECU 5 in the operation support system 1 having the above configuration will be described with reference to FIG. FIG. 5 is a flowchart of the operation support processing program according to the present embodiment. Here, the operation support processing program is executed after ACC of the vehicle is turned on, and the touch pad 7 is vibrated virtually on the touch pad 7 by vibrating the touch pad 7 while the user is performing a touch operation on the touch pad 7. This is a program that allows the user to recognize the arrangement mode and shape of the arranged operation object. The programs shown in the flowcharts of FIGS. 5, 7, 8, 15, 21, and 23 below are stored in the RAM 12 and ROM 13 provided in the operation support system 1 and are executed by the CPU 11. The

  First, in step (hereinafter abbreviated as S) 1 in the operation support processing program, the CPU 11 performs an initialization process. In this initialization process, for example, a history of touch coordinates stored in the flash memory 14 is initialized.

  Next, in S <b> 2, based on the detection signal transmitted from the touch pad 7, the CPU 11 determines whether or not the user is touching at least one point on the touch pad 7 (hereinafter referred to as a touch state). judge. For example, when the touch pad 7 is a resistance film type or a capacitance type, when a pressure higher than a predetermined value is detected or a change in capacitance higher than a predetermined value is detected, the touch pad 7 is in a touch state. judge.

  And when it determines with it being in a touch state (S2: YES), it transfers to S3. On the other hand, when it is determined that the touch state is not established (S2: NO), the process proceeds to S12.

  In S <b> 3, based on the detection signal transmitted from the touch pad 7, the CPU 11 detects touch coordinates that are coordinates of a point where the user touches the touch pad 7. For example, when the touch pad 7 is a resistive film type or a capacitance type, the touch coordinates are detected by detecting the position where the pressure has changed and the position of the current flowing based on the change in the capacitance. To do. Note that S2 is repeatedly executed at predetermined detection intervals until it is determined that the acceptance of the operation is terminated in S13 described later, and the touch coordinates are detected.

  Next, in S4, the CPU 11 determines whether or not touch coordinates have been detected in the process of S3 executed before the previous detection cycle.

  If it is determined that the touch coordinates have been detected before the previous one detection cycle (S4: YES), that is, if it is determined that the touch state has been continuously continued for more than one detection cycle, the process proceeds to S6. Transition. On the other hand, when it is determined that the touch coordinates have not been detected before the previous detection cycle (S4: NO), that is, the user is newly touched on this time (the state is shifted from the state that the user has not touched to the state that the user has touched). ), The process proceeds to S5.

  In S5, the CPU 11 cumulatively stores the touch coordinates detected in S3 as a touch coordinate history in a storage medium such as the flash memory 14. Thereafter, the process proceeds to S12.

In S6, the CPU 11 reads the history of touch coordinates stored in the flash memory 14, and based on the touch coordinates detected before the previous one detection cycle and the touch coordinates newly detected in S3 this time, The moving speed V is calculated. Specifically, when the distance between the previous and present two touch coordinates is L and the touch coordinate detection cycle is T, the following equation (1) is used.
V = L / T (1)

  Next, in S7, the CPU 11 reads the history of touch coordinates stored in the flash memory 14, and determines whether or not a flick operation has been performed. Here, the flick operation is a touch operation that performs a touch-off (a transition from a state touched by the user to a state where the user has not touched) while moving the touch coordinates.

  If it is determined that the flick operation has been performed (S7: YES), the process proceeds to S8. On the other hand, when it is determined that the flick operation has not been performed (S7: NO), the process proceeds to S9.

  In S8, the CPU 11 executes a process based on the flick operation. Specifically, the display screen displayed on the display 6 is switched. For example, the display screen is switched between an operation screen for operating a navigation device, an operation screen for operating an air conditioner, and an operation screen for operating audio. When the display screen is switched, the type (that is, the shape) and the arrangement mode of the operation target object arranged on the touch pad 7 change, and accordingly, the vibration reference information 15 (see FIG. 4) is also new. Switch to information corresponding to the display screen. Thereafter, the process proceeds to S12. In S8, the CPU 11 may change only the type (that is, the shape) of the operation object to be arranged without changing the arrangement of the operation object. Further, the type (that is, the shape) of the operation object to be arranged may not be changed, and only the arrangement of the operation object may be changed.

  On the other hand, in S9, the CPU 11 determines whether or not the touch coordinate movement speed V calculated in S6 is equal to or higher than a predetermined threshold. Here, the threshold value used as the determination criterion in S9 is stored in the RAM 12 or the like, for example, and is set based on the touch coordinate detection cycle. The threshold value is set higher as the touch coordinate detection cycle is shorter. More specifically, the displacement mode of the vibration reference value based on the detected touch coordinate history (for example, the displacement amount per unit time of the vibration reference value, the timing at which the displacement amount per unit time increases or decreases, etc.) The upper limit value of the movement speed of the touch coordinates that can specify the threshold value is set as a threshold value.

Hereinafter, the threshold value will be described in more detail with specific examples.
FIG. 6 is a diagram illustrating a displacement mode of the vibration reference value of the touch coordinates with respect to time when the user performs a drag operation in the right direction from the point S with the air conditioner operation screen 20 displayed on the display 6. is there. As shown in the graph of FIG. 6, when the movement speed of the touch coordinates is slow, the displacement of the vibration reference value is specified even when moving the same distance than when the movement speed of the touch coordinates is fast. Increase the number of samples. Therefore, when the movement speed of the touch coordinates is low, the displacement mode of the vibration reference value is based on the touch coordinates detected in each detection cycle (more specifically, the vibration reference value specified based on the touch coordinates). Can be specified.
As the displacement mode of the vibration reference value to be specified, the displacement amount of the vibration reference value per unit time (for example, the slope angle of the button) θ and the displacement amount θ of the vibration reference value per unit time increase. Timing (for example, timing of reaching the uphill slope of the button or timing of ending the downhill slope) t1, timing of decreasing the displacement θ of the vibration reference value per unit time (for example, timing of ending or downhill of the button uphill) There is a timing t2 when it reaches the slope).
On the other hand, when the movement speed of the touch coordinates is fast, the number of samples for specifying the displacement of the vibration reference value is small, so the touch coordinates detected at each detection cycle (more specifically, specified based on the touch coordinates). It is difficult to specify the displacement mode (t1, t2, θ) of the vibration reference value based on the vibration reference value).

  The number of samples for specifying the displacement of the vibration reference value also depends on the detection period of touch coordinates. That is, the shorter the detection cycle, the greater the number of samples for specifying the displacement of the vibration reference value even at the same touch coordinate movement speed. It becomes possible to specify. For example, for a section in which the displacement amount per unit time of the vibration reference value is other than 0, the upper limit value of the moving speed of touch coordinates that can acquire a plurality of samples in each section is set as a threshold value.

  If it is determined as a result of the determination in S9 that the moving speed V of the touch coordinates is less than a predetermined threshold (S9: NO), the process proceeds to S10. On the other hand, when it is determined that the moving speed V of the touch coordinates is equal to or higher than the predetermined threshold (S9: YES), the process proceeds to S11.

  In S10, the CPU 11 executes a first vibration process (FIG. 7) described later. Here, the first vibration process specifies a displacement mode of the vibration reference value accompanying the movement of the touch coordinates based on the vibration reference value corresponding to the touch coordinates detected in S3, and the specified vibration reference value This is a process of vibrating the touch pad 7 based on the displacement mode.

  On the other hand, in S11, the CPU 11 executes second vibration processing (FIG. 10) described later. Here, the second vibration process predicts the displacement mode of the vibration reference value accompanying the movement of the touch coordinates from the touch coordinate history detected in S3, and touches based on the predicted displacement mode of the vibration reference value. This is processing for vibrating the pad 7.

  Next, in S12, the CPU 11 executes other processing based on the touch coordinates detected in S3. For example, based on the detected touch coordinates and the type of touch operation (drag operation, flick operation, etc.), the operation content of the vehicle-mounted device or system that is the operation target (for example, scrolling the map image displayed on the navigation device, The set temperature rise of the air conditioner and the audio channel change) are specified, and the specified operation content is transmitted to various vehicle-mounted devices and the control ECU of the vehicle via the CAN. Specifically, an area in which the operation target is virtually arranged on the touch pad 7 is an operation area that receives a user's operation, and various types of vehicles mounted via the operation input device 2 based on the user's operation in the operation area. The device and system will be operated.

  Thereafter, in S <b> 13, the CPU 11 determines whether to finish accepting the user's operation by the operation input device 2. For example, when the shift position is changed to “P”, when the ACC is turned off, it is determined that the acceptance of the user operation is finished.

  And when it determines with accepting the user's operation by the operation input apparatus 2 (S13: NO), it returns to S2. On the other hand, when it is determined that the acceptance of the user's operation by the operation input device 2 is finished (S13: YES), the operation support processing program is finished.

  Next, the sub-process of the first vibration process executed in S10 will be described with reference to FIG. FIG. 7 is a flowchart of the sub processing program of the first vibration processing.

  First, in S21, the CPU 11 reads the vibration reference information 15 stored in the flash memory 14, and specifies the vibration reference value corresponding to the touch coordinates newly detected in S3 this time. The vibration reference value is set according to the arrangement mode and shape of the operation object virtually arranged on the touch pad 7 as described above, and the coordinate system of the touch pad 7 is associated with the vibration reference value. Information (see FIG. 4).

  Next, in S22, the CPU 11 reads the vibration reference information 15 stored in the flash memory 14, and similarly specifies the vibration reference value corresponding to the touch coordinates detected before the previous one detection cycle.

  Subsequently, in S23, the CPU 11 calculates a difference between the vibration reference value corresponding to the previous touch coordinate specified in S22 and the vibration reference value corresponding to the current touch coordinate specified in S21.

  Next, in S24, the CPU 11 determines whether or not the vibration reference value corresponding to the previous touch coordinate is different from the vibration reference value corresponding to the current touch coordinate based on the difference calculated in S23, that is, vibration. It is determined whether or not the reference value has been displaced.

  If it is determined that the vibration reference value has been displaced (S24: YES), that is, if it is determined that the touch coordinates have moved since the previous detection and the vibration reference values before and after the movement are different. To S25. For example, when moving between the sections A and B shown in FIG. 4 in the x-axis positive direction or the negative direction, the vibration reference value is displaced. On the other hand, if it is determined that the vibration reference value is not displaced (S24: NO), that is, even if the touch coordinates have not moved or moved since the previous detection, the vibration reference before and after the movement. If it is determined that the values are the same, the process proceeds to S27.

  In S25, the CPU 11 executes a first vibration waveform calculation process (FIG. 8) described later. Here, the first vibration waveform calculation process is a process of calculating a vibration waveform when vibrating the touch pad based on the vibration reference value corresponding to the previous touch coordinates and the vibration reference value corresponding to the current touch coordinates. It is. The vibration waveform is defined by the vibration direction, amplitude, and vibration cycle. Since the vibration cycle is determined by the type and installation mode of the piezoelectric element 8, the vibration direction and amplitude are calculated particularly in S25.

  Next, in S26, the CPU 11 vibrates the touch pad 7 based on the vibration waveform calculated in S25. Specifically, the CPU 11 applies a signal voltage corresponding to the vibration waveform calculated in S25 to the piezoelectric element 8, thereby distorting the piezoelectric element 8 and vibrating the touch pad 7. As a result, the touch pad 7 is vibrated by the vibration waveform calculated in S25. Then, by generating vibration, a tactile sensation as if the user touches the touchpad is touching an actual operation target (for example, a finger is placed on the corner of the button or the slope of the button is moved up). Give the sensation of going down).

  Thereafter, in S27, the CPU 11 cumulatively stores the touch coordinates newly detected in S3 this time in a storage medium such as the flash memory 14 as a history of touch coordinates. Thereafter, the process proceeds to S12.

  Next, the sub-process of the first vibration waveform calculation process executed in S25 will be described with reference to FIG. FIG. 8 is a flowchart of a sub-processing program of the first vibration waveform calculation process.

  First, in S31, the CPU 11 reads the difference D between the vibration reference value corresponding to the previous touch coordinate calculated in S23 and the vibration reference value corresponding to the current touch coordinate.

Next, in S32, the CPU 11 calculates a vibration component Uh based on the difference D read in S31. Specifically, when the moving speed of the touch coordinates calculated in S6 is V, the following calculation is performed using the following equation (2).
Uh = D × V × A (2)
“A” is a unique coefficient for each apparatus.

Subsequently, in S33, the CPU 11 determines the vibration reference value angle θ1 corresponding to the previous touch coordinate calculated in S23 (the displacement amount of the vibration reference value per unit time) and the vibration reference value corresponding to the current touch coordinate. An angle difference Δθ with respect to the angle θ2 is calculated. Specifically, it is calculated by the following equation (3).
Δθ = θ2−θ1 (3)

  For example, when the vibration reference value is displaced over time as shown in FIG. 9, if the angle difference Δθ is calculated at the timing shown in (A), the angle θ1 of the vibration reference value corresponding to the previous touch coordinates and Since the angle θ2 of the vibration reference value corresponding to the touch coordinates this time is both “0”, the angle difference Δθ is “0”. Further, when the angle difference Δθ is calculated at the timing of crossing the edge of the operation target object (that is, the vibration boundary that is the boundary of the operation region) shown in (B), the angle θ1 of the vibration reference value corresponding to the previous touch coordinate is “ Since the angle θ2 of the vibration reference value corresponding to the current touch coordinates is “α”, the angle difference Δθ is “α”. Further, when the angle difference Δθ is calculated at the timing shown in (C), the angle θ1 of the vibration reference value corresponding to the previous touch coordinate is “α”, and the angle θ2 of the vibration reference value corresponding to the current touch coordinate is Since it is “0”, the angle difference Δθ is “−α”.

Next, in S34, the CPU 11 calculates a vibration component Uθ based on the angle difference Δθ calculated in S33. Specifically, it is calculated by the following equation (4).
Uθ = sin Δθ × B (4)
“B” is a coefficient specific to each apparatus.

After that, in S35, the CPU 11 calculates the final vibration waveform U by combining the vibration component Uh based on the difference between the vibration reference values calculated in S32 and the vibration component Uθ based on the angle difference between the vibration reference values. To do. Specifically, it is calculated by the following equation (5).
U = Uh + Uθ (5)

  As a result, the final vibration waveform U also changes depending on the moving direction and moving speed of the touch coordinates. The larger the displacement amount per unit time of the vibration reference value and the faster the moving speed of the touch coordinates, the higher the touch pad 7. As a result, the amplitude of the vibration that vibrates is increased. Further, when the amount of displacement per unit time of the vibration reference value increases with respect to the previous detection time by crossing the edge of the operation target (that is, the vibration boundary that is the boundary of the operation region) or the like (0 ° <Δθ < In the case of 90 °), the amplitude of vibration that vibrates the touch pad 7 in accordance with the amount of displacement increases in the positive direction. Similarly, when the amount of displacement per unit time of the vibration reference value decreases with respect to the previous detection time by crossing the edge of the operation target (that is, the vibration boundary which is the boundary of the operation region) (−90 ° < In the case of Δθ <0 °), the amplitude of vibration that vibrates the touch pad 7 in accordance with the amount of displacement increases in the negative direction. As shown in FIG. 10, the positive direction of the vibration waveform is a direction that vibrates in the front direction of the touch pad 7 (that is, the touch pad 7 is convex), and the negative direction is the back direction of the touch pad 7 (that is, the touch pad). 7 is a direction to vibrate in a concave shape.

  Thereafter, the process proceeds to S26, and as described above, the CPU 11 vibrates the touch pad 7 with the vibration waveform U calculated in S35. The vibration process in S26 is repeatedly executed at predetermined detection cycles until it is determined in S13 that the acceptance of the operation is finished.

Hereinafter, the vibration mode of the touch pad 7 by the operation support processing program S26 will be described with reference to FIGS.
First, a vibration mode when the vibration reference value increases with time, that is, when the touch coordinates move in the direction of increasing the tilt of the operation target will be described with reference to FIG. In the example shown in FIG. 11, at a timing t <b> 1 when the displacement amount of the vibration reference value per unit time increases (a timing when the operation object crosses the edge (vibration boundary) and reaches the ascending slope of the operation object), t <b> 1. Causes vibration. In this case, the vibration waveform has vibration components Uh, Uθ> 0, and thus has a larger amplitude in the positive direction. As a result, it is possible to provide a tactile sensation with a finger on the corner of the button.
Thereafter, while the vibration reference value is displaced, vibration is repeatedly generated by the vibration waveform having the same shape according to the rising angle. The vibration component Uθ during that time is zero. This makes it possible to give a sense of touching the slope of the button at a certain angle.
Next, at the timing t2 when the displacement amount of the vibration reference value per unit time decreases to “0” (timing when the ascending slope of the operation target is finished) t2, the vibration is finished after the last vibration is generated. . Since the vibration waveform at that time is vibration component Uh = 0 and Uθ <0, the vibration waveform has an amplitude in the negative direction, which is the opposite direction. Accordingly, it is possible to give a tactile sensation in which the upward inclination ends and the finger moves on the plane.

  When t2 is located between touch coordinate detection periods as shown in FIG. 12, the last vibration waveform is not a negative vibration waveform as shown in FIG. 11, but a positive direction with a reduced amplitude. It becomes a waveform.

Next, a vibration mode when the vibration reference value is lowered with time, that is, when the touch coordinates are moved downward in the inclination of the operation target will be described with reference to FIG. In the example shown in FIG. 13, vibration is first generated at a timing t <b> 1 at which the displacement amount of the vibration reference value per unit time decreases (timing when the operation target reaches the down slope). In this case, the vibration waveform has vibration components Uh and Uθ <0, and thus the vibration waveform has a larger amplitude in the negative direction. As a result, it is possible to provide a tactile sensation with a finger on the corner of the button.
Thereafter, while the vibration reference value is displaced, the vibration is repeatedly generated by the vibration waveform having the same shape corresponding to the descending angle. The vibration component Uθ during that time is zero. This makes it possible to give a tactile sensation down the slope of the button at a certain angle.
Next, at a timing t2 when the amount of displacement of the vibration reference value per unit time increases and becomes “0” (timing when the descending slope of the operation object ends across the edge (vibration boundary) of the operation object). The vibration is finished after the last vibration is generated. Since the vibration waveform at that time is vibration component Uh = 0 and Uθ> 0, the vibration waveform has an amplitude in the positive direction, which is the opposite direction. Accordingly, it is possible to give a tactile sensation in which the downward inclination is finished and the finger moves on the plane.

  As shown in FIG. 14, when t2 is located between touch coordinate detection cycles, the last vibration waveform is not a positive vibration waveform as shown in FIG. 13, but a negative direction with a reduced amplitude. It becomes a waveform.

11 to 14, when the position of the touch point moves out of the operation region across the vibration boundary from within the operation region, and when the position of the touch point crosses the vibration boundary from outside the operation region. The vibration mode for vibrating the touch pad 7 is different from when it is moved inward. In addition, the position of the touch point crosses the vibration boundary from the outside of the operation area as compared with the case where the position of the touch point moves outside the operation area across the vibration boundary from within the operation area (vibration waveform in FIGS. 13 and 14). When moving into the operation area (vibration waveforms in FIGS. 11 and 12), it is desirable to increase the amplitude of vibration that vibrates the touch pad 7.
Furthermore, in the vibration mode shown in FIGS. 11 and 12, the magnitude of the amplitude for vibrating the touch pad 7 at the timing when the displacement amount per unit time of the vibration reference value increases is the displacement amount per unit time of the vibration reference value. It becomes larger than the magnitude of the amplitude that vibrates the operation surface at the timing of decreasing.

  In the embodiment described above, the detection cycle of touch coordinates and the vibration cycle completely coincide with each other. However, when the detection cycle is longer, the above-described S25 is performed until the next detection cycle comes. It is configured to repeatedly perform vibration based on the vibration waveform calculated in step (1). If the vibration period is longer, the next vibration is performed with the vibration based on the vibration waveform calculated in the most recent S25 every time the vibration period ends.

  Subsequently, the sub-process of the second vibration process executed in S11 will be described with reference to FIG. FIG. 15 is a flowchart of a sub-processing program for the second vibration processing.

  First, in S41, the CPU 11 reads the vibration reference information 15 stored in the flash memory 14, and specifies the vibration reference value corresponding to the touch coordinates newly detected in S3 this time. The vibration reference value is set according to the arrangement mode and shape of the operation object virtually arranged on the touch pad 7 as described above, and the coordinate system of the touch pad 7 is associated with the vibration reference value. Information (see FIG. 4).

  Next, in S42, the CPU 11 predicts the position of the touch coordinates detected after the next one detection cycle based on the touch coordinates newly detected in S3 this time and the history of the touch coordinates stored in the flash memory 14. .

Here, the touch coordinate prediction process in S42 will be described with reference to FIGS.
<First Method>
For example, in the first method shown in FIG. 16, the separation distance P and the azimuth γ of the current touch coordinates with respect to the previous touch coordinates are specified, and the next touch coordinates are predicted by moving the same distance in the same direction thereafter. Predict.
<Second method>
In the second method shown in FIG. 17, the separation distance P1 and the orientation γ1 of the previous touch coordinates with respect to the previous touch coordinates are specified, respectively, and the separation distance P2 and the orientation γ2 of the current touch coordinates with respect to the previous touch coordinates are determined. Identify each one. Then, the next touch coordinate is predicted by predicting that the movement will continue with the same azimuth change amount and distance change amount. For example, the separation distance of the previous touch coordinates with respect to the previous touch coordinates is “2” and the orientation is “10 °”, the separation distance of the current touch coordinates with respect to the previous touch coordinates is “4”, and the orientation is “20”. If it is “°”, a point that is “6” away from the current touch coordinate in the direction of “30 °” is predicted as the next touch coordinate.
<Third method>
Further, in the third method shown in FIG. 18, the touch coordinates predicted by the first method shown in FIG. 16, the touch coordinates predicted by the first method shown in FIG. 16, and the second coordinates shown in FIG. The intermediate point with the touch coordinates predicted by the method is predicted as the next touch coordinates.

  Next, in S42, the CPU 11 determines whether or not the detection period of the touch coordinates is twice or more the vibration period. The detection cycle is basically determined based on the type of the touch pad 7, and the vibration cycle is determined depending on the type of the piezoelectric element 8 and the installation mode.

  And when it determines with the detection period of a touch coordinate being less than twice a vibration period (S43: NO), it transfers to S44. On the other hand, when it is determined that the touch coordinate detection period is twice or more the vibration period (S43: YES), the process proceeds to S48.

  In S44, the CPU 11 executes a waveform specifying process (FIG. 21) described later. The waveform specifying process is a process of selecting a basic shape of a vibration waveform for vibrating the touch pad 7 from a plurality of patterns based on a touch coordinate movement mode predicted in the future.

  In S45, the CPU 11 executes a second vibration waveform calculation process (FIG. 23) described later. Here, the second vibration waveform calculation process is based on the vibration waveform whose basic shape is selected in S44 based on the vibration reference value corresponding to the current touch coordinate and the vibration reference value corresponding to the next touch coordinate. This is a process of calculating a specific shape. The vibration waveform is defined by the vibration direction, amplitude, and vibration period. Since the vibration period is determined by the type and installation mode of the piezoelectric element 8, the vibration direction is determined by S44. In particular, the amplitude is calculated.

  Thereafter, in S46, the CPU 11 vibrates the touch pad 7 based on the vibration waveform selected and calculated in S44 and S45. Specifically, the CPU 11 applies a signal voltage corresponding to the vibration waveform selected and calculated in S44 and S45 to the piezoelectric element 8, thereby distorting the piezoelectric element 8 and vibrating the touch pad 7. As a result, the touch pad 7 is vibrated by the vibration waveform selected and calculated in S44 and S45. Then, by generating vibration, a tactile sensation as if the user touches the touchpad is touching an actual operation target (for example, a finger is placed on the corner of the button or the slope of the button is moved up). Give the sensation of going down).

  Thereafter, in S47, the CPU 11 cumulatively stores the touch coordinates newly detected in S3 this time in a storage medium such as the flash memory 14 as a history of touch coordinates. Thereafter, the process proceeds to S12.

  On the other hand, in S48, the CPU 11 equally divides the touch coordinates newly detected in S3 this time and the touch coordinates detected after the next one detection period predicted in S42 according to the vibration period, Specify the touch coordinate position of the segment point. Specifically, if the detection period of the touch coordinates is twice the vibration period, it is divided into two equal parts, and if the detection period of the touch coordinates is three times the vibration period, it is divided into three equal parts. Note that the fractional part is rounded down (for example, if it is 2.5 times, it is divided into two equal parts). As a result, it is possible to predict the position of the touch coordinates for each starting point of the subsequent vibration cycle for each vibration cycle.

  For example, as shown in FIG. 19, when the current touch coordinate and the next touch coordinate are located and the detection period of the touch coordinate is three times the vibration period, the current touch coordinate and the next touch coordinate Divide the interval into three equal parts. Then, the two touch coordinates R1 and R2 of each division point are specified. As a result, R1 is the position of the touch coordinate at the start time of the next vibration cycle (one vibration cycle after the current time), and R1 is the touch coordinate at the start time of the next vibration cycle (two vibration cycles after the current time). It becomes the position.

  The subsequent processes of S49 to S53 are repeatedly executed as processing targets in order from the touch coordinates closest to the current touch coordinates for all the touch coordinates specified in S48. Then, after the processes of S49 to S53 are executed for all the touch coordinates specified in S48, the process proceeds to S54.

  First, in S49, the vibration reference information 15 stored in the flash memory 14 is read, and the vibration reference value corresponding to the touch coordinates to be processed is specified.

  Next, in S50, the vibration reference information 15 stored in the flash memory 14 is read, and the vibration reference value corresponding to the touch coordinates to be processed last time is specified. When the process of S50 is performed first (when there is no touch coordinate to be processed last time), the vibration reference value corresponding to the current touch coordinate specified in S41 is used instead.

  Subsequently, in S51, the CPU 11 calculates a difference between the vibration reference value corresponding to the touch coordinates of the current processing target specified in S49 and the vibration reference value corresponding to the touch coordinates of the previous processing target specified in S50. To do.

In S <b> 52, the CPU 11 corrects the movement speed V of the touch coordinates calculated in S <b> 6 based on the touch coordinates to be processed this time and the touch coordinates to be processed last time. Specifically, when the distance between the previous and present touch coordinates is L ′, the detection period of the touch coordinates is T ′, and the number equally divided in S48 is N, the following equation (6) Is calculated by
V = L ′ / (T ′ / N) (6)

  Next, in S53, the CPU 11 executes the first vibration waveform calculation process (FIG. 8) described above. However, in S31, the difference calculated in S51 is read out, and in S33, the angle difference between the angle of the vibration reference value corresponding to the touch coordinates of the previous processing target and the angle of the vibration reference value corresponding to the touch coordinates of the current processing target is calculated. To do. As a result, the vibration waveform U is calculated for each vibration period included until the next touch coordinate detection period.

  Thereafter, in S54, which is executed after the processing of S49 to S53 is executed for all the touch coordinates specified in S48, the vibration waveform U for each vibration period calculated in S53 is synthesized. As a result, a vibration waveform as shown in FIG. 20 is calculated.

  Thereafter, in S55, the CPU 11 vibrates the touch pad 7 based on the vibration waveform calculated in S54. Specifically, the CPU 11 distorts the piezoelectric element 8 and vibrates the touch pad 7 by applying a signal voltage corresponding to the vibration waveform calculated in S54 to the piezoelectric element 8 continuously every vibration period. For example, in the example shown in FIG. 20, the vibration for three vibration periods included until the next touch coordinate detection period is sequentially performed in S46. As a result, the touch pad 7 is vibrated by the vibration waveform calculated in S54. Then, by generating vibration, a tactile sensation as if the user touches the touchpad is touching an actual operation target (for example, a finger is placed on the corner of the button or the slope of the button is moved up). Give the sensation of going down).

  Thereafter, in S47, the CPU 11 cumulatively stores the touch coordinates newly detected in S3 this time in a storage medium such as the flash memory 14 as a history of touch coordinates. Thereafter, the process proceeds to S12.

  Subsequently, the sub-process of the waveform specifying process executed in S44 will be described with reference to FIG. FIG. 21 is a flowchart of a sub-processing program for waveform identification processing.

  First, in S <b> 61, the CPU 11 reads a waveform pattern that is a candidate for a vibration waveform for vibrating the touch pad 7 from the RAM 12 or the like. In the present embodiment, the three waveform patterns A to C shown in FIG. 22 are read out. As shown in FIG. 22, the waveform pattern A is a pattern that vibrates once in the front direction of the touch pad 7 (that is, the touch pad 7 is convex), and the waveform pattern B is the back direction of the touch pad 7 (that is, the touch pad 7). Is a pattern that vibrates once in a concave shape, and the waveform pattern C is a pattern that continuously vibrates in the front direction after being vibrated in the depth direction of the touch pad 7. In addition, you may prepare patterns other than said AC as a waveform pattern used as the candidate of a vibration waveform.

  Next, in S <b> 62, the CPU 11 determines whether or not the touch coordinates newly detected in S <b> 3 this time are located on any of the operation objects virtually arranged on the touch pad 7. Specifically, the arrangement area of the operation target is stored in advance for each display screen, and the determination is made by comparing the touch coordinates with the arrangement area. When the vibration reference value corresponding to the touch coordinates is a value preset for each display screen (for example, “h1” and “h2” in the example shown in FIG. 4), it is determined that the position is on the operation target. May be.

  If it is determined that the touch coordinates newly detected in S3 this time are located on any of the operation objects virtually arranged on the touch pad 7 (S62: YES), the process proceeds to S63. On the other hand, when it is determined that the touch coordinates newly detected in S3 this time are not located on any operation target virtually arranged on the touch pad 7 (S62: NO), the process proceeds to S68. Transition.

  In S63, the CPU 11 determines whether or not the touch coordinates after the next one detection cycle predicted in S42 are located on the same operation target as the current touch coordinates. Since the specific determination method is the same as that in S62, a description thereof will be omitted.

  When it is determined that the touch coordinates after the next one detection cycle are located on the same operation target as the current touch coordinates (S63: YES), the process proceeds to S64. On the other hand, when it is determined that the touch coordinates after the next one detection cycle are not located on the same operation target as the current touch coordinates (S63: NO), the process proceeds to S65.

  In S64, the CPU 11 estimates that the touch coordinates will move on the same operation target in the future (that is, the vibration reference value will not be displaced), and does not select the vibration waveform that vibrates the touch pad 7 (that is, does not vibrate). Decide). Thereafter, the process proceeds to S45.

  On the other hand, in S65, the CPU 11 determines whether or not the touch coordinates after the next one detection cycle predicted in S42 are located on an operation target different from the current touch coordinates. Since the specific determination method is the same as that in S62, a description thereof will be omitted.

  If it is determined that the touch coordinates after the next one detection cycle are located on an operation object different from the current touch coordinates (S65: YES), the process proceeds to S66. On the other hand, when it is determined that the touch coordinates after the next one detection cycle are not located on the operation target different from the current touch coordinates (S65: NO), the process proceeds to S67.

  In S <b> 66, the CPU 11 estimates that the touch coordinates will move from the current operation object to another operation object (i.e., the vibration reference value increases again after once decreasing), and vibrates the touch pad 7. The waveform pattern C is selected as the basic shape of the vibration waveform to be performed. Thereafter, the process proceeds to S45. The waveform pattern C is a pattern that vibrates continuously in the front direction after being vibrated in the back direction of the touch pad 7 as described above. By applying vibration based on the waveform pattern C, it is possible to provide a tactile sensation that once descends from the operation target and then moves up to another operation target.

  In S67, the CPU 11 estimates that the touch coordinates will move from the operation target to a surface without the operation target (that is, the vibration reference value decreases), and the vibration waveform that vibrates the touch pad 7 is determined. The waveform pattern B is selected as the basic shape. Thereafter, the process proceeds to S45. The waveform pattern B is a pattern that vibrates once in the back direction of the touch pad 7 as described above. By applying the vibration based on the waveform pattern B, it is possible to give a tactile sensation descending from the operation target.

  On the other hand, in S <b> 68, the CPU 11 determines whether or not the touch coordinates after the next one detection cycle predicted in S <b> 42 are located on any operation target virtually arranged on the touch pad 7. Since the specific determination method is the same as that in S62, a description thereof will be omitted.

  If it is determined that the touch coordinates after the next one detection cycle are located on any of the operation objects virtually arranged on the touch pad 7 (S68: YES), the process proceeds to S69. On the other hand, when it is determined that the touch coordinates after the next one detection cycle are not located on any operation target virtually arranged on the touch pad 7 (S68: NO), the process proceeds to S70. Transition.

  In S <b> 69, the CPU 11 estimates that the touch coordinates will move from the surface without the operation target onto the operation target (that is, the vibration reference value increases), and sets the basic shape of the vibration waveform for vibrating the touch pad 7. Select waveform pattern A. Thereafter, the process proceeds to S45. The waveform pattern A is a pattern that vibrates once in the front direction of the touch pad 7 as described above. By applying vibration based on the waveform pattern A, it is possible to impart a tactile sensation that rises on the operation target.

  On the other hand, in S70, the CPU 11 estimates that the touch coordinates will not move onto the operation target in the future (that is, the vibration reference value will not be displaced) and does not select the vibration waveform that vibrates the touch pad 7 (that is, does not vibrate). To decide). Thereafter, the process proceeds to S45.

  Next, the sub-process of the second vibration waveform calculation process executed in S45 will be described with reference to FIG. FIG. 23 is a flowchart of a sub-processing program for the second vibration waveform calculation process.

  First, in S71, the CPU 11 reads the vibration reference information 15 stored in the flash memory 14, and specifies the vibration reference value corresponding to the touch coordinates after the next one detection cycle predicted in S42. Then, a difference D between the vibration reference value corresponding to the current touch coordinate specified in S41 and the vibration reference value corresponding to the touch coordinate after the next one detection cycle is calculated.

Next, in S72, the CPU 11 calculates a vibration component Uh based on the difference D calculated in S71. Specifically, when the moving speed of the touch coordinates calculated in S6 is V, the following calculation is performed using the following equation (7).
Uh = D × V × A (7)
“A” is a unique coefficient for each apparatus.

  Subsequently, in S73, the CPU 11 predicts the timing at which the vibration reference value starts to be displaced from the vibration reference value corresponding to the current touch coordinates. For example, when the vibration reference value corresponding to each touch coordinate is specified as shown in FIG. 24, s1 is predicted as the timing at which the vibration reference value starts to be displaced from the vibration reference value corresponding to the current touch coordinate. The For prediction, vibration reference information 15 stored in the flash memory 14 is used in addition to the vibration reference value corresponding to each touch coordinate.

  Thereafter, in S74, the CPU 11 applies the vibration component Uh based on the difference between the vibration reference values calculated in S72 to the waveform pattern selected in S44, and further, the vibration reference value predicted in S73 shifts the displacement. A vibration waveform that starts vibration at the start timing is calculated as a vibration waveform U. However, if the waveform pattern is not selected in S44, U = 0 (does not vibrate).

  As a result, the final vibration waveform U vibrates in the direction according to the waveform pattern (FIG. 22) selected in S44, and further, the larger the displacement amount per unit time of the vibration reference value, and the touch coordinate. The faster the moving speed, the larger the amplitude of vibration that vibrates the touch pad 7.

  Thereafter, the process proceeds to S46, and as described above, the CPU 11 vibrates the touch pad 7 with the vibration waveform U calculated in S74. If the next touch coordinate detection cycle is reached before the vibration executed in S46 is completed, the next vibration is performed after the current vibration is completed.

Hereinafter, the vibration mode of the touch pad 7 according to S46 of the operation support processing program will be described with reference to FIGS.
First, referring to FIG. 25, a description will be given of a vibration mode when it is predicted that the touch coordinates will move on the operation target from a surface without the operation target in the future. In the example shown in FIG. 25, the timing at which the vibration reference value is predicted to start displacement (the timing at which the operation object reaches the upslope across the edge of the operation object (that is, the vibration boundary that is the boundary of the operation area)). ) At s1, vibration is generated. The vibration waveform at that time is a vibration waveform that vibrates once in the front direction of the touch pad 7 (that is, the touch pad 7 is convex). As a result, it is possible to apply a tactile sensation in which the finger is applied to the corner of the button, and then goes up the slope of the button, and finally the upward inclination finishes and the finger moves to the plane.

  Next, a vibration mode when it is predicted that the touch coordinates will move from the operation target to a surface without the operation target in the future will be described with reference to FIG. In the example shown in FIG. 26, the vibration is generated at the timing (timing when the vibration reference value reaches the downward slope of the operation target) s2 when the vibration reference value is predicted to start displacement. The vibration waveform at that time is a vibration waveform that vibrates once in the back direction of the touch pad 7 (that is, the touch pad 7 is concave). As a result, it is possible to give a tactile sensation that a finger is applied to the corner of the button, then descends the slope of the button, and finally the descending slope ends and the finger moves to the plane.

  Next, a vibration mode when it is predicted that the touch coordinates will move from the current operation object to another operation object in the future will be described with reference to FIG. In the example shown in FIG. 27, the vibration is generated at the timing (timing that the vibration reference value reaches the down slope of the button) s3 when the vibration reference value is predicted to start displacement. In this case, the vibration waveform is oscillated in the depth direction of the touch pad 7 (that is, the touch pad 7 is concave) and then continuously in the front direction of the touch pad 7 (that is, the touch pad 7 is convex) each time (total (Twice) vibration waveform to vibrate. As a result, a finger is applied to the corner of the button, and then the slope of the button is lowered, and further, the slope of the other button is raised. It becomes possible.

As described above in detail, according to the operation support system 1, the operation support method by the operation support system 1, and the computer program executed by the operation support system 1 according to the present embodiment, they are virtually arranged on the touch pad 7. A vibration reference value is set for the coordinate system of the touch pad 7 based on the arrangement form and shape of the operation target, the touch coordinates where the user touches the touch pad 7 is detected (S3), and the touch coordinates are moved. The displacement mode of the vibration reference value corresponding to the touch coordinates is specified (S21 to S25), and the touch pad 7 is vibrated by the waveform based on the specified displacement mode (S26). Even if the user is not visually recognizing the user, the user can accurately grasp the arrangement mode and shape of the operation object virtually arranged on the touch pad 7. So, it becomes possible to perform precise operations with no errors.
Further, since the touch pad 7 is continuously vibrated while the vibration reference value is displaced, an actual operation is performed when the touch coordinates move around the operation object virtually arranged on the touch pad 7. It is possible to give the user a tactile sensation equivalent to moving the slope or step of the object. In addition, it is possible to allow the user to grasp that the touch coordinates have moved onto the operation target or have deviated from the operation target.
In addition, since the amplitude of vibration that vibrates the touch pad 7 increases as the displacement amount per unit time of the vibration reference value increases, the actual operation target is considered in consideration of the movement speed of the touch coordinates and the shape of the operation target. It is possible to give the user a tactile sensation equivalent to moving an inclined surface or a step of an object more accurately.
In addition, the higher the movement speed of the touch coordinates, the larger the amplitude of vibration that vibrates the touch pad 7, so that it is equivalent to moving the slope or step of the actual operation target in consideration of the movement speed of the touch coordinates. The tactile sensation can be given to the user more accurately.
Further, at the timing when the displacement amount per unit time of the vibration reference value increases or decreases, the touch pad 7 is vibrated based on the increase amount or the decrease amount. In the case of moving, it is possible to give the user a tactile sensation equivalent to crossing the edge or corner of the actual operation target.
Moreover, since the touch pad 7 is vibrated based on the displacement mode of the vibration reference value specified for each touch coordinate detection cycle, it is possible to apply appropriate vibration according to the current touch coordinate for each detection cycle. It becomes.

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in the present embodiment, the case of performing a touch operation with a touch pad is described, but a touch panel other than the touch pad, a tablet, or the like may be used as a means for receiving the touch operation. In that case, the touch panel and the tablet are configured to vibrate. Moreover, when using a touch panel, the display 6 which displays the operation target virtually arranged on the touch panel is superimposed on the touch panel.

  In the present embodiment, in the waveform specifying process shown in FIG. 21, the vibration waveform for vibrating the touch pad 7 is selected from the three waveform patterns shown in FIG. 22, but the three waveform patterns shown in FIG. It is good also as a structure made to select from other than.

  In the present embodiment, the touchpad 7 is not vibrated when a flick operation is performed as a touch operation. However, when a specific touch operation other than the flick operation is performed, the touchpad 7 is not vibrated. It is good also as a structure which does not perform a vibration.

  Further, the present invention can be applied to various devices having means for accepting a touch operation in addition to various on-vehicle devices mounted on the vehicle and devices for operating the system by a touch operation of a driver who drives the vehicle. It is. For example, the present invention can be applied to a mobile phone, a smartphone, a tablet terminal, a personal computer, a portable music player, etc. (hereinafter referred to as a portable terminal).

1 Operation support system 5 Operation support ECU
6 Display 7 Touchpad 8 Piezoelectric element 11 CPU
12 RAM
13 ROM
14 Flash memory 15 Vibration standard information 21 Power button 22 Temperature increase button 23 Temperature decrease button

Claims (9)

In an operation support system comprising an operation surface that receives a user's operation and a vibration mechanism that vibrates the operation surface at a predetermined vibration cycle.
A vibration reference information acquisition unit that acquires vibration reference information that is set according to the arrangement mode and shape of the operation target that is virtually arranged on the operation surface and associates the coordinate system of the operation surface with a vibration reference value. When,
Touch point detection means for detecting the position of the touch point where the user touches the operation surface for each predetermined detection cycle longer than the vibration cycle;
A touch point prediction unit that predicts a future position of the touch point for each vibration period based on a history of the position of the touch point detected by the touch point detection unit;
Based on the position of the touch point predicted by the touch point prediction means and the vibration reference information, a displacement mode of the vibration reference value corresponding to the position of the touch point accompanying the future movement of the touch point is predicted. Displacement mode prediction means;
An operation support system comprising: an operation surface vibration unit that vibrates the operation surface by the vibration mechanism based on a displacement mode of the vibration reference value predicted by the displacement mode prediction unit. The touch point prediction means predicts the position of the touch point at the start time of the vibration cycle,
The displacement mode prediction means includes
Identifying the vibration reference value corresponding to the position of the touch point at the start of the vibration cycle;
The operation support system according to claim 1, wherein a displacement mode of the vibration reference value is predicted based on the specified vibration reference value.   The operation support system according to claim 1, wherein the detection cycle is twice or more than the vibration cycle.   4. The operation surface vibration means continuously vibrates the operation surface while the vibration reference value predicted by the displacement mode prediction means is displaced. 5. The operation support system according to any one of the above.   The operation surface vibration means increases the amplitude of vibration that vibrates the operation surface as the displacement amount per unit time of the vibration reference value predicted by the displacement mode prediction means increases. 4. The operation support system according to 4.   The operation support system according to claim 4, wherein the operation surface vibration unit increases the amplitude of vibration that vibrates the operation surface as the moving speed of the touch point increases.   The operation surface vibration means vibrates the operation surface based on the increase amount or the decrease amount at a timing when the displacement amount per unit time of the vibration reference value predicted by the displacement mode prediction means increases or decreases. The operation support system according to any one of claims 1 to 6, wherein In an operation support method for an operating device, comprising: an operation surface that receives a user operation; and a vibration mechanism that vibrates the operation surface at a predetermined vibration cycle.
A vibration reference information acquisition step that acquires vibration reference information that is set according to the arrangement mode and shape of the operation target that is virtually arranged on the operation surface and associates the coordinate system of the operation surface with a vibration reference value. When,
A touch point detecting step of detecting the position of the touch point where the user touches the operation surface at every predetermined detection cycle longer than the vibration cycle;
Based on the history of the position of the touch point detected by the touch point detection step, the touch point prediction step of predicting the position of the future touch point for each vibration cycle;
Based on the position of the touch point predicted by the touch point prediction step and the vibration reference information, a displacement mode of the vibration reference value corresponding to the position of the touch point accompanying the future movement of the touch point is predicted. A displacement mode prediction step;
An operation surface vibration step of vibrating the operation surface by the vibration mechanism based on the displacement mode of the vibration reference value predicted by the displacement mode prediction step. In a computer program for operating an operation device including an operation surface that receives a user's operation and a vibration mechanism that vibrates the operation surface at a predetermined vibration cycle,
A vibration reference information acquisition function that acquires vibration reference information that is set according to the arrangement mode and shape of the operation target that is virtually arranged on the operation surface and associates the coordinate system of the operation surface with a vibration reference value. When,
A touch point detection function for detecting the position of the touch point touched by the user at each predetermined detection cycle longer than the vibration cycle;
Based on the history of the position of the touch point detected by the touch point detection function, a touch point prediction function that predicts the future position of the touch point for each vibration cycle;
Based on the position of the touch point predicted by the touch point prediction function and the vibration reference information, the displacement mode of the vibration reference value corresponding to the position of the touch point accompanying the future movement of the touch point is predicted. A displacement mode prediction function;
Based on a displacement mode of the vibration reference value predicted by the displacement mode prediction function, an operation surface vibration function for vibrating the operation surface by the vibration mechanism;
A computer program for executing

Images