JP5766398B2 - Tactile presentation device - Google Patents

Description

  The present invention relates to an apparatus that receives an input to a touch sensor and presents a tactile sensation.

  2. Description of the Related Art In recent years, in portable terminals such as mobile phones, there are some devices that employ an input device including a touch sensor such as a touch panel or a touch switch as an input device such as an operation unit or a switch that receives an operation input by a user. In addition to portable terminals, input devices equipped with touch sensors are widely used in information devices such as calculators and ticket machines, household appliances such as microwave ovens, televisions, and lighting equipment, and industrial equipment (FA devices). .

  As such a touch sensor, various systems such as a resistive film system, a capacitance system, and an optical system are known. However, any type of touch sensor accepts a touch input by a finger or a stylus pen, and the touch sensor itself is not physically displaced like a push button switch even when touched.

  For this reason, even if the touch sensor accepts a touch input, the operator cannot obtain feedback for the input. Since the operator does not have a feeling of operation when performing the operation input, there is no way to check whether the input operation has been accepted by the terminal, and repeated input such as touching the same position many times. It is likely to occur and may stress the operator.

  As a device for preventing such unnecessary repeated input, for example, a device that receives a touch input and makes a sound is known. In addition, for example, the display mode is changed by changing the display color of an input user interface object (hereinafter simply referred to as “object”) such as an input button displayed on the display unit corresponding to the input position. There are also known things to do. If such a thing is used, the operator can confirm the reception status of his input operation by hearing or vision.

  However, in the case of a feedback method that works on hearing, confirmation in a noisy environment becomes difficult. Furthermore, when the device in use is in a silent state such as the manner mode, feedback that works on hearing cannot be handled. Also, in the case of a feedback method that works visually, if the size of the object displayed on the display unit is small, especially when inputting with the finger, the object is hidden under the finger and the display mode change cannot be confirmed There is.

  Therefore, a feedback method using a device that vibrates the touch sensor and presents a tactile sensation to the fingertip of the operator when the touch sensor accepts input regardless of hearing or vision has been proposed (for example, Patent Document 1 and Patents). Reference 2).

JP 2003-288158 A JP 2008-130055 A

  According to the devices disclosed in Patent Document 1 and Patent Document 2, when an operation input to the touch sensor is received, a tactile sensation due to vibration can be presented to the fingertip of the operator. Therefore, an operator who performs input using such a device can recognize by tactile sense that the operation input performed on the touch sensor has been properly received by the device.

  However, in order to present a tactile sensation when an operation input is received using such an apparatus, a process for presenting a tactile sensation according to a predetermined condition is performed in advance by application software (hereinafter simply referred to as “application”). Need to be built in. That is, in the above-described apparatus, if an application developed in advance assuming that vibration is generated is used, a tactile sensation can be presented according to a predetermined condition. However, even if an application that is not supposed to generate vibration is applied as it is to the above-described apparatus, it is not possible to present a tactile sensation appropriately.

  Therefore, in order to present a tactile sensation appropriately according to a predetermined condition in the apparatus as described above, a new application is developed or a process for presenting a tactile sensation is added to an existing application. Etc. need to be modified.

  Specifically, when an input is made to a touch sensor corresponding to the position of an object such as a key or button displayed on the display unit of the tactile sensation presentation apparatus, the apparatus presents a predetermined tactile sensation. Processing needs to be added to the application. In addition, it is necessary to prevent the device from presenting a tactile sensation even when an input is made to a touch sensor corresponding to a position where an object such as a key or a button is not displayed on the display unit.

  By controlling the presence or absence of the tactile sensation in this way, when an operation input is received by a user interface using a touch sensor and a GUI (Graphical User Interface) for input / output, the tactile sensation due to vibration is applied to the fingertip of the operator. Can be presented. That is, it is possible to notify the operator of a tactile sensation that the input is accepted by an input to an object such as a key or button displayed as an image. On the other hand, it is possible to prevent feedback even if an input is made to a position where there is no object such as a key or button.

  When performing such control, the tactile sensation providing apparatus generally manages the state of the GUI when an input is made to the touch sensor, that is, the state of an object such as a key or button displayed on the display unit. It is necessary to link with existing applications.

  Hereinafter, processing assumed when presenting a tactile sensation in the tactile sensation providing apparatus will be described.

  FIG. 8 is a block diagram illustrating a schematic configuration of an assumed tactile sensation providing apparatus. As shown in FIG. 8, the tactile sensation providing apparatus 100 includes a control unit 110, an application execution unit 120, a display unit 130, a touch sensor 140, and a tactile sensation providing unit 150. The control unit 110 includes a display control unit 112 and a tactile sensation control unit 114.

  The control unit 110 controls the entire tactile sensation providing apparatus 100 including each functional unit of the tactile sensation providing apparatus 100. The application execution unit 120 reads various applications from a storage unit (not shown) and executes processing based on the applications. In the following description, the application executed by the application execution unit 120 is a process of presenting a predetermined tactile sensation when an input is made to the touch sensor 140 corresponding to the position of the object displayed on the display unit 130. Are added (incorporated) in advance.

  The display unit 130 is configured by a liquid crystal display, an organic EL display, or the like, and displays an image of an object such as a key or button for inputting. The touch sensor 140 is normally disposed on the front surface of the display unit 130 and receives touch input by an operator's finger or the like on the object displayed on the display unit 130 from the touch surface of the corresponding touch sensor 140. The touch sensor 140 notifies the application execution unit 120 of the position of the touch input received through the touch surface via the control unit 110. The tactile sensation providing unit 150 is configured by a small eccentric motor, a piezoelectric element, or the like, and presents a tactile sensation to a touch target such as an operator's finger by vibrating the touch surface of the touch sensor 140.

  Based on the display data supplied from the application execution unit 120, the display control unit 112 of the control unit 110 displays an image such as an object on the display unit 130, or changes the display mode of the image. Control the display. The tactile sensation control unit 114 controls the tactile sensation providing unit 150 to present a vibration pattern based on a predetermined condition based on an instruction from the application execution unit 120.

  FIG. 9 is a flowchart schematically illustrating an example of an operation of presenting a tactile sensation based on an input by the tactile sensation providing apparatus 100.

  When touch input to the touch surface of the touch sensor 140 is detected, an operation for controlling the presentation of tactile sensation by the tactile sensation providing apparatus 100 is started. When this operation starts, first, the touch sensor 140 notifies the application execution unit 120 of the position of the touch input on the touch surface (step S100). Upon receiving this notification, the application execution unit 120 determines whether or not the input position corresponds to a position where an image such as a key or a button is displayed on the display unit 130 (step S102). In step S102, when the input position does not correspond to the position where an image of an object such as a key or a button is displayed on the display unit 130, the application execution unit 120 ends the operation without giving an instruction regarding display or touch. In this case, no tactile sensation is presented.

  On the other hand, in step S102, when the input position corresponds to a position where an image of an object such as a key or button is displayed, the application execution unit 120 displays to the display control unit 112 that the input has been accepted. Display data for performing Here, the display indicating that the input has been accepted can be visually recognized by the operator, for example, by highlighting an object such as a key or a button for a moment or displaying an image showing a state where a key or a button is pressed. It is to display like this. Upon receiving such display data from the application execution unit 120, the display control unit 112 changes the image displayed on the display unit 130 based on the display data (step S104).

  Subsequently, the application execution unit 120 instructs the tactile sensation control unit 114 to present a predetermined tactile sensation in order to notify the operator that the input has been accepted. Here, the presentation of the predetermined tactile sensation is to provide feedback that can be recognized by the tactile sense of the operator, such as vibrating the touch surface of the touch sensor 140 for a moment. Upon receiving such a tactile sensation presentation instruction, the tactile sensation control unit 114 controls the tactile sensation presentation unit 150 to present a predetermined vibration pattern in accordance with the instruction. Accordingly, the tactile sensation providing unit 150 presents a tactile sensation with a predetermined vibration pattern to the touch target touching the touch surface of the touch sensor 140 (step S106). By performing such processing, the tactile sensation providing apparatus 100 can present a tactile sensation only when an operation input for an object that accepts an input is received by a user interface that employs a touch sensor and a GUI at the time of input. it can.

  As described above, according to the tactile sensation providing apparatus 100, the above-described operation can be performed without any problem as long as the application supports the operation of presenting the tactile sensation. In other words, if the application incorporates a process for presenting a tactile sensation in advance, a predetermined tactile sensation can be appropriately presented by the above-described operation.

  However, in the tactile sensation providing apparatus 100 described above, when an application that does not support tactile sensation is applied, even if the display of the object changes based on the input, the tactile sensation can be appropriately presented in the state as it is. Can not. Therefore, in order to effectively use various application resources that have been developed up to now in the tactile sensation providing apparatus 100, each application is modified and a tactile sensation is presented according to a predetermined condition. Processing must be added. Such modification to add processing must be performed for each of various applications, and there is a concern that it takes enormous effort and cost. In addition, if a different tactile sensation is to be presented for each object, a process for presenting a different tactile sensation for each object is added, so that labor and cost are extremely increased.

  Under such circumstances, it may become a factor that hinders vendors from entering the development of applications for the tactile sensation presentation device in the future, or may be a factor that delays the spread of platforms that employ touch sensors and GUIs.

  For example, the techniques disclosed in Patent Document 1 and Patent Document 2 merely vibrate the touch sensor according to the input when the touch sensor accepts the input. For this reason, in the technologies disclosed in Patent Document 1 and Patent Document 2, it is assumed that the tactile sensation cannot be appropriately presented even when an application corresponding to the tactile sensation is applied. In particular, in a device in which a button switch such as a mechanical push button switch (push button switch) is drawn on the touch sensor, a finger or the like touches the touch sensor lightly when the threshold value for accepting the touch sensor input is low. A tactile sensation is presented only by (touching). For this reason, since the operator reacts to a light touching operation (touch) in the previous stage of pushing the touch sensor, unlike the reaction of the drawn image (push button), there is a possibility that an erroneous operation is induced.

  In addition, even if the operator does not intend to push the drawn button switch (even if power is not applied), the operator feels tactile according to the touch operation (touch) of the touch sensor. It will also make you feel uncomfortable. Here, the threshold value for accepting an input by touch of the touch sensor is a threshold value at which the touch sensor reacts. For example, in the resistive film method, the threshold value is a threshold value of the pressing force with which the upper conductive film contacts the lower conductive film. In the method, it is a detection threshold value of an electrical signal due to contact.

  Furthermore, even if a user interface based on an application in which a process for presenting a tactile sensation with a touch sensor is adopted in a conventional device that presents a tactile sensation, when an input is received, it is like a mechanical push button switch. It was not possible to provide a comfortable feeling of operation. That is, the tactile sensation presented by the conventional tactile sensation presentation device is only to notify the operator that the input has been accepted, and merely performs an operation such as vibrating the touch surface. Therefore, the tactile sensation presented by the conventional tactile sensation presentation device does not provide the operator with a realistic operational feeling as when operating a mechanical push button switch.

  However, in a user interface employing a touch sensor and a GUI, an image of an object imitating an actual mechanical push button switch is often displayed. Therefore, when an input to such an object is received, it is preferable to present a realistic tactile sensation following the mechanical push button switch as an operation feeling given to the operator. When a pressing input is received for the displayed object, if feedback can be returned as if an actual key or button was clicked, the operational feeling given to the operator can be improved.

  In the case of a mechanical push button switch, an operator obtains a tactile sensation (hereinafter referred to as a click tactile sensation) when the switch is depressed and a metal dome switch or the like is buckled by applying a certain amount of pressing force. Can do. Similarly, in the case of a mechanical push button switch, the operator does not press the original metal dome switch or the like by reducing the pressing force to some extent in the process of releasing the pressing force to the pressed switch. A tactile sensation (hereinafter referred to as a release tactile sensation) that returns to the state can be obtained. Further, in the case of a mechanical push button switch, even if the operator touches the switch with a very weak pressing force (even if touched), the click feeling is not presented, and from the state where the click feeling is not presented, Release feel is not presented. In a tactile sensation providing apparatus that employs a conventional touch sensor and GUI, it has not been possible to provide a click tactile sensation and a release tactile sensation like the mechanical push button switch described above.

  Further, in the case of inputting to a mechanical push button switch, various different tactile sensations can be obtained according to the difference in the physical configuration of the switch mechanism or the like. For example, even when the degree of tactile strength obtained by pressing differs depending on the size of the button, or when the stroke depth when pushing by the button is different, the feeling of different modes when pressed is different. It is obtained.

  The devices disclosed in Patent Documents 1 and 2 cannot present such a different tactile sensation for each object, that is, a tactile sensation suitable for each object. The tactile sensation presented by the devices disclosed in Patent Documents 1 and 2 is primarily intended to notify the operator that the input has been accepted, so the same is true regardless of which object receives the press input. A tactile sensation was generally presented. Therefore, if a different tactile sensation is to be presented depending on the object, as described above, it is necessary to make a modification such as adding a process for presenting each tactile sensation to each application. It will be very laborious and costly.

  Accordingly, an object of the present invention made in view of such circumstances is to provide a tactile sensation providing apparatus that presents a tactile sensation commensurate with an object.

The invention of the tactile sensation providing apparatus according to the first aspect to achieve the above object is as follows:
A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A display unit;
An analysis unit for analyzing information on the height of an object to be displayed on the display unit , or information on a shaded part;
Based on the result of analyzing the information on the height of the object or the information on the shaded part , the touch surface presented to the object by the tactile sensation providing unit is set, and when the input to the object is detected, the touch surface A control unit that controls the tactile sensation providing unit so as to present a tactile sensation according to the object with respect to a pressing target that is pressing
It is characterized by providing.

The invention of the tactile sensation providing apparatus according to the second aspect is as follows:
A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A display unit;
An analysis unit for analyzing an image displayed on the display unit ;
Based on the result of analyzing the image, sets a tactile the tactile sensation providing unit to objects in the image is presented, when detecting an input to the object, relative to the pressing subjects presses the touch surface A control unit that controls the tactile sensation providing unit to present a tactile sensation according to the object;
It is characterized by providing .

The invention according to a third aspect is the tactile sensation device according to the first or second aspect,
A load detecting unit for detecting a pressing load on the touch surface of the touch sensor;
When the control unit detects a pressing load that satisfies a load criterion for the tactile sensation in a state of detecting an input to the object, the tactile sensation is presented so as to present the tactile sensation to the pressing target. The presenting unit is controlled.

The invention according to a fourth aspect is the tactile sensation device according to the third aspect,
The said control part changes the load reference | standard which shows the said tactile sensation to the said object based on the said analysis result.

  According to the present invention, a tactile sensation suitable for an object can be presented.

It is a block diagram which shows schematic structure of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the mounting structure of the tactile sense presentation apparatus shown in FIG. It is a figure explaining the analysis of the user interface by this invention. It is a flowchart which shows click tactile sensation presentation operation | movement from the input reception of the tactile sensation presentation apparatus shown in FIG. It is a flowchart which shows the release tactile sense presentation operation | movement of the tactile sense presentation apparatus shown in FIG. It is a figure explaining the analysis of the user interface by this invention. It is a figure explaining the analysis of the user interface by this invention. It is a block diagram which shows schematic structure of the tactile sense presentation apparatus assumed. It is a flowchart which shows operation | movement of the tactile sense presentation apparatus assumed.

  Embodiments of the present invention will be described below with reference to the drawings.

  According to the present invention, by analyzing information for generating a user interface to be displayed on a display unit, a tactile sensation to be presented when an input is made is set according to the size of each object. In addition, when presenting the tactile sensation in this way, the present invention does not present a tactile sensation to a pressure that touches (touches) the touch sensor lightly, and the operator intentionally presses the touch sensor. Present tactile sensation to the action of pushing (pushing).

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of the tactile sensation providing apparatus according to the first embodiment of the present invention.

  As shown in FIG. 1, the tactile sensation providing apparatus 1 includes a control unit 10, an application execution unit 20, a display unit 30, a touch sensor 40, a tactile sensation providing unit 50, and a load detection unit 60. . The control unit 10 includes a display control unit 12, a tactile sensation control unit 14, and a user interface analysis unit 16. In the drawings and the following description, “user interface” is simply abbreviated as “UI” as appropriate.

  The control unit 10 controls the entire tactile sensation providing apparatus 1 including each functional unit of the tactile sensation providing apparatus 1. The application execution unit 20 reads various applications from a storage unit (not shown) and executes processing based on the applications. In the present embodiment, a case will be described in which the application execution unit 20 executes an application that does not originally support tactile presentation.

  The display unit 30 displays an object such as a push button switch (push button switch) as an image. Note that the push button switch is an input button, a key, or the like (hereinafter simply referred to as “key or the like”). The display unit 30 is configured using, for example, a liquid crystal display panel, an organic EL display panel, or the like. The application execution unit 20 supplies information constituting the UI to the display control unit 12 and the UI analysis unit 16 when generating a UI or changing an object image. When the information constituting the UI is supplied in this way, the display control unit 12 can perform display based on the information on the display unit 30.

  The touch sensor 40 is usually arranged on the front surface of the display unit 30, and a touch input by the operator's finger or the like (press target) on the object displayed on the display unit 30 is touched by the corresponding touch sensor 40. Accept by face. The touch sensor 40 detects the position of the touch input on the touch surface, and notifies the application execution unit 20 of the detected input position via the tactile sensation control unit 14. The touch sensor 40 is configured by a known method such as a resistive film method, a capacitance method, and an optical method.

  The tactile sensation providing unit 50 is configured using a piezoelectric vibrator or the like, and generates vibration on the touch surface of the touch sensor 40. The tactile sensation providing unit 50 presents a tactile sensation to the pressing target pressing the touch surface by generating vibration on the touch surface of the touch sensor 40. The pressing target pressing the touch surface is, for example, an operator's finger. The load detection unit 60 detects a pressing load on the touch surface of the touch sensor 40, and is configured using an element that reacts linearly with a load such as a strain gauge sensor or a piezoelectric element, for example. In addition, the load detection unit 60 notifies the tactile sensation control unit 14 of the detected pressing load on the touch surface of the touch sensor 40.

  Based on the display data supplied from the application execution unit 20, the display control unit 12 of the control unit 10 displays an object or the like on the display unit 30 or changes the image display. Take control. The display control unit 12 receives display data of an object used when generating a UI to be displayed on the display unit 30 from the application execution unit 20. The display control unit 12 also displays object display data indicating that an input has been received when the position of the operator's press input to the touch sensor 40 is a position corresponding to an object that starts a predetermined operation. Received from the application execution unit 20.

  The tactile sensation control unit 14 notifies the application execution unit 20 and the like of the position of the touch input on the touch surface detected by the touch sensor 40.

  The UI analysis unit 16 analyzes information constituting a UI displayed on the display unit 30 supplied from the application execution unit 20. Analysis of information constituting the UI performed by the UI analysis unit 16 will be described later. The UI analysis unit 16 notifies the tactile sensation control unit 14 of information on the result of analysis in this way.

  The tactile sensation control unit 14 that has received the information of the result analyzed by the UI analysis unit 16 in this manner is presented by the tactile sensation presentation unit 50 when the input position with respect to the touch sensor 40 is the position of the object that starts a predetermined operation. Set the tactile sensation. The tactile sensation control unit 14 is set when the pressing load detected by the load detection unit 60 satisfies a load standard for presenting a tactile sensation in a state where an input to an object for executing a predetermined process is detected. The tactile sensation providing unit 50 is controlled to generate a predetermined pattern of vibration.

  2A and 2B show an example of the mounting structure of the tactile sensation providing apparatus 1 shown in FIG. 1, FIG. 2A is a main part sectional view, and FIG. 2B is a main part plan view. The display unit 30 is housed and held in the housing 61. The touch sensor 40 is held on the display unit 30 via an insulator 62 made of an elastic member. In the tactile sensation providing apparatus 1 according to the present embodiment, the display unit 30 and the touch sensor 40 are rectangular in plan view. In addition, the tactile sensation providing apparatus 1 holds the touch sensor 40 on the display unit 30 via the insulators 62 arranged at the four corners deviated from the display area A of the display unit 30 indicated by virtual lines in FIG. To do.

  The casing 61 is provided with an upper cover 63 so as to cover the surface area of the touch sensor 40 that is out of the display area of the display unit 30, and is made of an elastic member between the upper cover 63 and the touch sensor 40. An insulator 64 is provided. In the touch sensor 40 shown in FIG. 2, the surface member having the touch surface 40a is made of, for example, a transparent film or glass, and the back member is made of glass or acrylic. The touch sensor 40 has a structure in which, when the touch surface 40a is pressed, the pressed portion is slightly bent (distorted) in accordance with the pressing force, or the structure itself is bent slightly.

  On the surface of the touch sensor 40, a strain gauge sensor 51 for detecting a load (pressing force) applied to the touch sensor 40 is provided in the vicinity of each side covered with the upper cover 63 by adhesion or the like. Further, on the back surface of the touch sensor 40, piezoelectric vibrators 52 for vibrating the touch sensor 40 are provided by adhesion or the like in the vicinity of two opposing sides. That is, in the tactile sensation providing apparatus 1 shown in FIG. 2, the load detecting unit 60 shown in FIG. 1 is configured using four strain gauge sensors 51, and the tactile sensation providing unit 50 is configured using two piezoelectric vibrators 52. doing. The touch surface 40a is vibrated by vibrating the touch sensor 40 by the tactile sensation providing unit 50. In FIG. 2B, the casing 61, the upper cover 63, and the insulator 64 shown in FIG. 2A are not shown.

  Next, analysis of information constituting the UI according to the present embodiment will be described.

  In the present invention, the UI analysis unit 16 generates object UI data from the application execution unit 20 when the UI is first generated and displayed on the display unit 30 or when the image of the displayed object is changed. Receive various attribute information. The various attribute information including the image data of the object is information necessary for generating a UI or an object constituting the UI. This information includes, for example, information on the position at which the image of the object is displayed in the display area of the display unit 30 in addition to the image data used when displaying the image of each object constituting the UI. . Furthermore, this information can be various types of information indicating the attributes of each object, such as information on the size of the image of the object and information on the height of the object represented by the image of the object.

  In the present embodiment, the UI analysis unit 16 analyzes information obtained by extracting information other than the image of the object displayed on the display unit 30 among the information constituting the UI. Specifically, information such as the size and height of the button represented by the object supplied as the attribute information of the object is extracted (acquired) and analyzed.

  In general, the size of an object can generally be obtained directly as size information by HTML. Further, when the area of the region where the same color is closed among the colors used for the object can be determined, the size of the button represented by the object can be estimated from the determination result. Similarly, when the information on the height of the button indicated by the object is also included in the attribute information, the height information is also acquired. For example, depending on the operating system (OS), the height of the button represented by the object may be standardized by 3D control or the like. In such a case, information can be obtained directly from the OS.

  For example, in a style sheet such as HTML / CSS, buttons and characters can be shaded with a Shadow tag. In such a case, the height of the displayed button is estimated from the width of the three-dimensional effect portion of the image by using the shadow width of the shadow effect, etc., by acquiring the attribute of the shadowed portion. You can also

  The above analysis itself can be performed using various existing analysis techniques. In this way, the UI analysis unit 16 analyzes the attribute information of each object constituting the UI and notifies the tactile sensation control unit 14 of the analysis result. The tactile sensation control unit 14 that has received the analysis result of the attribute information of the object constituting the UI sets a tactile sensation suitable for the object based on the analysis result. In this way, a tactile sensation suitable for each object can be presented.

  When setting the tactile sensation in this way, depending on the object, the type of tactile sensation (for example, the tactile sensation of a metal dome switch or rubber switch), the tactile sensation (for example, the strength of feedback), or the load standard for presenting the tactile sensation Various settings can be made. The load standard for presenting a tactile sensation is the predetermined load standard for presenting a tactile sensation to the pressed object when the load detection unit 60 detects a pressing load that satisfies the predetermined load standard.

  FIG. 3 is a diagram illustrating a specific example of the tactile sensation set by the tactile sensation control unit 14 when the UI analysis unit 16 analyzes the information constituting the UI.

  As shown in FIG. 3A, when the UI analysis unit 16 determines that the object is a small object, a relatively small (weak feedback) tactile sensation can be set. Alternatively, in this case, the frequency of the vibration to be generated can be increased, or the time for generating the vibration can be shortened. On the other hand, as shown in FIG. 3B, when the analysis result of the UI analysis unit 16 is found to be a large object, a relatively large (strong feedback) tactile sensation can be set. Alternatively, in this case, the frequency of vibration to be generated can be lowered, or the time for generating vibration can be lengthened. Further, in the case of a small object as shown in FIG. 3A, the load standard for presenting the tactile sensation can be set to be relatively small so that the tactile sensation is presented by a relatively weak press by the operator. In the case of a large object as shown in FIG. 3B, the load standard for presenting the tactile sensation can be set to be relatively large so that the tactile sensation is not presented unless the operator presses a relatively strong pressure.

  As shown in FIG. 3C, when it is determined that the object is a low (thin) object as an analysis result of the UI analysis unit 16, a relatively small (weak feedback) tactile sensation may be set. it can. Alternatively, in this case, the frequency of the vibration to be generated can be increased, or the time for generating the vibration can be shortened. On the other hand, as shown in FIG. 3D, when it is determined that the object is a high (thick) object as an analysis result of the UI analysis unit 16, a relatively large (strong feedback) tactile sensation is set. be able to. Alternatively, in this case, the frequency of vibration to be generated can be lowered, or the time for generating vibration can be lengthened. Further, in the case of a thin object as shown in FIG. 3C, the load standard for presenting the tactile sensation can be set relatively small so that the tactile sensation is presented by a relatively weak press by the operator. Further, in the case of a thick object as shown in FIG. 3D, it is possible to set a relatively large load standard for presenting tactile sensation so that the tactile sensation is not presented unless the operator has a relatively strong pressure. The tactile sensation setting described above is merely a description of some examples of assumed tactile sensation settings, and various other tactile sensations and load standards that present tactile sensations such as tactile sensation modes can be set.

  Next, the operation of the tactile sensation providing apparatus 1 will be described. Hereinafter, an operation when the tactile sensation providing apparatus 1 displays a UI on the display unit 30 and then receives an operator input and presents a tactile sensation will be described.

  The tactile sensation providing apparatus 1 according to the present embodiment analyzes information constituting the UI when the touch sensor 40 detects an operator's touch input on the object displayed on the display unit 30. In addition, the tactile sensation providing apparatus 1 presents a tactile sensation when the pressing load detected by the load detection unit 60 satisfies a predetermined standard in a state where an operator's touch input to the object is detected by the touch sensor 40. The tactile sensation is presented when the pressing load detected by the load detection unit 60 satisfies a predetermined standard. In such a case, the operator not only touches the touch sensor but also intentionally presses (pushes). ) Based on what can be considered to have performed the action.

  FIG. 4 is a flowchart showing the operation of the tactile sensation providing apparatus 1 according to the present embodiment. In performing the operation of the tactile sensation providing apparatus 1, it is assumed that a UI composed of objects such as keys is image-displayed on the display unit 30 in advance in order to receive an input with an operator's finger or stylus. . At this time, the application execution unit 20 supplies display data of each object constituting the GUI to the display control unit 12. Then, the display control unit 12 controls the display unit 30 to display (draw) an image of each object based on the display data of the object supplied from the application execution unit 20. As described above, hereinafter, it is assumed that an application executed by the UI in the tactile sensation providing apparatus 1 does not support tactile sensation presentation. It is assumed that this application displays the object image with some change when a touch input is made to the touch sensor 40 corresponding to the position of the object image displayed on the display unit 30. However, in the present embodiment, such an image change is not essential.

  The operation of the tactile sensation providing apparatus 1 according to the present embodiment starts with the above-described preparations completed. When the operation of the tactile sensation providing apparatus 1 starts, first, the UI analysis unit 16 monitors whether or not there is a touch input on the touch surface of the touch sensor 40 with an operator's finger or the like (step S11). When a touch input to the touch surface of the touch sensor 40 is detected in step S11, the UI analysis unit 16 notifies the application execution unit 20 of the position of the touch input (step S12).

  When the position of the touch input in the touch sensor 40 is notified, the application execution unit 20 determines whether or not the position of the input corresponds to a position where an image of an object such as a key is displayed in the current GUI ( Step S13). When the input position corresponds to the position where the image of the object is displayed, the information constituting the UI is analyzed (step S14).

  When the input position corresponds to the position where the object image is displayed, the application execution unit 20 supplies display data for changing the object image according to the setting based on the application to the display control unit 12. . When receiving this display data, the display control unit 12 controls the display unit 30 to change the image of the displayed object (step S15). That is, when there is a touch input to an object such as a key, the display image of the object is changed. When changing the display image of an object, for example, the operator makes a display as if the object is selected, or changes the color or brightness of the object, so that the operator touches (touches) a key or the like. It is preferable to make it visible. There are various modes of display for changing the image of such an object depending on the setting based on the application.

  If the display of the object is changed in step S15 (that is, if there is a touch input to the object such as a key), the operation proceeds to step S16. In step S <b> 16, the tactile sensation control unit 14 sets a tactile sensation corresponding to the input input object based on the result of the analysis performed by the UI analysis unit 16 based on the information constituting the UI. When the tactile sensation corresponding to the object receiving the input is set, the tactile sensation control unit 14 presents the tactile sensation while the pressing load detected by the load detection unit 60 is increased by the pressure on the touch surface of the touch sensor 40. It is determined whether or not a predetermined standard is satisfied (step S17). For example, the load detection unit 60 detects the load from the average value of the outputs of the four strain gauge sensors 51. Here, the load satisfying the predetermined standard is set in advance to a value such as 1.5 N (Newton) based on the pressing load when the operator performs a normal pressing operation, and the setting can be changed thereafter. It is preferable to do this. In addition, the predetermined standard is set so as not to set an excessively low standard in consideration of a pressing load at the time of pressing input based on the operator's intention (for example, an average value). This is to prevent an operation when the operator touches lightly unintentionally as a push (push) input, and to give the operator a pressure sensation for realistic tactile sensation described later. It is.

  When the pressing load satisfies a predetermined standard in step S17, the tactile sensation control unit 14 generates a predetermined vibration on the touch surface of the touch sensor 40 and controls the tactile sensation providing unit 50 so as to present the tactile sensation (step). S18). That is, the tactile sensation control unit 14 presents a tactile sensation when the load detection unit 60 detects a pressing load that satisfies a load criterion for presenting a tactile sensation in a state where an input to an object that executes a predetermined process is detected. The tactile sensation providing unit 50 is controlled. By this control, the tactile sensation providing unit 50 presents a tactile sensation to the pressing target pressing the touch surface of the touch sensor 40. The tactile sensation presented here is a tactile sensation set for each object based on the analysis of the information constituting the UI described above.

  In the present embodiment, the tactile sensation presented to the pressed object by the tactile sensation providing unit 50 in step S18 can be the click sensation described above. In order to present a realistic click tactile sensation to the operator, the tactile sensation providing apparatus 1 stimulates the tactile sense in a state where the pressure sense of the operator is stimulated. That is, the tactile sensation providing apparatus 1 stimulates pressure sense until the load applied to the touch sensor 40 satisfies a reference (eg, 1.5N) for presenting the tactile sensation, and when the load satisfies the reference, the piezoelectric vibrator 52 is driven by a predetermined drive signal to vibrate the touch surface 40a to stimulate the sense of touch. Thereby, the tactile sensation providing apparatus 1 can present to the operator a click tactile sensation similar to that obtained when a button switch such as a push button switch (push button switch) is pressed. Therefore, even if the operator is a virtual push button switch drawn on the touch sensor, the operator can perform an input operation on the touch sensor 40 while obtaining a realistic click feeling similar to that when the actual push button switch is operated. So you do n’t feel uncomfortable. Further, since the input operation can be performed in conjunction with the consciousness of “pressing” the touch sensor 40, an input error due to a simple touch can be prevented.

  The drive signal when presenting the click sensation described above, that is, the constant frequency, period (wavelength), waveform, and amplitude for stimulating the sense of touch can be set as appropriate according to the click sensation presented. For example, when a tactile sensation represented by a metal dome switch used in a portable terminal is presented, for example, the tactile sensation providing unit 50 is driven by a drive signal for one cycle including a sine wave having a constant frequency of 170 Hz. The tactile sensation providing unit 50 is driven by such a drive signal, and the touch surface 40a is vibrated by about 15 μm in a state where a reference pressing load is applied. Thereby, it is possible to present the operator with a realistic click feeling as if the user had clicked on an actual key.

  Further, when the pressing load does not satisfy the predetermined standard in step S <b> 17, the tactile sensation control unit 14 reduces the pressing load detected by the load detecting unit 60 to zero due to the weak pressing of the touch sensor 40 against the touch surface. It is determined whether or not (step S19). If the pressure load detected by the load detection unit 60 is not yet zero in step S19, the touch input to the touch sensor 40 by the operator has not been released (released), and the process returns to step S17. On the other hand, when the pressing load detected by the load detection unit 60 in step S19 becomes zero, it can be determined that the operator has released (released) the touch input to the touch sensor 40, and thus the operation according to this embodiment is performed. finish. Note that the operation in step S19 is an operation for determining whether or not an input to the touch panel 40 is still detected, instead of an operation for determining whether or not the pressing load detected by the load detection unit 60 has become zero. You can also

  On the other hand, if the input position does not correspond to the position where the image of the object is displayed in step S13, the application execution unit 20 ends the process. That is, when there is no touch input for an object such as a key, or when the touch input is not for an object such as a key, the application execution unit 20 ends the process without changing the display image of the object. Furthermore, in this case, the UI analysis unit 16 ends the process without giving an instruction to the tactile sensation control unit 14 to present the tactile sensation. As described above, the tactile sensation providing apparatus 1 can make the operator recognize that the pressing input to the object is not accepted in the case of an input to a position other than the object such as a key by not presenting the tactile sensation.

  As described above, according to the present embodiment, when there is an input to an object that executes a predetermined motion and a pressing load that satisfies a predetermined load standard is detected, analysis of information constituting the UI is performed. Present the tactile sensation set based on. Therefore, even when the application to be activated does not correspond to the operation related to the presentation of tactile sensation, the tactile sensation providing apparatus 1 analyzes the information constituting the UI, so that the tactile sensation for each object is based on the press input to each object. Can be presented appropriately. Therefore, according to the tactile sensation providing apparatus 1, a tactile sensation commensurate with the object can be presented. Also, existing application resources that are executed using the UI can be diverted. In addition, the tactile sensation providing apparatus 1 according to the present embodiment allows the operator to recognize that an input to the object is accepted by an input of pressing (pushing) the object such as a key by a real click tactile sense. Can do.

  Next, after the tactile sensation providing apparatus 1 receives an input from the operator and presents a click tactile sense, an operation when presenting a release tactile sense will be described.

  When a press input is made to a normal mechanical push button switch, the tactile sensation (hereinafter referred to as release tactile sense) returns to the state in which the metal dome switch or the like is not depressed, as the pressing force on the switch subsequently weakens. ) Is presented. Since the tactile sensation providing apparatus 1 according to the present embodiment can present a real click tactile sensation in response to a press input, the real touch is being released (released) during the release of the pressed touch sensor. Presenting a release tactile sensation can provide a natural feeling of operation.

  FIG. 5 is a flowchart showing the operation of the tactile sensation providing apparatus 1 according to the present embodiment. The operation shown in FIG. 5 starts after the operation (step S18) for presenting the click feeling described in FIG. When this operation starts, the tactile sensation control unit 14 changes the pressing load detected by the load detection unit 60 from a predetermined reference value or more to less than the predetermined reference value due to a weakening of the input pressing force applied to the touch panel 40 by the operator. It is determined whether or not (step S31). Here, “change from a predetermined reference or more to less than the predetermined reference” means that the predetermined load is released from a state where the pressing load exceeds the predetermined reference (or more than the predetermined reference) at the time of release. It means that the state becomes below (or less than a predetermined standard).

  When it is determined in step S31 that the pressing load has changed from a predetermined reference value to less than the predetermined reference value, the application execution unit 20 performs display control on display data for changing the image of the object in accordance with the setting based on the application. To the unit 12. When receiving the display data, the display control unit 12 controls the display unit 30 to change the image of the displayed object (step S32). That is, the display control unit 12 changes the display image of the object in the process of releasing the input from the object such as a key. When changing the display image of an object, for example, a display that returns before an already selected object is selected, or the color or brightness of the object can be changed. It is preferable that the operator can visually recognize that the touch on the key or the like has been released by such a display change.

  If the display of the object is changed in step S52, the tactile sensation control unit 14 then controls the tactile sensation providing unit 50 to present the release tactile sensation set based on the analysis of information constituting the UI (step S33). ). Note that it is preferable to set the release tactile sensation presented here as well as the click tactile sensation set in step S16 based on the analysis of the information constituting the UI in step S14 of FIG.

  Here, the load standard for presenting the release tactile sensation can be set the same as the above-described load standard for presenting the click tactile sensation at the time of pressing, but preferably 50% to the load standard for presenting the click tactile sensation at the time of pressing. Set to 80% lower value. In this way, when the same position (object) is continuously input (sequentially hit), the sequential input matches the tactile sensation presentation timing, and a realistic click tactile sensation with no sense of incongruity can be presented. In other words, by making the standard load that presents a tactile sensation at the time of release smaller than the standard load that presents a tactile sensation at the time of pressing, the reference load that presents a tactile sensation at the time of release is pressed. Occasionally, the operability during continuous input can be significantly improved by setting the load to be approximately 50% or more of the reference load that sometimes presents a tactile sensation. In addition, by making the reference load that presents a tactile sensation at the time of release approximately 80% or less of the reference load that presents a tactile sensation when pressed, it is possible to cope with minute load changes in the hold state during continuous input.

  In this way, it is possible to present a more realistic click tactile sensation that can be felt as “cut” when pressed and “click” when released. Of course, the release tactile sensation may use the same drive signal as the click tactile sensation, but in this embodiment, for example, as described with reference to FIG.

  As described above, the tactile sensation providing apparatus 1 according to the present embodiment can set the tactile sensation for each object for each object based on the analysis of information constituting the UI. Therefore, the tactile sensation providing apparatus 1 can divert existing application resources and present an appropriate tactile sensation for each object. Further, the tactile sensation providing apparatus 1 according to the present embodiment can present various tactile sensations such as a click tactile sensation and a release tactile sensation in accordance with an input to the object. For this reason, the tactile sensation providing apparatus 1 can utilize existing application resources without wasting them, and can provide a realistic operational feeling that can be obtained when operating an actual push button switch. it can.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the second embodiment, the aspect of analysis of information constituting the UI in step S14 (FIG. 4) is changed in the first embodiment described above.

  In the first embodiment described above, the UI analysis unit 16 analyzed information obtained by extracting information other than the image of the object displayed on the display unit 30 among the information configuring the UI. However, the information constituting the UI supplied from the application execution unit 20 does not always include various types of information such as the size and height of the object depending on the application. Therefore, in consideration of such a case, in the second embodiment, the image of the object displayed on the display unit 30 among the information constituting the UI is analyzed.

  In step S14, the UI analysis unit 16 analyzes the image of the object, so that the size of the object can be determined first. For example, as shown in FIG. 6A, an area in which the same color as the color near the position where the touch of the operator's finger is detected on the touch sensor 40 is determined as an object such as one button. can do. In addition, for example, in the case of an image display in which an object imitates a button and has a shadow, the height (width) of an area followed by a dark color (shadow) compared to the color of the area near the center of the object ) To estimate the height of the object. For example, as shown in FIG. 6B, the height of the object can be estimated from the width (x) or (y), which is a shadow portion compared to the color of the object central region.

  Further, for example, as shown in FIG. 7, when the unevenness is expressed by a color tone change like a gradation as a whole, the object image expresses the unevenness expressed by the object by detecting the change in the tone of the gradation. The condition, that is, the height can be estimated.

  For example, the object shown in FIG. 7A can be determined to represent an object having a relatively low height because the change in color tone of light and dark gradations is relatively gentle. In the figure, the change in brightness in the vertical direction of the object is represented by the horizontal axis direction in the adjacent graph, and the vertical axis direction represents the lightness and darkness of the color tone. In the graph, a portion where the straight line is inclined is a portion expressing a shadow by gradation, and a horizontal straight line in the center of the graph is a portion expressing the upper surface (key top) of the key. The height represented by the object can be estimated based on the difference in color tone before and after such gradation. On the other hand, the object shown in FIG. 7B can be determined to represent a relatively high object because the change in color tone of light and dark gradations is relatively severe. If the information constituting the UI includes the information on the size of the object, but does not include the height information, the size information is used as it is, and the height information is used in the present embodiment. It is also possible to use an appropriate combination of analysis methods such as determination by image analysis.

  Depending on the object, there may be a case where a highlight indicating gloss or gloss is displayed on a part of the image, for example, near the upper end of the object shown in FIG. In such a case, if the image analysis is performed as it is like the graph shown next to the object shown in FIG. 7C, the height of the object may not be correctly estimated. In the graph, the part α is a highlighted part representing the gloss of the object. Therefore, it is preferable that such a rapid change in the minute section is not considered in the estimation of the height of the object by processing it as noise. That is, for example, by applying a low-pass filter, the height of the object is estimated after removing a portion such as α shown in FIG. 7C.

  The image analysis described above can be performed using various existing image analysis techniques. Based on the analysis result, the tactile sensation control unit 14 sets a tactile sensation suitable for each object and a load standard that presents the tactile sensation, and can be implemented in the same manner as in the first embodiment. When setting the tactile sensation for each object and the load standard for presenting the tactile sensation, the object size information and height information obtained by the above-described image analysis can be used according to the same conditions as in the first embodiment. it can. Moreover, in this Embodiment, you may set the load reference | standard which shows a tactile sensation and a tactile sensation according to the original conditions based on image analysis. For example, the overall color of an object is discriminated by image analysis of the object, and a “feeling heavy color” object that has a profound feeling is set so that a strong tactile sensation is presented or a tactile sensation is presented. The load standard can be set high. Further, for example, a weak tactile sensation can be set for an object of “color that looks light”, or a load standard for presenting a tactile sensation can be set low.

  The UI analysis unit 16 is provided with an algorithm for performing image analysis so that an appropriate determination can be made in accordance with a predetermined condition so that the image analysis based on the setting as described above can be performed.

  Thus, according to the present embodiment, it is possible to set so that an appropriate tactile sensation corresponding to the appearance of an object such as a button is presented.

  In addition, this invention is not limited only to the said embodiment, Many deformation | transformation or a change is possible. For example, when analyzing the information constituting the UI in the second embodiment described above, the tactile sensation setting and the tactile sensation are compared by comparing the image before the object is clicked with the image after the object is clicked. You may set the load standard to present. By comparing the images before and after the object is clicked in this way, it is possible to estimate the stroke depth when the button or the like represented by the object is pressed.

  Further, in each of the above-described embodiments, the display is changed to the display of the image in which the object is clicked in step S15 in FIG. 4, but this process may be performed before or after step S18. In this way, the tactile sensation presented by the press input and the display of the object that has received the press input are synchronized, and the UI also looks natural.

  Further, in each of the above-described embodiments, information constituting the UI is started after an operator's pressing input to the object is detected, and an appropriate tactile sensation is set. However, when the UI is first generated and each UI object is displayed, it is also possible to analyze the UI information and set the tactile sensation in advance. In this case, the tactile sensation control unit 14 must temporarily store the set tactile sensation. However, as compared with the case where the tactile sensation is set every time, the UI analysis unit 16 and the tactile sensation control unit 14 There is an advantage that the processing load is reduced. Therefore, even if the processing speed of the processing device mounted on the tactile sensation providing apparatus 1 is not high, there is no possibility that the presentation of the tactile sensation will be delayed because the processing is not in time.

  For example, in each of the above-described embodiments, the configuration in which the control unit 10 includes the display control unit 12, the tactile sensation control unit 14, and the UI analysis unit 16 has been described. However, the tactile sensation providing apparatus according to the present invention is not limited to such a configuration, and various configurations can be adopted according to the requirements in designing. For example, by causing one function unit to function as another function unit, the other function unit can be omitted, or one function unit can be subdivided into a plurality of function units.

  Furthermore, in each embodiment mentioned above, a load detection part can be comprised according to the input detection system in a touch sensor. For example, in the case of the resistance film method, if a load can be detected from a change in an output signal based on a change in resistance due to a contact area, it can be configured without using a strain gauge sensor. Alternatively, in the case of the electrostatic capacity method, even when the load can be detected from the change in the output signal based on the change in the electrostatic capacity, it can be configured without using the strain gauge sensor.

  The tactile sensation providing unit can be configured using an arbitrary number of piezoelectric vibrators, or can be configured by providing a transparent piezoelectric element over the entire surface of the touch sensor, or can express vibration that presents tactile sensation. Can be configured to rotate once in one cycle of the drive signal. Furthermore, when the load detection unit and the tactile sensation providing unit are configured using a piezoelectric element, the load detection unit and the tactile sensation providing unit can be configured by sharing the piezoelectric element.

  In each of the above-described embodiments, the description has been made assuming the configuration in which the touch sensor is disposed so as to overlap the upper surface of the display unit. However, the tactile sensation providing apparatus according to the present invention does not necessarily have such a configuration, and the touch sensor and the display unit may be separated from each other.

  The present invention can also be effectively applied to an input device that functions as a touch switch in which a touch sensor performs an on / off operation.

  The tactile sensation providing apparatus according to the present invention drives the piezoelectric element when the pressing load detected based on the output of the piezoelectric element satisfies a load standard for presenting a tactile sensation. Here, the case where the pressing load detected based on the output of the piezoelectric element satisfies the load standard for presenting a tactile sensation may be when the detected pressing load reaches a reference value for presenting a tactile sensation. Then, it may be when the detected pressing load exceeds a reference value for presenting tactile sensation, or when a reference value for presenting tactile sensation is detected based on the output of the piezoelectric element.

DESCRIPTION OF SYMBOLS 1 Tactile sensation presentation apparatus 10 Control part 12 Display control part 14 Tactile sense control part 16 UI analysis part 20 Application execution part 30 Display part 40 Touch sensor 40a Touch surface 50 Tactile sensation presentation part 52 Piezoelectric vibrator 60 Load detection part 61 Case 62 Insulator 63 Upper cover 64 insulator

Claims (4)

A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A display unit;
An analysis unit for analyzing information on the height of an object to be displayed on the display unit , or information on a shaded part;
Based on the result of analyzing the information on the height of the object or the information on the shaded part , the touch surface presented to the object by the tactile sensation providing unit is set, and when the input to the object is detected, the touch surface A control unit that controls the tactile sensation providing unit so as to present a tactile sensation according to the object with respect to a pressing target that is pressing
A tactile sensation presentation apparatus comprising: A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A display unit;
An analysis unit for analyzing an image displayed on the display unit;
Based on the result of analyzing the image, the tactile sensation presented by the tactile sensation providing unit is set for the object in the image, and when an input to the object is detected, the touch target pressing the touch surface is detected. A control unit that controls the tactile sensation providing unit to present a tactile sensation according to the object;
A tactile sensation presentation apparatus comprising: A load detecting unit for detecting a pressing load on the touch surface of the touch sensor;
When the control unit detects a pressing load that satisfies a load criterion for the tactile sensation in a state of detecting an input to the object, the tactile sensation is presented so as to present the tactile sensation to the pressing target. Control the presentation part,
The tactile sensation providing apparatus according to claim 1.   The tactile sensation providing apparatus according to claim 3, wherein the control unit changes a load standard for presenting the tactile sensation to the object based on the analysis result.

Images