JP2012113644A - Tactile sense presentation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for presenting a tactile sense of each layer by layer-displaying images on a display unit and detecting a touch corresponding to an image of each layer by a touch sensor.SOLUTION: A tactile sense presentation device 1 comprises: a touch sensor 40; a tactile sense presentation unit 50 for vibrating a touch surface of the touch sensor 40; a display unit 30; and a control unit 10 for controlling the display unit 30 to display images with plural layers. The control unit 10 further controls a tactile sense to be presented by the tactile sense presentation unit 50 to be set for each layer, and when the touch sensor 40 detects a touch at a position corresponding to an image displayed on a layer with a tactile sense set therefor on the display unit 30, controls an operation allocated to the image to be performed and controls the tactile sense presentation unit 50 to present a tactile sense set for the layer.

Description

  The present invention relates to an apparatus for presenting a tactile sensation. More particularly, the present invention relates to an apparatus that presents a tactile sensation based on a touch operation on a touch sensor.

  2. Description of the Related Art In recent years, in mobile terminals such as mobile phones, some input devices including touch sensors such as touch panels and touch switches have been adopted as input devices such as operation units and switches that detect user operations. In addition to portable terminals, input devices equipped with touch sensors are widely used in information devices such as calculators and ticket machines, household appliances such as microwave ovens, televisions, and lighting equipment, and industrial equipment (FA devices). .

  As such a touch sensor, various systems such as a resistive film system, a capacitance system, and an optical system are known. However, any type of touch sensor detects contact with a finger or a stylus pen, and the touch sensor itself is not physically displaced like a push button switch even when touched.

  Therefore, even if the operation by contact is recognized by the input device, the operator cannot obtain feedback for the operation. For this reason, the operator who operates with respect to a touch sensor cannot obtain the operation feeling obtained when pressing a key or a button constituted by a mechanical push button switch. As a result, there is a problem that the operator has no way of knowing whether or not his / her contact operation has been properly recognized by the input device unless the input device starts a predetermined operation based on contact with the touch sensor. .

  Therefore, conventionally, a feedback method has been proposed in which when the touch sensor detects contact regardless of hearing or vision, the touch sensor is vibrated to generate vibration at the fingertip of the operator (for example, Patent Documents). 1 and Patent Document 2).

JP 2003-288158 A JP 2008-130055 A

  According to the techniques disclosed in Patent Literature 1 and Patent Literature 2, when an operation on the touch sensor is detected, vibration can be generated at the fingertip of the operator. Therefore, an operator who performs an operation using such an input device can recognize by touch that the operation performed on the touch sensor is correctly recognized by the input device.

  By the way, an input device provided with a touch panel displays an object such as a key or a button as an image on a display unit (for example, LCD) arranged on the back surface of the touch sensor, so that an operator should touch the touch sensor. There are many configurations that suggest a position. Recently, some control units (for example, LCD controllers) that perform control such that such a display unit displays an image of an object or the like can perform layer display.

  The layer display is to synthesize and display an image as a superposition of a plurality of virtual layers when displaying the image on the display unit. When an image that is not a transparent color is superimposed and displayed by a layer, only the image of the highest layer is displayed on the display unit, and the superimposed portion of the images of the lower layer is not displayed. On the other hand, when a transparent color image is superimposed and displayed by a layer, an image of a lower layer is also displayed through the uppermost layer in the display unit.

  In the input device configured to display objects in layers on the display unit arranged on the back surface of the touch sensor in this way, there are several problems that are expected to be caused by the occurrence of vibration described above.

  First, in layer display, a plurality of hierarchical structures are only virtual, and only one image is displayed on one display unit as an actual display unit display. For this reason, even if the touch sensor tries to present a tactile sensation by detecting a contact at a position corresponding to the object area displayed on the display unit, the input device determines which layer the input corresponds to the object. I can't figure it out.

  In general input devices, particularly when an object is displayed with an image that is not a transparent color, the touch sensor detects contact at a position corresponding to the object area displayed on the top layer among the plurality of layers. Sometimes it is enough. However, when the touch is detected and the tactile sensation is presented in consideration of only the object displayed on the top layer in this way, the element that executes the application program is always displayed on the top layer in the display unit. The object needs to be recognized.

  In addition, when considering only the object displayed on the top layer in this way, if multiple objects are superimposed on the same position on multiple layers, the object displayed on a layer other than the top layer is displayed. It is not possible to detect a contact with respect to. In particular, when a touch panel is employed in a small terminal such as a cellular phone, for example, the display unit and the touch sensor often have to be small. Accordingly, as described above, if a plurality of objects can be superimposed and displayed on a plurality of layers with a transparent color on the display unit and contact with an object of an arbitrary layer can be detected, the touch panel can be effectively used even if the area is small. it can.

  An object of the present invention made in view of such circumstances is to provide an apparatus that presents a tactile sensation of each layer by displaying an image in a layer on a display unit and detecting a touch corresponding to the image of each layer by a touch sensor. There is to do.

The invention of the tactile sensation providing apparatus according to the first aspect to achieve the above object is as follows:
A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A display for displaying an image;
A control unit that controls the display unit to display an image in a plurality of layers;
A tactile sensation presentation device comprising:
The control unit further controls to set the tactile sensation presented by the tactile sensation providing unit for each layer, and the touch sensor is located at a position corresponding to an image displayed on the layer where the tactile sensation is set in the display unit. When contact is detected, control is performed so as to perform an operation assigned to the image, and the tactile sensation providing unit is controlled so as to present the tactile sensation set in the layer.

The invention according to the second aspect is the tactile sensation providing apparatus according to the first aspect,
The control unit is configured to set validity or invalidity of presentation of tactile sensation set in another layer other than the one layer in one of the plurality of layers.

The invention according to a third aspect is the tactile sensation providing apparatus according to the first or second aspect,
The control unit determines a layer on which an image corresponding to a position where the touch is detected is displayed according to a touch mode detected by the touch sensor.

Moreover, the invention of the tactile sensation providing apparatus according to the fourth aspect of achieving the above object is as follows:
A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A load detection unit for detecting a pressing load on the touch surface;
A display for displaying an image;
A control unit that controls the display unit to display an image in a plurality of layers;
A tactile sensation presentation device comprising:
The control unit performs control so that a pressing load serving as a reference for the tactile sensation providing unit to provide a tactile sensation is set for each layer, and the touch sensor is displayed on the layer where the tactile sensation is set in the display unit. While detecting a contact at a position corresponding to the image, if the load detection unit detects a pressing load that satisfies the criteria set for the layer, the control is performed so as to perform the operation assigned to the image. The tactile sensation providing unit is controlled to present the tactile sensation set for the layer.

The invention according to a fifth aspect is the tactile sensation providing apparatus according to the fourth aspect,
The control unit is configured to set validity or invalidity of presentation of tactile sensation set in another layer other than the one layer in one of the plurality of layers.

The invention according to a sixth aspect is the tactile sensation providing apparatus according to any one of the first to fifth aspects.
The said control part is controlled to change the said setting of the tactile sensation set for every layer based on generation | occurrence | production of a predetermined event.

  According to the present invention, the image is displayed on the display unit in a plurality of layers, and the tactile sensation presented by the tactile sensation presentation unit is set for each layer, so that the touch sensor detects a contact corresponding to the image of each layer. The tactile sensation set for each layer can be presented.

It is a block diagram which shows schematic structure of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the mounting structure of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure explaining the structure of the display data which added the tactile sense control information which concerns on embodiment of this invention. It is a figure explaining the example of operation | movement of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure explaining the example of operation | movement of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure explaining the example of operation | movement of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure explaining the other example of operation | movement of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure explaining the other example of operation | movement of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a figure explaining the other example of operation | movement of the tactile sense presentation apparatus which concerns on embodiment of this invention. It is a block diagram which shows schematic structure of the tactile sensation presentation apparatus which concerns on the modification of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of a tactile sensation providing apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, the tactile sensation providing apparatus 1 includes a display / tactile sensation control unit 10 and an application execution unit 20. The tactile sensation providing apparatus 1 includes a display unit 30, a touch sensor 40, a tactile sensation providing unit 50, a load detection unit 60, and a storage unit 80. The display / tactile sensation control unit 10 and the application execution unit 20 are included in the control unit 70.

  The storage unit 80 can be configured by, for example, a NAND flash memory. The memory area of the storage unit 80 includes an area for storing various application software and an area for storing display resources such as objects and image files. As will be described later, tactile sensation control information is added to the display resource image data supplied from the storage unit 80.

  The application execution unit 20 reads a specified one of various applications from the storage unit 80, executes processing based on the application, and controls each functional unit based on the application. The application execution unit 20 has a RAM that temporarily stores an application read from the storage unit 80.

  The display unit 30 displays a background image or the like, or displays an object such as a button or a key (hereinafter simply referred to as “key etc.”). This object is an image that suggests to the operator the area or position to be touched. Such image data is supplied from the display / tactile sensation control unit 10 under the control of the application execution unit 20. The display unit 30 is configured using, for example, a liquid crystal display panel (LCD), an organic EL display panel, or the like.

  The touch sensor 40 is usually disposed on the front surface of the display unit 30, and the touch (or release of contact) of the object displayed on the display unit 30 by the operator's finger or the like (contact target) is detected by the corresponding touch sensor 40. Detect on the touch surface. The touch sensor 40 detects the position of contact with the touch surface and notifies the display / tactile sensation control unit 10 of the detected position of contact. The touch sensor 40 is configured by a known method such as a resistive film method, a capacitance method, and an optical method. Note that it is not essential for the touch target to physically touch the touch sensor 40 when the touch sensor 40 detects a touch by the touch target. For example, when the touch sensor 40 is an optical type, the touch sensor 40 detects a position where the infrared light on the touch sensor 40 is blocked by the touch target, and thus it is not necessary for the touch target to touch the touch sensor.

  The tactile sensation providing unit 50 is configured using, for example, a piezoelectric vibrator or the like, and generates a vibration according to a predetermined vibration pattern on the touch surface of the touch sensor 40. The tactile sensation providing unit 50 presents a tactile sensation with respect to a contact target in contact with the touch surface by generating vibration on the touch surface of the touch sensor 40.

  The load detection unit 60 detects a pressing load on the touch surface of the touch sensor 40, and is configured using an element that reacts linearly with a load such as a strain gauge sensor or a piezoelectric element, for example. Further, the load detection unit 60 notifies the display / tactile sensation control unit 10 of the detected pressing load on the touch surface of the touch sensor 40.

  The display / tactile sensation control unit 10 displays a background image, an image of an object, or the like on the display unit 30 based on a display instruction from the application execution unit 20 or changes the display of the image. Control relating to display at 30 is performed. When the application execution unit 20 issues a display instruction to the display / tactile sensation control unit 10, image data stored in the storage unit 80 can be read as display data of an image to be displayed on the display unit 30. it can. The display / tactile sensation control unit 10 receives the supply of the image data read from the storage unit 80 in this way, and controls the display unit 30 to draw a predetermined image at a predetermined position. At this time, the display / tactile sensation control unit 10 controls the display unit 30 to display an image in a plurality of layers.

  Further, as will be described later, the display / tactile sensation control unit 10 includes tactile sensation control information added to the image data designated by the application execution unit 20, the display position of the display resource and the display layer information, and the like. Based on the tactile setting. In the present embodiment, the tactile sensation to be presented is set for each layer of the image displayed on the display unit 30. Therefore, the display / tactile sensation control unit 10 performs control so that the tactile sensation presented by the tactile sensation presentation unit 50 is set for each layer.

  Further, the display / tactile sensation control unit 10 controls the tactile sensation providing unit 50 to generate a predetermined pattern of vibration according to a predetermined condition based on the above-described setting of the tactile sensation. For example, when the touch sensor 40 detects contact at a position corresponding to a region of an image such as a predetermined object displayed on the display unit 30, the display / tactile sensation control unit 10 performs a predetermined operation assigned to the image. In addition, the tactile sensation providing unit 50 is controlled so as to present the tactile sensation set in the image with respect to the contact target in contact with the touch surface of the touch sensor 40. In particular, in the present embodiment, the display / tactile sensation control unit 10 assigns to the image when the touch sensor 40 detects contact at a position corresponding to the image displayed on the layer for which the tactile sensation is set on the display unit 30. The tactile sensation providing unit 50 is controlled so as to present the tactile sensation set in the layer.

  The tactile sensation providing apparatus 1 according to the present embodiment further distinguishes which layer the operator is making contact with according to the pressing force when the operator presses the touch sensor 40. Therefore, the display / tactile sensation control unit 10 determines whether or not the pressing load detected by the load detection unit 60 satisfies a predetermined criterion for presenting a tactile sensation. In this case, the display / tactile sensation control unit 10 causes the tactile sensation so as to generate a predetermined pattern of vibration based on a predetermined condition when the load detection unit 60 detects a pressing load that satisfies the predetermined criterion for presenting the above-mentioned tactile sensation. The presenter 50 is controlled.

  For example, the display / tactile sensation control unit 10 performs control so that a pressing load that is a reference for the tactile sensation providing unit 50 to provide a tactile sensation is set for each layer. Then, when the touch sensor 40 detects a contact at a position corresponding to the image displayed on the layer for which the tactile sensation is set on the display unit 30, the display / tactile sensation control unit 10 When a pressing load that satisfies the criteria set in (2) is detected, control is performed so as to perform the operation assigned to the image, and the tactile sensation providing unit 50 is controlled so as to present the tactile sensation set in the layer. At this time, the display / tactile sensation control unit 10 controls the tactile sensation providing unit 50 so as to present a tactile sensation, and also displays information such as position information detected by the touch sensor 40 and load detected by the load detection unit. The application execution unit 20 is notified at an appropriate timing.

  As described above, in the present embodiment, the tactile sensation to be presented is set for each layer of the image displayed on the display unit 30. Therefore, in this case, the display / tactile sensation control unit 10 performs control so that a pressing load that is a reference for the tactile sensation providing unit 50 to provide a tactile sensation is set for each layer. For example, a plurality of pressing load standards can be set as the load stage, and the setting corresponding to the contact with the lower layer can be made as the reference load increases. In addition, when the touch sensor 40 detects a contact at a position corresponding to an image displayed on the layer for which the tactile sensation is set on the display unit 30, the display / tactile sensation control unit 10 When the pressing load satisfying the standard set in the above is detected, the tactile sensation providing unit 50 is controlled so as to present the tactile sensation set in the layer.

  Further, in the present embodiment, depending on the contact mode when the operator performs a contact operation on the touch sensor 40, it is possible to distinguish which layer the operator is making contact with, So that different contacts can be distinguished. Therefore, the display / tactile sensation control unit 10 determines a layer on which an image corresponding to the position where the touch is detected is displayed according to the touch mode detected by the touch sensor 40.

For example, the display / tactile sensation control unit 10 recognizes contact with an object displayed on different layers when the touch sensor 40 detects a single tap contact and when the touch sensor 40 detects a double tap contact.
The display / tactile sensation control unit 10 is different types of contact to the same object displayed on the same layer depending on whether the touch sensor 40 detects a single tap contact or a double tap contact. It can also be recognized.

  Further, for example, the display / tactile sensation control unit 10 displays objects on different layers depending on whether the touch sensor 40 detects a contact by a normal slide operation or a contact by a flick operation faster than the slide operation. Recognize contact with Also in this case, the display / tactile sensation control unit 10 displays on the same layer when the touch sensor 40 detects contact by a normal slide operation and when it detects contact by a flick operation faster than the slide operation. It can also be recognized that they are different types of contact with the same object.

  Therefore, in the present embodiment, the display / tactile sensation control unit 10 constitutes a control unit.

  FIG. 2 shows an example of a mounting structure centering on the display unit 30, the touch sensor 40, the tactile sensation providing unit 50, and the load detecting unit 60 in the tactile sensation providing device 1 shown in FIG. FIG. 2A is a cross-sectional view of the main part, and FIG. 2B is a plan view of the main part. The display unit 30 is housed and held in the housing 71. The touch sensor 40 is held on the display unit 30 via an insulator 72 made of an elastic member. In the tactile sensation providing apparatus 1 according to the present embodiment, the display unit 30 and the touch sensor 40 are rectangular in plan view. However, it can be made into a shape according to various conditions such as the configuration of the touch sensor 40 or the display unit 30 included in the tactile sensation providing apparatus 1. In addition, the tactile sensation providing apparatus 1 holds the touch sensor 40 on the display unit 30 via insulators 72 disposed at the four corners deviated from the display area A of the display unit 30 indicated by virtual lines in FIG. To do.

  Further, the casing 71 is provided with an upper cover 73 so as to cover the surface area of the touch sensor 40 that is out of the display area of the display unit 30, and is made of an elastic member between the upper cover 73 and the touch sensor 40. An insulator 74 is provided. In the touch sensor 40 shown in FIG. 2, the surface member having the touch surface 40a is made of, for example, a transparent film or glass, and the back member is made of glass or acrylic. The touch sensor 40 has a structure in which, when the touch surface 40a is pressed, the pressed portion is slightly bent (distorted) in accordance with the pressing force, or the structure itself is bent slightly.

  On the surface of the touch sensor 40, in the vicinity of each side covered with the upper cover 73, a strain gauge sensor 62 for detecting a load (pressing force) applied to the touch sensor 40 is provided by bonding or the like. Further, on the back surface of the touch sensor 40, piezoelectric vibrators 52 for vibrating the touch sensor 40 are provided by adhesion or the like in the vicinity of two opposing sides. That is, in the tactile sensation providing apparatus 1 shown in FIG. 2, the load detecting unit 60 shown in FIG. 1 is configured using four strain gauge sensors 62, and the tactile sensation providing unit 50 is configured using two piezoelectric vibrators 52. is doing. The touch surface 40a is vibrated by vibrating the touch sensor 40 by the tactile sensation providing unit 50. In FIG. 2B, the casing 71, the upper cover 73, and the insulator 74 shown in FIG. 2A are not shown.

  Next, information relating to tactile sensation control (hereinafter referred to as “tactile sensation control information” as appropriate) added to image data supplied by an application used in the tactile sensation providing apparatus 1 according to the present invention will be described.

  FIG. 3 is a diagram illustrating an example of a configuration in which tactile sensation control information is added to image data of a display resource. In the present embodiment, as described above, an image is displayed on the display unit in a plurality of layers, and the tactile sensation presented in each of the plurality of layers is set. In FIG. 3A, in order to facilitate the explanation, the plurality of layers when displaying an image on the display unit 30 are three layers: an upper layer, a middle layer, and a lower layer. However, display of an image by a plurality of layers according to the present embodiment is not limited to three layers, and an arbitrary plurality of layers can be configured, for example, four or more layers.

As shown in FIG. 3A, each of the upper, middle, and lower layers includes the following information.
(1) Touch control information (2) Image data That is, in each layer, the touch control information of (1) is added to the image data of (2). The image data (2) includes image data displayed on the layer, such as a background image or an object representing a key. The tactile sensation control information (1) includes information for setting the tactile sensation presented in the layer.

  FIG. 3B is a diagram for explaining the details of the content of the tactile sensation control information (1) in each layer. In FIG. 3B, the content of the tactile sensation control information of the middle layer is described as an example, but the information configuration is the same in other layers even though the content of the information is different. As shown in FIG. 3B, the tactile sensation control information of each layer includes the following information.

(1) Tactile sensation activation flag—a flag indicating whether or not tactile sensation is to be presented in the layer (for example, if “ON”, a predetermined tactile sensation is presented in response to contact corresponding to the layer, and “OFF”. If not, no tactile sensation will be presented even if contact corresponding to the layer is detected)

(2) Lower layer control flag − When an image is displayed on a plurality of layers on the display unit 30, a flag (for example, “ON” indicating whether or not contact corresponding to a layer existing below the layer is permitted. ", The contact corresponding to the lower layer is detected, but if it is" OFF ", the contact corresponding to the lower layer is not detected. Can enable / disable the tactile sensation set for other layers other than the layer)

(3) Click tactile sensation threshold—a threshold of a pressing load that is a reference for recognizing the press corresponding to the layer and presenting the click tactile sensation (that is, the load detection unit 60 by the operator's press on the touch sensor 40) When the pressing load detected by the touch reaches the click tactile sensation threshold, the tactile sensation providing apparatus 1 presents the click tactile sensation)

(4) Release tactile sensation threshold—a threshold of a pressing load that is a reference for recognizing that the pressing corresponding to the layer has been released and presenting a releasing tactile sensation (that is, the operator's pressing on the touch sensor 40 is weakened and the load is detected. When the pressing load detected by the unit 60 falls below the release tactile sensation threshold, the tactile sensation providing apparatus 1 presents a release tactile sensation (even if contact is still detected).

(5) Vibration waveform-The vibration waveform data itself used when the tactile sensation providing unit 50 presents the tactile sensation set for the layer, or a vibration pattern prepared in advance (ie, the vibration waveform) Based on the information, the display / tactile sensation control unit 10 can control the tactile sensation providing unit 50 to present various predetermined tactile sensations)

(6) Number of stages of tactile sensation-For example, in the case where the reference of the pressing load is set as a plurality of load stages to detect the pressing step by step, or the contact is detected as a different contact depending on the contact mode, , A number indicating how many steps to distinguish and detect as a different press or touch and present the corresponding tactile sensation

  The “click tactile sensation” is a tactile sensation in which the key is cramped, such as presented when a key or a button configured by a mechanical push button switch is pressed. “Release tactile sensation” means when a key that is composed of a mechanical push button switch or a key that has been cramped, such as that obtained when the pressing force is weakened after the button is pressed, is restored. It is the tactile sensation as presented.

  As described above, the tactile sensation providing apparatus 1 according to the present embodiment adds the tactile sensation control information to each layer, thereby distinguishing and detecting contact or press corresponding to each layer, and varies depending on the detection. A tactile sensation can be presented. Further, as described above, the tactile sensation providing apparatus 1 according to the present embodiment can freely determine whether or not to perform a tactile sensation providing operation in an arbitrary layer by including the tactile sensation activation flag (1) in the tactile sensation control information. Can be set. In addition, the tactile sensation providing apparatus 1 according to the present embodiment can freely set whether or not to perform the tactile sensation providing operation in a lower layer than the layer by including the lower layer control flag (2) in the tactile sensation control information. can do. In other words, in the tactile sensation providing apparatus 1, the display / tactile sensation control unit 10 sets validity or invalidity of tactile sensation set in a lower layer than the one layer in one of the plurality of layers.

  In a general user interface (UI), for example, when an application is started or when an operation based on an application is started based on a key press, the arrangement of objects is changed or the UI configuration is changed. It is often done. Therefore, the display / tactile sensation control unit 10 sets the tactile sensation in each layer when a predetermined event occurs, such as when the application is started or when an operation based on the application is started based on a key press as described above. It is preferable to be at the time of occurrence. That is, in the tactile sensation providing apparatus 1, it is preferable that the display / tactile sensation control unit 10 performs control so that the setting of the tactile sensation set for each layer is changed based on the occurrence of a predetermined event.

  As described above, according to the tactile sensation providing apparatus 1 according to the present embodiment, it is possible to easily perform control related to tactile sensation by adding tactile sensation control information to each layer. Conventionally, even if an object or the like is superimposed on a plurality of layers on the display unit, detection of contact or pressure and presentation of a tactile sensation according to the detection are limited to two-dimensional control. However, according to the tactile sensation providing apparatus 1 of the present embodiment, three-dimensional control can be performed by detecting contact or pressing and presenting a tactile sensation according to the detection in units of layers.

  Next, the operation of the tactile sensation providing apparatus 1 according to the present embodiment will be described using a specific example. FIG. 4 is a diagram illustrating a state in which images are superimposed on the display unit 30 of the tactile sensation providing apparatus 1 using three layers. In the following description, the display of the image on the display unit 30 that is originally superimposed is illustrated separately for each layer for the sake of explanation. In the following description, the contact or press detected by the touch sensor 40 at a position corresponding to the area of the object displayed on the display unit 30 is simply referred to as “contact or press against the object” as appropriate.

  In FIG. 4, the layer shown in FIG. 4A represents the lowest layer, and is simply referred to as “lower layer” below. Further, the layer shown in (B) represents a layer that is immediately above the lower layer described above, and is simply referred to as “middle layer” hereinafter. Furthermore, the layer shown in (C) represents a layer that overlaps the most recent upper layer of the above-described middle layer, and is simply referred to as “upper layer” hereinafter. In addition, in this example, in order to simplify description, the case where the display of the display part 30 is comprised by three layers is demonstrated, but in the tactile sensation presentation apparatus 1 by this Embodiment, the number of layers is arbitrary. It can be.

  As shown in FIG. 4A, in the lower layer, by executing the application, the background color is displayed as an image, or an object such as a key for performing an operation based on the application is displayed as an image. Also, as shown in FIG. 4B, in the middle layer, for example, a one-segment broadcasting application is executed to reproduce a video of a news program. Further, as shown in FIG. 4C, in the upper layer, for example, a pop-up display object is displayed as an image so that the operator can select whether to end the application.

  Note that, in the upper layer of FIG. 4C, the state in which the pop-up display is performed is illustrated, but when the pop-up display is not performed in the normal operation, as illustrated in FIG. 5C. In addition, no image is displayed. In this example, the objects displayed in each layer as shown in FIG. 4 are not transparently displayed, but the layers in which no object is displayed as shown in FIG. A transparent display shall be performed.

  FIG. 5 is a diagram showing a normal display when a one-segment broadcasting application is activated in the tactile sensation providing apparatus 1 based on the layer display shown in FIG. As described above, the normal display is a case where the pop-up display that alerts the operator on the display unit 30 is not performed.

  As shown in FIG. 5C, in the upper layer when the pop-up display is not performed, the touch to the layer is detected because the tactile sensation activation flag described in FIG. 3B is OFF. And no tactile sensation is presented. Further, (2) since the lower layer control flag is ON, no contact is detected in the upper layer, but contact with a lower middle layer is detected. Note that the number of stages of tactile sensation presentation is set to zero because contact is not detected in the upper layer.

  Next, as shown in FIG. 5B, in the middle layer when the pop-up display is not performed on the upper layer, (1) the touch activation flag is OFF, so that contact with the layer is detected. And no tactile sensation is presented. That is, no contact is detected with respect to a location where a moving image by one-segment broadcasting is displayed, and no tactile sensation is presented even if contact is made with the location. Further, (2) since the lower layer control flag is ON, contact with a lower layer that is lower than the middle layer is detected without detecting contact in the middle layer. Note that the number of stages of tactile sensation presentation is set to zero because contact is not detected even in the middle layer.

  Furthermore, as shown in FIG. 5A, in the lower layer when the pop-up display is not performed on the upper layer, (1) the touch activation flag is ON, so that contact with the layer is detected, A tactile sensation is presented in response to the detection. That is, contact with a portion where an object such as a key displayed at the bottom of the screen is displayed is detected, and a corresponding tactile sensation is presented in response to the detection. In the lower layer, the number of stages of tactile sensation presentation is set to 1 in order to detect one-stage contact or press on each object. That is, in this example, only one type of contact or pressure is detected without distinguishing between pressures based on multi-stage load standards for the lower layer or according to contact or pressure modes.

  Next, for example, when an operator performs an operation to end the 1Seg broadcasting application from the menu, or an event that asks whether or not to end the 1Seg broadcasting application occurs due to a timer setting or an interrupt operation of another application. And

  When such an event occurs, as described above, the arrangement of objects, the configuration of UI, and the like are often changed. Therefore, when such a change is made, a new display on the display unit 30 is configured by (2) image data shown in FIG. 3, and (1) display / tactile control is performed by tactile control information. The setting when the unit 10 controls the tactile sensation providing unit 50 is also changed. In this case, in order to confirm with the operator whether or not to end the one-segment broadcasting application, a pop-up display as shown in FIG. 6C is performed in the upper layer.

  FIG. 6 is a diagram illustrating a state in which a pop-up is displayed on the display unit 30 by a layer while the one-seg broadcasting application is activated in the tactile sensation providing apparatus 1.

  As shown in FIG. 6C, when the display is changed, in the upper layer in the case of performing the pop-up display, (1) the tactile sensation activation flag is set to ON and contact with the layer is detected. A tactile sensation is presented in response to the detection. In addition, (2) since the lower layer control flag is set to OFF, contact with the lower middle layer and the lower layer than the upper layer is not detected. Accordingly, as a result, no contact is detected with respect to anything other than the pop-up object displayed on the upper layer, and even if contact is made, no tactile sensation is presented. Note that, in the upper layer, (6) the number of stages of tactile sensation is set to 1 so that one level of contact or press is detected for each object. That is, in this example, only one type of contact or press is detected for this upper layer pop-up display.

  Next, the operation of the tactile sensation providing apparatus 1 according to the present embodiment will be further described using another specific example. FIG. 7 is a diagram illustrating a state in which images are superimposed on the display unit 30 of the tactile sensation providing apparatus 1 using three layers. Also in the following description, the display of the image on the display unit 30 that is originally superimposed is illustrated separately for each layer for the sake of explanation. In FIG. 7 and subsequent figures, the layer shown in (A) is simply written as “lower layer”, the layer shown in (B) is simply written as “middle layer”, and the layer shown in (C) is simply “upper layer”. It is written as “Layer”.

  As shown in FIG. 7A, in the lower layer, for example, when a music player application is executed, a background color is displayed as an image, or an object such as a key for performing an operation based on the application is displayed as an image. Is done. As shown in FIG. 7B, in the middle layer, for example, a character input screen is displayed by executing an e-mail creation application. Further, as shown in FIG. 7C, in the upper layer, for example, when an incoming call is received, a pop-up display object is displayed as an image so that the operator can select whether or not to connect the incoming call. Is done.

  Note that, in the upper layer of FIG. 7C, the state in which the pop-up display is performed is illustrated, but when the pop-up display is not performed in the normal operation, as illustrated in FIG. 8C. In addition, no image is displayed. Also, in this example, unlike the example described above, the objects displayed on each layer as shown in FIG. In other words, even if objects of multiple layers are displayed overlapping each other, each object is displayed in a light transparent color, so that the objects displayed in each layer are displayed transparently. To. Further, as shown in FIG. 8C, the layer on which no object is displayed is transparently displayed in the same manner as the above-described example.

  The tactile sensation providing apparatus 1 in this example can perform a multitasking operation that simultaneously executes a plurality of applications, and displays operations on different layers for each application so as to distinguish and detect operations performed by the operator for each layer. Furthermore, in this example, it is assumed that the operation of pressing or touching can be detected as differently distinguished in several stages. That is, in this example, the tactile sensation providing apparatus 1 performs a pressing on a different layer or a pressing at a different stage in a predetermined layer depending on the pressing stage when the operator performs a touch operation on the touch sensor 40. To be able to detect. Furthermore, in this example, the tactile sensation providing apparatus 1 is configured as a contact with a different layer or a contact at a different stage in a predetermined layer depending on a contact mode when the operator performs a touch operation on the touch sensor 40. It can also be detected.

  FIG. 8 is a diagram showing a normal display when the application for creating an e-mail is activated while the application for the music player is activated in the tactile sensation providing apparatus 1 based on the layer display shown in FIG. As described above, the normal display is a case where the pop-up display that prompts the operator to pay attention is not performed on the display unit 30. In this case, an object (image) related to an application for creating an e-mail is displayed in the middle layer, and an object (image) related to a music player is displayed in the lower layer.

  As shown in FIG. 8C, in the upper layer when the pop-up display is not performed, the touch to the layer is detected because the tactile sensation activation flag described in FIG. 3B is OFF. And no tactile sensation is presented. Further, (2) since the lower layer control flag is ON, contact is not detected in the upper layer, but contact with a lower middle layer is detected. Note that the number of stages of tactile sensation presentation is set to zero because contact is not detected in the upper layer.

  Next, as shown in FIG. 8B, in the middle layer when the pop-up display is not performed on the upper layer, (1) the tactile sensation activation flag is ON, so that contact with the layer is detected, A tactile sensation is presented in response to the detection. That is, a contact with a place where an object is displayed by the e-mail creation application is detected, and when the contact is detected, a predetermined operation is performed and a set tactile sensation is presented. Further, (2) since the lower layer control flag is ON, the contact with the lower layer that is lower than the middle layer is further detected while detecting the contact with the middle layer. Note that the number of stages of (6) tactile sensation when detecting contact in the middle layer is set to 1.

  Further, as shown in FIG. 8A, in the lower layer when the pop-up display is not performed on the upper layer, (1) since the tactile sensation activation flag is ON, contact with the layer is detected, A tactile sensation is presented in response to the detection. That is, contact with a portion where an object such as a key displayed at the bottom of the screen is displayed is detected, and a corresponding tactile sensation is presented in response to the detection. In the lower layer, the number of stages of tactile sensation presentation is set to 2 in order to detect a two-stage contact or press on each object (6). In other words, in this example, two types of contact or press are detected by distinguishing between the pressures of the lower layer based on the multi-stage load criteria and the contact or press mode. In this example, as shown in FIG. 8B, since the first level contact or press is already detected in the middle layer, the second level is actually detected in the lower layer shown in FIG. 8C. And the contact or press of the 3rd step will be detected.

  Therefore, the operator can perform an operation on the e-mail creation application in the middle layer by performing the first-stage contact or pressing. Furthermore, the operator can perform operations on the music player application in the lower layer by performing contact or pressing at the second and third stages. In this case, for example, the next music piece can be reproduced in the music player by detecting the second stage contact or press, and the previous music piece can be reproduced by detecting the third stage contact or press. Can be assigned, and so on. Note that the control unit 10 may switch the middle layer to the upper layer and display it on the display unit 30 by the operator performing contact or pressing in the first stage. Similarly, the control unit 10 may switch the lower layer to the upper layer and display it on the display unit 30 by the operator performing contact or pressing at the second stage.

  For example, when an object related to an application for creating an e-mail displayed in the middle layer and an object related to a music player displayed in the lower layer overlap, when the operator presses a position corresponding to the object related to the music player, Before the process corresponding to the object related to the music player is performed, the process corresponding to the object related to the e-mail creation application is performed. As described above, the tactile sensation providing apparatus 1 can perform the following processing in order to prevent the processing unintended by the operator from being performed due to the objects of the respective layers overlapping.

  That is, when the operator touches or presses the first step on the position where the object related to the e-mail creating application displayed in the middle layer and the object related to the music player displayed in the lower layer overlap. In other words, the tactile sensation providing apparatus 1 only presents a tactile sensation associated with the middle layer to the operator, and does not perform processing related to an object related to an e-mail creation application. The processing related to the object relating to the e-mail creation application is performed after the operator performs a predetermined determination operation without performing the second stage contact or pressing after the first stage contact or pressing is performed. Limited to cases. Here, examples of the predetermined determination operation include a slide operation, a flick operation, and an operation for releasing contact with the touch sensor 40.

  Next, it is assumed that the telephone call application received by the tactile sensation providing apparatus 1 has received a telephone call, and therefore an event asking whether or not to connect the incoming call has occurred due to the interruption operation of the call application.

  When such an event occurs, a new display on the display unit 30 is configured by (2) image data shown in FIG. 3, and (1) the display / tactile control unit 10 presents a tactile sensation based on the tactile control information. Settings for controlling the unit 50 are also changed. In this case, a pop-up display as shown in FIG. 9C is performed in the upper layer in order to confirm with the operator whether or not to connect the incoming call. In the upper layer shown in FIG. 9C, it is assumed that the tactile sensation providing apparatus 1 performs an operation of connecting an incoming call in response to detection of contact with an object indicating “incoming”. That is, the operator can make a call by connecting the incoming call by touching the object indicating “incoming” displayed on the upper layer with a finger or the like.

  FIG. 9 is a diagram illustrating a state in which a pop-up indicating that a call has been received is displayed on the display unit 30 by a layer while different applications are being executed in the middle layer and the lower layer in the tactile sensation providing apparatus 1.

  As shown in FIG. 9C, along with the display change, in the upper layer in the case of performing the pop-up display, (1) the tactile sensation activation flag is set to ON and contact with the layer is detected. A tactile sensation is presented in response to the detection. Further, (2) since the lower layer control flag remains set to ON, contact with the middle layer and the lower layer lower than the upper layer is detected. Note that in the upper layer, the number of tactile sensation presentation stages is set to 1 in order to detect one-stage contact or press on each object. Therefore, the first-stage contact with the pop-up object displayed on the upper layer is detected, a tactile sensation is presented in response to the detection of the contact, and the incoming call is connected.

  As shown in FIG. 9B, in the middle layer, (1) since the tactile sensation activation flag is OFF, contact with the layer is not detected, and tactile sensation is not presented. Accordingly, as a result, contact with an object based on the e-mail creation application displayed on the middle layer is not detected, and the operator cannot perform an operation on the e-mail creation application. However, as shown in FIG. 9B, (2) since the lower layer control flag is ON, contact with a lower layer that is lower than the middle layer is not detected even if contact is not detected in the middle layer. Is detected.

  As shown in FIG. 9A, in the lower layer, (1) since the tactile sensation activation flag is ON, contact with the layer is detected, and tactile sensation is presented according to the detection. In the lower layer, the number of stages of tactile sensation presentation is set to 2 in order to detect a two-stage contact or press on each object (6). That is, in this example, two types of contact or pressure are detected by distinguishing between the pressures of the lower layers based on the multi-stage load criteria and the distinction according to the contact or pressure mode. In this example, as shown in FIG. 9C, since the first level contact or press has already been detected in the upper layer, the second level is actually detected in the lower layer shown in FIG. And the contact or press of the 3rd step will be detected. In this case, as described above, it is possible to reproduce the next song on the music player by detecting the second stage contact or press, and it is possible to reproduce the previous song by detecting the third stage contact or press. Can be played, and so on.

  As described above, according to the tactile sensation providing apparatus 1, when a telephone call is received while the music player and the e-mail creation application are simultaneously executed, for example, the operation for the e-mail creation application is invalidated and the music player is invalidated. The operation on can be left valid.

  As shown in the above-described example, according to the tactile sensation providing apparatus 1 of the present embodiment, an image is displayed in a layer on the display unit, and the touch sensor detects a contact corresponding to the image of each layer, The tactile sensation of each layer can be presented. Therefore, according to the tactile sensation providing apparatus 1 of the present embodiment, the tactile sensation can be controlled three-dimensionally by detecting contact or pressing and presenting the tactile sensation according to the detection in units of layers. The touch sensor 40 and the display unit 30 can be effectively used.

  In addition, this invention is not limited only to the said embodiment, Many deformation | transformation or a change is possible. For example, in the embodiment described above, the configuration in which the display / tactile sensation control unit 10 controls the display unit 30 and the tactile sensation presentation unit 50 has been described. However, the present invention is not limited to the configuration. Various configurations can be adopted according to the design requirements. For example, as shown in FIG. 10, the display control unit 12 controls the display unit 30, and the tactile sensation control unit 14 controls the tactile sensation providing unit 50. In this case, the display control unit 12 performs control so that the display unit 30 displays an image based on the image data of (2) shown in FIG. Control is performed so that the tactile sensation providing unit 50 displays a predetermined tactile sensation based on the tactile sensation control information of (1) shown in FIG.

  Further, for example, the display / tactile sensation control unit may include the application execution unit 20.

  Moreover, in the said embodiment, although the contact with respect to the touch surface of the said touch sensor was detected using the touch sensor 40, when the reference | standard of a predetermined pressing load was satisfy | filled using the load sensor (load detection part). It can also be determined that contact has been made. Such a load detection unit can be configured by using an arbitrary number of strain gauge sensors and the like, similarly to the load detection unit of the above-described embodiment.

  Moreover, in embodiment mentioned above, a load detection part can be comprised according to the contact detection system in a touch sensor. For example, in the case of the resistance film method, if a load can be detected from a change in an output signal based on a change in resistance due to a contact area, it can be configured without using a strain gauge sensor. Alternatively, in the case of the electrostatic capacity method, even when the load can be detected from the change in the output signal based on the change in the electrostatic capacity, it can be configured without using the strain gauge sensor.

  In addition, the tactile sensation providing unit may be configured using an arbitrary number of piezoelectric vibrators, or may be configured by providing a transparent piezoelectric element on the entire surface of the touch sensor, or can express vibration that presents tactile sensation. Can be configured to rotate once in one cycle of the drive signal. Furthermore, when the load detection unit and the tactile sensation providing unit are configured using a piezoelectric element, the load detection unit / tactile sensation providing unit can also be configured by sharing the piezoelectric element.

  In embodiment mentioned above, the case where the load detection part 60 which detects the press load with respect to the touch surface of the touch sensor 40 was provided was demonstrated. However, in the present invention, the detection of the pressing load on the touch sensor 40 is not an essential matter, and the operation described above can be performed simply by the touch sensor 40 detecting only the presence / absence of the contact and the position of the contact. In this case, since different actions cannot be assigned by stepwise detection of the load, different actions are assigned according to the manner of contact with the touch sensor 40.

  Further, the tactile sensation providing apparatus according to the present invention has been described on the assumption that the piezoelectric element is driven when a pressing load detected based on the output of the piezoelectric element satisfies a predetermined standard for presenting a tactile sensation. . Here, the case where the pressing load detected based on the output of the piezoelectric element satisfies the standard for presenting tactile sensation may be the time when the detected pressing load reaches a reference value for presenting tactile sensation. The detected pressing load may exceed a reference value that provides tactile sensation, or may be detected when a reference value that provides tactile sensation is detected based on the output of the piezoelectric element.

  Further, the “display unit” and “touch sensor” in the description of the embodiment of the present invention described above are configured by an integrated device, for example, by providing both functions of the display unit and the touch sensor on a common substrate. Also good. As an example of the configuration of the device in which both functions of the display unit and the touch sensor are integrated, a plurality of photoelectric conversion elements such as photodiodes are regularly arranged in a matrix electrode array of pixel electrodes of the liquid crystal panel. Some are mixed. This device displays an image with a liquid crystal panel structure, while reflecting the light of a backlight for liquid crystal display at the tip of a pen that touches and inputs a desired position on the surface of the panel. By receiving the light, the touch position can be detected.

  Further, in the present invention, the “image displayed on the layer on which the tactile sensation is set in the display unit” does not include only the image actually displayed on the display unit of the actual tactile sensation presentation apparatus, The image of the lower layer which overlaps with the image of the layer and is not actually displayed on the display unit is also included.

DESCRIPTION OF SYMBOLS 1 Tactile sensation presentation apparatus 10 Display / tactile sensation control part 20 Application execution part 30 Display part 40 Touch sensor 40a Touch surface 50 Tactile sensation presentation part 52 Piezoelectric vibrator 60 Load detection part 62 Strain gauge sensor 70 Control part 71 Case 72 Insulator 73 Upper cover 74 Insulator 80 Storage Unit

Claims (6)

A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A display for displaying an image;
A control unit that controls the display unit to display an image in a plurality of layers;
A tactile sensation presentation device comprising:
The control unit further controls to set the tactile sensation presented by the tactile sensation providing unit for each layer, and the touch sensor is located at a position corresponding to an image displayed on the layer where the tactile sensation is set in the display unit. When a contact is detected, the tactile sensation providing apparatus controls the operation assigned to the image and controls the tactile sensation providing unit to present the tactile sensation set in the layer.   The tactile sensation presenting apparatus according to claim 1, wherein the control unit sets validity or invalidity of tactile sensation set in a layer other than the one layer in one of the plurality of layers.   The tactile sensation providing apparatus according to claim 1, wherein the control unit determines a layer on which an image corresponding to a position where the touch is detected is displayed according to a touch mode detected by the touch sensor. A touch sensor;
A tactile sensation providing unit that vibrates the touch surface of the touch sensor;
A load detection unit for detecting a pressing load on the touch surface;
A display for displaying an image;
A control unit that controls the display unit to display an image in a plurality of layers;
A tactile sensation presentation device comprising:
The control unit performs control so that a pressing load serving as a reference for the tactile sensation providing unit to provide a tactile sensation is set for each layer, and the touch sensor is displayed on the layer where the tactile sensation is set in the display unit. While detecting a contact at a position corresponding to the image, if the load detection unit detects a pressing load that satisfies the criteria set for the layer, the control is performed so as to perform the operation assigned to the image. The tactile sensation providing apparatus controls the tactile sensation providing unit to present the tactile sensation set in the layer.   The tactile sensation providing apparatus according to claim 4, wherein the control unit sets validity or invalidity of presentation of tactile sensation set in a layer other than the one layer in one of the plurality of layers.   The tactile sensation providing apparatus according to claim 1, wherein the control unit performs control so that the setting of tactile sensation set for each layer is changed based on occurrence of a predetermined event.

Images