JP2015181055A - User interface device, tactile vibration application method and program for applying tactile vibration corresponding to depth/height of tactile object image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a user interface device for applying three-dimensional information included in an image displayed on a screen to a finger with which an operation is performed.SOLUTION: A user interface device includes: depth quantity allocation means for allocating depth quantity as a distance from a reference point in a virtual space to each position on an object set in a virtual space in a screen, that is, an object to which a tactile object image has been adhered; height quantity allocation means for allocating height quantity corresponding to the tactile object image to each position on the tactile object image; depth/height acquisition means for, when a contact corresponding position corresponding to the contact position of a finger is superimposed with the display position range of the tactile object image, acquiring the depth quantity and height quantity allocated to the contact corresponding position; and tactile vibration control means for applying tactile vibration corresponding to the acquired depth quantity and a changed portion of the acquired height quantity accompanied by the movement of the contact position of the finger to the finger.

Description

  The present invention relates to a user interface device including a display that displays a display target on a screen and a touch panel that enables operation with a finger.

  2. Description of the Related Art Conventionally, user interface devices equipped with a touch panel that accepts operations by finger contact have been widely used. In particular, in recent years, touch panels have been actively employed as input means in user interface devices such as smartphones, tablet computers, electronic books, PDAs (Personal Digital Assistants), and so-called portable information devices.

  If a touch panel is adopted, various input operations corresponding to the multi-functionality of the device can be performed. However, on the other hand, erroneous input operations tend to occur. Specifically, an input operation occurs even when the user unintentionally touches the touch panel, or even if the user performs an input operation, a desired function is not activated and a redo operation is required.

  There is a technique for giving a tactile response by vibration or the like to the operated finger as a measure for dealing with the problem of the erroneous operation. The user can perform an operation with reference to obtaining a response (feedback) to the operation performed by himself / herself, and can obtain the response with a finger to confirm the completion of the operation.

  For example, Patent Document 1 discloses a touch panel device that vibrates a multi-touch panel that can detect a plurality of contacts that occur simultaneously and performs feedback of touch confirmation. In this device, when the first finger touches, the first vibration is applied to the finger, and when the second finger touches while the first finger is touching, the vibration level is higher than the first vibration. A large second vibration is applied.

  Further, Patent Document 2 discloses an input device that performs notification in a notification mode set in a button area when one of a plurality of button areas allocated on the detection surface of the touch panel is touched. . Here, this notification form is any one of vibration, sound, color, and brightness, or a combination thereof.

  Further, Patent Document 3 discloses an input / output device in which a piezoelectric actuator is incorporated in a component storage portion of an acrylic panel and combined with a touch panel and a liquid crystal display device. Again, tactile feedback is provided to the operator's finger during an information input operation. As an example of this tactile feedback, a tactile sense A is given by the first vibration pattern Pa when the button icon is pressed, and a tactile sense B is given by the second vibration pattern Pb when the button icon is released.

JP 2010-55282 A JP 2010-9321 A JP 2006-215738 A

  Currently, portable information devices can provide services for handling three-dimensional objects such as various games, online shopping, virtual museums, and museums via the Internet. In these services, a 3D (three-dimensional) image related to an actual three-dimensional object is often displayed on the screen. There are also cases where a standby screen in which operation target images such as icons are arranged and a list screen in which thumbnails such as photos and video files are arranged are displayed as a 3D image.

  Consider performing an operation by touching a finger on such a 3D image. At this time, it is difficult for the user to obtain three-dimensional information such as a sense of depth of the 3D image through the finger that performs the operation only with the conventional feedback technique as described in Patent Documents 1 to 3. For example, information about the position of the 3D image part in contact with the finger in the depth direction of the screen and whether there is an unevenness or a step in the part is obtained through the operating finger. It is very difficult.

  Therefore, even if 3D technology is adopted for the corner and the input / display target, the three-dimensional information included in the input / display target can be acquired only through vision.

  Therefore, an object of the present invention is to provide a user interface device, a tactile vibration imparting method, and a program capable of imparting three-dimensional information included in an image displayed on a screen to an operating finger.

According to the present invention, there is provided a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a contact position of a finger,
A tactile vibration mechanism that imparts tactile vibration that brings tactile sensation to a finger that touches the touch panel;
From the reference point in the virtual space to each position on the object that is the display target set in the virtual space set in the screen, and the tactile target image to be touched is attached A depth amount assigning means for assigning a depth amount which is a distance of
High and low amount assigning means for assigning a high and low amount corresponding to the tactile object image to each position on the tactile object image;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to this contact corresponding position and the tactile sense corresponding to this contact corresponding position Depth / high / low acquisition means for acquiring the high / low amount assigned to the position on the target image;
The haptic vibration mechanism unit is instructed to apply a tactile vibration according to the acquired depth amount and the acquired amount of change in height according to the movement of the contact position of the finger to the finger via the touch panel. A user interface device having haptic vibration control means is provided.

As one embodiment of the user interface device according to the present invention,
The first object and the second object are at least partially overlapped when viewed from the reference point side, and the first object is positioned in front of the first object and the second object. Display of the second tactile target image affixed to the second object without overlapping the contact corresponding position within the superimposition region with the display position range of the first tactile target image affixed to the first object When overlapping with the position range,
Depth / high / low acquisition means
It is also preferable to acquire the depth amount assigned to the contact corresponding position of the second object and the height amount assigned to the position on the second tactile target image corresponding to the contact corresponding position.

  Further, according to the user interface device of the present invention, the haptic vibration control means is configured so that the intensity of the haptic vibration becomes a monotonically decreasing function for the acquired depth amount and becomes a monotone increasing function for the acquired amount of change in height. It is also preferable to determine to.

Furthermore, as another embodiment of the user interface device according to the present invention,
It further includes a pressing force detection unit that detects a pressing force applied to the touch panel by a finger,
It is also preferable that the tactile vibration control means gives the finger a tactile vibration corresponding to the acquired depth amount, height change amount, and pressing force value. In this case, it is also preferable that the haptic vibration control means determines the intensity of the haptic vibration to be a monotonically increasing function with respect to the pressing force value.

In addition, according to the user interface device of the present invention, as viewed from the user's operation, the finger moves while touching so as to cross the tactile target image pasted on the object displayed on the screen of the image display unit. When tracing, it is also preferable that the finger is given a tactile vibration having a smaller intensity as the tactile object image is located further in the virtual space.

According to the present invention, there is further provided a program installed in a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a finger contact position. ,
It has a tactile vibration mechanism that gives tactile vibration that brings tactile sensation to the finger that touched the touch panel.
From the reference point in the virtual space to each position on the object that is the display target set in the virtual space set in the screen, and the tactile target image to be touched is attached A depth amount assigning means for assigning a depth amount which is a distance of
High and low amount assigning means for assigning a high and low amount corresponding to the tactile object image to each position on the tactile object image;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to this contact corresponding position and the tactile sense corresponding to this contact corresponding position Depth / high / low acquisition means for acquiring the high / low amount assigned to the position on the target image;
The haptic vibration mechanism unit is instructed to apply a tactile vibration according to the acquired depth amount and the acquired amount of change in height according to the movement of the contact position of the finger to the finger via the touch panel. A program for a user interface device that causes a computer to function as tactile vibration control means is provided.

According to the present invention, there is further provided a tactile vibration imparting method in a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a contact position of a finger. ,
A tactile vibration mechanism for applying a tactile vibration that provides a tactile sensation to a finger touching the touch panel, and the tactile vibration applying method includes
From the reference point in the virtual space to each position on the object that is the display target set in the virtual space set in the screen, and the tactile target image to be touched is attached A first step of assigning a depth amount which is a distance of
A second step of assigning a height corresponding to the tactile target image to each position on the tactile target image;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to this contact corresponding position and the tactile sense corresponding to this contact corresponding position A third step of obtaining a height amount assigned to a position on the target image;
The haptic vibration mechanism unit is instructed to apply a tactile vibration according to the acquired depth amount and the acquired amount of change in height according to the movement of the contact position of the finger to the finger via the touch panel. There is provided a tactile vibration imparting method having a fourth step.

  According to the user interface device, the tactile vibration imparting method, and the program of the present invention, three-dimensional information included in an image displayed on a screen can be imparted to a finger that performs an operation.

1 is a front view illustrating an embodiment of a portable information device according to the present invention, a schematic diagram illustrating a depth amount of a displayed object, and a graph illustrating a height of a tactile target image pasted on the object. It is the schematic and graph explaining the procedure which acquires the depth amount S and the height amount M corresponding to the contact position of a finger | toe, and determines the vibration intensity I of the tactile vibration provided to a finger | toe. It is the schematic and graph explaining other embodiment about determination of the vibration strength I. FIG. 6 is a front view showing an embodiment in which a plurality of objects are displayed on the screen, and a schematic diagram for explaining the depth amount and the height amount when the plurality of objects are displayed. It is a front view which shows other embodiment of the object displayed on the screen. FIG. 2 is a perspective view and a functional configuration diagram schematically showing a configuration of a portable information device according to the present invention. 3 is a flowchart illustrating an embodiment of a tactile vibration imparting method according to the present invention. It is a flowchart which shows other embodiment of the tactile vibration provision method by this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

In the user interface device according to the present invention,
(A) Assign a depth amount to each position on the object displayed on the screen,
(B) Assign a high and low amount to each position on the tactile target image affixed to this object,
(C) obtaining the depth and height corresponding to the contact position of the finger;
(D) It is characterized in that tactile vibration having vibration intensity corresponding to the amount of depth and the amount of change in height according to the movement of the contact position of the finger is applied to the finger. Here, the tactile vibration is a vibration or vibration pattern that brings a tactile sensation to the finger.

  Many of the user interface devices according to the present invention are portable information devices such as smartphones and tablet computers that are normally carried around and can be operated with fingers. Therefore, hereinafter, a portable information device will be described as an embodiment of the present invention.

  FIG. 1 is a front view showing an embodiment of a portable information device according to the present invention, a schematic diagram for explaining the depth amount of a displayed object, and a graph for explaining the height of a tactile target image pasted on the object. It is.

  1A, the portable information device 1 includes a display 101 that displays an object on a screen, and a touch panel that is arranged on the screen of the display 101 and sequentially detects a finger contact position 106 over time. 100, and a tactile vibration mechanism 102 that applies a tactile vibration v to a user's finger in contact with the touch panel 100 (screen of the display 101).

  In this portable information device 1, a menu plate 104 is displayed as an object. The menu plate 104 is disposed obliquely with respect to the screen in the virtual space set in the screen of the display 101, and the right end of the menu plate 104 is retracted further back as viewed from the screen.

  On the menu plate 104, icons 105a, 105b, and 105c are pasted as tactile object images. Here, the icons 105a, 105b, and 105c are sequentially positioned on the far side as viewed from the screen. Note that the positions of these icons can be changed, for example, by an operation with a predetermined finger (for example, a procedure of dragging after a finger has been in contact for a predetermined time or more).

  The icons 105a to 105c may be virtual switches in which a function such as an application given in advance is activated by, for example, a tap operation with a finger, a push-in operation, or a long-press operation. Alternatively, it may be simply an area where a tactile accent is provided on the menu plate 104.

  The user touches the touch panel 100 (the screen of the display 101) with a finger and slides the finger so as to trace the menu plate 104. Here, when the finger traces any of the icons 105a to 105c, that is, when the contact position 106 of the finger is superimposed on any of the display position ranges of the icons 105a to 105c, it is assumed that this icon is selected. The unit 102 applies the tactile vibration v to the finger.

  The tactile vibration v has a vibration intensity I corresponding to the depth amount and the height amount, which is three-dimensional information included in the menu plate 104 and the icons 105a to 105c. Thereby, the user can acquire these three-dimensional information through his / her finger performing the operation.

  Hereinafter, allocation of the depth amount and the height amount to the menu plate 104 and the icons 105a to 105c will be described.

According to FIG. 1B, a depth amount S is assigned to each position on the displayed menu plate 104. Here, a straight line passing through the assumed camera position O _c and the position on the menu plate 104 in the virtual space set in the screen is assumed. At this time, the depth amount S is defined as the length (distance) from the intersection (reference point) between the straight line and the touch panel surface 100a to the position on the menu plate 104.

A contact corresponding position 107 that is a position on the menu plate 104 corresponding to the finger contact position 106 is defined as an intersection of the menu plate 104 and a straight line passing through the assumed camera position O _c and the contact position 106. Accordingly, the depth amount S assigned to the contact corresponding position 107 is the length of the line segment connecting the contact position 106 and the contact corresponding position 107.

The depth amount S is not limited to that shown in FIG. For example, assuming that the assumed camera position O _c is at infinity, and the vertical foot that is lowered from one position on the menu plate 104 to the touch panel surface 100a is a reference point, the length of the vertical line is assigned to this one position. It can also be a quantity.

Furthermore, the distance (S ′ in FIG. 1B) between the assumed camera position O _c and one position on the menu plate 104 may be adopted as the depth amount with the assumed camera position O _c as a reference point. Is possible.

  Next, according to FIG. 1C, a high and low amount M is assigned to each position on each of the icons 105a to 105c. The height M can be set to a value corresponding to the thickness (height, level difference, unevenness) assumed to be included in each of the icons 105a to 105c, for example. In the present embodiment, the icon 105c is thicker than the icons 105a and 105b, and correspondingly, a larger height M is assigned to each position of the icon 105c.

Icons 105 a to 105 c are affixed to the menu plate 104. As a result, the height M is assigned to the position where the icons 105 a to 105 c exist on the menu plate 104. Here, the graph of FIG. 1C shows the height M at a position on the X _ob axis in the menu plate 104.

  FIG. 2 is a schematic diagram and a graph for explaining a procedure for obtaining the depth amount S and the height amount M corresponding to the finger contact position 106 and determining the vibration intensity I of the tactile vibration applied to the finger.

First, FIG. 2A shows the depth amount S and the height amount M corresponding to the finger contact position 106. The user moves (slides) the finger while touching the touch panel 100. Accordingly, the contact corresponding position 107 corresponding to the contact position 106 moves on the X _ob axis in the menu plate 104. Here, the contact corresponding position 107 sequentially passes through the display position range of the icons 105a to 105c.

Next, FIG. 2B shows the depth amount S _i and the height amount M _i acquired at each sampling time interval Δt, and the height amount change ΔM _i and the vibration intensity I _i calculated from these values. .

The depth amount S _i is the distance between the contact position 106 and the contact corresponding position 107 at time t = Δt × i (i = 0, 1, 2,...). The depth amount S _i increases as the user drags the far side of the menu plate 104 with his / her finger as viewed from the user's operation.

The height M _i is the height assigned to the contact corresponding position 107 at time t = Δt × i (i = 0, 1, 2,...). If the corresponding contact corresponding position 107 is not superimposed on any display position range in the icons 105a to 105c, M _i = 0 is set. As a result, the height M _i in FIG. 2B takes a finite value only in the region corresponding to each of the icons 105a to 105c.

Further, the height change ΔM _{i at} time t = Δt × i is calculated using the following equation.
ΔM _i = M _i −M _i−1 (i = 0, 1, 2,...)
Accordingly, in the present embodiment (FIG. 2B), the amount of change ΔM _i takes a positive value when the touch-corresponding position 107 reaches the display position range of the tactile object images (icons 105a to 105c). Moreover, when the contact corresponding position 107 leaves | separates from the display position range of a tactile object image (icon 105a-105c), a negative value is taken.

Here, the absolute value | ΔM _i | of the amount of change ΔM _i of the height is expressed by the following expression | ΔM _i | ≧ M _th
It is also preferable to determine a vibration intensity I _i to be described below and to apply a tactile vibration to the finger only when the condition is satisfied. Here, M _th is a predetermined height change threshold. As a result, an operational feeling that fits the real sense of being sensed as a tactile sensation can be obtained only with a certain level of elevation (thickness, step, unevenness).

As described above, the vibration intensity I _i of the tactile vibration at time t = Δt × i is determined from the depth amount S _i acquired and calculated and the amount of change ΔM _i using the following equation.
(1) I _i = C · | ΔM _i | / S _i
Here, C is a proportionality constant. In actual vibration application, it is also preferable that tactile vibration of vibration intensity I _i is applied to the finger from time t to time Δt _I.

The vibration intensity I _i determined by the above equation (1) takes a value proportional to | ΔM _i |. As a result, when the contact position of the finger enters and exits the tactile object image (icons 105a to 105c), the higher (thick) the tactile object image, the greater the vibration intensity I _i is given to the finger. Thereby, the user can obtain a tactile sensation according to the height (three-dimensional information) of the tactile target image.

Furthermore, according to the above equation (1), the vibration intensity I _i takes a smaller value as the depth amount S _i increases. As a result, a tactile vibration having a lower strength is applied to the finger as the tactile object image (icons 105a to 105c) traced across the finger is located on the back side. Thereby, the user can obtain a feeling of depth (three-dimensional information) according to the position in the screen depth direction of the tactile object image. The screen depth direction is a direction substantially perpendicular to the screen or a direction including a component perpendicular to the screen.

Here, it is also preferable to determine the frequency f in the tactile vibration of the vibration intensity I _i as follows.
When ΔM _i > 0, f = f ₁
When ΔM _i <0, f = f ₂ (<f ₁ )
In this case, the contact position corresponding 107 upon approaching the display position range of a haptic object image (icon 105 a to 105 c), tactile vibrations of frequency _{f 1} is given. On the other hand, when the contact corresponding position 107 is detached from the display position range of a haptic object image (icon 105 a to 105 c), (lower than the frequency _{f 1)} different from the frequency _{f 1} tactile vibrations of frequency _{f 2} Is granted.

  As a result, the user can distinguish between entering and exiting the tactile object image by vibration applied to the finger tracing the tactile object image, and a more realistic sense of height (thickness, step, unevenness). Can be obtained.

Further, the height S is directly assigned to the tactile object images (icons 105a to 105c). As a result, when the tactile object image moves, and even when the assumed camera position O _c (FIG. 1B) is changed, the user always touches the finger when touching the finger. The height S assigned to the target image is acquired. Thus, without being influenced by the change of movement and assumed camera position O _c of the haptic object image, it is possible to obtain a tactile supposed height haptic object image (thickness, stepped, uneven).

The vibration intensity I _i is not limited to the above formula (1). For example, it is possible to determine the vibration intensity I _i to be a monotonically decreasing function for the depth amount S _i and to be a monotonically increasing function for the height change amount ΔM _i (absolute value thereof).

  As described above, it is understood that the portable information device 1 can give three-dimensional information such as depth and height (thickness, step, unevenness) included in the tactile target image as tactile vibration to the finger that performs the operation. The

  FIG. 3 is a schematic view and a graph for explaining another embodiment of the determination of the vibration intensity I.

In the embodiment shown in FIG. 3, the setting of the menu plate 104 and the tactile object images (icons 105a to 105c) and the acquisition of the depth amount S and the height amount M corresponding to the finger contact position 106 are performed in the embodiment of FIG. It is the same. However, in the present embodiment, in consideration of the pressing force _pc with respect to the touch panel 100 by the finger, the touch when the finger is pressed and the tactile object image is traced is realized. Hereinafter, the present embodiment will be described with a focus on differences from the embodiment of FIG.

As shown in FIG. 3A, the user pushes the finger into the touch panel 100 between the icon 105a and the icon 105b while sliding the finger in contact. Pressure p _c by the finger, as shown in the graph of pressure p _c in FIG. 3 (B), it is increasing in depressed position. The size of the pressing force p _c by the finger is detected and measured by the pressure detecting unit 103 to be described later.

Here, in the present embodiment, when the vibration intensity I _i is determined, the measured pressing force _pc value is used together with the | ΔM _i | / S _i value used in the embodiment of FIG. That is, the vibration intensity _{I i} is 3 as shown in (B) | ΔM _i | by the pressing force _{p c} value in / _{S i} and the time t (= Δt × i), is determined by the following equation .
_{(2) I i = C '} · p c · | ΔM i | / S i
Here, C ′ is a proportionality constant. In actual vibration application, as in the embodiment of FIG. 2, it is also preferable that tactile vibration of vibration intensity I _i is applied to the finger from time t to time Δt _I elapses.

The vibration intensity I _i determined by the above equation (2) takes a larger value as the pressing force _pc value increases. For example, in the graph of the vibration intensity I _i in FIG. 3B, since the icon 105b is located on the back side of the icon 105a, the vibration intensity I _i when the finger contacts the icon 105b is the icon 105a. Should be smaller than. However, since the finger slides while pressing the icon 105b more strongly, in this embodiment, the vibration intensity I _i when touching the icon 105b is rather larger than that of the icon 105a.

  Thus, according to this embodiment, even if it is a tactile object image located in the back side, if a finger is pushed more strongly, a strong tactile vibration will be given to a finger. Accordingly, it is possible to realize a real feeling that the tactile sensation changes depending on the degree to which the finger is pressed when tracing a three-dimensional object in an actual space. Further, it is possible to acquire a sense of depth (three-dimensional information) of the tactile object image from the change in tactile sensation at the time of pressing.

In the present embodiment, the vibration intensity I _i is not limited to the above formula (2). For example, the vibration intensity I _i, can be determined to be a monotonically increasing function for pressure p _c.

  FIG. 4 is a front view showing an embodiment in which a plurality of objects are displayed on the screen, and a schematic diagram for explaining a depth amount and a height amount when a plurality of objects are displayed.

  The portable information device 1 shown in FIG. 4A has the same configuration as that shown in FIG. 1A, but the first menu plate 104a and the first menu are displayed on the screen of the display 101 as objects. 2 menu plate 104b is displayed.

  The first menu plate 104a and the second menu plate 104b are at least partially overlapped when viewed from the screen side (touch panel 100 side). Further, the first menu plate 104a is positioned in front of the second menu plate 104b. Furthermore, the first menu plate 104a and the second menu plate 104b are translucent images. As a result, the icon 105g of the second menu plate 104b is seen through the first menu plate 104a even though it is covered by the first menu plate 104a when viewed from the screen side.

  Here, a case where the user selects the icon 105g by sliding the touched finger and superimposing the finger contact position 106 on the see-through icon 105g will be described.

  As shown in FIG. 4B, the touch corresponding position 107a on the first menu plate 104a corresponding to the finger touch position 106 is a tactile object image (icons 105d to 105f) pasted on the first menu plate 104a. ) Is not superimposed on the display position range. On the other hand, the contact corresponding position 107b on the second menu plate 104b corresponding to the finger contact position 106 is superimposed on the tactile object image pasted on the second menu plate 104b, that is, the display position range of the icon 105g. Yes.

In this case, the user amount of depth S ₂ and altitude amount M ₂ of the icon 105g corresponding selected in the finger is acquired, these values are used to calculate the vibration intensity I _i. The acquired depth amount S ₂ and height amount M ₂ are as follows, as shown in FIG.
(A) Depth amount S ₂ : Distance between the contact position 106 on the touch panel surface 100a and the contact corresponding position 107b on the second menu plate 104b. That is, the depth amount assigned to the contact corresponding position 107b of the second menu plate 104b.
(B) High / low amount M ₂ : The high / low amount assigned to the icon 105g on the contact corresponding position 107b.

  As a result, the user can touch the icon 105g located on the back side of the first menu plate 104a and select the icon. As a result, a plurality of objects (menu plate, etc.) are prepared, and a large number of tactile target images (icons, etc.) are three-dimensionally arranged on each object, and the user can select a desired icon among them. It becomes possible. At that time, since a tactile vibration according to the position of the icon in the depth direction of the screen is given to the finger, the user may confirm whether or not the desired icon is surely selected from the three-dimensionally arranged icons. And operability is improved.

  FIG. 5 is a front view showing another embodiment of the object displayed on the screen.

  According to FIG. 5, a staircase object 2 that is a staircase-shaped three-dimensional object image is displayed on the screen of the display 101 of the portable information device 1. The staircase object 2 is composed of a large number of polygons, and a tactile object image is pasted (mapped) as a texture to the polygons 200, 201, 210 and 211.

A depth amount S and a height amount M are assigned to the tactile object images pasted on these polygons. Specifically, for example, it is as follows.
Tactile target image (polygon 200): depth amount S1, height amount M1
Tactile target image (polygon 201): depth amount S1, height amount M2
Tactile object image (polygon 210): depth amount S2, height amount M1
Tactile object image (polygon 211): depth amount S2, height amount M2

  Here, the depth amount S1> the depth amount S2. This corresponds to the fact that the polygons 200 and 201 are affixed to the stair step above the polygons 210 and 211 (on the front side as viewed from the screen). Further, the height M2> the height M1. As a result, a bulge is expressed on the stair step where the polygons 200 and 201 (polygons 210 and 211) are located.

  When the staircase object 2 set in this way is traced with a finger, the finger touch position 106 is superimposed on the display position range of the polygons 200, 201, 210 and 211, and each time the finger moves away from the display position range, a tactile sensation is applied to the finger. Vibration is applied. The vibration intensity I at this time is determined from the depth amount S1 or S2 assigned to the polygons 200, 201, 210, and 211 and the amount of change in the height amount S1 or S2.

  As a result, three-dimensional information included in the target displayed on the screen, such as “projection of a central protruding rhombus provided at each stair step of the stair step object 2”, can be given to the touching finger. It becomes.

  Of course, the object is not limited to the staircase object. For example, three-dimensional images displayed on various applications, search / standby screens, and three-dimensional object images handled in services such as distribution games, online shopping, and virtual museums / museums can be employed as objects.

  FIG. 6 is a perspective view and a functional configuration diagram schematically showing the configuration of the portable information device 1 according to the present invention.

  According to FIG. 6, the portable information device 1 includes a touch panel 100, a display 101, a tactile vibration mechanism 102, and a processor memory as a functional component. Here, the function configuration unit (processor memory) realizes its function by executing a program that causes the computer mounted on the portable information device 1 to function. Further, as shown in FIG. 6, the portable information device 1 preferably further includes a pressing force detection unit 103.

  The display 101 displays an object (such as a menu plate) to be displayed on the screen. The touch panel 100 is arranged on the screen of the display 101, sequentially detects the contact position of the user's finger as time passes, and outputs the contact position information to the contact position determination unit 122 described later. As this touch panel 100, for example, a projected capacitive touch panel, a surface capacitive touch panel, a resistive touch panel, an ultrasonic surface acoustic wave touch panel, an infrared scanning touch panel, or the like can be employed.

  The tactile vibration mechanism unit 102 applies tactile vibration to the finger touching the touch panel 100 by vibrating the touch panel 100. The tactile vibration mechanism 102 may be a piezoelectric actuator formed using a piezoelectric material such as PZT (lead zirconate titanate).

Pressing force detection unit 103 detects the pressing force _{p c} applied to the touch panel 100 by the finger. Pressing force detection unit 103 is, for example, are placed under the four corners of the touch panel 100, the touch panel 100 flexed pressed against the finger the total pressing on itself, is detected as the pressing force p _c. The pressing force signal output from the pressing force detection unit 103 is input to a pressing force measurement unit 123 described later. The pressing force detection unit 103 can be, for example, a piezoelectric sensor formed using a piezoelectric material such as PZT. In addition to or in place of the tactile vibration mechanism unit 102 formed of a piezoelectric actuator, the pressing force detection unit 103 can be used as a tactile vibration mechanism unit.

  Similarly, according to FIG. 6, the functional configuration unit (processor memory) includes a depth amount assigning unit 120, a height amount assigning unit 121, a contact position determining unit 122, a depth / step difference obtaining unit 124, and a tactile vibration control unit. 125, a display control unit 110, and an application processing unit 111. Further, as shown in FIG. 6, it is preferable that the functional configuration unit (processor memory) further includes a pressing force measurement unit 123.

  The depth amount assigning unit 120 displays a screen from a reference point in the virtual space set in the screen to each position on each object (such as a menu plate) to which a tactile target image (such as an icon) is attached. A depth amount S which is a distance in the depth direction is assigned. Further, this allocation information is sequentially output to the depth / step difference acquisition unit 124.

  The height allocation unit 121 allocates a level corresponding to the tactile target image to each position on the tactile target image (such as an icon). Further, this allocation information is sequentially output to the depth / step difference acquisition unit 124.

  The contact position determination unit 122 inputs a contact position signal from the touch panel 100. Further, the tactile target image and object information are input from the application processing unit 111. Next, the contact position determination unit 122 calculates the contact corresponding position 107 from the finger contact position 106 based on these signals and information. Further, it is determined whether or not the contact corresponding position 107 is superimposed on the display position range of the tactile object image. Next, the determination result and information on the finger contact position 106 (contact corresponding position 107) are sequentially output to the depth / step acquisition unit 124.

The pressing force measurement unit 123 inputs the pressing force signal from the pressing force detection unit 103. Then, to calculate the magnitude of the pressing force p _c with the finger from the signal, sequentially outputting a pressure p _c value information in the depth-step acquisition unit 124.

  The depth / step acquisition unit 124 inputs information signals from the depth amount assignment unit 120, the height amount assignment unit 121, the contact position determination unit 122, and the pressing force measurement unit 123. Next, based on these pieces of information, when the contact corresponding position 107 is superimposed on the display position range of the tactile target image, the depth amount S assigned to the contact corresponding position 107 and the tactile sense corresponding to the contact corresponding position 107 are displayed. The height M assigned to the position on the target image is acquired. Subsequently, the acquired depth amount S and height amount M are sequentially output to the haptic vibration control unit 125.

Here, the first object (first menu plate 104a) and the second object (second menu plate 104b) as shown in FIGS. 4A to 4C are displayed, and the contact corresponding position is Consider a case where the display position range of the tactile target image of the first object is not superimposed on the display position range of the tactile target image (icon 105g) of the second object. At this time, the depth-step acquisition unit 124, the amount of depth _{S 2} assigned to the contact position corresponding 107b of the second object is assigned to the position of the tactile target image corresponding to the contact position corresponding 107b (icon 105 g) to get the high and low amount of M ₂ was.

  The tactile vibration control unit 125 includes a height change calculation unit 125a and a vibration intensity determination unit 125b. The height change calculation unit 125a sequentially calculates the change ΔM associated with the movement of the finger contact position 106 in the acquired height M. Further, the vibration intensity determination unit 125b determines the vibration intensity I according to the acquired depth amount S and the calculated height change ΔM.

Next, the haptic vibration control unit 125 instructs the haptic vibration mechanism unit 102 to apply the haptic vibration having the determined vibration intensity I to the finger via the touch panel 100. Further, as another embodiment, the haptic vibration control unit 125 applies tactile vibration according to the acquired pressing force _pc value to the finger in addition to the acquired depth amount S and the height change ΔM. It is also preferable that

  The display control unit 110 inputs application processing information from the application processing unit 111 and causes the display 101 to display an image corresponding to the execution of the application. The application processing unit 111 further causes the display control unit 110 to display an object (such as a menu plate) to which a tactile target image (such as an icon) is attached.

  FIG. 7 is a flowchart showing an embodiment of a tactile vibration applying method according to the present invention.

(S700) An object (such as a menu plate) including a tactile target image (such as an icon) is displayed.
(S701) A depth amount S is assigned to each position on the object (including on the tactile target image).
(S702) A high and low amount M is assigned to each position of the tactile object image.

(S710) A tactile vibration application loop (steps S710 to S722) is started. Here, the start time t is set to zero, the sampling time interval of the finger contact position 106 is set to Δt, and the initial value of the parameter i of the number of times of sampling (elapsed time) is set to zero. Further, the height M ₀ at time t = ₀ is set to zero. Further, the height change threshold M _th is set to a predetermined value.

(S711) The parameter i is incremented by 1.
(S712) Wait until time t reaches Δt × i.
(S713) The finger contact position 106 is measured.
(S714) A contact corresponding position 107 corresponding to the measured contact position 106 is calculated.

(S715) It is determined whether or not the contact corresponding position 107 is superimposed on a tactile object image (such as an icon). Here, when a false determination is made, it is assumed that the finger is not in contact with the tactile target image at the present time and tactile vibration is not activated, and the process proceeds to step S722.
(S716, S717) On the other hand, when the true determination is made in step S715, the depth amount S _i and the height amount M _i assigned to the contact corresponding position 107 are acquired assuming that the finger is in contact with the tactile object image. .

(S718) A height change ΔM _i (= M _i −M _i−1 ) is calculated from the acquired heights M _i and M _i−1 .
(S719) It is determined whether or not the calculated height change ΔM _i is greater than or equal to a predetermined height change threshold M _th (ΔM _i ≧ M _th ). Here, when a false determination is made, the process proceeds to step S722 on the assumption that the tactile vibration is not activated instead of the level (thickness, step, unevenness) that can be captured as a tactile sense.

(S720) On the other hand, when the true determination is made in step S719, the vibration intensity I _i is calculated from the acquired and calculated depth amount S _i and the height change ΔM _i . In this calculation, it is also preferable to use the above-described formula (1) I _i = C · | ΔM _i | / S _i .
(S721) A tactile vibration having the calculated vibration intensity I _i is applied to the finger.

(S722) One loop of the tactile vibration imparting loop is terminated, the process proceeds to step S710, and if the finger is in contact with the touch panel 100, the tactile vibration imparting loop (steps S710 to S722) is repeated. On the other hand, if the finger is separated from the touch panel 100, it is determined that the operation with the finger is finished, and the tactile vibration imparting method is completed.

  FIG. 8 is a flowchart showing another embodiment of the tactile vibration imparting method according to the present invention.

The embodiment shown in FIG. 8, in the flowchart shown in FIG. 7, between step S719 and step S720, insert the pressure _{p c} measuring step S720a ', further, the step S720, the pressing force _{p c} Step S720b ′ is also taken into consideration.

(S720a ') for measuring the pressing force _{p c} with the finger.
(S720b ′) When the true determination is made in step S719, the vibration intensity I _i is calculated from the acquired and calculated depth amount S _i, height change ΔM _i , and the measured pressing force _pc value. During this calculation, the above-mentioned equation _{(2) I i = C '} · p c · | it is also preferable to use a / _{S i} | ΔM _i.

Thus, by applying a tactile vibration having a vibration intensity in consideration of the pressing force p _c with the finger, to achieve a realistic tactile varies depending on the degree of pushing the finger when tracing the three-dimensional object in the actual space Can do.

  As described above in detail, according to the user interface device, the haptic vibration imparting method, and the program of the present invention, the depth S and the height M allocated to the object displayed on the screen and the haptic target image are determined. The vibration intensity I is determined. As a result, three-dimensional information such as depth and height included in the object and the tactile target image can be given to the finger that performs the operation.

  It should be noted that all the embodiments described above are merely illustrative of the present invention and are not intended to be limiting, and the present invention can be implemented in other various modifications and changes. Therefore, the scope of the present invention is defined only by the claims and their equivalents.

1 Portable information device (user interface device)
DESCRIPTION OF SYMBOLS 100 Touch panel 100a Touch panel surface 101 Display 102 Tactile vibration mechanism part 103 Pressing force detection part 104, 104a, 104b Menu plate (object)
105a, 105b, 105c, 105d, 105e, 105f, 105g Icon (tactile object image)
106 Contact Position 107 Contact Corresponding Position 110 Display Control Unit 111 Application Processing Unit 120 Depth Amount Allocation Unit 121 High / Low Amount Allocation Unit 122 Contact Position Determination Unit 123 Pressing Force Measurement Unit 124 Depth / Level Step Acquisition Unit 125 Tactile Vibration Control Unit 2 Stair Object 200 , 201, 210, 211 Polygon

According to the present invention, there is provided a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a contact position of a finger,
The image display unit may be an object to be displayed on installed in the virtual space is set to the screen, the position on the object that haptic object image as a haptic object is attached, reference in the virtual space position and depth amount is allocated corresponding to the distance to the position on the object from the position of the tactile target image, and displays the objects that have high and low weight is assigned for this haptic object image,
This user interface device
A tactile vibration mechanism that imparts tactile vibration that brings tactile sensation to a finger that touches the touch panel;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to this contact corresponding position and the tactile sense corresponding to this contact corresponding position Depth / high / low acquisition means for acquiring the high / low amount assigned to the position on the target image;
There is provided a user interface device including tactile vibration control means for instructing a tactile vibration mechanism unit to apply a tactile vibration corresponding to the acquired depth amount and height amount to a finger via a touch panel. Here, the tactile vibration control means applies tactile vibration to the finger via the touch panel according to the acquired depth amount and the acquired amount of change in height according to the movement of the finger contact position. It is also preferable to instruct the tactile vibration mechanism portion as much as possible.

As one embodiment of the user interface device according to the present invention,
The first object and the second object are at least partially overlapped when viewed from the reference position side, and the first object is positioned in front of the first object and the second object. Display of the second tactile target image affixed to the second object without overlapping the contact corresponding position within the superimposition region with the display position range of the first tactile target image affixed to the first object When overlapping with the position range,
Depth / high / low acquisition means
It is also preferable to acquire the depth amount assigned to the contact corresponding position of the second object and the height amount assigned to the position on the second tactile target image corresponding to the contact corresponding position.

Further, as another embodiment of a user interface device according to the present invention further comprises a pressure detector for detecting the pressing force applied to the touch panel by a finger, tactile vibration control means, acquired pressed pressure value preferred also Turkey is assigned to the finger tactile vibration in response to. In this case, it is also preferable that the haptic vibration control means determines the intensity of the haptic vibration to be a monotonically increasing function with respect to the pressing force value. Furthermore, according to the user interface device according to the present invention, as viewed from the user operation, the finger moves while touching and crosses the tactile target image pasted on the object displayed on the screen of the image display unit. When the finger is traced as described above, it is preferable that the finger is given a tactile vibration having a smaller strength as the tactile object image is located in the virtual space.

According to the present invention, there is also provided a user interface device including an image display unit that displays a display target on a screen, and a touch panel that sequentially detects a finger contact position,
  The image display unit is an object that is a display target installed in a virtual space set in the screen, and a tactile target image to be a tactile target is pasted, and the tactile target image is positioned at a position on the tactile target image. Display objects with high and low amounts corresponding to the target image,
  This user interface device
  A tactile vibration mechanism that imparts tactile vibration that brings tactile sensation to a finger that touches the touch panel;
  A pressure detection unit that detects pressure applied to the touch panel by a finger;
  When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the height assigned to the position on the tactile target image corresponding to this touch corresponding position is High and low acquisition means to acquire;
  Tactile vibration control means for instructing the tactile vibration mechanism unit to apply a tactile vibration corresponding to the acquired height and pressure value to the finger via the touch panel;
A user interface device is provided.

According to the present invention, there is further provided a program installed in a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a finger contact position ,
The user interface device includes a tactile vibration mechanism that applies tactile vibration that provides tactile sensation to the finger that touches the touch panel.
The image display unit may be an object to be displayed on installed in the virtual space is set to the screen, the position on the object that haptic object image as a haptic object is attached, reference in the virtual space position and depth amount is allocated corresponding to the distance to the position on the object from the position of the tactile target image, and displays the objects that have high and low weight is assigned for this haptic object image,
This program
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to this contact corresponding position and the tactile sense corresponding to this contact corresponding position Depth / high / low acquisition means for acquiring the high / low amount assigned to the position on the target image;
A program for a user interface device that causes a computer to function as tactile vibration control means for instructing a tactile vibration mechanism unit to apply a tactile vibration according to the acquired depth amount and height amount to a finger via a touch panel. Provided.

According to the present invention, there is further provided a tactile vibration imparting method in a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a contact position of a finger ,
The user interface device includes a tactile vibration mechanism that applies tactile vibration that provides tactile sensation to a finger that touches the touch panel.
The image display unit may be an object to be displayed on installed in the virtual space is set to the screen, the position on the object that haptic object image as a haptic object is attached, reference in the virtual space position and depth amount is allocated corresponding to the distance to the position on the object from the position of the tactile target image, and displays the objects that have high and low weight is assigned for this haptic object image,
This tactile vibration application method is:
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to this contact corresponding position and the tactile sense corresponding to this contact corresponding position and Luz step to obtain the high and low amount allocated to a position on the target image,
A tactile vibration in response to the acquired amount of depth and height amount, tactile vibrating method and a Luz step instructs the haptic vibration mechanism to impart through the touch panel with respect to a finger is provided.

Claims (8)

A user interface device including an image display unit that displays a display target on a screen, and a touch panel that sequentially detects a contact position of a finger,
A tactile vibration mechanism that imparts tactile vibration that provides tactile sensation to the finger that has touched the touch panel;
An object that is a display target installed in a virtual space set in the screen, and each position on the object to which a tactile target image to be touched is pasted from a reference point in the virtual space. A depth amount assigning means for assigning a depth amount which is a distance to the position;
High and low amount assigning means for assigning a high and low amount corresponding to the tactile target image to each position on the tactile target image;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to the contact corresponding position and the contact corresponding position Depth / height acquisition means for acquiring a height amount assigned to a position on the tactile object image
The tactile sensation to apply to the finger via the touch panel the tactile vibration according to the acquired depth amount and the acquired amount of change in height according to the movement of the contact position of the finger. A tactile vibration control means for instructing the vibration mechanism unit. The first object and the second object are at least partially overlapped when viewed from the reference point side, and the first object is positioned in front of the first object and the second object. The contact-corresponding position in the overlapping area with the object does not overlap the display position range of the first tactile target image attached to the first object, and the second object attached to the second object. When overlapping with the display position range of the tactile target image of 2
The depth / height acquisition means is:
The depth amount assigned to the contact corresponding position of the second object and the height amount assigned to the position on the second tactile target image corresponding to the contact corresponding position are acquired. Item 4. The user interface device according to Item 1.   The tactile vibration control means determines the intensity of the tactile vibration to be a monotonically decreasing function for the acquired depth amount and to be a monotonically increasing function for the acquired amount of change in height. Item 3. The user interface device according to Item 1 or 2. A pressing force detection unit for detecting a pressing force applied to the touch panel by the finger;
The haptic vibration control means causes the haptic vibration corresponding to the acquired depth amount, the amount of change in height and the pressing force value to be applied to the finger. The user interface device according to any one of the above.   The user interface device according to claim 4, wherein the haptic vibration control means determines the intensity of the haptic vibration so as to be a monotonically increasing function for the pressing force value. When viewed from the user's operation, the finger moves while touching, and the finger is traced across the tactile target image pasted on the object displayed on the screen of the image display unit. 6. The user according to claim 1, wherein a tactile vibration having a smaller intensity is applied as the tactile object image is located further in the virtual space. Interface device. A program installed in a user interface device that includes an image display unit that displays a display target on a screen and a touch panel that sequentially detects a contact position of a finger, the user interface device comprising:
A tactile vibration mechanism that provides tactile vibration that provides tactile sensation to the finger touching the touch panel;
An object that is a display target installed in a virtual space set in the screen, and each position on the object to which a tactile target image to be touched is pasted from a reference point in the virtual space. A depth amount assigning means for assigning a depth amount which is a distance to the position;
High and low amount assigning means for assigning a high and low amount corresponding to the tactile target image to each position on the tactile target image;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to the contact corresponding position and the contact corresponding position Depth / height acquisition means for acquiring a height amount assigned to a position on the tactile object image
The tactile sensation to apply to the finger via the touch panel the tactile vibration according to the acquired depth amount and the acquired amount of change in height according to the movement of the contact position of the finger. A program for a user interface device, which causes a computer to function as tactile vibration control means for instructing a vibration mechanism unit. A tactile vibration imparting method in a user interface device including an image display unit that displays a display target on a screen and a touch panel that sequentially detects a contact position of a finger, the user interface device comprising:
A tactile vibration mechanism that provides tactile vibration that provides tactile sensation to the finger that has touched the touch panel, and the tactile vibration imparting method includes:
An object that is a display target installed in a virtual space set in the screen, and each position on the object to which a tactile target image to be touched is pasted from a reference point in the virtual space. A first step of assigning a depth amount which is a distance to the position;
A second step of assigning a height corresponding to the tactile target image to each position on the tactile target image;
When the contact corresponding position, which is the position on the object corresponding to the contact position of the finger, is superimposed on the display position range of the tactile target image, the depth amount assigned to the contact corresponding position and the contact corresponding position A third step of obtaining a height amount assigned to a position on the tactile object image to be
The tactile sensation to apply to the finger via the touch panel the tactile vibration according to the acquired depth amount and the acquired amount of change in height according to the movement of the contact position of the finger. A tactile vibration imparting method comprising: a fourth step for instructing the vibration mechanism unit.

Images