WO2011128959A1 - Dispositif de génération d'information de commande de sensation tactile - Google Patents

Dispositif de génération d'information de commande de sensation tactile Download PDF

Info

Publication number
WO2011128959A1
WO2011128959A1 PCT/JP2010/007604 JP2010007604W WO2011128959A1 WO 2011128959 A1 WO2011128959 A1 WO 2011128959A1 JP 2010007604 W JP2010007604 W JP 2010007604W WO 2011128959 A1 WO2011128959 A1 WO 2011128959A1
Authority
WO
WIPO (PCT)
Prior art keywords
gui
control information
gui element
acquired
touch panel
Prior art date
Application number
PCT/JP2010/007604
Other languages
English (en)
Japanese (ja)
Inventor
光洋 阿曽
秦　秀彦
雄基 篠本
紹二 大坪
Original Assignee
パナソニック株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニック株式会社 filed Critical パナソニック株式会社
Publication of WO2011128959A1 publication Critical patent/WO2011128959A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1626—Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1643—Details related to the display arrangement, including those related to the mounting of the display in the housing the display being associated to a digitizer, e.g. laptops that can be used as penpads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G06F3/04855—Interaction with scrollbars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text

Definitions

  • the present invention relates to an information terminal having a touch panel display that operates a graphical user interface (GUI) displayed on a display screen by a touch operation, and simulates the type and operation of the GUI to the fingertip of the user who operates the GUI.
  • GUI graphical user interface
  • the present invention relates to a technique for making GUI operation on a touch panel without unevenness more intuitive and easy to use by reproducing as typical tactile information.
  • an information terminal equipped with a touch panel display such as a PDA (Personal Digital Assistant), a bank ATM (Automatic Teller Machine), or a mobile phone
  • a touch panel display such as a PDA (Personal Digital Assistant), a bank ATM (Automatic Teller Machine), or a mobile phone
  • the user can directly operate the GUI on the screen with a fingertip. Therefore, it is possible to operate the GUI more intuitively than a mouse or keyboard that indirectly operates the GUI.
  • the information terminal is provided with buttons for volume adjustment with unevenness and buttons for music selection operation, so that the user can adjust the volume and perform music selection operation by groping while the music player is in a pocket or bag.
  • buttons for volume adjustment with unevenness and buttons for music selection operation, so that the user can adjust the volume and perform music selection operation by groping while the music player is in a pocket or bag.
  • the touch panel type display unless the GUI is visually observed, it is difficult to determine the display position and display area of the GUI for adjusting the sound volume and the GUI for music selection operation. Met.
  • Non-Patent Document 1 there is a technique for making a fine texture on the surface, such as an object image displayed on a display, or a feeling of roughness as a pseudo tactile sensation on the fingertip of the user who touched the dill play.
  • a technique for making a fine texture on the surface such as an object image displayed on a display, or a feeling of roughness as a pseudo tactile sensation on the fingertip of the user who touched the dill play
  • Non-Patent Document 1 by realizing a display that reproduces a tactile sense, a user can feel the texture of an object displayed on the display with a fingertip. It was not disclosed to apply to the operation of.
  • the present invention solves the above-described conventional problems, and reproduces a pseudo tactile sensation on the user's fingertip in accordance with the characteristics of the user's operation target GUI in an information terminal provided with a touch panel. It is an object of the present invention to provide a tactile control information generation device that can inform a user of the presence and characteristics of a user and realize an intuitive and easy-to-use GUI operation.
  • An object of the present invention is directed to a haptic control information generation device that controls a haptic reproduction device that reproduces a haptic sensation on a part of a user's body that operates a touch panel display.
  • the haptic control information generation apparatus of the present invention includes a GUI attribute management table that holds attribute values of GUI elements displayed on the touch panel display, and a GUI attribute that manages the GUI attribute management table.
  • a drawing processing unit that acquires an attribute value of a GUI element to be drawn from the management unit and the GUI attribute management unit, draws the GUI element based on the acquired attribute value, and outputs the drawn GUI element to a touch panel display
  • the input information from the touch panel display notifies the drawing processing unit of the acquired input information, and instructs the drawing processing unit to draw a GUI element, and the input information acquired by the input processing unit.
  • the attribute value of the GUI element determined as the operation target by the operation target extraction unit and the operation target extraction unit is acquired from the GUI attribute management unit, and the acquired attribute value of the GUI element and the input information acquired by the input acquisition unit From the above, when a part of the body of the user who operates the touch panel display touches the GUI element displayed on the touch panel display, the tactile reproduction device is configured to reproduce the tactile sensation on the part of the user's body.
  • a control information generation unit that generates control information to be instructed.
  • the haptic control information generation device of the present invention it is possible to reproduce a haptic according to the characteristics of the GUI on the fingertip of the user who operates the touch panel. Thereby, it is possible to provide an intuitive and easy-to-use GUI operation for a user who traces the GUI on the touch panel.
  • FIG. 1 is a diagram illustrating an example of a configuration of an information terminal 100 including a haptic control information generation device according to the first embodiment of the present invention.
  • FIG. 2A is a diagram showing an overlapping relationship between the touch panel 1011 and the display screen 200 according to the first embodiment of the present invention.
  • FIG. 2B is a diagram showing an example of a GUI element displayed on the display screen 200 according to the first embodiment of the present invention.
  • FIG. 3A is a diagram showing an example of the GUI attribute management table 108 according to the first embodiment of the present invention.
  • FIG. 3B is a diagram showing an example of the GUI attribute management table 108b according to the first embodiment of the present invention.
  • FIG. 4 is a flowchart showing an example of processing of the information terminal 100 according to the first embodiment of the present invention.
  • FIG. 5A is a diagram illustrating a correspondence relationship between GUI elements and frictional forces according to the first embodiment of the present invention.
  • FIG. 5B is a diagram illustrating a correspondence relationship between GUI elements and frictional forces according to the first embodiment of the present invention.
  • FIG. 5C is a diagram illustrating a correspondence relationship between GUI elements and frictional forces according to the first embodiment of the present invention.
  • FIG. 6 is a diagram illustrating an example of the relationship between the GUI element and the frictional force according to the second embodiment of the present invention.
  • FIG. 7 is a diagram illustrating an example of the relationship between the GUI element and the frictional force according to the second embodiment of the present invention.
  • FIG. 5A is a diagram illustrating a correspondence relationship between GUI elements and frictional forces according to the first embodiment of the present invention.
  • FIG. 5B is a diagram illustrating a correspondence relationship between GUI elements and frictional forces
  • FIG. 8 is a diagram illustrating an example of the relationship between the GUI element and the frictional force according to the second embodiment of the present invention.
  • FIG. 9 is a diagram illustrating an example of table information indicating a correspondence relationship between the attribute value of the GUI element and the control information calculation method according to the second embodiment of the present invention.
  • FIG. 1 is a diagram illustrating an example of a configuration of an information terminal 100 including a haptic control information generation device according to the first embodiment of the present invention.
  • the information terminal 100 includes an input device 101, a display device 102, a haptic control information generation device, and a haptic reproduction device 109.
  • the display device 102 includes a display screen 200.
  • FIG. 2A is a diagram illustrating an overlapping relationship between the touch panel 1011 and the display screen 200 included in the display device 102.
  • FIG. 2B is a diagram illustrating an example of a GUI element displayed on the display screen 200 of the display device 102.
  • a structure in which the touch panel 1011 and the display screen 200 as illustrated in FIG. 2A are overlapped may be called a touch panel type display.
  • the input device 101 includes a touch panel 1011 in its configuration.
  • the touch panel 1011 is arranged so as to overlap the display screen 200 of the display device 102 as shown in FIG. 2A.
  • the input apparatus 101 uses the touch position of the user's fingertip on the touch panel 1011 as the coordinate value Pin (Xin, Yin) of the XY coordinate system to the input processing unit 103.
  • the XY coordinate system defines the upper left end of the touch panel 1011 as the origin, the right direction as the positive direction of the X axis, and the downward direction as the positive direction of the Y axis.
  • the touch panel 1011 coincides with the xy axes on the display screen 200 shown in FIG. 2B.
  • a plurality of GUI elements 201 to 208 are displayed on the display screen 200 shown in FIG. 2B.
  • the touch panel 1011 is touched by the user's fingertip, but is not limited to the fingertip and may be a part of the user's body.
  • the tactile control information generation device includes an input processing unit 103, a drawing processing unit 104, a control information generation unit 105, an operation target extraction unit 106, a GUI attribute management unit 107, and a GUI attribute management table 108.
  • the haptic control information generation device may further include a haptic reproduction device 109 described later.
  • the input processing unit 103 receives the touch position Pin (Xin, Yin) of the user's finger from the input device 101 as input information
  • the input processing unit 103 receives the drawing processing unit 104 and the control information generation unit 105. Then, the contact position Pin (Xin, Yin) is notified and the start of each process is instructed.
  • the drawing processing unit 104 displays the display from the GUI attribute management table 108 managed by the GUI attribute management unit 107 via the GUI attribute management unit 107.
  • the attribute value of the GUI element to be displayed on the device 102 is acquired.
  • the drawing processing unit 104 draws a GUI on the display screen 200 of the display device 102 according to the acquired attribute value of the GUI element. Details of the GUI attribute management table 108 will be described later.
  • the control information generation unit 105 When the contact position Pin (Xin, Yin) is notified, the control information generation unit 105 notifies the operation target extraction unit 106 of an acquisition request for a GUI element to be operated. In addition, the control information generation unit 105 generates control information for the haptic reproduction device 109 to reproduce a haptic from the attribute value of the GUI element extracted by the operation target extraction unit 106 and the input information acquired by the input processing unit 103. Generate. In addition, the control information generation unit 105 notifies the tactile reproduction device 109 of the generated control information, thereby instructing the tactile reproduction device 109 to reproduce the tactile sensation.
  • the tactile reproduction device 109 reproduces a pseudo tactile sensation on the fingertip of the user who touched the touch panel 1011 according to the control information notified from the control information generation unit 105.
  • the frictional force generated on the contact surface between the user's fingertip and the touch panel or the strength of the propulsive force can be designated as control information to the haptic reproduction device 109 for reproducing the haptics. Shall.
  • the frictional force is a force generated in a direction opposite to the tracing direction when the user performs a tracing operation on the touch panel.
  • the propulsive force is a force that moves the user's fingertip in contact with the touch panel in an arbitrary direction. Therefore, the control information generation unit 105 rubs the direction indicating the direction opposite to the user's drag direction (for example, the direction of the unit vector with respect to the XY axes on the touch panel 1101) as the control information, along with the strength of the frictional force.
  • the tactile playback device 109 is notified of the force generation direction.
  • the direction in which the frictional force is generated is changed from the current contact position Pin to the previous contact position by the control information generation unit 105 storing the contact position notified from the input processing unit 103 during the previous process. It can be calculated as a heading unit vector. Moreover, what is necessary is just to invert the sign of said unit vector, when generating a driving force. In the following description, a tactile reproduction device 109 that generates a frictional force will be described as an example.
  • the operation target extraction unit 106 acquires all the GUI elements existing on the contact position Pin (Xin, Yin) from the GUI attribute management unit 107, and the overlapping order is the highest.
  • a GUI element is extracted as an operation target.
  • the highest-level GUI element is a GUI element that is displayed on the forefront (user side) on the display screen when the GUI elements are displayed on the display screen in an overlapping manner.
  • the GUI element is a component of the GUI such as a button, an image, text, or a scroll bar.
  • the GUI attribute management unit 107 manages the GUI attribute management table 108.
  • the GUI attribute management unit 107 receives a GUI element acquisition request, and extracts a GUI element that matches the condition of the acquisition request from the GUI attribute management table 108.
  • the attribute value of the GUI element managed by the GUI attribute management table 108 includes the element ID, the overlapping order value indicating the overlapping relationship between the GUI elements, the element type indicating the type of the GUI element, the display position of the GUI element, and the display Defines size, background type, and supplementary information. Next, details of each attribute value will be described in order.
  • Element ID is an ID uniquely determined for each GUI element.
  • GUI elements corresponding to element ID 3201 to element ID 3208 are defined.
  • the overlapping order value is the order in which the GUI is displayed, and is defined so that the larger the overlapping order value, the closer to the front (user side) is displayed on the display screen. Note that GUI elements with the same overlap order value are shown in a non-overlapping state.
  • the element type of the GUI element indicates a display or behavior characteristic of the GUI element such as a button, a scroll bar, an image, or a container.
  • the display position (x, y) of the GUI element is represented by the coordinate value of the xy coordinate system on the display screen 200 of the display device 102, and is the coordinate value of the upper left point of the GUI element.
  • the display size represents the width and height of the GUI element.
  • the background type of the GUI element indicates how the background of the GUI element is drawn, such as whether it is painted with a single color, painted as a gradation, or whether a specified image is displayed. Information is defined. Further, the supplemental information of the GUI element is information defined individually for each GUI element depending on the element type or background type of the GUI element.
  • GUI element ID 3201 in FIG. 3A “left / right scroll bar” is defined as the element type.
  • element ID 3207 “up / down scroll bar” is defined as the element type.
  • the element ID 3201 and the element ID 3207 define “scroll position, scroll direction” as supplementary information.
  • buttons is defined as the element type.
  • the element ID 3205 and the element ID 3206 define “gradation” as the background type, the start color and end color of the gradation are defined as supplementary information.
  • GUI element for displaying an image
  • GUI element ID 3202 or 3203 in which the background type is designated as an image
  • image for acquiring an image to be displayed on the background is used as supplementary information.
  • Data name "is defined.
  • the operation target extraction unit 106 has the largest overlap order value among all the GUI elements that include the contact position Pin (Xin, Yin) in the display area of the GUI element. Extract an element as an operation target.
  • GUI elements 201 to 208 in FIG. 2B represent the result of the rendering processing unit 104 displaying the element IDs 3201 to 3208 in the GUI attribute management table 108 in FIG. 3A on the display device 102.
  • the GUI element with the element ID 3201 represents the left / right scroll bar 201 in FIG. 2B.
  • the GUI elements 202 to 208 in FIG. 2B are display examples.
  • the operation target extraction unit 106 may determine a GUI element to be extracted as an operation target by an operation target availability flag for determining whether the attribute of the GUI element can be an operation target.
  • an operation target propriety flag indicating whether “operation is possible” or “operation is not possible” is set.
  • the operation target extraction unit 106 refers to the GUI attribute management table 108b, can determine that the operation target availability flag is “operation possible”, and extracts the GUI element with the highest overlapping order as the GUI element to be operated. To do.
  • FIG. 4 is a flowchart showing an example of processing of the information terminal 100 according to the first embodiment of the present invention.
  • step S ⁇ b> 401 when input device 101 detects that the user touches touch panel 1011, input processing unit 103 causes touch position Pin (Xin, Yin) of the user's fingertip on touch panel 1011 from input device 101. ) And notifies the drawing processing unit 104 and the control information generation unit 105 of the received contact position Pin (YES in step S401).
  • the contact position Pin 530, 130
  • step S ⁇ b> 402 the drawing processing unit 104 acquires the attribute value of the GUI element to be displayed on the display device 102 from the GUI attribute management table 108 managed by the GUI attribute management unit 107 via the GUI attribute management unit 107. Then, the GUI is drawn on the display screen 200 of the display device 102.
  • the GUI element drawn on the display screen 200 is assumed to be FIG. 2B as described above.
  • step S403 the control information generation unit 105 notifies the contact position Pin (530, 130) notified from the input processing unit 103, and extracts the operation target GUI element so as to extract the operation target GUI element. Is notified of the operation target extraction instruction.
  • step S404 the operation target extraction unit 106 extracts a GUI element including the contact position Pin (530, 130) in the display area from the GUI attribute management table managed by the GUI attribute management unit 107.
  • the operation target extraction unit 106 extracts the element ID 3205 and the element ID 3208.
  • the operation target extraction unit 106 extracts, from the element ID 3205 and the element ID 3208, the GUI element of the element ID 3205 having the higher overlapping order value as the operation target.
  • step S ⁇ b> 405 the control information generation unit 105 generates control information from the attribute value of the GUI element extracted by the operation target extraction unit 106 and the input information acquired by the input acquisition unit 103. Notify the playback device 109.
  • step S406 the tactile reproduction device 109 generates a frictional force as a tactile sensation on the touch panel 1011 in contact with the user's finger according to the control information notified from the control information generating unit 105.
  • control information generated by the control information generation unit 105 in step S405 will be described with reference to FIGS. 5A to 5C.
  • 5A to 5C show a correspondence relationship between the scroll position X of the scroll bar element (for example, the GUI element 201 in FIG. 2B) and the frictional force F.
  • FIG. As shown in FIGS. 5A to 5C, the control information generation unit 105 dynamically changes the frictional force F according to the scroll position X of the scroll bar element.
  • the graph of FIG. 5A shows a state in which the center of the scroll bar element is the minimum value of the friction force F, and the friction force F gradually increases toward the left end and the right end.
  • the control information generation unit 105 can calculate the control information from the contact position Pin by holding such a relational expression. Thereby, the user can sense the scroll position with the fingertip by the change of the frictional force.
  • the control information generation unit 105 calculates the friction force F by holding the correspondence relationship between the friction force F and the scroll position X as table information instead of holding the relational expression as described above. May be.
  • the control information generation unit 105 may set the center of the scroll bar element as the maximum value of the frictional force and gradually decrease the frictional force F toward the left end and the right end.
  • the control information generation unit 105 sets the left end or the right end as the minimum value of the frictional force and gradually increases the frictional force F toward the right end or the left end. Good.
  • the frictional force can be changed according to the slider position in the same manner as the scroll bar element.
  • the slider element showing the frictional force as described above can be applied not only to a screen scroll operation as shown in FIG. 2B but also to an operation such as volume adjustment.
  • the control information generation unit 105 may change the frictional force in accordance with the volume level reproduced by the information terminal 100.
  • Non-Patent Document 1 For the basic principle of presenting a tactile sensation to the user in the tactile reproduction device 109, for example, the technique disclosed in Non-Patent Document 1 can be applied.
  • Non-Patent Document 1 the deformation on the skin surface of the fingertip that occurs when the user traces the object surface with a fine roughness at a velocity v using the fingertip is expressed as a superposition formula of sinusoidal traveling waves.
  • this sine traveling wave superposition formula and a similar sine traveling wave in Non-Patent Document 1, an elastic wave called a Lamb wave is used) are intentionally generated and sent to the user's fingertip. Tell.
  • deformation of the skin surface similar to when a rough object is traced occurs at the fingertip of the user who has detected the elastic wave, and the user feels a tactile sensation as if he / she traced the rough object.
  • the haptic reproduction device 109 converts the strength of the frictional force notified from the control information generation unit 105 into a rough feeling, and copes with the rough feeling.
  • a tactile sensation is given to the user's fingertip that has touched the touch panel 1011.
  • the traveling direction of the elastic wave is a direction defined by the unit vector notified from the control information generation unit 105.
  • the tactile reproduction device 109 holds, as an index for defining a rough object surface, a correspondence relationship among the friction coefficient ⁇ of the object surface, the spatial frequency k of the elastic wave, and the time frequency ⁇ as table information in advance. It may be left.
  • a large friction coefficient is associated with an object having a large roughness
  • a small friction coefficient is associated with an object having a small roughness.
  • the haptic control information generation device As described above, according to the haptic control information generation device according to the first embodiment of the present invention, it is possible to reproduce a haptic according to the feature of the GUI on the fingertip of the user who operates the touch panel. Thereby, an intuitive GUI operation can be provided to a user who performs a tracing operation on the GUI on the touch panel.
  • FIG. 2B is used also about the GUI element displayed on the display screen 200.
  • FIG. 6 is a diagram for explaining a correspondence relationship between a tactile sensation reproduced on a user's fingertip and a GUI element in the second embodiment of the present invention.
  • FIG. 6 illustrates the relationship between the element ID 3205 (the enlargement button 205 in FIG. 2B) extracted by the operation target extraction unit 106, the touch position Pin of the touch panel by the user, and the strength of the frictional force generated at the touch position Pin. ing.
  • the frictional force F that gradually increases from the display center portion (point O) of the GUI element toward the display boundary direction, the frictional force F at the contact position Pin, and the contact position Pin from the display center portion. It is illustrated as a relationship with the distance D to.
  • the control information generation unit 105 can calculate the control information from the contact position Pin by holding such a relational expression.
  • the control information generation unit 105 may calculate the frictional force F from any relational expression other than those described above, and the correspondence information between the frictional force F and the distance D from the display center to the contact position is table information. As a result, the frictional force F may be calculated.
  • the control information generation unit 105 generates a frictional force F that gradually increases in the GUI element display boundary direction from the attribute value of the GUI element extracted by the operation target extraction unit 106 and the input information acquired by the input acquisition unit 103. Is generated and notified to the tactile playback device 109 as control information. Based on the notified frictional force F, the tactile reproduction device 109 generates a frictional force as a tactile sensation on the touch panel 1011 in contact with the user's finger. Thereby, the user who operates the touch panel can feel the display boundary of the GUI element with the fingertip even if the touch panel is a flat touch panel.
  • the direction in which the frictional force is generated may be reversed. This is because the frictional force F gradually changes toward the display boundary direction of the GUI element, so that the user can feel the display boundary of the GUI element with the fingertip.
  • control information generation unit 105 may generate the propulsive force.
  • the control information generation unit 105 can control to guide the user's fingertip to a specific GUI element by generating a driving force toward the center of the area of the specific GUI element. Further, the direction in which the propulsive force is generated may be reversed.
  • the control information generation unit 105 is based on the relationship between the movement distance length from the previous user contact position to the current contact position Pin and the elapsed time from the previous input.
  • the control information generation part 105 can also change a frictional force according to a user's tracing speed.
  • the user can perform the tracing operation of the GUI element too early, thereby suppressing the early operation on the GUI element.
  • the slider element controls the volume, it is possible to suppress a sudden increase in volume due to a quick slider operation by the user.
  • the control information generation unit 105 may determine whether to multiply the friction force by the tracing speed according to the element type of the GUI element. As this determination method, for example, “multiply by the tracing speed” is set to the supplementary information of the table information shown in FIG. 9 described later. The control information generation unit 105 can determine that the friction force is multiplied by the tracing speed by referring to the supplementary information of the table information.
  • the control information generation unit 105 may generate a luminance distribution from the background color of the GUI element, and may generate control information that increases the frictional force as the luminance of the acquired background color increases.
  • FIG. 7 shows an example in which a luminance distribution is generated from a GUI element whose background color is gradation. If the gradation is such that the background color is brighter from the center of the display area of the GUI element toward the periphery of the area, the luminance distribution is as shown in FIG. In this case, the control information generation unit 105 generates control information such that the frictional force of the GUI element increases from the center of the GUI element with low luminance toward the region boundary with high luminance.
  • the correspondence relationship between the luminance and the frictional force may be reversed, or a driving force may be generated instead of the frictional force.
  • FIG. 8 shows a state in which the center of the GUI element 209 is set to the minimum value of the frictional force, and the frictional force F gradually increases toward the GUI element 202 and the GUI element 203.
  • the user who operates the touch panel can estimate the positions of the buttons (GUI elements 202 and 203) arranged in pairs by the change in the frictional force of the GUI element 209 even in the case of a flat touch panel. it can.
  • the frictional force F generated on the GUI element 209 is not limited to the above example, and may change as shown in FIGS. 5A to 5C, for example.
  • the control information generation unit 105 may switch the control information calculation method according to the GUI element extracted by the operation target extraction unit 106.
  • FIG. 9 is a diagram illustrating an example of table information indicating a correspondence relationship between an attribute value of a GUI element and a control information calculation method.
  • FIG. 9 shows a control information calculation method and supplementary information corresponding to the element type and the background type. In the example shown in FIG. 9, it is described that the relational expressions described in FIGS. 5A, 5B, 6, and 7 described above are used as the control information calculation method.
  • the control information generation unit 105 refers to the table information in FIG. 9 and, if the element type of the GUI element is “scroll bar”, as shown in the graph of FIG. 5A. Control information is calculated using a relational expression. Further, if the element type of the GUI element is “slide bar”, the control information generation unit 105 says that the supplementary information is “multiply the tracing speed”, so the relational expression as shown in the graph of FIG. Control information is calculated by a relational expression multiplied by the tracing speed. In addition, if the element type of the GUI element is “button” and the background type is “gradation”, the control information generation unit 105, as described with reference to FIG. Generate control information that increases the frictional force of the elements.
  • Each processing procedure performed by the haptic control information generation device is predetermined program data that can execute the processing procedure described above stored in a storage device (ROM, RAM, hard disk, etc.). May be realized by being interpreted and executed by the CPU.
  • the program data may be introduced into the storage device via the storage medium, or may be directly executed from the storage medium.
  • the storage medium refers to a semiconductor memory such as a ROM, a RAM, and a flash memory, a magnetic disk memory such as a flexible disk and a hard disk, an optical disk memory such as a CD-ROM, a DVD, and a BD, and a memory card.
  • the storage medium is a concept including a communication medium such as a telephone line or a conveyance path.
  • each component of the haptic control information generation apparatus (FIG. 1) according to the embodiment of the present invention described above can be realized as an LSI which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include all or part of each. Although referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
  • the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor.
  • An FPGA Field Programmable Gate Array
  • a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.
  • the tactile control information generation device is useful for improving the operability of the GUI in an information terminal equipped with a touch panel display.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un dispositif de génération d'information de commande de sensation tactile qui réalise une opération d'interface utilisateur graphique qui est intuitive et facile à utiliser sur un écran tactile. Une unité de traitement d'entrée acquiert une information d'entrée provenant d'un affichage d'écran tactile, notifie l'information d'entrée acquise à une unité de traitement de dessin, et instruit également l'unité de traitement de dessin de dessiner un élément d'interface utilisateur graphique. Une unité d'extraction de cible d'opération détermine l'élément d'interface utilisateur graphique tactile qui est la cible d'opération de l'utilisateur, ladite détermination étant réalisée à partir de l'information d'entrée acquise par l'unité de traitement d'entrée et des valeurs d'attributs de l'élément d'interface utilisateur graphique qui sont gérés par une unité de gestion d'attributs d'interface utilisateur graphique. Une unité de génération d'information de commande acquiert à partir de l'unité de gestion d'attributs les valeurs d'attributs de l'élément d'interface utilisateur graphique déterminé comme étant la cible d'opération par l'unité d'extraction de cible d'opération. L'unité de génération d'information de commande génère ensuite, à partir des valeurs d'attributs d'élément d'interface utilisateur graphique et de l'information d'entrée acquise par l'unité d'acquisition d'entrée, une information de commande qui instruit un dispositif de reproduction de sensation tactile pour la reproduction d'une sensation tactile dans la partie du corps de l'utilisateur qui commande l'affichage d'écran tactile, lorsque ladite partie du corps de l'utilisateur touche l'élément d'interface utilisateur graphique affiché sur l'écran tactile.
PCT/JP2010/007604 2010-04-13 2010-12-28 Dispositif de génération d'information de commande de sensation tactile WO2011128959A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2010-091984 2010-04-13
JP2010091984 2010-04-13

Publications (1)

Publication Number Publication Date
WO2011128959A1 true WO2011128959A1 (fr) 2011-10-20

Family

ID=44798353

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2010/007604 WO2011128959A1 (fr) 2010-04-13 2010-12-28 Dispositif de génération d'information de commande de sensation tactile

Country Status (1)

Country Link
WO (1) WO2011128959A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013061605A1 (fr) * 2011-10-27 2013-05-02 京セラ株式会社 Dispositif de retour de force
JP2017189153A (ja) * 2016-04-15 2017-10-19 小橋工業株式会社 農作業機

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1185400A (ja) * 1997-09-11 1999-03-30 Sony Corp 表示装置
JP2005284416A (ja) * 2004-03-26 2005-10-13 Sony Corp 触覚機能付き入力装置、情報入力方法及び電子機器
JP2008516348A (ja) * 2004-10-08 2008-05-15 イマージョン コーポレーション タッチ式入力装置におけるボタンおよびスクロール動作シミュレーション用の触覚フィードバック

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1185400A (ja) * 1997-09-11 1999-03-30 Sony Corp 表示装置
JP2005284416A (ja) * 2004-03-26 2005-10-13 Sony Corp 触覚機能付き入力装置、情報入力方法及び電子機器
JP2008516348A (ja) * 2004-10-08 2008-05-15 イマージョン コーポレーション タッチ式入力装置におけるボタンおよびスクロール動作シミュレーション用の触覚フィードバック

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013061605A1 (fr) * 2011-10-27 2013-05-02 京セラ株式会社 Dispositif de retour de force
US20140320440A1 (en) * 2011-10-27 2014-10-30 Kyocera Corporation Tactile sensation providing device
JPWO2013061605A1 (ja) * 2011-10-27 2015-04-02 京セラ株式会社 触感呈示装置
US9519347B2 (en) 2011-10-27 2016-12-13 Kyocera Corporation Tactile sensation providing device
JP2017189153A (ja) * 2016-04-15 2017-10-19 小橋工業株式会社 農作業機

Similar Documents

Publication Publication Date Title
US12001650B2 (en) Music user interface
AU2021201419B2 (en) Device, method, and graphical user interface for adjusting the appearance of a control
US20230185447A1 (en) Information Processing Device, Operation Input Method And Operation Input Program
DK179034B1 (en) Devices, methods, and graphical user interfaces for dynamically adjusting presentation of audio outputs
US20190391658A1 (en) Device, Method, and Graphical User Interface for Forgoing Generation of Tactile Output for a Multi-Contact Gesture
KR102034584B1 (ko) 포터블 디바이스 및 그 제어 방법
JP4577428B2 (ja) 表示装置、表示方法及びプログラム
US9600120B2 (en) Device, method, and graphical user interface for orientation-based parallax display
EP3130999A1 (fr) Clavier d'ordinateur virtuel
WO2013114792A1 (fr) Dispositif et procédé visant à offrir une sensation tactile
US10908797B2 (en) Systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display
KR20180098166A (ko) 가상 감성 터치를 위한 시스템들 및 방법들
JP2018113025A (ja) 触覚によるコンプライアンス錯覚のためのシステム及び方法
JP2018506803A (ja) 湾曲ディスプレイとのユーザ相互作用のためのシステム及び方法
CN111684402B (zh) 触摸输入表面上的触觉效果
US11237709B2 (en) Systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display
KR20150135820A (ko) 디스플레이 장치 및 그 제어 방법
JP6205043B1 (ja) キーボード、情報処理装置、フィードバック方法、及びプログラム
WO2011128959A1 (fr) Dispositif de génération d'information de commande de sensation tactile
WO2018081112A1 (fr) Systèmes et procédés permettant à des utilisateurs malvoyants d'interagir avec un écran secondaire tactile
AU2017101426B4 (en) Systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display
JP2013114481A (ja) 情報処理装置、情報処理方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10849798

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10849798

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP