JP3937682B2 - Information processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an information processing apparatus, and more particularly to an information processing apparatus capable of transmitting information to a user by presenting a force / tactile sensation.
[0002]
[Prior art]
In information processing equipment such as personal computers, the user's vision is used as a means for the user to recognize and indicate graphical user interfaces such as application software displayed on the display and objects such as images and text. In order to reduce the visual burden in addition to the means for designating with the mouse cursor, the following apparatus has been proposed that includes means for presenting tactile information to the fingertip of the user holding the mouse. ing.
[0003]
Japanese Patent Laid-Open No. 63-111521 proposes a device that presents tactile information to a mouse-type device when the position of a selectable element displayed on the display screen matches the cursor position. This device has the effect of the accuracy of element selection.
[0004]
Japanese Patent Laid-Open No. 06-230916 discloses a non-visual method for the state of a graphic object such as a scroll bar, scroll control, push button, or window used in a graphical user interface. The technique of determining by this is described, and it is described using a pointing device such as a mouse that presents a force-tactile sensation as one means of the method.
[0005]
As a device that presents a specific force / tactile sensation, for example, a mouse that presents a force / tactile sensation is provided by Immersion, Inc., and an application programming interface ( Hereinafter, it is referred to as API). Therefore, for example, when developing application software, by using these, it is possible to develop application software of a graphical user interface having a force / tactile sense presentation function.
[0006]
In recent years, documents called hypertext format homepages written in HTML (Hyper Text Markup Language) have become widespread on the Internet, and even general end users who are not software experts tend to view and edit homepages. There is. In addition, objects such as links, texts, images, buttons, and the like are displayed on these home pages, and an object may be specified with a cursor by operating a mouse. At this time, it is desirable that these objects can be easily specified.
[0007]
[Problems to be solved by the invention]
However, in the above-described prior art, the function of presenting a force / tactile sensation with respect to a graphical user interface applied by application software or the like is a function previously incorporated in the software. Therefore, the end user uses only a function prepared in advance, and gives a function of presenting a tactile sensation in an object in a document edited and created by the end user such as a home page. There is a problem that it is impossible to set to present a tactile sensation when contacting the object.
[0008]
In addition, when using the above-mentioned Immersion API, it is necessary to incorporate the software into the program when developing the application software itself, so it is used by software developers who have specialized knowledge of software program development. Although it is possible, there is a problem that it is difficult for an end user to use.
[0009]
The present invention has been made to solve the above-described problems, and provides an information processing apparatus that can easily set a force / tactile sense for an object in document data and can present the force / tactile sense to a user. The purpose is to provide.
[0010]
[Means for Solving the Problems]
  The invention according to claim 1, Force defined using tagsShould present tactile sensationKi tactile senseBjekPower againstAttribute information about tactile presentationWrite newsThe script mentioned and the previousWriting tactile senseObjects andThe tactile senseBjekFor presenting force and touch to theForce-tactile presentationToIncluded document dataAndHenGatherEditing means; document holding means for storing and holding the document data; script holding means for storing and holding the script; interpreting the document data and executing the script;Display force haptic objectBrowsing means to show and previousEditing means and the viewerSet by execution of the script in the browsing means, pointing means for instructing a part of the document data displayed in a row, force / tactile sense presenting means for presenting force / sense to the userBeforeControl means for controlling the force / tactile sense presenting means based on a writing force / tactile sense presenting condition.
[0011]
  According to the first aspect of the present invention, the user can edit by the editing means., Force defined using tagsShould present tactile sensationKi tactile senseBjekPower againstAttribute information about tactile presentationWrite newsThe script mentioned and the previousWriting tactile senseObjects andThe tactile senseBjekFor presenting force and touch to theForce-tactile presentationToIncluded document dataAndHenGatherRu.The document data and script edited and created by the user are stored and held in the document holding unit and the script holding unit, respectively. The browsing means interprets the document data, executes the script, and allows the user to browseDisplay force haptic objectShow. The control means is set by executing the script in the browsing means.BeforeThe force / tactile sense presenting means is controlled based on the writing force / tactile sense presenting condition. This allows the user to use pointing means, Browsing equipmentDisplayedForceShould present tactile sensationKi tactile senseBy instructing the object, a force / tactile sensation is presented to the user by the force / tactile sense presenting means.
According to a second aspect of the present invention, there is provided script holding means for storing and storing a script describing attribute information relating to force / tactile sense presentation for a force / tactile object to be presented with force / tactile sense defined using a tag, and the force / haptic object And document holding means for storing and holding document data including force / haptic presentation conditions for presenting the force / haptic object to the force / haptic object, interpreting the document data, executing the script, and setting the attribute information A browsing means for displaying the force / tactile object so that the user can browse; a pointing means for indicating a part of the document data displayed on the browsing means; and a force / haptic for presenting the force / tactile sensation to the user Execution of the script when the presenting means, the pointing means instruction, and the force / tactile sense presentation condition correspond to each other Based on a more set the attribute information, and a, and a control means for controlling the force tactile display means.
According to a third aspect of the present invention, the information processing apparatus according to the second aspect further comprises an editing means for editing the script and the document data.
The attribute information may be represented by at least one of a function and a variable.
According to a fifth aspect of the present invention, the script further includes attribute information for changing the display of the force / tactile object when presenting the force / tactile sense, and the browsing means is included in the script. The force-tactile object may be displayed based on attribute information for changing the display of the force-tactile object.
[0012]
  ClaimItem 6The described invention is claimed.Any one of claims 1 to 5In the described invention, a force / tactile sensation should be presented by the script.Ki tactile senseThe attribute information related to the presentation of the force / tactile sensation set for the object is information representing the force / tactile intensity taking different values in a plurality of stages.
[0013]
  ClaimItem 6According to the described invention, the tactile sensation should be presented by the script.Ki tactile senseThe attribute information regarding the presentation of the force / tactile sensation set for the object represents a force / tactile intensity having different values in a plurality of stages. As a result, if the force / tactile intensity value is set to 0, 1, 2, or 3, for example, according to each intensity level including the intensity level 0, the control means outputs the force / haptic corresponding to each intensity value. The force / tactile sense presenting means can be controlled to present. In this way, if the strength level 0 is used, it is possible to simultaneously set the presence or absence of the force / tactile sensation only by the force / tactile sense.
[0014]
  ClaimItem 7The described invention is claimed.As described in any one of 1 to 6In the invention described above, it is necessary to present a tactile sensation with the script.Ki tactile senseThe attribute information regarding the presentation of the force / tactile sensation set for the object is information indicating presence / absence of the force / tactile sensation to be presented to the user.
[0015]
  ClaimItem 7According to the described invention, the tactile sensation should be presented by the script.Ki tactile senseThe attribute information regarding the presentation of the force / tactile sensation set for the object represents the presence / absence of the force / tactile sensation to be presented to the user, that is, the presence / absence of the force / tactile sensation. As a result, the control means can control the force / tactile sense presenting means so as to present or not present the force / tactile sense according to each value.
[0016]
In addition, you may set attribute information combining the information showing the presence or absence of a force haptic and the information showing a force haptic intensity.
[0017]
  ClaimItem 8The invention described in claims 1 toItem 7In any one of the inventions, the pointing means is a mouse-type input device for moving a cursor, and the force / tactile sensation providing means is arranged in a part of a casing of the input device.
[0018]
  ClaimItem 8According to the described invention, the pointing means is a mouse-type input device that moves the cursor. In general, when moving the cursor, the user places a finger on the casing and performs an operation. Therefore, by disposing the tactile sensation providing means on the part where the fingers of the housing of the input device are placed, the force haptic sensation is presented to the user operating the input device without impairing the operability of the input device. Can do.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0020]
  As shown in FIG. 1, the information processing apparatus 10 according to the present embodiment includes a display device 14 that displays various document data, and a force / tactile sensation providing unit 12a that displays a force / tactile sensation on an upper portion of a mouse-type pointing device 12b. A mouse-type force / tactile sensation presentation device 12 (refer to FIG. 2), a keyboard 16 for inputting data, a main storage device 20 such as a RAM and a ROM, an external storage device 22 such as a hard disk device, and a CPU 24 are provided. They are connected to each other so that data can be exchanged via the bus 18. The mouse-type force / tactile sensation presentation device 12 corresponds to the force / haptic angle presentation means and the pointing means of the present invention.do itYes.
[0021]
Further, the main configuration of the information processing apparatus 10 according to the present embodiment can be classified into the functional blocks shown in FIG. 3, and force / tactile document editing means 30 for editing and creating force / tactile documents and force / haptic scripts (details will be described later). A force / tactile script holding unit 32 for storing and holding the force / tactile script; a force / tactile document holding unit 34 for storing / holding the force / tactile document; and a force / tactile document browsing for browsing the force / tactile document It comprises means 36 and device control means 38 for controlling the mouse-type force haptic presentation device 12. The force / tactile document editing means 30, force / tactile script holding means 32, force / tactile document holding means 34, force / haptic document browsing means 36, and device control means 38 are the editing means, script holding means, document holding means of the present invention. , Corresponding to browsing means and control means.
[0022]
The force / tactile document editing unit 30 is connected to the force / tactile script holding unit 32 and the force / tactile document holding unit 34 so as to be able to exchange data. The force / tactile document browsing means 36 is connected so as to be able to read data from the force / tactile script holding means 32 and the force / haptic document holding means 34, and can exchange data with the device control means 38. It is connected.
[0023]
The force / tactile document editing unit 30 and the force / tactile document browsing unit 36 are configured to include a software program that is stored and held in the external storage device 22 and read and executed by the CPU 24 in the main storage device 20.
[0024]
In other words, the force / tactile document editing unit 30 applies force / tactile documents, which are document data including an object (hereinafter referred to as force / tactile object) to be set for force / haptics to be presented to the user, to the force / tactile object. And software for editing and creating a force / tactile script describing a setting procedure of attribute information for presenting force / tactile sense.
[0025]
Further, the force / tactile document browsing unit 36 reads the force / tactile document and the force / tactile script edited and created by the force / tactile document editing unit 30, displays the force / tactile document on the display device 14, and uses a mouse-type force / tactile sense presentation device. Data of the pointing position of 12 pointing cursors (hereinafter referred to as a cursor) is acquired from the device control means 38 described later, and the positional relationship between the force-tactile object and the cursor in the force-tactile document displayed on the display device 14 is obtained. Correspondingly, it is configured to include software for executing the force / tactile script.
[0026]
The force / tactile document holding unit 34 and the force / tactile script holding unit 32 store and hold the force / tactile document and the force / tactile script edited by the force / tactile document editing unit 30 as data files. It consists of
[0027]
The device control means 38 acquires data of the pointing position of the cursor of the mouse-type force / tactile sensation presentation device 12 or based on the attribute information of the force / tactile object set by the force / tactile script executed by the force / tactile document browsing means 36. Driver software for operating the force / tactile sensation providing unit 12a and hardware (not shown) controlled by the driver software.
[0028]
Here, specific examples of the above-described force-tactile document and force-tactile script will be described in detail.
[0029]
The force / tactile document of the present embodiment is described in HTML for creating document data in a hypertext format. Usually, document data described in HTML can be arranged with objects such as various image data, text, text set with hyperlinks, and the like, for example, as layers using LAYER tags. Defined. This layer includes various attribute information for setting and referring to its own state, including a layer identifier for identifying itself. As shown in FIG. 4, in addition to the layer identifier 40, the force / tactile object according to the present embodiment includes a force / tactile level 42 that is attribute information that specifies the strength of the force / tactile sensation to be presented to the user, and a force to the user. A force / tactile sensation presenting state 44 which is attribute information for designating whether or not to present a tactile sensation. By appropriately setting these attribute information, the force / haptic object is described in the force / haptic document data. In addition, as specific setting values of the force / tactile sensation level 42, for example, three types of high intensity “high”, medium intensity “mid”, and low intensity “low” are used, and the specific setting of the force / tactile sense presentation state 44 is set. As the values, presentation start “on” and presentation stop “off” are used.
[0030]
Further, as shown in FIG. 5, the force / tactile script according to the present embodiment stops the function forceOn () for setting attribute information for starting the force / tactile sensation and the force / tactile sense presentation in the force / tactile object. A function forceOff () for setting attribute information to be described is described.
[0031]
In the function forceOn (), the variable “color” is used as an argument, and the value of the variable is set to the attribute value “bgColor”. Here, the attribute value “bgColor” is an attribute value representing the color of the layer itself. For example, an attribute value prepared in the specification of the script description language “JavaScript” provided by Netscape can be used. . Also, the attribute value “forceLevel” is an attribute value representing the force / tactile sensation level 42, and for example, a medium strength “mid” is set. Also, the attribute value “forceStatus” is an attribute value representing the force / tactile sensation presentation state 44, and for example, the presentation start “on” is set.
[0032]
In the function forceOff (), as in the function forceOn (), the variable “color” is used as an argument, and the value of the variable is set to the attribute value “bgColor”. In addition, in the attribute value “forceStatus”, for example, a presentation stop “off” is set as a value representing the force / tactile sense presentation state 44.
[0033]
When the force / tactile document editing means 30 sets the layer identifier 40 of the force / tactile object in the document data to “example 1” and uses the LAYER tag, for example, as shown in FIG. The above-mentioned functions forceOn () and forceOff () are set for “onMouseOver” which is an event handler to be executed and “onMouseOut” which is an event handler which is executed when the cursor leaves the object. Note that “pink” as an argument of the function forceOn () means that pink is set as the color of the layer itself, and “white” as an argument of the function forceOff () is set as white as the color of the layer itself Is meant to do.
[0034]
As a result, the function forceOn () is executed when the cursor touches an object that presents a force haptic, and the function forceOff () is executed when the cursor leaves.
[0035]
Next, the operation of the present embodiment will be described using the specific example described above.
[0036]
The force / tactile document browsing means 36 of the information processing apparatus 10 according to the present embodiment displays the force / tactile document described above on the display device 14 and sets the attribute information of the force / tactile object by executing the force / haptic script described above. To do. Further, the device control means 38 presents a force / tactile sense based on the attribute information of the force / tactile object set by the force / haptic script described above. Note that processing routines described below executed in the force / tactile document browsing unit 36 and the device control unit 38 are executed in parallel as individual processes of the CPU 24.
[0037]
The force / tactile document browsing means 36 executes a processing routine shown in FIG.
[0038]
First, in step 100, it is determined whether or not the user has instructed the force / tactile document browsing means 36 to end the process. If the determination in step 100 is negative, the processing routine is ended because an instruction to end the processing has been given. On the other hand, if the determination in step 100 is affirmative, the process proceeds to step 102, where the cursor of the mouse-type force / tactile sensation presentation device 12 moved by the user moves to an object on the document data in which the force / tactile sense presentation attribute information is set. Judge whether contact has occurred. Here, for example, as shown in FIG. 8, it is determined whether or not the cursor 52 has touched the object 50 in which the attribute information of the force / tactile sensation presentation on the document data is set. If the determination in step 102 is negative, there is no contact of the cursor 52 with the object 50, so the process returns to step 100 and the above processing is repeated thereafter. On the other hand, if the determination in step 102 is affirmative, since the cursor 52 touches the object 50, the process proceeds to step 104 to call and execute the force / tactile script.
[0039]
At this time, “onMouseOver” which is an event handler set in the LAYER tag describing the object in the document data shown in FIG. 6 is executed. Thereby, the function forceOn () of the force / tactile script shown in FIG. 5 is called and executed. By executing the function forceOn ('pink'), pink is set as the color of the object itself, and the force / tactile sensation level 42 and the force / tactile sense presenting state 44 shown in FIG.
[0040]
In the next step 106, the setting information in step 104 is written into the main storage device 20 which is a memory shared with the device control means 38, and the process proceeds to step 108.
[0041]
In the next step 108, it is determined whether or not the cursor in contact with the object for which the attribute information of the force / tactile sensation presentation is set on the document data has been detached from the object by the user. Since the cursor is in a state of being moved or stopped while being in contact with the object, this step 108 is repeated. On the other hand, if the determination in step 108 is affirmative, the user has moved the cursor away from the object. Therefore, the process proceeds to step 110 to call and execute the force / tactile script.
[0042]
At this time, “onMouseOut” which is an event handler set in the LAYER tag describing the object in the document data shown in FIG. 6 is executed. Thereby, the function forceOff () of the force / tactile script shown in FIG. 5 is called and executed. By executing the function forceOff ('white'), white is set as the color of the object itself, the set content of the force / tactile sensation level 42 is deleted, and the force / tactile sense presenting state 44 is set to stop presenting.
[0043]
In the next step 112, the setting information in step 110 is written in the main storage device 20, which is a memory shared with the device control means 38, the process returns to step 100, and the above processing is repeated thereafter.
[0044]
Through the above processing, the force / tactile document browsing means 36 is in contact with the object of the mouse-type force / tactile sensation presentation device 12 moved by the user with respect to the object set with the force / tactile sense presentation attribute information on the document data. And when it leaves | separates, the memory content of the main memory 20 which is a memory shared with the device control means 38 is updated by executing a force tactile script.
[0045]
On the other hand, the device control means 38 executes the processing routine shown in FIG.
[0046]
First, in step 200, it is determined whether or not there is an instruction to end the process by the user to the device control means 38. If the determination in step 200 is negative, the processing routine is ended because an instruction to end the processing has been given. On the other hand, if the determination in step 200 is affirmative, the process proceeds to step 202, where the force / tactile level 42, which is attribute information of the force / tactile object, is stored in the main memory 20 which is a memory shared with the force / tactile document browsing means 36. And the force tactile sensation presentation state 44 is read.
[0047]
In the next step 204, it is determined whether or not the read stored contents are updated and changed. If the determination in step 204 is negative, the process returns to step 200, and thereafter the above processing is repeated. On the other hand, if the determination in step 204 is affirmative, the process proceeds to step 206 to determine whether or not the force / tactile sensation presentation state 44 of the changed attribute information is a presentation start. If the determination in step 206 is affirmative, the process proceeds to step 208 where the force / tactile sensation providing unit 12a is driven to present a force / tactile sensation to the user, the process returns to step 200, and the above processing is repeated thereafter. On the other hand, in the case of negative determination in step 206, the force / tactile sensation presentation state 44 is the presentation stop, so the process proceeds to step 210, the force / tactile sensation providing unit 12a is stopped, and the presentation of the force / tactile sensation to the user is stopped. After that, the above process is repeated.
[0048]
As a result of the above processing, when the cursor touches the force-tactile object in the force-tactile document displayed on the display device 14, the user performs a normal mouse operation with the mouse-type force / tactile sense presentation device 12, and also uses the fingertip to A force tactile sensation can be obtained by touching the tactile sense providing unit 12a.
[0049]
Therefore, for various objects such as links, figures, buttons, and texts found on documents such as websites, editing documents such as websites that are end users who do not have specialized knowledge of software program development. When a person edits a document, a description of a force / tactile sensation presentation instruction is easily given to the document, and a viewer of a document such as a home page can present a force / tactile sensation through the mouse-type force / tactile sense presentation device 12. Can be received at your fingertips.
[0050]
In this embodiment, the processing routines shown in FIGS. 7 and 9 executed by the force / tactile document browsing unit 36 and the device control unit 38 are executed in parallel, but are executed by the force / tactile document browsing unit 36. In the processing routine, for example, after step 106 and step 112, the processing routine executed by the device control unit 38 may be executed.
[0051]
【The invention's effect】
As described above, according to the present invention, when editing document data, attribute information related to the presentation of force / tactile sensation is set for an object in the document data, and the force / tactile sense is presented based on the attribute information. Therefore, there is an excellent effect that it is possible to provide an information processing apparatus that can easily set a force / tactile sense for an object in document data and can present the force / tactile sense to a user.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of an information processing apparatus according to an embodiment of the present invention.
FIG. 2 is a schematic diagram for explaining the configuration of an example of the input device according to the embodiment of the invention.
FIG. 3 is a schematic diagram showing a configuration in which the main configuration of the information processing apparatus according to the embodiment of the present invention is classified into functional blocks.
FIG. 4 is a diagram for explaining data to be set for an object that presents a force-tactile sensation.
FIG. 5 is a diagram for explaining an example of a script described for an object presenting a force / tactile sensation;
FIG. 6 is a diagram for explaining an example of a user interface when editing document data by the force / tactile document editing unit according to the embodiment of the present invention;
FIG. 7 is a flowchart showing a flow of processing in the force / tactile document browsing means according to the embodiment of the present invention.
FIG. 8 is a diagram for explaining an example of display document data according to the embodiment of the present invention.
FIG. 9 is a flowchart showing the flow of processing in the device control means according to the embodiment of the present invention.
[Explanation of symbols]
10 Information processing equipment
12 Mouse-type force tactile display device
12a Force / tactile sense presentation part
12b pointing device
14 Display device
16 keyboard
18 Bus
20 Main memory
22 External storage device
24 CPU
30 Force-tactile document editing means
32 Force / tactile script holding means
34 Force / tactile document holding means
36 Force-tactile document browsing means
38 Device control means

Claims (8)

And scripts that the attribute information and statements regarding the presentation of force tactile force sense of touch, which is defined using the tag for the presentation all-out force tactile Oh Brzeg door, before Symbol force tactile object and the force tactile Oh Brzeg door to force tactile and editing means you edit, and document data including the force tactile display conditions for presenting,
Document holding means for storing and holding the document data;
Script holding means for storing and holding the script;
When you execute the script interprets the document data for setting the attribute information together, and table Shimesuru browsing means pre Symbol force haptic object so that the user viewable,
And pointing means for directing a portion of the document data that has been displayed before Symbol browsing hand stage,
Force tactile sensation presenting means for presenting force tactile sensation to the user;
Wherein when an instruction to the force tactile presentation condition of the pointing means corresponds, on the basis of the set front Symbol attribute information by the execution of the previous SL scripts, and control means for controlling the force tactile display means,
An information processing apparatus comprising:   Script holding means for storing and holding a script describing attribute information related to force / tactile sense presentation for a force / tactile object to be presented with force / tactile sense defined using a tag;
  Document holding means for storing and holding document data including the force / tactile object and a force / tactile sense presentation condition for presenting the force / tactile object to the force / tactile object;
  Reading means for interpreting the document data and executing the script to set the attribute information and displaying the force-tactile object so that the user can view it;
  Pointing means for indicating a part of the document data displayed on the browsing means;
  Force tactile sensation presenting means for presenting force tactile sensation to the user;
  Control means for controlling the force / tactile sensation providing means based on the attribute information set by execution of the script when an instruction from the pointing means corresponds to the force / tactile sense presenting condition;
  An information processing apparatus comprising:   The information apparatus according to claim 2, further comprising editing means for editing the script and the document data.   The information processing apparatus according to claim 1, wherein the attribute information is represented by at least one of a function and a variable.   The script further includes attribute information for changing a display of the force-tactile object when presenting the force-tactile sense.
  5. The information processing according to claim 1, wherein the browsing unit displays the force-tactile object based on attribute information for changing a display of the force-tactile object included in the script. apparatus. Attribute information relating to the presentation of the force tactile set the force tactile to presentation all-out force haptic object by the script claims 1 to 5 and information indicating the force tactile intensity take multiple stepwise different values serial mounting of the information processing apparatus or to one of. Attribute information relating to the presentation of the force tactile set the force tactile to presentation all-out force haptic object by the script, one of claims 1 to 6 and information indicating the presence or absence of force tactile be presented to the user serial mounting of the information processing apparatus to (1). The information processing according to any one of claims 1 to 7, wherein the pointing means is a mouse-type input device that moves a cursor, and the force / tactile sense presenting means is arranged in a part of a casing of the input device. apparatus.

Images