JP5713400B2 - User interface device capable of operation with finger using pointer, operation invoking method, and program - Google Patents

Description

  The present invention relates to a user interface device including a display for displaying an image and a touch panel as a user interface.

  2. Description of the Related Art Conventionally, user interface devices equipped with a touch panel that perform an input operation by touching a screen with a finger or a pointing device such as a stylus are widely used. In particular, in user interface devices such as smartphones, tablet computers, electronic books, and PDAs (personal digital assistants), so-called portable information devices, touch panels are actively employed as user interfaces that accept operations with fingers and the like.

  In such a portable information device, it is generally possible to select an object such as an icon or a link by bringing a finger into contact with the touch panel and activate its function. In addition, many handwritten character data or drawing data can be created by handwriting input of characters or figures with a finger. For example, a handwritten memo is created and a description is added to the photographed data, a drawing information indicating a route or a destination is added to an image such as a map, or an application that requires handwritten input is used. be able to. The handwriting input function can be applied in various ways, and various devices have been conventionally made.

  For example, Patent Document 1 discloses a drawing apparatus capable of easily drawing a line segment connecting two end points while drawing a locus with a finger. In this apparatus, the first end point and the second end point are determined using two fingers, and drawing data indicating a line segment connecting these end points is generated.

  Patent Document 2 discloses a handwriting input display device that captures drawing information in accordance with a drawing operation of an input pen on a tablet unit. In this apparatus, whether or not the drawing information is to be erased is determined according to the user ID associated with the drawing information and the inputted erasure target range. At this time, the range to be erased is defined by drawing a closed curve with the input pen.

JP 2011-100503 A Special table 2010-239469

  However, even with the conventional technology as described above, in a portable information device, a finger is brought into contact with the vicinity of the edge / corner of the display screen (upper touch panel) or an object having a fine display position range is placed. It has been difficult to perform a desired operation in contact. In addition, it is difficult to input details of input information such as characters and figures with a finger.

  In fact, in portable information devices, there are many cases where it is desired to easily perform operations such as object selection and handwriting input with a finger without using other tools. In addition, for example, a situation in which handwriting input is performed before and after another operation using a finger frequently occurs. Therefore, it is often required to perform an operation with a finger directly without using a pointing device as disclosed in Patent Document 2.

  On the other hand, in the portable information device, because of its portability, the area of the touch panel that is operated with a finger is limited. As a result, the display position range of an object such as a displayed link tends to be small. Therefore, for example, it is very difficult to select a desired one with a finger from among a large number of links closely arranged.

  Furthermore, because of the limitation of the touch panel area, for example, in the case of handwriting input, it is often desired to secure a desired amount of information by inputting characters or figures of the smallest possible size. However, for example, when a trajectory is input by a finger as described in Patent Document 1, the finger contact point, that is, the most advanced position of the drawn trajectory is hidden behind the finger and cannot be visually recognized. As a result, it becomes difficult to input details of small-sized characters or figures accurately with a finger as intended.

  The problems described above apply to all portable information devices that perform operations for selecting and activating an object, handwriting input, and the like by bringing a finger into contact with the touch panel. Therefore, an object of the present invention is to provide a user interface device, an operation activation method, and a program that can specify a desired position in a place or details that are difficult to reach by a finger operation on the touch panel.

According to the present invention, an image display unit that displays a pointer that points to an input position with its tip, a touch panel that sequentially outputs a contact position of each finger, and a pressing force applied to the touch panel by the finger. A user interface device comprising a pressing force detection unit for detecting,
Pointer position determining means for sequentially determining the position of the tip of the pointer in accordance with the contact position at each time point of the finger;
Pointer display means for displaying a pointer so that the tip is located at each time point;
A pointer is displayed at a position corresponding to the contact position of the finger at each point in time, and a push button for determining whether or not the pressing force detected by a series of pressing operations while the finger is in contact is equal to or greater than a predetermined threshold value. Pressure determining means;
There is provided a user interface device having operation control means for activating an action associated with the position of the tip of the pointer when the pressing force determination means makes a true determination.

  According to this user interface device of the present invention, when the pressing force determining means makes a true determination, the action control means performs an action associated with an object that includes the position of the tip of the pointer within its display position range. It is also preferable to activate.

In addition, according to the user interface device of the present invention, it further includes contact position determination means for determining whether or not the contact position of the finger is superimposed on the display position range of the pointer.
The pointer position determination means sequentially determines the position of the tip of the pointer according to the contact position at each time point of the finger when the contact position determination means makes a true determination,
It is also preferable that the operation control means displays the input result by the pointer corresponding to the locus of the position of the tip, which corresponds to the locus in which the true determination is made by the pressing force determination means.

  Further, in the above embodiment in which the input result by the pointer is displayed, the finger is brought into contact with the pointer to hold the pointer and moved while being pushed into the pointer as viewed from the user's operation. At this time, it is also preferable that the operation control means displays an input result by the pointer corresponding to the locus of the position of the tip that moves in accordance with the movement of the finger.

  Further, in the above-described embodiment in which an input result by the pointer is displayed, the pointer position determination unit determines the contact position of the finger at the initial time point when the contact position of the finger is superimposed on the display position range of the pointer. It is also preferable to calculate a translational conversion to be converted into the position of the tip at the point, and at each subsequent time point, a position obtained by converting the contact position of the finger by the translational conversion is set as the position of the tip of the pointer.

Further, in the above embodiment in which the input result by the pointer is displayed , the input result by the pointer is a result of a predetermined object displayed on the screen of the image display unit being changed by the action of the tip of the pointer. Is also preferable.

  Furthermore, in the above-described embodiment in which an input result by the pointer is displayed, the input result by the pointer is preferably a handwriting image including a locus of the position of the tip.

  Further, in the above embodiment in which the input result by the pointer is displayed, the pointer display means displays a right-handed pointer and a left-handed pointer as the pointer according to a predetermined operation by the finger. Is also preferable.

According to the present invention, the image display unit that displays a pointer that points to the input position with the tip of itself, the touch panel that sequentially outputs the contact position of each finger, and the press applied to the touch panel by the finger. A program installed in a user interface device including a pressure detection unit for detecting pressure,
Pointer position determining means for sequentially determining the position of the tip of the pointer in accordance with the contact position at each time point of the finger;
Pointer display means for displaying a pointer so that the tip is located at each time point;
A pointer is displayed at a position corresponding to the contact position of the finger at each point in time, and a push button for determining whether or not the pressing force detected by a series of pressing operations while the finger is in contact is equal to or greater than a predetermined threshold value. Pressure determining means;
A program for a user interface device is provided that causes a computer to function as an operation control unit that activates an operation associated with the position of the tip of the pointer when the pressing force determination unit makes a true determination.

According to the present invention, furthermore, an image display unit that displays a pointer that indicates an input position with the tip of itself, a touch panel that sequentially outputs a contact position at each time point of the finger, and the touch panel provided to the touch panel by the finger An operation activation method in a user interface device including a pressing force detection unit for detecting a pressing force,
A first step of sequentially determining the position of the tip of the pointer according to the contact position of each finger at each time point;
A second step of displaying a pointer so that the tip is positioned at each time point;
A pointer is displayed at a position corresponding to the contact position at each time point of the finger, and it is further determined whether or not the pressing force detected by a series of operations of pushing the finger while keeping the finger in contact is equal to or greater than a predetermined threshold value. 3 steps,
When the true determination is made in the third step, there is provided an action invoking method including a fourth step of invoking an action associated with the position of the tip of the pointer.

  According to the user interface device, the operation activation method, and the program of the present invention, it is possible to specify a desired position in a place where the finger is difficult to reach or in detail by an operation with a finger on the touch panel.

FIG. 6 is a front view of a portable information device showing a handwritten input using a pointer according to the present invention, and a schematic diagram of an operation by a finger. A pressing force p _C value with the finger, a front view and a graph of a portable information device for explaining the relationship between the display of the input result of the pointer (handwriting image). Schematic showing the relationship between the contact position of the finger and the tip position of the pointer when the finger holds the pointer, a schematic showing the case where the finger moves (holds again) the pointer, and right-handed and left-handed It is the schematic which shows the case where a pointer is each handled. It is a front view of the portable information device which shows other embodiment of operation | movement activation (information input) using the pointer by this invention. FIG. 1 is a perspective view and a functional configuration diagram schematically showing an embodiment of a portable information device according to the present invention. It is a flowchart which shows one Embodiment of the operation | movement activation (information input) method by this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  In the user interface device according to the present invention, the pointer displayed on the screen of the display is held by the touched finger, and the tip of the pointer serving as the input position moves in accordance with the movement of the contact position of the finger. Furthermore, when the pressing force with respect to the pointer (touch panel) by the finger is equal to or greater than a predetermined threshold value, the position of the tip of the pointer is designated, and an operation associated with the position of the tip is activated.

  In the user interface device according to the present invention, information is input when a user's finger contacts the touch panel. Therefore, most of the devices are portable information devices such as smartphones and tablet computers that can be carried and operated by hand. Therefore, hereinafter, a portable information device will be described as an embodiment of the present invention.

  1A and 1B are a front view of the portable information device 1 and a schematic diagram of the operation with the finger, showing the operation with the finger using the pointer according to the present invention.

Referring to FIG. 1A, the portable information device 1 includes a display 101 that displays a pointer 104 and a touch panel 100 that is disposed on the screen of the display 101. Here, the pointer 104 is an image for a user interface that indicates an input position by the tip 104e of the pointer 104e. The portable information device 1 also includes a pressing force detection unit 103 (see FIG. 5) that detects a pressing force p _{C applied} to the touch panel 100 by a finger.

  Here, first, an embodiment will be described in which a desired one is selected with a finger from among a large number of links (objects) 109 displayed using the pointer 104. The pointer 104 is not limited to the form shown in FIG. For example, an arrow shape may be used as long as it indicates a position different from the finger contact position 105.

  According to FIG. 1A, the user makes a finger touch the pointer 104 (touch panel 100). That is, the finger contact position 105 is superimposed on the display position range of the pointer 104. As a result, the finger holds the pointer 104. Here, “hold” refers to a state in which the pointer 104 moves together with the finger as the finger moves as long as the finger is in contact with the touch panel 100.

Next, the user moves (slides) the finger in contact with the finger, and places the tip 104e of the pointer 104 that moves with the finger within the display position range of the desired link 109. Thereafter, the finger pushes contact position 105 (touch panel 100) with a pressing force _{p C.}

When the pressing force p _C is equal to or greater than a predetermined pressing force threshold value p _TH , an operation control unit 125 (FIG. 5) described later is associated with a link 109 that includes the position of the tip 104e within its own display position range. Activate the action. Specifically, the hypertext function of the link 109 is activated, and it becomes possible to browse a desired link destination document or image.

  As described above, according to the present embodiment, the position on the screen is designated with the tip 104e separated from the contact position 105 of the finger, and the function of the object such as the link 109 is activated. As a result, it is easy to designate a desired position and activate a desired function in details displayed on the screen of the display 101, and also in places where fingers are difficult to reach, such as near the edges and corners of the screen. .

In the above embodiment, the pointer 104 is displayed before the finger contacts. As a change mode for this, the pointer 104 is also preferably displayed when the finger touches the screen (touch panel 100) of the display 101. In this case, the position displaced from the finger contact position 105 by a predetermined translation vector is the position of the tip 104e in the displayed pointer 104. Further, the pressing force threshold value p _TH can be set to a value in the range of 0.5 N (Newton) to 2.0 N, for example.

  Next, an embodiment in which handwriting input is performed with a finger using the pointer 104 will be described. FIG. 1B shows the state of this handwriting input. In FIG. 1B, the pointer 104 is an image of a writing instrument, and the tip 104e is a pen (brush) tip. The memo paper image occupies the entire screen of the display 101. This memo paper image is an object that changes (characters and the like are drawn) under the action of the tip (pen nib) 104e of the pointer 104, but may occupy a predetermined area that forms part of the screen. .

  According to FIG. 1B, first, the user brings a finger into contact with the pointer 104 (touch panel 100). That is, the finger contact position 105 is superimposed on the display position range of the pointer 104. As a result, the finger holds the pointer 104.

  At the time of holding, the user brings the finger into contact with a position where the tip 104 e can be seen without being hidden by the finger, for example, a middle position of the pointer 104. Next, the user moves (slides) the finger in contact with the finger, and places the tip 104e of the pointer 104 that moves together with the finger at a desired position at the beginning of writing the character.

  Next, the user moves (slides) while pushing the finger into the pointer 104 (touch panel 100). Along with this, the pointer 104 (the tip 104e of the pointer 104) also moves. As a result, the position of the tip 104e forms a certain trajectory as a section where the finger is pushed. At this time, the handwriting image 106 including the locus (which is a section where the finger is pressed) is displayed on the screen. The handwriting image 106 becomes an input result by the pointer 104.

  At this time, the finger contact position 105 and the tip 104e generating the handwriting image 106 are separated from each other by a predetermined distance. Therefore, the user can always visually recognize the tip 104e without being blocked by a finger.

  Further, once displayed, the handwriting image 106 remains on the screen unless a predetermined command is generated, and the tip 104e of the pointer 104 moves, and the handwriting image 106 is preferably extended so as to follow the tip 104e. Thereby, the user can write down information on the intended character, figure, etc. on the screen (memo paper image). After writing the intended information, the user removes his / her finger from the touch panel 100 or stops pushing while touching the finger. Thereby, the handwriting operation ends.

  The handwriting image 106 formed by the above handwriting input can be stored in the memory in the portable information device 1 as handwritten character data.

As described above, the condition that the finger holds the pointer 104 and the handwriting action is activated is as follows.
a) The finger contact position 105 overlaps the display position range of the pointer 104, and then the finger continues to touch the touch panel 100,
b) upon pushing with a finger, the pressing force _{p C} on the touch panel 100, a predetermined pressing force threshold as _{p TH,} the following equation (1) _{p C} ≧ _{p TH}
Is to satisfy. The pressing force threshold p _TH can be set to a value within the range of 0.5N to 2.0N, for example.

  By providing the above condition b), it is possible to avoid malfunctions such as unintentional writing that occurs when the user's finger or the like touches the touch panel 100 without intention. That is, it becomes easy to activate a handwriting operation as intended by the user. Furthermore, as long as pressing with the finger (condition b) is stopped, the pointer 104 can be moved to the next handwriting start position without activating the handwriting operation even with the finger kept in contact.

  Furthermore, it is also preferable that the tactile response mechanism unit 102 (FIG. 5), which will be described later, gives a tactile response to the finger while the handwriting action is activated. Here, while the handwriting operation is activated, the above-described conditions a) and b) are satisfied, and the handwritten image 106 is displayed while being newly expanded. Thereby, the user can experience from the finger the feeling of actually writing or drawing on a medium such as paper with a writing instrument, and can obtain a realistic operational feeling.

It is also preferable that the user can set and adjust the threshold value p _TH of the pressing force p _C by the finger as appropriate. Thus, the user can reliably execute the operation of moving the finger while keeping it in contact with the handwriting operation and the operation of performing the handwriting according to his / her sense and preference. At this time, a slide switch image 107 for adjusting the pressing force threshold value for changing the threshold value p _TH may be displayed and used.

  As described above, according to the present embodiment, the pointer 104 is held by bringing the finger into contact with the pointer 104, and the tip 104e of the pointer moves in accordance with the movement of the finger contact position 105. Here, the contact position 105 and the tip 104e translate while being spaced apart from each other with a fixed positional relationship. Thereby, the user can always visually recognize the tip 104e as the input position without being blocked by the finger. As a result, it is possible to input the input information in detail as intended, even if the character or figure has a small size.

  Even in a drawing operation in which a watercolor image or the like can be generated by the pointer 104, the tip 104e serving as the input position can always be visually recognized without being blocked by a finger. As a result, a finer pen (brush) operation can be performed, and a desired input result (watercolor image or the like) can be easily obtained.

  Furthermore, the contact position 105 and the tip 104e are separated from each other with a certain positional relationship. Accordingly, the tip 104e of the pointer 104 can be applied to an area where the fingertip is difficult to reach, such as near the edge or corner of the screen. As a result, it is possible to display desired input results such as writing or drawing in almost all areas of the screen.

2A and 2B are a front view and a graph of the portable information device 1 for explaining the relationship between the pressing force p _C value by the finger and the display of the input result (handwriting image 106) by the pointer. is there.

  According to FIG. 2 (A), the pointer 104 starts from the point 2a and reaches the point 2h sequentially through the point 2b, the point 2c, the point 2d, the point 2e, the point 2f, and the point 2g in the screen of the display 101. An example of the path (trajectory) of the position of the tip 104e is shown. Of these journeys, the trajectory from point 2b to point 2c is displayed on the screen as a handwriting image 106 representing “no” of hiragana.

  In addition, the trajectory from the point 2e to the point 2f and the trajectory from the point 2g to the point 2h are displayed on the screen as a handwriting image 106 representing Hiragana “ri”. In this embodiment, when the tip 104e of the pointer 104 reaches the point 2d, the user holds the pointer 104 again (holds it).

FIG. 2B shows the transition of the magnitude of the pressing force p _{C applied to} the touch panel 100 by the finger in the path (trajectory) of the position of the tip 104e. According to FIG. 2 (B), the pressing force _{p C} by a finger becomes larger as the position of the tip 104e proceeds toward from point 2a to point 2b, reaches a predetermined pressing force threshold _{p TH} at the point 2b.

Thereafter, the pressing force p _C satisfies the above-described equation (1) p _C ≧ p _TH until the position of the tip 104e reaches the point 2b to the point 2c. In response to this, the handwriting image 106 including the locus from the point 2b to the position of the tip 104e is drawn while expanding along with the movement of the tip 104e. Next, when the position of the tip 104e reaches the point 2c, the drawing of the handwriting image 106 representing "no" of hiragana is completed.

Thereafter, the pressing force p _C decreases as the position of the tip 104e of the pointer 104 moves away from the point 2c, and becomes zero when the position of the tip 104e reaches the point 2d. This is because the finger is separated from the pointer 104 (touch panel 100) when the user moves (holds) the pointer 104 again. After this dimensional worlds, the pressing force _{p C} becomes larger as the position of the tip 104e is directed to the point 2e, reaches a predetermined pressing force threshold _{p TH} at the point 2e.

Thereafter, the pressing force p _C satisfies the above-described equation (1) p _C ≧ p _TH until the position of the tip 104e reaches the point 2e to the point 2f. In response to this, the handwriting image 106 including the locus from the point 2e to the position of the tip 104e is drawn while expanding along with the movement of the tip 104e. Next, when the position of the tip 104e reaches the point 2f, a handwriting image 106 representing the left part of “Hi” in Hiragana is drawn.

Further, the pressing force p _C satisfies the condition of the above-described formula (1) p _C ≧ p _TH until the position of the tip 104e further advances and reaches from the point 2g to the point 2h. In response to this, the handwriting image 106 including the locus from the point 2g to the position of the tip 104e is drawn while expanding along with the movement of the tip 104e. Next, when the position of the tip 104e reaches the point 2h, the handwriting image 106 representing the right portion of the “RI” of the hiragana is drawn, and the drawing of the “RI” of the hiragana is completed.

As described above, according to the present invention, the locus of the position of the tip 104e, which is a locus in which the pressing force p _C by the finger is equal to or greater than the predetermined threshold p _TH (point 2b−point 2c, point 2e). -The input result (handwriting image 106) corresponding to point 2f, point 2g-point 2h) is displayed. As a result, malfunctions are less likely to occur, and handwriting operations as intended by the user can be activated. Further, it is possible to select whether or not to activate the handwriting operation while keeping the finger in contact with the touch panel 100. That is, a handwriting operation can be performed while tracing the screen with a finger by an operation in which the movement of the finger in the direction perpendicular to the screen is almost suppressed.

  FIG. 3A is a schematic diagram showing the relationship between the finger contact position 105 and the tip 104e position when the finger holds the pointer 104. FIG. FIG. 3B is a schematic diagram illustrating a case where a finger changes (holds again) the pointer 104. Further, FIG. 3C is a schematic diagram showing a case where the right-handed and left-handed pointers 104 are respectively handled.

  As shown in FIG. 3A, the finger contact position 105 first overlaps the display position range of the pointer 104. Here, the display position range is an area within an outline defining the image of the pointer 104 in FIG. The finger is brought into contact with a position separated from the tip 104e so that the tip 104e of the pointer 104 is not hidden by the finger and cannot be seen.

Here, an xy orthogonal coordinate system is provided on the screen, the coordinates of the finger contact position 105 are (x _C0 , y _C0 ), and the coordinates of the position of the tip 104 e of the pointer 104 are (x _P0 , y _P0 ). Then, it is possible to determine the coordinates _{(x C0, y} C0) coordinate transformation of the translational movement from the coordinates _{(x P0, y} P0) _{(translational} transform) _{f T.} In particular,
(1) Δx = x _P0 −x _C0
(2) Δy = y _P0 −y _C0
The translational transformation f _T is
(3) f _T : (x, y) → (x + Δx, y + Δy)
Is calculated.

Next, the user moves the finger while touching it, and as a result, the coordinates of the contact position 105 change to (x _C1 , y _C1 ). Position of the leading end 104e of the pointer 104 at this time _is determined by translating the translational transform _{f T} a (x _{C1, y} C1). Specifically, the coordinates (x _P1 , y _P1 ) of the position of the tip 104e at this time are expressed by the following equation (4) x _P1 = x _C1 + Δx
(5) y _P1 = y _C1 + Δy
Is calculated by

When the finger contact position 105 moves to the position of the coordinates (x _C1 , y _C1 ), the tip 104e of the pointer 104 is moved to the position of the calculated coordinates (x _P1 , y _P1 ). Thus, the user can move the tip 104e of the pointer 104 as intended while obtaining the feeling of holding the pointer 104 with a finger.

  Next, a method for determining the position of the tip 104e of the pointer 104 when the user changes (holds) the pointer 104 again will be described.

  According to FIG. 3B, first, the finger contacts the pointer 104 at the contact position 105. Thereafter, in order to change the pointer 104, the user moves the finger away from the pointer 104 (touch panel 100) and touches the pointer 104 again. At this time, the finger contact position 105 ′ is different from the contact position 105.

When the finger first contacts the pointer 104 at the contact position 105, that is, when the finger 104 is held, the position of the tip 104e of the pointer 104 is, as described above, the translational transformation f _T : (x, y) → (x + Δx, y + Δy). The translational transform f _T is during contact without separation finger, stored, used for calculating the position of the tip 104e.

Then, when away from the pointer 104 (touch panel 100) for re-holding the finger, translational transform f _T which has been stored is erased (reset). Thereafter, when the finger again contacts at the contact position 105 ′, a new translational transformation f _T ′ is determined again. This translational transformation f _T ′ has the coordinates of the contact position 105 ′ as (x _C0 ′, y _C0 ′) and the coordinates of the position of the tip 104 e of the pointer 104 as (x _P0 , y _P0 ).
(6) Δx ′ = x _P0 −x _C0 ′
(7) Δy ′ = y _P0 −y _C0 ′
As
_{(8) f T ': (} x, y) → (x + Δx', y + Δy ')
Is calculated.

Thereafter, the position of the tip 104e of the pointer 104 when the user moves the finger while making contact with the finger is determined using the translational transformation f _T ′. In this way, the user can change the pointer 104, and as a result, the relationship between the finger contact position 105 and the tip 104e position of the pointer 104 can be adjusted so that it can be easily operated.

  Next, an embodiment in which the user can handle the right-handed and left-handed pointers 104 will be described.

According to the embodiment shown in FIG. 3C, the changeover switch image 108 for switching the pointer is displayed on the screen. When the user presses the “R” portion of the changeover switch image 108 with a pressing force p _{C equal to} or higher than a predetermined threshold value p _TH with a finger, the right-handed pointer 104 is displayed at a predetermined position. The right-handed pointer 104 extends obliquely so that the pen (brush tip) (tip 104e) faces the lower left when the screen is viewed from the front. Therefore, the specification is suitable for writing / drawing with the finger of the user's right hand. The user can hold the pointer 104 by preferably touching the displayed pointer 104 with the finger of the right hand.

On the other hand, when the user presses the “L” portion of the changeover switch image 108 with his / her finger with a pressing force p _{C equal to} or higher than a predetermined threshold value p _TH , a left-handed pointer 104 ′ is displayed at a predetermined position. This left-handed pointer 104 'extends obliquely such that the pen (brush tip) (tip 104e') faces the lower right when the screen is viewed from the front. Therefore, the specification is suitable for writing and drawing with the finger of the left hand of the user. The user can hold the pointer 104 ', preferably by touching the displayed pointer 104' with the finger of the left hand.

  When the right-handed (left-handed) pointer 104 (104 ′) is already displayed, if the “L” portion (“R” portion) of the changeover switch image 108 is pressed, the pointer 104 (104 ′) ) Is preferably converted into a pointer 104 ′ (104) by changing the direction of its pen (brush) tip (tip 104e ′) to the lower left (lower right).

Further, the pointer can be switched by a predetermined operation with a finger without using the changeover switch image 108 as described above. For example, a right-handed (left-handed) pointer 104 (104 ′) is pushed twice with a finger with a pressing force p _{C equal to} or greater than a predetermined threshold value _pTH within a predetermined time. The pointer 104 ′ (104) may be set so as to be converted.

  FIGS. 4A and 4B are front views of the portable information device 1 showing another embodiment of information input (operation activation) using a pointer according to the present invention.

  According to FIG. 4A, the portable information device 1 includes a display 101 that displays a pointer 204 and a touch panel 100 that is arranged on the screen of the display 101. In this embodiment, the pointer 204 is an image of a surgical knife, and the tip 204e is a cutting edge. A patient's left arm image 206 is displayed on the screen of the display 101. The left arm image 206 is a target that changes (cuts) under the action of the tip (blade edge) 204e of the pointer 204.

  First, the user overlaps the contact position 205 of the finger with the display position range of the pointer (female image) 204. As a result, the finger holds the pointer 204. Next, the user moves the finger in contact with the finger, and places the tip (blade edge) 204e of the pointer 204 that moves with the finger at a desired incision start position.

The user then the finger, the pointer 204 is moved while pressing a predetermined threshold _{p TH} or more of the pressing force _{p C} (the touch panel 100). Along with this, the tip 204e of the pointer 204 also moves. As a result, the position of the tip 204e forms a trajectory that is a section in which the finger is pressed (p _C ≧ p _TH ). At this time, an incision image 207 including this locus is displayed on the screen.

  Here, the finger contact position 205 and the tip (blade edge) 204e generating the incision image 207 are separated from each other by a predetermined distance. Therefore, the user can always visually recognize the tip (blade edge) 204e without being blocked by the finger.

  In the present embodiment, the incision image 207 is displayed by moving a finger while pushing it into the pointer 204 as viewed from the user's operation. That is, an incision is possible only after a certain amount of force is applied to the subject. Thereby, the user can act on the target image while obtaining a realistic operation feeling.

  According to FIG. 4B, the portable information device 1 includes a display 101 that displays a pointer 304 and a touch panel 100 that is arranged on the screen of the display 101. In the present embodiment, the pointer 304 is an image of a fossil scoop, and the tip 304e is a cutting edge. In addition, on the screen of the display 101, a formation image 306 of the excavation site is displayed. The formation image 306 is a target that is changed (excavated) under the action of the tip (cutting edge) 204e of the pointer 304.

  First, the user overlaps the contact position 305 of the finger with the display position range of the pointer (scoop image) 304. As a result, the finger holds the pointer 304. Next, the user moves the finger in contact with the finger, and places the tip (blade edge) 304e of the pointer 304 that moves together with the finger at a desired excavation start position.

The user then the finger is moved while pressing a predetermined threshold _{p TH} or more of the pressing force _{p C} pointer 304 (touch panel 100). Along with this, the tip 304e of the pointer 304 also moves. As a result, the position of the tip 304e forms a trajectory that is a section in which the finger is pressed (p _C ≧ p _TH ). At this time, the excavation trace image 307 including this trajectory is displayed on the screen. The excavation trace image 307 may include a portion where a part of the fossil image previously buried in the formation image 306 is exposed depending on the position.

  Here, the finger contact position 305 and the tip (blade edge) 304e generating the excavation trace image 307 are separated by a predetermined distance. Therefore, the user can always visually recognize the tip (blade edge) 304e without being blocked by a finger.

  As described above, in this embodiment, the excavation trace image 307 is displayed by moving a finger while pushing it into the pointer 304 as viewed from the user's operation. That is, excavation is possible only after a certain amount of force is applied to the object. Thereby, the user can act on the target image while obtaining a realistic operation feeling.

  FIG. 5 is a perspective view and a functional configuration diagram schematically showing an embodiment of the portable information device 1 according to the present invention.

  Referring to FIG. 5, the portable information device 1 includes a touch panel 100, a display 101, a tactile response mechanism unit 102, a pressing force detection unit 103, and a processor memory. Here, the processor memory realizes its function by executing a program.

  The display 101 displays the pointer 104 (, 204, 304) on the screen. The touch panel 100 is disposed on the screen of the display 101 and sequentially outputs the contact position 105 of the user's finger as time passes. As this touch panel 100, a projected capacitive touch panel, a surface capacitive touch panel, a resistive touch panel, an ultrasonic surface acoustic wave touch panel, an infrared scanning touch panel, or the like can be employed.

The tactile response mechanism unit 102 gives a tactile response to the finger touching the touch panel 100 by vibrating the touch panel 100. For example, when a user presses a finger with a pressing force p _C equal to or greater than a predetermined threshold value p _TH and a pointer input result (handwriting image 106 or the like) is displayed while being newly expanded, vibration with a predetermined intensity is applied to the finger. Given. The tactile response mechanism 102 can be a piezoelectric actuator formed using a piezoelectric material such as PZT (lead zirconate titanate).

The pressing force detection unit 103 detects a pressing force p _{C applied} to the touch panel 100 by a finger. The pressing force detection unit 103 is installed, for example, under the four corners of the touch panel 100, and detects the total pressure exerted on the touch panel 100 that is bent by pressing a finger as the pressing force p _C. The pressing force detection unit 103 can be, for example, a piezoelectric sensor formed using a piezoelectric material such as PZT. In addition to or in addition to providing the tactile response mechanism unit 102 formed of a piezoelectric actuator, the pressing force detection unit 103 can be used as a tactile response mechanism unit.

  The processor memory inputs a finger contact position signal output from the touch panel 100 and a pressing force signal output from the pressing force detection unit 103, and a pointer 104 (, 204) by the user's finger based on these signals. 304) is recognized. Next, input results (handwriting image 106, incision image 207, excavation trace image 307) by the pointer are displayed in accordance with the contents of this operation.

  Similarly, according to FIG. 5, the processor memory includes a contact position determination unit 121, a pointer position determination unit 122, a pointer display unit 123, a pressing force determination unit 124, an operation control unit 125, and a tactile response control unit. 126, a display control unit 111, and an application processing unit 112.

  The contact position determination unit 121 receives a finger contact position signal output from the touch panel 100 and determines whether or not the finger contact position 105 overlaps the display position range of the pointer 104. Further, the determination result is output to the pointer position determination unit 122. Further, information regarding the presence or absence of a finger touching the touch panel 100 is also sequentially output to the pointer position determination unit 122.

  When the pointer position determination unit 122 inputs a true determination from the contact position determination unit 121, that is, a determination that the finger contact position 105 is superimposed on the display position range of the pointer 104, the pointer position determination unit 122 performs subsequent finger contact at each time point. The position of the tip 104e of the pointer 104 is sequentially determined according to the position 105.

At this time, the pointer position determining unit 122 uses the coordinates (x _C0 , y _C0 ) of the contact position 105 at the initial superimposed time as the coordinates (x _P0 , y _P0 ) of the position of the tip 104e of the pointer 104 at the same point. determining the translational transform f _T to convert to. Then, in each subsequent time point, the position of the result of a contact position 105 of the finger is converted by the translational transform f _T, the position of the tip 104e. Further, the pointer position determination unit 122 sequentially outputs the determined position information of the tip 104e to the pointer display unit 123 and the operation control unit 125.

  On the other hand, when the information indicating that the finger contact position does not exist, that is, the finger is not in contact with the touch panel 100, is input from the contact position determination unit 121, the pointer position determination unit 122 at that time is the position of the tip 104e. Stop making decisions. In this case, the position of the tip 104e is maintained at the position immediately before the determination stop.

  The pointer display unit 123 receives position information of the tip 104e from the pointer position determination unit 122. Further, using this input position information, the display control unit 111 is instructed to display the pointer 104 so that the tip 104e is positioned at each time point.

The pressing force determination unit 124 receives the pressing force signal output from the pressing force detection unit 103, and determines whether or not the pressing force p _C by the finger is _{equal to} or greater than a predetermined pressing force threshold value pTH. Further, the determination result is output to the operation control unit 125.

The operation control unit 125 inputs the position information of the tip 104e from the pointer position determination unit 122, and further inputs the determination result from the pressing force determination unit 124. Next, the motion control unit 125 is a locus of the position of the tip 104e of the pointer 104 based on the input position information and determination result, and the true determination (p _C ≧ p _TH ) by the pressing force determination unit 124. The trajectory that becomes the section where is made is determined.

  The operation control unit 125 further instructs the display control unit 111 to display an input result (handwriting image 106 or the like) by the pointer 104 corresponding to the determined locus. At this time, it is also preferable that the operation control unit 125 instructs the tactile response control unit 126 to activate a tactile response simultaneously with this instruction. The input result by the pointer 104 may be determined by the operation control unit 125, or may be determined by the display control unit 111 based on the determined locus and receiving information from the application processing unit 112. . The tactile response control unit 126 controls the tactile response mechanism unit 102 to give a tactile response (for example, vibration) to the finger based on the instruction of tactile response activation input from the operation control unit 125.

  The display control unit 111 inputs application processing information from the application processing unit 112 and causes the display 101 to display an image corresponding to the execution of the application. Further, the display control unit 111 inputs instruction information for displaying the pointer 104 from the pointer display unit 123, and causes the display 101 to display the pointer 104 so that the tip 104e is positioned at each time point. Further, the display control unit 111 inputs instruction information for displaying an input result (handwriting image 106 or the like) by the pointer 104 from the operation control unit 125, and the input result by the pointer 104 corresponding to the locus of the tip 104e is displayed on the display 101. Display.

  FIG. 6 is a flowchart showing an embodiment of an operation activation (information input) method according to the present invention.

  According to FIG. 6, first, the pointer 104 (204, 304) is displayed on the display 101 (step S601). Next, the touch of the finger is detected by the touch panel 100 (step S602). Next, the contact position determination unit 121 determines whether or not the finger contact position 105 overlaps the display position range of the pointer 104 (step S603).

When the contact position 105 of the finger is determined to overlap with the display position range of the pointer 104 in step 603, the pointer position determination unit 122, translational transform _{f T} is determined (step S604). Translational transform _{f T} is for converting coordinates of the contact position 105 at the initial time obtained by superimposing _{(x C0, y} C0) to the coordinates _{(x P0, y} P0) of the tip 104e position of the pointer 104 at the same time is there. On the other hand, when it is determined that the pointer 104 does not overlap with the display position range of the pointer 104, the process returns to step S602 to detect finger contact.

After the determination of the translational transform _{f T} (step S604), enters a loop for displaying an input result by the pointer 104 (step S605), by the contact position determination unit 121, the contact position 105 of the finger is measured (step S606 ). Next, the pointer position determination unit 122 determines the position of the tip 104e of the pointer 104 according to the finger contact position 105 (step S607). Specifically, a contact position 105 of the finger, the position of the result of converting the determined translational transform f _T, is the position of the tip 104e.

Next, the pointer 104 is displayed by the pointer display unit 123 so that the tip 104e is located at the position determined in step S607 (step S608). Next, the pressing force detection unit 103 measures the pressing force p _C by the finger (step S609).

Next, the pressing force determination unit 122 determines whether or not the pressing force p _C is _{equal to} or greater than a predetermined pressing force threshold value p _TH (step S610). When it is determined that the pressure is equal to or greater than the predetermined pressing force threshold value p _TH , the operation control unit 125 inputs an input result by the pointer 104 corresponding to the locus of the position of the tip 104e and the locus of the section where p _C ≧ p _TH. (Handwriting image 106 etc.) is displayed (step S611). Thereafter, the contact position determination unit 121 measures the presence or absence of finger contact (step S612). On the other hand, if it is determined to be less than a predetermined pressure threshold _{p TH} in step S610, the without going through step S611, the process proceeds to step S612.

Next, the contact position determination unit 121 determines whether or not the finger is in contact (step 613). Here, when it is determined that the finger is in contact, the input result display (operation control) loop including the steps S605 to S613 described above is repeated. On the other hand, when it is determined that the finger is not in contact (the finger contact position does not exist), the information input method ends. Incidentally, as shown in FIG. 3 (B), if the user's finger dimensional worlds pointer 104, and ends the flow described above is once started again, in step S604, the translational transform f _T is updated The

  For the various embodiments of the present invention described above, various changes, modifications, and omissions in the technical idea and scope of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

1 Portable information device (user interface device)
DESCRIPTION OF SYMBOLS 100 Touch panel 101 Display 102 Tactile response mechanism part 103 Pressing force detection part 104,104 ', 204,304 Pointer 104e, 104e', 204e, 304e Pointer 105,205,305 Point of finger contact 106 Handwriting image (input by pointer result)
107 Slide switch image 108 Changeover switch image 109 Link (object)
DESCRIPTION OF SYMBOLS 111 Display control part 112 Application process part 121 Contact position determination part 122 Pointer position determination part 123 Pointer display part 124 Pressing force determination part 125 Operation | movement control part 126 Tactile response control part 206 Left arm image (it changes according to the effect | action by the front-end | tip of a pointer Target)
207 Incision image (input result by pointer)
306 Geologic image (object that changes under the action of the tip of the pointer)
307 Excavation image (input result by pointer)

Claims (10)

An image display unit that displays a pointer that indicates an input position with its tip, a touch panel that sequentially outputs a contact position of each finger, and a pressing force detection unit that detects a pressing force applied to the touch panel by the finger A user interface device comprising:
Pointer position determining means for sequentially determining the position of the tip of the pointer according to the contact position at each time point of the finger;
Pointer display means for displaying the pointer so that the tip is determined at each time point;
Whether or not the pressing force detected by a series of operations in which the pointer is displayed at a position corresponding to the contact position at each time point of the finger and the finger is pressed while being in contact is greater than or equal to a predetermined threshold value. A pressing force determining means for determining;
An operation control means for activating an action associated with the position of the tip of the pointer when the pressing force determination means makes a true determination. Contact position determination means for determining whether or not the contact position of the finger overlaps the display position range of the pointer;
The pointer position determination means sequentially determines the position of the tip of the pointer according to the contact position at each time point of the finger when the contact position determination means makes a true determination,
The operation control means displays an input result by the pointer corresponding to a locus of the position of the tip, which corresponds to a locus that is a section in which a true determination is made by the pressing force determination means. Item 4. The user interface device according to Item 1.   When viewed from the user's operation, when the finger is brought into contact with the pointer and the pointer is held, and the finger is moved while being pushed into the pointer, the motion control means adjusts the movement of the finger. The user interface device according to claim 2, wherein an input result by the pointer corresponding to a locus of the position of the moving tip is displayed.   The pointer position determining means calculates a translational conversion for converting the finger contact position at the initial time when the finger contact position overlaps the display position range of the pointer into the position of the tip at the same point. 3. The user interface device according to claim 1, wherein at each subsequent time point, a position obtained by converting the contact position of the finger by the translation conversion is set as the position of the tip of the pointer.   5. The input result by the pointer is a result of a predetermined object displayed on the screen of the image display unit being changed by the action of the tip of the pointer. 6. The user interface device according to item.   The user interface device according to claim 2, wherein the input result by the pointer is a handwriting image including a locus of the position of the tip.   The said pointer display means displays the right-handed pointer and the left-handed pointer separately as said pointer according to predetermined operation with the said finger | toe, The any one of Claim 2 to 6 characterized by the above-mentioned. The user interface device described in 1.   The motion control means activates an action associated with an object including the position of the tip of the pointer within its display position range when the pressing force determination means makes a true determination. The user interface device according to claim 1. An image display unit that displays a pointer that indicates an input position with its tip, a touch panel that sequentially outputs a contact position of each finger, and a pressing force detection unit that detects a pressing force applied to the touch panel by the finger A program installed in a user interface device comprising:
Pointer position determining means for sequentially determining the position of the tip of the pointer according to the contact position at each time point of the finger;
Pointer display means for displaying the pointer so that the tip is determined at each time point;
Whether or not the pressing force detected by a series of operations in which the pointer is displayed at a position corresponding to the contact position at each time point of the finger and the finger is pressed while being in contact is greater than or equal to a predetermined threshold value. A pressing force determining means for determining;
A program for a user interface device, which causes a computer to function as an operation control unit that activates an operation associated with the position of the tip of the pointer when the pressing force determination unit makes a true determination. An image display unit that displays a pointer that indicates an input position with its tip, a touch panel that sequentially outputs a contact position of each finger, and a pressing force detection unit that detects a pressing force applied to the touch panel by the finger An operation activation method in a user interface device comprising:
A first step of sequentially determining the position of the tip of the pointer according to the contact position at each time point of the finger;
A second step of displaying the pointer so that the tip is positioned at each time point;
Whether or not the pressing force detected by a series of operations in which the pointer is displayed at a position corresponding to the contact position at each time point of the finger and the finger is pressed while being in contact is greater than or equal to a predetermined threshold value. A third step of determining;
And a fourth step of activating an action associated with the position of the tip of the pointer when a true determination is made in the third step.

Images