JP5501715B2 - User interface device, control method for user interface device, and program - Google Patents

Description

  The present invention relates to a user interface device, a user interface device control method, and a program.

  When the user touches the screen of the touch panel as the user interface device with a fingertip, the coordinates of the point touched with the fingertip are sent from the touch panel to the gesture analysis unit. The gesture analysis unit determines what gesture it is from the coordinates and the history of coordinates sent so far, and the gesture determination result (page feed, rotation, zoom, etc.) and its parameters (if page feed, moves) The amount, the angle for rotation, and the magnification for zoom). If no gesture can be determined from the history of coordinates up to that point, the next touch is awaited. If it is determined that the gesture has been completed or interrupted, the coordinate history data up to that point is erased and the next touch is awaited.

  The application performs an operation according to the received gesture and its parameters. For example, in the image display software, a photograph displayed on the screen is switched by a page turning gesture, a display image is rotated by a rotation gesture, and an image is enlarged (reduced) by a zoom gesture.

This type of user interface device is disclosed in Patent Document 1.
The invention described in Patent Document 1 is an invention related to “user interface device and user interface method”. Specifically, “a guide locus that stores a gesture to which a predetermined command is assigned and guides the gesture” Storage means for storing, a trajectory analysis means for analyzing the user's operation trajectory for the trajectory input device, and each time the trajectory analysis means analyzes the user's operation trajectory, from the gestures stored in the storage means, Gesture search means for searching for one or more gestures that match the operation trajectory, and obtaining the guidance trajectory of the gesture searched for by the gesture search means from the storage means and displaying the guidance trajectory, while searching for the previous time by the gesture search means Does not match the user's operation trajectory Then, a guidance trajectory display means for deleting the guidance trajectory of the displayed gesture, and a command calling means for calling a command assigned to the gesture when the gesture searched by the gesture search means is narrowed down to one. It's a thing.
That is, the invention described in Patent Document 1 searches for one or more gestures that match the user's operation locus, and displays the guidance locus. When a gesture is narrowed down to one, a command associated with the gesture is called.

JP 2006-331210 A

  However, the above-described conventional technique guides what kind of gesture operation can be performed, but what commands are associated with the movement amount and the movement direction with respect to a gesture that performs processing according to the movement amount and the movement direction. It is not considered at all to indicate whether or not

  In addition, in the prior art, in order to guide a gesture such as a fixed figure with one finger, both fingers are moved by multi-touch, and processing according to the relative movement (amount and direction) of both fingers is performed. When gesturing, it is considered to change the guidance of the gesture depending on how much of which finger is moved (whether the first finger is fixed and the second finger is moved, or both fingers are moved by the same amount). There is room for improvement in operability.

  Accordingly, an object of the present invention is to provide a user interface device, a user interface device control method, and a program with improved operability in order to solve the above-described problems.

The apparatus of the present invention includes a touch panel for detecting a finger trajectory from the start of contact, a storage unit for storing a correspondence between a gesture that is the finger trajectory having a predetermined regularity and a predetermined process, and the touch panel. When the finger trajectory from the start of contact to the end of contact corresponds to the regularity of the gesture, processing means for executing the predetermined process according to a parameter determined according to the contact end point and detected by the touch panel It is determined whether or not the finger trajectory from the start of touching corresponds to the regularity of the gesture, and in this case, a guide trajectory obtained by extending the finger trajectory according to the regularity and a plurality of the trajectories on the guide trajectory characterized by comprising a control means for controlling so that each predicted process results displayed on the touch panel when the touch end at point

  According to the present invention, a guidance trajectory is displayed on the touch panel based on the trajectory of the finger, and on the guidance trajectory, a point to be a point and a result expected when operating up to that point are displayed. It is possible to realize provision of a user interface device, a user interface device control method, and a program with improved performance.

It is a flowchart which shows one Embodiment of the control method of the user interface apparatus which concerns on this invention. It is a block diagram which shows one Embodiment of the user interface apparatus which concerns on this invention. (A), (b) is explanatory drawing which shows one Example of the control method of the user interface apparatus based on this invention. (A), (b) is explanatory drawing which shows the other Example of the control method of the user interface apparatus based on this invention. (A), (b) is explanatory drawing which shows the other Example of the control method of the user interface apparatus based on this invention. (A), (b) is explanatory drawing which shows the other Example of the control method of the user interface apparatus based on this invention. (A)-(c) is explanatory drawing which shows the other Example of the control method of the user interface apparatus based on this invention.

Next, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a flowchart showing an embodiment of a control method for a user interface device according to the present invention.
FIG. 2 is a block diagram showing an embodiment of a user interface device according to the present invention.

<Configuration>
The user interface device shown in FIG. 2 includes a touch panel 1, a CPU 2, a ROM 3, a RAM 4, an I / O port 5, and a bus line 6.

The touch panel 1 is an input device in which a liquid crystal display device and a panel switch are integrated, and the panel switch has a function of generating a signal at a point where a finger is touched.
For example, the panel switch may use any one of a resistive film type, an infrared type, an ultrasonic surface acoustic wave type, an acoustic wave collation type, and a projected capacitive type.

  The resistive touch panel has a parallel transparent electrode (sensor wire) on a glass substrate, a spacer on the parallel transparent electrode, and another transparent parallel electrode (on the spacer so as to be orthogonal to the transparent parallel electrode ( Sensor wire), and the coordinates of the intersection point can be understood by contacting the transparent electrodes by pressing the uppermost transparent parallel electrode.

  An infrared touch panel is configured so that the coordinates of the intersection point can be known by running an infrared beam vertically and horizontally and detecting that the finger is shielded from light.

  The ultrasonic surface acoustic wave type touch panel utilizes surface acoustic waves that propagate in a concentrated manner near the surface of an object.

  The acoustic wave type touch panel detects an acoustic pulse generated by touching the surface of the touch panel and is transmitted through the touch panel, and determines a touch position by signal analysis.

  The projected capacitive touch panel is the same as the resistive film type in that the electrodes run vertically and horizontally, but the touched location does not conduct like the resistive film, but the electrostatic capacitance between the electrodes It captures changes in capacity.

  The CPU 2 has a function including a coordinate detection unit 2a, a gesture completion determination unit 2b, a gesture type determination determination unit 2c, and a guidance display touch determination unit 2d. For example, a microprocessor is used.

The coordinate detection unit 2 a is a function that detects the coordinates of the point of the finger touched on the screen of the touch panel 1.
The gesture completion determination unit 2b is a functional unit that determines whether or not a gesture by moving while a finger is touched on the touch panel 1 is completed.
The gesture type determination determination unit 2c is a functional unit that determines whether it is determined whether the type of gesture by moving while the finger is touched on the touch panel 1 is page turning, rotation, enlargement, or reduction. .
The guidance display touch determination unit 2d is a functional unit that determines whether the guidance display displayed on the screen of the touch panel 1 has been touched.

  The ROM 3 is a memory that stores a control program for overall control of the user interface device and gesture pattern data.

  The RAM 4 is a memory that stores coordinate data input by the touch panel 1.

  The I / O port 5 is a port for connecting to an external electronic device such as a personal computer. For example, a USB (Universal Serial Bus), RS232C, or the like is used.

<Operation>
Next, the operation of the user interface device shown in FIG. 2 will be described with reference to FIG.
The subject of operation is the CPU 2.
First, the CPU 2 detects the coordinates of the point where the finger is touched on the screen of the touch panel 1 (step S1).
The CPU 2 stores the coordinates of the point where the finger on the screen of the touch panel 1 is touched in the RAM 4 (step S2).
The CPU 2 determines a gesture from the coordinate history (step S3).
The CPU 2 determines whether or not the gesture has been completed (step S4).
If the CPU 2 determines that the gesture has been completed (step S4 / Y), it executes the gesture process and ends (step S9).

If the CPU 2 determines that the gesture has not been completed (step S4 / N), it determines whether or not the type of gesture has been determined (step S5).
When the CPU 2 determines that the type of gesture has been determined (step S5 / Y), the CPU 2 displays a guidance trajectory on the screen of the touch panel 1 (step S6), and displays guidance for the point to be a point on the screen of the touch panel 1. (Step S7).
The CPU 2 determines whether or not the guidance display displayed on the screen of the touch panel 1 has been touched (step S8). If the CPU 2 determines that the guidance display has been touched (step S8 / Y), it executes the gesture process and ends. (Step S9).
When the CPU 2 determines that the guidance display on the screen of the touch panel 1 is not touched (step S8 / N), the CPU 2 returns to step S1 and repeats the following.

<Program>
The user interface device according to the present invention described above is realized by a program that causes a computer to execute processing. Examples of the computer include general-purpose computers such as a microprocessor, but the present invention is not limited to this. Therefore, as an example, a case where the present invention is realized by a program will be described below.

As a program, on a computer,
(1) A procedure for determining whether the touched finger is a gesture,
(2) When a finger motion is a gesture, a procedure for displaying a guidance trajectory on the touch panel based on the trajectory associated with the gesture,
(3) On the guidance trajectory, a procedure for displaying the point to be a point and the expected result when operating to that point,
A program that executes

Thus, the user interface device according to the present invention can be realized anywhere as long as there is a computer environment capable of executing the program.
Such a program may be stored in a computer-readable storage medium.

<Storage medium>
Here, examples of the storage medium include computer-readable storage media such as a CD-ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (CD Recordable), and a DVD (Digital Versatile Disk). , Flash memory, RAM (Random Access Memory), ROM (Read Only Memory), semiconductor memory such as FeRAM (ferroelectric memory), and HDD (Hard Disc Drive).

  The above-described embodiment shows an example of a preferred embodiment of the present invention, and the present invention is not limited thereto, and various modifications can be made without departing from the scope of the invention. is there.

FIGS. 3A and 3B are explanatory views showing an embodiment of a control method of the user interface device according to the present invention.
When a photo (face photo) is displayed on the screen 1a of the touch panel 1 (see FIG. 2), the user touches with a finger (for example, the right index finger) 10 and traces in the direction of the arrow 11 (so-called panning), thereby gesturing The type determination determining unit 2c determines that the user's operation is panning, and the page of the screen 1a is paged (FIG. 3A).

  When the finger 10 keeps touching the screen 1a, the screen is switched to the screen 1b, and the shooting history 12 as a guidance display is superimposed on the screen 1a. In this case, it is shown that the position of the finger 10 is around the end of March in the shooting history 12.

  By displaying guidance on the screen 1a, the user grasps when the current photograph is a photograph taken and to what extent the finger should be moved to display the photograph on the target photographing date. In addition, the photograph of the photographing month can be displayed by directly touching the photographing month of the photographing history 12, so that the operability is improved.

4A and 4B are explanatory views showing another embodiment of the control method of the user interface device according to the present invention.
When a face photo is displayed sideways on the screen 1 a of the touch panel 1, the user touches the center of the screen 1 a with one finger (for example, the finger of the left hand) and the other finger (for example, the finger of the right hand) 10. If the upper side of the screen 1a is touched (multi-touch) and moved in the direction of the arrow 14, the gesture type determination determining unit 2c determines that the user's operation is rotation, and displays a guidance trajectory associated with the rotation and a guidance display. A reduced image after rotation is displayed (FIG. 4A).
That is, the touch panel 1 displays a guidance trajectory T1 of the finger 10 on the screen 1a, and further displays a reduced image 1b of the current orientation on the upper side of the screen 1a on the guidance trajectory T1,
A reduced image (in this case, an upright image) 1c rotated 90 degrees on the right side of the screen 1a is displayed, a reduced image 1d after 180 degrees rotated is displayed on the lower side of the screen, and rotated 270 degrees on the left side. A later reduced image (inverted image in this case) 1e is displayed (FIG. 4B).

  Here, when the finger 13 is fixed and the finger 10 is moved to the target position on the guidance trajectory T1, the image is rotated to the angle of the moved position, but the finger 10 and the finger 13 are moved from the screen 1a. When it is released, the guidance display 1b to 1e is kept displayed for a certain period of time (for example, 2 to 3 seconds), and it is selected by touching any one of the guidance displays 1b to 1e on the screen 1a during that time. The rotation processing in the selected state may be performed.

  With these guidance displays, the user can grasp how the finger is displayed to what position, and touch the reduced images 1b, 1c, 1d, and 1e by directly touching them. Therefore, the operability is improved.

FIGS. 5A and 5B are explanatory views showing another embodiment of the control method of the user interface device according to the present invention.
When the finger 10 and the finger 13 are simultaneously moved on the screen 1a of the touch panel 1 in the directions of arrows 16 and 17 so as to leave at substantially the same speed (FIG. 5A), the gesture type determination determining unit 2c zooms in on the user's operation. The guidance trajectory T2, the rectangular frame lines L1 to L3, and the explanations 18 and 19 are displayed as guidance displays on the screen 1a.
That is, the explanation display of “fit on screen” is linked to the frame line L1, the frame line L2 is linked to the fingers 10 and 13, and the comment display of “actual size” is linked (pixel same size) to the frame line L3. The size is selected by moving the fingers 10 and 13 to a desired size or touching the guidance display.
Since the user can select a desired size based on the guidance display, the operability is improved.

FIGS. 6A and 6B are explanatory views showing another embodiment of the method for controlling the user interface device according to the present invention.
The difference between the embodiment shown in FIGS. 6A and 6B and the embodiment shown in FIGS. 5A and 5B is that the explanation display in the guidance display is an icon.
That is, the icon 20 corresponds to the frame line L1, and the icon 21 corresponds to the frame line L3.
Even if configured in this way, the user can select a desired size based on the guidance display, so that the operability is improved.

FIGS. 7A to 7C are explanatory views showing another embodiment of the method for controlling the user interface device according to the present invention.
The difference between the embodiment shown in FIGS. 7A to 7C and the embodiment shown in FIGS. 5A and 5B is that the fingers 10 and 13 are simultaneously moved in the directions of arrows 16 and 17 at substantially the same speed. In the middle of moving away, the finger 13 is further moved away in the direction of the arrow 22 while the finger 10 is fixed.
In this case, guidance display is performed based on the most recent ratio of the movement amounts of both fingers. That is, while the fingers 10 and 13 are moving at the same time, the moving ratio of the finger 10 and the finger 13 is the same, so the frame lines L1 to L3 are connected to the finger 10 and the finger 13 as shown in FIG. When the finger 10 is stopped and the finger 13 is moved, the position where the frame lines L1 to L3 are displayed is displayed as shown in FIG. 7C. Thus, it switches to the position when only the finger 13 has moved.
Even if configured in this way, the user can select a desired size based on the guidance display, so that the operability is improved.

As described above, according to the present embodiment, it is possible to perform a guidance display as to how much and in what amount the gesture is performed according to the movement amount and the movement direction.
In the case of multi-touch, it is possible to display an appropriate guidance according to the ratio of moving both fingers.
The displayed guidance can be directly touched to perform processing resulting in the point.

1 Touch Panel 1a Screen 1b-1e Reduced Image 2 CPU
2a Coordinate detection unit 2b Gesture completion determination unit 2c Gesture type determination determination unit 2d Guidance display touch determination unit 3 ROM
4 RAM
5 I / O port 6 Bus line 10, 13 Finger 11, 14, 16, 17, 22 Arrow 12, 18, 19 Guidance display 20, 21 Icon L1-L4 Frame line T1 Guidance locus associated with the gesture of rotation T2 Guide trajectory associated with zoom gesture

Claims (8)

A touch panel that detects the trajectory of the finger from the start of contact;
Storage means for storing a correspondence relationship between a gesture that is a trajectory of the finger having a predetermined regularity and a predetermined process;
Processing means for executing the predetermined processing according to a parameter determined according to a contact end point when a finger trajectory from the start of contact to the touch panel corresponds to the regularity of the gesture;
It is determined whether or not the finger trajectory detected from the contact start detected by the touch panel corresponds to the regularity of the gesture, and if so, the guide trajectory obtained by extending the finger trajectory according to the regularity and the guidance each expected processing results and control means for controlling so as to display on the touch panel when the touch end by a plurality of points on the trajectory,
A user interface device comprising:   The user interface device according to claim 1, wherein the control unit displays the expected result in a predetermined vicinity on the guide locus. A finger trajectory from the start of contact is detected on the touch panel, a correspondence relationship between the gesture that is the trajectory of the finger having a predetermined regularity and a predetermined process is stored, and the finger from the start of contact to the touch panel to the end of contact is stored. When the trajectory of the touch corresponds to the regularity of the gesture, the predetermined processing is executed according to a parameter determined according to the contact end point, and the trajectory of the finger detected from the touch start detected by the touch panel is the gesture Whether or not it corresponds to the regularity of the information, and in the case of corresponding, the guidance trajectory obtained by extending the trajectory of the finger according to the regularity and respective prediction processes when the contact is ended at a plurality of points on the guidance trajectory the method of the user interface and displaying the result in the touch panel.   4. The method of controlling a user interface device according to claim 3, wherein when a finger contact position on the guide locus changes, an expected display is changed based on the latest position.   After the finger is released from the touch panel, the guidance trajectory is displayed for a certain period of time, and when the point where the expected result is displayed is directly touched, the process that results in the result is performed. The control method of the user interface device according to claim 3 or 4.   The method for controlling a user interface device according to any one of claims 3 to 5, wherein, when there are a plurality of fingers, a guide locus is displayed according to a relative movement of each finger.   7. The method of controlling a user interface device according to claim 6, wherein when the second finger is operated while the first finger is fixed among the plurality of fingers, a guide locus is displayed for the second finger. On the computer,
The procedure for the touch panel to detect the trajectory of the finger from the start of contact,
A procedure for storing a correspondence relationship between a gesture that is a trajectory of the finger having a predetermined regularity and a predetermined process;
A procedure in which processing means executes the predetermined process according to a parameter determined according to a contact end point when a finger trajectory from the start of contact to the touch panel to the end of contact corresponds to the regularity of the gesture. ,
The control means determines whether or not the finger trajectory detected from the touch start detected by the touch panel corresponds to the regularity of the gesture, and in the case of the guidance, the finger trajectory is extended according to the regularity. procedure for controlling so that each predicted process results when in contact ends a locus and a plurality of points on the guide track displayed on the touch panel,
A program characterized by having executed.

Images