JP2005352738A - Pointing operation system using target position prediction - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interface capable of shortening a time of a pointing operation by a mouse, a 3D tracker, etc., in a GUI environment on a large-sized, high-resolution screen and in a virtual space. <P>SOLUTION: In the pointing operation, the peak speed in pointer movement and the distance up to a target at the point of time are correlative, and thereby the distance to the target is obtained from the peak speed and the target position is predicted from a pointer position and a direction, that an object position to which a user desires to move the pointer can be predicted by a system side before operation completion. Consequently, the system can shorten the operation time by actuating an application that an icon present there indicates and so on. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a pointing operation prediction technique on a screen of an information terminal device or in a virtual reality space, and more particularly to an operation of a pointing device such as a mouse or a three-dimensional tracker.

  In recent years, the information processing apparatus has a large screen and a high resolution display. In such a GUI (Graphical User Interface) environment, the time required for the pointing operation tends to increase, and it is estimated that this tendency will become more prominent in the future. For this reason, an interface capable of reducing the time for pointing operation such as a mouse is required.

  Conventionally, based on the current pointer position, the neighborhood area is used as a search area, and it is checked whether there is an object to be operated in the search area range. If there is, the pointer is automatically moved to the object position. There was something to move (patent document 1).

  In addition, there is a device that checks whether an object to be operated exists on a straight line indicating a pointer movement direction on the screen, and automatically moves the pointer to the position of the object if there exists (Patent Document 2). ).

  Also, when moving the pointer on the screen, the search range of the pointer destination is assumed according to the information on the position, moving direction, and moving speed of the pointer, and is closest to the current pointer position from within the search range Some objects predict a moving object as a destination object and automatically move the pointer (Patent Document 3).

  In addition, some mouse speed profile functions were prepared in advance, the most approximate speed profile function was extracted from the slope of the mouse movement speed, and the target object was predicted based on that, and the pointer was moved automatically. (Patent Document 4).

JP-A-8-76956 JP-A-6-83540 JP-A-10-222307 JP-A-9-244808

  The first problem is that, in the technique described in Patent Document 1, in the method in which the vicinity of the current pointer position is used as the search area, the distance between the object to which the user wants to move the pointer and the current pointer position is large. In many cases, the pointing device must be operated many times, and the operability is not preferable.

  The second problem is that, in the technique described in Patent Document 2, in the method of searching for an object on an extension line in the current pointer movement direction, the distance between the object to which the user wants to move the pointer and the current pointer position. If the object is small, the pointer may move to a distant object that is not intended at all depending on the direction in which the pointer is moved even though the object to which the pointer is to be moved is near the current pointer position. The property is not preferable.

A third problem is that, in the technique described in Patent Document 3, the method of predicting an object assuming a search range of a pointer movement destination according to information on the movement speed of the pointer performs a search according to the movement speed. If the user wants to move the pointer and another object exists between the current pointer position and the range is set, the object cannot be selected and the operability is not preferable.
Further, in the method disclosed in Patent Document 3, a prediction operation is performed when the moving speed becomes zero, that is, when the speed profile is completed. I had to stop it.
Furthermore, in the method disclosed in Patent Document 3, since the pointer movement direction is an extension of the movement direction just before the pointer stops, the movement direction just before the stop is noise when the pointing device is operated quickly. Often increases and does not match the direction of the target object.

The fourth problem is that the speed profile is used in the technique described in Patent Document 4 above, but it is determined from the rise of the moving speed at the start of movement which speed profile is similar to the current movement. For this reason, blurring and errors are increasing. Since the initial movement is noisy and the human movement is particularly jagged, the method of judging by the first movement is not preferable.
In addition, it is necessary to prepare a function group of speed profiles according to the situation in advance.

  As a result of earnestly researching the human interface, the present inventor has found that an application indicated by an icon existing at the location can be predicted by the system side before the operation is completed by the system side when the user wants to move the pointer during the pointing operation It has been found that there is a possibility that the work time can be shortened by starting to start and the work burden on the user can be reduced, which is very effective.

  Further, the inventor has focused on the fact that there is a correlation between the peak speed during the pointer movement and the distance to the target at that point in the pointing operation. That is, the following formula can be used to represent the peak speed during the pointer movement and the distance to the target at that time, and the following formula is used to find that the target position can be predicted from the peak speed, completing the present invention. It came to do.

(Equation 1)
Predicted distance to target = a × peak speed + b

  Here, a and b are parameter coefficients obtained by trials such as calibration, and take different values depending on individual operators performing the pointing operation.

  According to the first aspect of the pointing operation system of the present invention, “when a pointing device such as a mouse is operated on a screen of an information terminal device to move a pointer such as a mouse cursor or three-dimensionally in a virtual reality space” When the pointer is moved by operating a tracker or the like, the information on the pointer position or the pointer speed is captured, and the maximum value (peak speed) of the pointer speed is detected as a point at which the pointer speed changes from an increase to a decrease. The predicted distance from the peak speed to the target is calculated by a linear correlation function, the target predicted position is calculated from the predicted distance to the target, the pointer position at the peak speed, and the pointer moving direction, and is within a certain range from the target predicted position. One of the existing objects (icons, etc.) The objects can be selected "function is provided.

  Here, the primary correlation function is a mathematical expression represented by the above-described equation (1). By indicating the peak velocity as a three-dimensional vector, the present invention can be applied not only to a two-dimensional information terminal screen but also to a pointing device in a three-dimensional virtual space.

  Further, the object existing within a certain range from the target predicted position refers to an object such as an icon within a certain distance range from the target predicted position. The parameter for determining the constant distance is a variable parameter because it affects the resolution closely.

  In addition, as a method for selecting one object among objects (icons or the like) existing in a certain range, for example, an object closest to the target predicted position is selected, or an object that is frequently used is selected. It is possible to select an object suitable for the situation and context of the current situation.

  According to a second aspect of the pointing operation system according to the present invention, in the first aspect, “a target position is predicted every time a maximum value (peak speed) of a pointer speed is detected. When multiple maximum values are detected in the means for detecting the value (peak speed), if the maximum value detected after the second time is smaller than the previous maximum value, that value is used as the peak speed. A “no” feature is provided.

  This is because it is considered that a plurality of maximum values are detected due to the maximum acceleration during the operation or the shaking of the hand.

  According to a third aspect of the pointing operation system of the present invention, in the first and second aspects, the pointer velocity vector at the time of peak velocity is expressed in the X-axis direction and the Y-axis direction on the screen or in space. And a function of calculating a predicted position of the target by predicting a distance to the target in each direction.

  When operating the pointing device quickly, the pointer velocity component is decomposed in the axial direction, taking into account that the direction of movement immediately before stopping is noisy and often does not match the direction of the target object. By recognizing the tangential direction of the pointer trajectory at the time of speed as the moving direction, the target predicted position is minimized.

  According to a fourth aspect of the pointing operation system of the present invention, in the first to third aspects, “when one object among objects existing within a certain range from the predicted position of the target can be selected, A function of automatically starting an application program corresponding to the selected object is provided.

  According to a fifth aspect of the pointing operation system of the present invention, in the first to third aspects, “when one object among objects existing within a certain range from the predicted position of the target can be selected, The function of automatically moving the pointer to the position of the selected object is provided.

  According to a sixth aspect of the pointing operation system of the present invention, in the first to third aspects, “when one object among objects existing in a certain range from the predicted position of the target can be selected, The function of “changing the icon form of the selected object” is provided.

  According to a seventh aspect of the pointing operation system according to the present invention, in the first aspect, when predicting the distance from the peak speed to the target, the distance from the peak speed to the target is determined for each individual user. A calibration function is provided in which the coefficient of the conversion formula to be obtained is determined in advance by several trials.

  According to the pointing operation system of the present invention, even when the distance between the object the user wants to move the pointer and the current pointer position is large, a small number of pointing device operations are sufficient, and the operability is improved.

  Further, according to the pointing operation system according to the present invention, when the object to which the pointer is to be moved is near the current pointer position, the pointing device is normally operated slowly, the peak speed is reduced, and the target prediction is performed. Since the position is in the vicinity, there is no possibility that the pointer moves to a distant object that is not intended at all, and the operability is improved.

  Further, according to the pointing operation system according to the present invention, even when another object exists between the object to which the user wants to move the pointer and the current pointer position, the object cannot be selected. Further, since the prediction can be performed before the speed profile is completed, it is possible to cope with a quicker pointing operation.

  Furthermore, since the predicted pointing position can be calculated directly from the peak speed of the pointing operation, it is not necessary to prepare a function group of the speed profile in advance.

  Embodiments of the present invention will be described below, but the present invention is not particularly limited thereto.

  Hereinafter, a pointing operation using target position prediction on the screen of the information terminal device will be described in detail with reference to a processing flowchart. Usually, in the GUI environment of the information terminal device, a pointer operation on the screen is typically performed with a mouse. While confirming the destination target of the pointer, the operator operates to bring the pointer closer to the target position.

  The present inventor has found that there is a correlation between the peak speed during the pointer movement and the distance to the target at that point in the pointing operation, and predicting the target position from the peak speed, the pointing operation time and the target A pointing operation system that can shorten the action time for an object (for example, the startup time of an application program) has been completed.

  The pointing operation system according to the present embodiment is composed of two processes. One is a process of detecting a change in the pointer that changes from moment to moment. Specifically, the program reads the position coordinates of the mouse periodically in 10 ms. The other is a process that periodically performs target position prediction algorithm processing, specifically, a program that performs target position prediction algorithm processing at a slower period than a program that reads mouse position coordinates, for example, 100 ms. .

  FIG. 1 shows an algorithm for target position prediction. The target position prediction algorithm calculates the peak velocity of the pointer from the position coordinates of the mouse pointer that changes from moment to moment, calculates the distance from the peak velocity to the target using a correlation function, and the pointer when the peak velocity is detected The target position is predicted from the coordinates and the progression vector.

  In the target position prediction algorithm, the peak speed is initialized as an initialization process, and the processes 1 to 4 in FIG. 1 are periodically repeated until the pointing task (program for performing the target position prediction algorithm process) is completed. Pointer peak speed detection processing 1 detects a peak speed and obtains a peak speed, a peak position, and a peak direction. Next, the target position prediction process 2 predicts the target position from the obtained peak speed, peak position, and peak direction. Then, the object estimation process 3 estimates an object closest to the target predicted position among the objects within a certain range from the target predicted position as a target. The object selection process 4 is a process for executing, for example, an activation command for the selected object. Also, the icon form of the selected object may be changed simultaneously with activation or without activation.

  FIG. 2 shows a flowchart of the peak speed detection process of the pointing operation system. Hereinafter, the peak speed detection process will be described. Step S201 initializes variables to be used. In step S202, it is determined whether or not the pointer has moved by detecting a signal from a pointing device such as a mouse. In step S203, the position and time of the pointer are obtained by using the current position () and the current time (), which are subroutine functions, and are set as position B and time B, respectively. These current position () and current time () are provided by the OS (operating system) of the computer that performs processing. In step S204, if the position A is an initial value, the position B and time B are substituted into the position A and time A in step S208. In step S205, a local direction starting from position A and ending at position B is calculated from the coordinates between the two positions A and B. Further, the direction of the pointer up to the previous time is set as the average direction, the vector sum with the local direction calculated this time is calculated, and the value of the average direction is updated accordingly. In step S206, the speed of the pointer is calculated from position A, position B, time A, and time B, and is substituted for speed B. In step S207, this speed B is compared with the peak speed value (pi speed) stored so far, and if the speed B is equal to or higher than the peak speed, the speed B, position B, and average direction are respectively determined in S209. , Peak speed (pi speed), peak speed position (pi position), and peak speed direction (pi direction) are substituted. If the speed B is smaller than the peak speed value (pi speed) stored so far, the peak speed detection process is terminated.

  FIG. 3 shows a flowchart of processing for predicting the target position. In step S301, each component of the peak speed in the X-axis direction, Y-axis direction, and Z-axis direction is calculated from the magnitude and direction of the peak speed. Here, in the case of a screen of a computer information terminal, since it is a two-dimensional coordinate, a horizontal component (X-axis direction) and a vertical component (Y-axis direction) are obtained. Next, a predicted distance from the speed to the target is obtained using a correlation formula between the peak speed and the distance to the target. Here, when a pointing operation of a mouse or the like is performed by a person, a correlation formula (distance to target = a × peak) between the case where the mouse is operated vertically on the screen and the case where the mouse is operated horizontally. The values of the parameter coefficients a and b of the speed + b) are different. For this reason, when the predicted distance is obtained using the correlation equation, the correlation equations of the parameter coefficients a and b which are different from each other are used for the velocity of the peak velocity in the X-axis direction component, the Y-axis direction component and the Z-axis direction component. I am going to do that. In step S302 of FIG. 3, the predicted distance to the target in the X-axis direction is calculated by substituting the X-axis direction component of the peak velocity into Equation 1. Similarly, the Y axis direction component of the peak velocity is substituted into Equation 1, the predicted distance to the target in the Y axis direction is calculated, the Z axis direction component of the peak velocity is substituted into Equation 1, and the Z axis The predicted distance to the target in the direction is calculated. In step S303, the target predicted coordinates are calculated from the pointer coordinates when the peak speed is detected and the predicted distance to the target in the XYZ-axis directions.

  FIG. 4 shows a flowchart of the object estimation process. First, in step S401, two variables (shortest distance and target) are initialized. Here, the shortest distance is a variable for obtaining the distance from the predicted position to the nearest object in the range, and the target is a variable for obtaining the ID of the object predicted with the target. In step S403, the distance from the target predicted position to the object is calculated from the target predicted position coordinates and the position coordinates of each object. Step S404 determines whether the distance from the target predicted position to the object is the shortest. In step S405, when the distance to the object is the shortest, the two variables, the shortest distance and the target value, are updated. Here, steps S403 to S405 are repeated so as to be performed on all objects existing within a certain distance range from the target predicted position coordinates.

  Next, calibration for obtaining the values of the parameter coefficients a and b of the correlation equation (distance to target = a × peak speed + b) will be described. FIG. 5 shows a flowchart of the calibration process. The value of the parameter coefficient varies depending on the individual and the operating environment. Decide the target in advance and repeat the analysis from each peak speed, each peak position, and each peak direction by repeating the mouse operation several times to select the target. An expression for predicting the distance to the target is generated. For determining the parameter coefficients a and b, the least square method or the like is used.

  Tables 1 and 2 below summarize the variables and subroutine functions used in the processing flowcharts of FIGS.

  The present invention provides an interface that uses prediction of a target position from a peak velocity, a pointing operation system such as a mouse accompanying an increase in the screen size of an information processing apparatus, and a three-dimensional in virtual reality (VR). It can be used for devices that manipulate objects, 3D trackers, and isotonic devices.

2 shows a target position prediction algorithm of a pointing operation system according to the present invention. An example of a flowchart of peak speed detection processing is shown. An example of the flowchart of a target position prediction process is shown. An example of the flowchart of an object estimation process is shown. 3 shows a flowchart of a calibration process of the pointing operation system.

Explanation of symbols

1 Pointer peak speed detection process 2 Target position prediction process 3 Object estimation process 4 Object selection process

Claims (7)

When moving a pointer such as a mouse cursor by operating a pointing device such as a mouse on the screen of the information terminal device, or when moving a pointer by operating a three-dimensional tracker in a virtual reality space,
Capture the pointer position or pointer speed information,
2) The maximum value of the pointer speed (peak speed) is detected as the point at which the pointer speed turns from increasing to decreasing,
3) Calculate the predicted distance from the peak velocity to the target using a first order correlation function;
4) Calculate the target predicted position from the predicted distance to the target, the pointer position at the peak speed, and the pointer movement direction,
A pointing operation system capable of selecting one object among objects (such as icons) existing within a certain range from the target predicted position.   The target position is predicted every time when the maximum value (peak speed) of the pointer speed is detected. When a plurality of maximum values are detected in the means for detecting the maximum value (peak speed) of the pointer speed, 2. The pointing operation system according to claim 1, wherein when the maximum value detected after the second time is smaller than the previous maximum value, the value is not adopted as the peak speed.   The vector of the velocity vector of the pointer at the peak velocity is decomposed into components in the X-axis direction, the Y-axis direction, and the Z-axis direction on the screen or in space, and the distance to the target is predicted for each direction, thereby predicting the target. 3. The pointing operation system according to claim 1, wherein a position is calculated.   4. The application program according to the selected object is automatically started when one of the objects existing within a certain range from the predicted position of the target can be selected. The pointing operation system described.   The pointer is automatically moved to the position of the selected object when one of the objects existing within a certain range from the predicted position of the target can be selected. Pointing operation system.   4. The pointing operation according to claim 1, wherein when one object among objects existing within a certain range from the predicted position of the target can be selected, the icon form of the selected object is changed. system. In the case of predicting the distance from the peak speed to the target, it has a calibration function for determining the coefficient of the conversion formula for obtaining the distance from the peak speed to the target for each individual user in advance by several trials. The pointing operation system according to claim 1.

Images