JP2003122506A - Coordinate input and operational method directing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a coordinate input and operational method directing device, capable of simultaneously indicating the coordinate position of an operational object and the classification of an operational method instruction. SOLUTION: This coordinate inputting and operational method direction device includes a two-dimensional pressure distribution sensor capable of indicating a plurality of coordinate positions at the same time, a recording means for temporarily recording the pressure distribution detected by the pressure distribution sensor, a means for computing one or more sets of the maximum value of pressure and the coordinate position in the detected pressure distribution, a means for comparing the temporarily recorded past pressure distribution with the present pressure distribution to obtain a change by operation, and a means for discriminating the classification of an operational method instruction by the above operation.

Description

Description: BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] The present invention relates to an information processing apparatus having a graphical user interface, and to an instruction device for instructing an operation on an arbitrary object on a screen. . 2. Description of the Related Art Conventionally, various planar position detecting devices have been used as devices for instructing a computer terminal of a coordinate position to be operated. However, the planar position detecting device indicates a coordinate (usually a single coordinate) on a screen, and is a general term for an input device for performing various operations based on the coordinate. It is used for designating a figure or character on the screen or selecting a function displayed on the screen by reflecting the movement (dimensionally), the change of the absolute coordinates, or the movement of the pointer on the screen. [0003] In general, devices such as a mouse, a trackball, a touch pen, a tablet, a touch panel, a joystick, and the like are meant. In addition, there is a touch panel as a device that can easily obtain a pseudo sensation such that an operator directly touches an operation target. In a resistive film system (pressure-sensitive system), which is a typical system of this touch panel, a coordinate input surface is formed by stacking two or more glass plates or resin films each having a (transparent) resistor, and an instruction is given on the surface. The two (transparent) resistors contact each other at the point where the contact is made. At this time, a method is employed in which a current is supplied to these resistors to detect a change in the resistance value, thereby determining the contact point and converting the contact point into the position coordinates. [0004] However, one finger is
As an operation using only the book, it can be considered that an operation such as designation of a target point, movement instruction, or the like is to be performed.
Also, although it is conceivable that an operation such as pressing an object in the image displayed at the target point is supposed,
A normal touch panel device has only a coordinate pointing interface by pressing a single fingertip, and has no left button, right button, etc. mounted on a mouse device, so it is possible to specify coordinates, but input commands at the same time. Can not do it. For this reason, as shown in FIG.
As long as only the book is used, the intentions of these operators cannot be conveyed accurately.Therefore, the operation instructions to the program are changed by operating the keyboard device while pressing a specific key, Operation of a plurality of steps, such as performing an operation on a target object after performing a selection operation on an operation menu displayed at a certain position on the screen, is cumbersome and intuitive. [0005] The mouse device usually has one to three buttons. Even in this case, it is necessary to select and drag an object, and to use both buttons and a keyboard for switching operations. There was annoying, such as two actions. [0006] Furthermore, since the operator only operates the object, not the object itself, but the pointer pointing to the object, there is a drawback that the operator does not have a feeling of directly operating the object (object) and is hardly familiar. SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object thereof is to provide a coordinate input and operation method instruction device capable of simultaneously indicating an operation target coordinate position and an operation method instruction type. It is in. [0008] In order to achieve the above object, the present invention provides a two-dimensional pressure distribution sensor which can simultaneously designate a plurality of coordinate positions, and a pressure distribution detected by the pressure distribution sensor. Recording means for temporarily recording; means for calculating a plurality of sets of the maximum value of the pressure in the detected pressure distribution and its coordinate position; and temporarily recorded past pressure distribution and current pressure distribution. A coordinate input and operation method instruction device is configured to include means for comparing and calculating the change by operation and means for determining the operation method instruction type by the operation. Embodiments of the present invention will be described below with reference to the accompanying drawings. As an application example requiring a feeling of directly operating an object (object), a system for deforming and editing a two-dimensional graphic model or the like is considered. FIG. 1 is a drawing that best illustrates the features of the present invention.
Is a display device, on which an object to be operated according to the present invention is displayed by an application for deforming and editing a 3D graphic model or the like. Reference numeral 102 denotes a touch panel, which comprises a number of pressure-sensitive sensors 107 arranged on vertical and horizontal grids. In the present embodiment, a 3 mm square pressure-sensitive sensor is 2 mm
Were arranged at intervals. A pressure distribution determining unit 103 scans the pressure-sensitive sensor 107 disposed on the touch panel unit 102 and determines the position of the pressure-sensitive sensor 107 at which pressure is detected and the pressure-sensitive value. Reference numeral 104 denotes a buffer for storing pressure distribution information for a plurality of scans determined by the pressure distribution determiner 103, and can extract pressure distribution information in order from the oldest information. Reference numeral 105 denotes a discriminator for the operation type and the coordinates to be operated. The latest pressure distribution information obtained from the pressure distribution determiner 103 based on the algorithm shown in FIG. Then, the coordinate position to be operated and the operation type information are determined from the change in the pressure distribution information. Reference numeral 106 denotes a pseudo mouse control driver software which stores information on the operation target coordinate position and the operation type information determined by the operation type and operation target coordinate discriminator 105 by the OS.
Or, convert to a format suitable for the application. FIG. 2 is a conceptual diagram illustrating a situation in which the system of the present invention is used, and an outline of the apparatus / system of the present invention will be described based on these drawings. In FIG. 2, reference numeral 202 denotes a computer to which a keyboard 203 for inputting character data and a display device 200 for image display (connected by a display cable 205) are connected. A touch panel 201 as a planar position detecting device for inputting an operation to a virtual space by an operator 207 is connected to a computer 202 via a data transfer cable 206. The computer 202 is connected to the touch panel 20
1 is scanned to determine the pressure distribution, the pressure distribution information is stored, the pressure distribution information is compared with the recorded pressure distribution information, and a coordinate position and an operation type to be operated are based on the comparison data. Based on the result, reflection of the operation on the model and the viewpoint in the virtual space, generation of a display display image, and other data input and command input from the operator are executed. When the system operator 207 activates the image display system, the computer 202 first reads data of the model 204 to be edited. Next, the operator 207 uses the keyboard 203 to input initial values related to the position detecting device. After the input of the initial value, the model 204 is displayed on the display device 200 and the touch panel can be edited. Here, the system operator 207
When a position is input from the touch panel 201, the detected position is sent to the computer 202 by serial transfer such as RS-232C. Then, the computer 202
Based on these detected positions, the vertex coordinates of the model 204 are changed or the model is rotated appropriately. Next, the computer 202 generates a two-dimensional image based on each data of the model and causes the display device 200 to display the two-dimensional image. When the operator 207 sends a command for terminating the system by operating the touch panel 201, the computer 202 writes the edited model data to a file, and then terminates the system. With the above execution, the operator 207 of the system can use the touch panel 201 to execute the deformation and turn of the model by a more intuitive operation. In FIG. 4 and FIG.
3 shows the state where the entire palm is pressed against 00
01. A pressure-sensitive sensor capable of independently detecting pressure is provided in each cell, but only the pressure-sensitive sensor 310 whose pressure is detected is shown as 310. By the way, the method which has been used as a method of giving some operation instruction by touching the touch panel 300 is a method of pressing only one fingertip as shown by 302. At this time, the pressure may be detected simultaneously by a plurality of pressure sensors. Here, the point 311 at which the detected pressure indicates the maximum value is determined as the operation target position. Reference numeral 312 denotes the origin of the x and y coordinates introduced to indicate the position of each pressure sensor on the touch panel 300. Reference numeral 313 denotes a palm pressed on the touch panel unit 300. When the entire palm is pressed, it is inevitable to apply even pressure to the entire portion touching the surface of the touch panel unit 300, so that only part of the range covered by the palm is detected or the detected pressure is Not uniform. Here, one finger is touched on the touch panel 300
In the case of pressing 303 to a certain position on the upper side and moving 304 to another position as it is, this operation can be replaced by the pseudo mouse control driver 106 with the operation of moving a pointer in a general mouse device. In addition, when a fingertip is pressed in a certain time series and then stopped without moving the position of the fingertip, it can be regarded as a click operation of the mouse device. Therefore, the present invention can be used simultaneously with a general application program already distributed. Reference numeral 305 denotes a state in which a wide area of the palm is pressed against the touch panel 300. When the palm is moved in the direction of the arrow while pressing the palm over a wide area 306, it is considered that this indicates an operation of moving the target object on the screen. In a case 322 where two or more points are pressed with a plurality of fingertips as in 321 and the point is rotated without changing the distance (or positional relationship) between the points, 322 It is assumed that an operation to rotate the target object on the screen is instructed. Reference numeral 320 denotes the center of gravity G of the maximum points a, b, and c in 321. When an operation of pressing two or more points using a plurality of fingertips as in H.323 to reduce the distance between each point and each local maximum point, or an operation 324 of bringing the fingertips closer to each other, Is instructing an operation to reduce the target object. As shown in 325, two or more points are pressed using a plurality of fingertips to increase the distance between each point and each local maximum point, or to perform an operation 326 to move the fingertips away from each other. In this case, it is considered that an operation for enlarging the target object is instructed. In any of these operation instructions, the operation target object and the operation type can be specified at the same time, so that the two-dimensional graphic model or the like can be easily modified and edited. These operations can be determined by the algorithm shown in FIG. The algorithm will be described below step by step. In step 401, the pressure distribution judging device 10
By the operation of 3, the respective pressure-sensitive sensors 107 disposed on the touch panel unit 102 are energized, and a change in the resistance value is obtained, thereby detecting the pressure from each of the pressure-sensitive sensors 107. In step 402, the pressure distribution information determined in step 401 is stored in the pressure distribution information storage buffer 10.
4 and at the same time, the oldest pressure distribution information remaining in the buffer 104 is extracted. The number of storages in the buffer 104 can be set freely according to the frequency of the pressure distribution determination in 401. In step 403, based on the pressure distribution information determined in step 401, the pressure value at each pressure detection point is compared with the pressure detection value of the pressure-sensitive sensor 107 adjacent to the detection point, and the values in the vicinity of 8 are detected. Calculate the local maximum. Next, the coordinates of these local maximum points are used as a vector (x, y) to obtain an average value, thereby obtaining the current center-of-gravity position coordinates G between the local maximum points. In step 404, the sum (average) of the number (area) of pressure-detected sensors, the pressure value, and the average pressure are calculated from the current pressure distribution information determined in step 401. In step 405, it is checked whether or not there is a pressure sensor whose pressure value has been detected from the previous pressure distribution information extracted in step 402. If the pressure value has not been detected before, this indicates that the touch panel has been touched for the first time, and thus this operation can be interpreted as a click operation 420. At this time, the current center of gravity point G calculated in step 403 is determined as the target coordinates of this operation. If a pressure value has been detected in the previous pressure distribution information, the process proceeds to the next step 406, where the maximum point coordinates and the barycentric position coordinates are obtained from the previous pressure distribution information as in step 403. . In step 407, the number of currently detected maximum points is checked. If there is only one maximum point, it is assumed that this is operated with one finger. When there are six or more maximum points, it is assumed that the operation is performed using a wide range such as the entire palm. In step 408, if it is determined in step 407 that the maximum point is 5 or less, it is determined whether or not the total pressure detection area is greater than 30 points. This is because when the pressure is detected at more than 30 points from the comparison between the size and interval of the pressure-sensitive sensor in the present embodiment and the number of sensors that can be generally pressed simultaneously by a human fingertip, Because it can be determined that a part other than is used. When the size and interval of the sensor are changed, the number of points used for this determination and the number of neighbors for determining the maximum point naturally change. In step 407, 30
In the case where it is determined that there is only one local maximum point as in 2, or that there are six or more as in 301, and in step 408 that the total pressure detection area is 30
If it is determined to be larger than the point, step 414
It is determined whether or not the center of gravity position formed by the current local maximum points and the center of gravity position formed by the previous local maximum points have changed. If it is determined that there is no change in the position of the center of gravity, it means that the part touched by the touch panel has not moved, so this operation can be interpreted as a click operation 421. . At this time, step 40
The current coordinates of the center of gravity G calculated in step 3 are determined as the target coordinates of this operation. If it is determined in step 414 that the coordinates of the center of gravity have moved, it is determined that the current number of pressure detection points is smaller than nine.
If it is determined in 5 that the current number of pressure detection points is smaller than 9, it indicates that the operation is performed with one finger as in the change from 303 to 304.
This operation can be interpreted 422 as a pointer operation. At this time, the previous barycenter coordinate calculated in step 406 is determined as the point operation start point coordinate, and the current barycenter coordinate calculated in step 403 is determined as the point operation end point coordinate. Since it is impossible to apply pressure to the pressure sensor at 9 points or more with one finger, step 415 is executed.
If 9 or more points are detected in
As a change to 06, this operation is interpreted as a movement operation.
3 you can. At this time, the previous coordinates of the center of gravity calculated in step 406 are determined as the coordinates of the moving operation start point, and the current coordinates of the center of gravity calculated in step 403 are determined as the coordinates of the moving operation end point. If it is determined in step 408 that the total pressure detection area is not more than 30 points, in step 409, the average of the distance between the center of gravity G and each local maximum point is determined based on the current and previous pressure information. calculate. In step 410, the average of the angle between the center of gravity G and each local maximum point is calculated based on the current and previous pressure information. The angle between the center of gravity G and a certain maximum point is
It can be calculated as a vector headed from the center of gravity G to the maximum point. In step 411, the average rotation angular velocity of each local maximum point, that is, the difference between the current average angle and the previous average angle is calculated. This can be determined as the difference between the current sum of the vectors calculated in step 410 and the previous sum. Also, this can be represented by the symbols in 321 and 322.
The average of the angles aGa ', bGb', and cGc 'is shown. In step 412, the average rotational angular velocity calculated in step 411 is a constant value, for example, π / 30.
If it is larger, it means that the operator is rotating the position of the finger, such as a change from 321 to 322, so this operation can be interpreted as a rotation operation 424. At this time, the current center of gravity G calculated in step 403 is determined as the center coordinate of this rotation operation, and the average rotation angular velocity calculated in step 411 is determined as the rotation amount. In step 413, the average of the distance between the current center of gravity point G calculated in step 409 and each local maximum point is calculated as follows:
The average of the distance between the previous center of gravity point G and each local maximum point is compared. If the average of the distances is smaller in the present than before, it can be determined that an operation of bringing a plurality of fingers closer to each other has been performed, as in a change from 323 to 324, and this operation is interpreted as a reduction operation 425. can do. At this time, the current center of gravity point calculated in step 403 is determined as the center coordinate of this reduction operation.
Is determined as the reduction coefficient based on the difference of the distance average from the current distance calculated in the above. When the average of the distance is larger in the present than before, as shown in the change from 325 to 326,
Since it can be determined that an operation of moving a plurality of fingers away from each other has been performed, this operation is interpreted as an enlargement operation 425.
can do. At this time, the current center-of-gravity point G calculated in step 403 is determined as the center coordinate of the enlargement operation, and the difference between the current and previous distance averages calculated in step 409 is determined as the enlargement coefficient. If the average of the distances has not changed between before and now, either the fingers have not moved at all, or the movement of each finger cancels out the effect of each other's movement. In this case, it is interpreted as a click operation 427 in this case. At this time, the current barycentric point coordinates calculated in step 403 are determined as the operation target coordinates. In step 416, since the operation target point coordinates suitable for each operation type are given in accordance with the operation method type interpreted in the above steps, these two-dimensional graphic models requiring these operation instructions are provided. Etc. can be smoothly executed. In the above embodiment, the touch panel 201 is installed independently of the display device 200. Instead, a transparent touch panel 221 is provided on the display surface of the display device 220 as shown in FIG. It is also possible to install so that it touches. In this case, the operator 227 of the system can perform each operation as described above with a feeling of directly touching the model 224 displayed on the display device 220, so that the movement of the model 224 can be further intuitively performed. And rotation operations can be performed. As is apparent from the above description, according to the present invention, a two-dimensional pressure distribution sensor capable of simultaneously indicating a plurality of coordinate positions and a pressure distribution detected by the pressure distribution sensor Recording means for temporarily recording; means for calculating a plurality of sets of the maximum value of the pressure in the detected pressure distribution and its coordinate position; and temporarily recorded past pressure distribution and current pressure distribution. Since the coordinate input and operation method instruction device is configured to include means for comparing and calculating the change by calculation and means for determining the operation method command type by the calculation,
The effect that the operation target coordinate position and the operation method command type can be simultaneously instructed is obtained.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing a configuration of the present invention. FIG. 2 is a conceptual diagram illustrating a situation in which the system of the present invention is used. FIG. 3 is a conceptual diagram illustrating a situation in which the system of the present invention is used. FIG. 4 is a diagram (part 1) showing an operation identified in the present invention. . FIG. 5 is a diagram (part 2) illustrating an operation identified in the present invention. FIG. 6 is a flowchart of the present invention. DESCRIPTION OF SYMBOLS 101 display device 102 touch panel unit 103 pressure distribution determination device 104 pressure distribution information storage buffer 105 operation type and operation target coordinate determination device 106 pseudo mouse control driver software 200 display device 201 touch panel 202 computer 203 keyboard device 204 displayed Model 205 Display cable 206 Data transfer cable 207 Operator

Claims (1)

Claims: 1. A two-dimensional pressure distribution sensor capable of simultaneously indicating a plurality of coordinate positions, recording means for temporarily recording a pressure distribution detected by the pressure distribution sensor, and Means for calculating a plurality of sets of the maximum value of the pressure in the pressure distribution and its coordinate position, means for comparing the temporarily recorded past pressure distribution and the current pressure distribution and calculating the change, and A coordinate input and operation method instruction device comprising means for determining an operation method instruction type by the calculation.