JP2014089592A - Information processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus capable of simultaneously receiving a plurality of touch inputs by a plurality of input devices under a situation where a framework capable of temporarily processing only an event related to one input device is installed.SOLUTION: An information processing apparatus includes: a first input device; a second input device; and a device driver. When indication by a first device and instruction by the second device are performed in parallel, the device driver combines position information on one position indicated by the first device and position information on one or more positions indicated by the second device to output event information on either the first device or the second device.

Description

  Embodiments described herein relate generally to an information processing apparatus capable of touch input.

  In recent years, information processing apparatuses capable of touch input on a display called a tablet or the like have been widely used. There are two types of touch input devices: a type that detects the magnetic force generated from the tip of the pen, which is called a stylus, and a type that detects the pressure and capacitance generated by the pressure of the pen tip or fingertip. There are two types. Recently, information processing apparatuses equipped with such two types of input devices and supporting two types of touch inputs are increasing.

  For example, in a notebook type personal computer, there are devices equipped with a touch pad and a stick-type pointing device as devices for moving a mouse cursor. For this reason, various mechanisms for controlling a plurality of input devices have been proposed.

JP 2000-250704 A

  The touch panel is a type of input device that detects pressure, capacitance, and the like generated by pressure bonding of a pen tip or a fingertip. Many touch panels mounted on tablets are multi-touch capable of accepting touch operations using two fingers called pinch-in and pinch-out. That is, according to the multi-touch compatible touch panel, it is possible to detect the positions of two places touched simultaneously.

  Here, consider a case where touch input by a stylus that generates magnetic force from the pen tip and touch input on the touch panel are performed simultaneously. Typically, events related to different input devices are treated as separate events, and a framework that processes events related to input devices can only handle events related to one input device at a time. Therefore, when events related to two input devices occur at the same time, either only one of them is processed or both are not processed.

  That is, a plurality of touch inputs from a single input device (for example, a touch panel) are accepted at the same time, but a plurality of touch inputs from a plurality of input devices (for example, a stylus and a touch panel) are not accepted at the same time.

  An object of the present invention is to provide an information processing apparatus capable of simultaneously accepting a plurality of touch inputs from a plurality of input devices in a situation where a framework capable of processing only an event related to one input device is mounted at a time. And

  According to the embodiment, the information processing apparatus includes a first input device, a second input device, and a device driver. The first input device indicates one position on the input area. The second input device indicates one or more positions on the input area. The device driver, when the instruction by the first device and the instruction by the second device are performed in parallel, position information of one position instructed by the first device and the second device Are combined with the position information of one or more positions specified in (1), and output as event information related to one of the first device and the second device.

1 is a diagram illustrating a schematic system configuration of an information processing apparatus according to an embodiment. FIG. 3 is a diagram illustrating a configuration example of coordinate data acquired by the information processing apparatus according to the embodiment using two input devices. The figure which shows the structural example of the coordinate data acquired when the information processing apparatus of embodiment is simultaneously input by two input devices. 6 is an exemplary diagram illustrating an example of information processing executed by a UI application operating on the information processing apparatus according to the embodiment. FIG. The figure which shows the flow of operation | movement in case the simultaneous input by two input devices of the information processing apparatus of embodiment is performed.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a schematic system configuration of an information processing apparatus according to the present embodiment. The information processing apparatus is realized as a tablet, for example.

  As shown in FIG. 1, the information processing apparatus includes a CPU 11, a memory 12, an IO controller 13, a video controller 14, and an LCD 15. The information processing apparatus includes a pen device controller 16, a touch device controller 17, a storage device 18, a pen device 19, and a touch device 20.

  The CPU 11 is a device responsible for overall operation control of the information processing apparatus. The CPU 11 functions as various means by loading various programs stored in the storage device 18 into the memory 12 and executing them. Let The various programs include the illustrated control program 101, a device driver 102 that operates under the control program 101, a framework 103, a user interface (UI) application 104, and the like. These will be described later.

  The IO controller 13 is a device that serves as a bridge for connecting the CPU 11 and each unit (other than the CPU 11) in the information processing apparatus. The IO controller 13 has a function of controlling access to the memory 12 and the storage device 18 and a function of executing communication with the video controller 14, the pen device controller 16, and the touch device controller 17.

  The memory 12 is a storage device that is a main storage of the information processing apparatus, while the storage device 18 is a storage device that is an external storage. The video controller 14 and the LCD 15 are devices responsible for the output side of the user interface provided by the information processing apparatus. On the other hand, the pen device controller 16 and the pen device 19, the touch device controller 17 and the touch device 20 are: It is a device responsible for the input side. That is, the information processing apparatus executes information processing corresponding to the input based on the input from the two input devices of the pen device 19 and the touch device 20 and displays the result on the LCD 15.

  The aforementioned device driver 102 is a module that acquires coordinate data from the pen device controller 16 that controls the pen device 19 or the touch device controller 17 that controls the touch device 20. When touch input is performed by the pen device 19, the pen device controller 16 detects the position, stores the coordinate data in its own register, and notifies the IO controller 13 of the occurrence of the event. This notification is transmitted from the IO controller 13 to the CPU 11, and as a result, an event that a touch input by the pen device 19 has been performed is notified to the device driver 102 executed by the CPU 11. Upon receiving this notification, the device driver 102 reads coordinate data from the register of the pen device controller 16.

  Similarly, when a touch input is performed by the touch device 20, the touch device controller 17 detects the position, stores the coordinate data in its own register, and notifies the IO controller 13 of the occurrence of the event. In response to this notification, the device driver 102 reads the coordinate data from the touch device controller 17. The touch device 20 is so-called multi-touch capable, and can input a plurality of points simultaneously.

  FIG. 2 is a diagram illustrating a configuration example of coordinate data read from the pen device controller 16 and the touch device controller 17 by the device driver 102.

  2, (A) is coordinate data acquired by the device driver 102 via the pen device controller 16 at the time of touch input by the pen device 19, and (B) is the touch device controller 17 at the time of touch input by the touch device 20. Coordinate data acquired by the device driver 102 via

  As shown in FIG. 2A, at the time of touch input by the pen device 19, the pen device controller 16 displays each of the X coordinate (X), Y coordinate (Y), and pressure (P) on the two-dimensional input area. Generate coordinate data including values. In addition, the pen device controller 16 in the present embodiment assigns an ID number (for identifying that it relates to a touch input by the pen device 19) described later.

  On the other hand, as described above, the touch device 20 is compatible with multi-touch and can simultaneously input a plurality of points. Therefore, as shown in FIG. The coordinate data including a plurality of sets of X), Y coordinates (Y), and pressures (P) (the number of touch inputs performed simultaneously) is generated. The touch device controller 17 assigns an ID number (ID (n)) to each set. For example, when a finger pressed on the touch device 20 is moved on the touch device 20 without being separated from the touch device 20, an X coordinate (X) and a Y coordinate (Y) detected with respect to the touch input by the finger. The same ID number continues to be assigned to each set of values of pressure and pressure (P). By this ID number, it is possible to recognize a touch operation such as pinch-in that narrows the distance between two fingers and pinch-out that widens the distance between two fingers. The ID number assigned by the pen device controller 16 is defined as a value for identifying that a value that does not overlap with the ID number assigned by the touch device controller 17 relates to touch input by the pen device 19.

  When the device driver 102 reads the coordinate data from the pen device controller 16 or the touch device controller 17, it passes this coordinate data to the framework 103 as input event information related to the pen device 19 or input event information related to the touch device 20. For example, when the framework 103 receives input event information related to the touch device 20, the framework 103 passes the input event information to the UI application 104 as event information. The UI application 104 executes, for example, gesture authentication processing (processing for recognizing the above-described pinch-in and pinch-out) using the coordinate data included in the input event information, for example, reducing / enlarging an image displayed on the LCD Various processes are performed.

  When the framework 103 receives input event information related to the pen device 19, the framework 103 delivers event information including the coordinate data to the UI application 104 as a single touch event. Further, when input event information related to the touch device 20 is received, and only one set of each value of the X coordinate (X), Y coordinate (Y), and pressure (P) is included, The framework 103 delivers event information including the coordinate data to the UI application 104 as a single touch event. When input event information related to the touch device 20 is received and a plurality of sets of values of X coordinate (X), Y coordinate (Y), and pressure (P) are included, the framework 103 Event information including the coordinate data is delivered to the UI application 104 as a multi-touch event.

  Incidentally, the framework 103 serially receives input event information from the device driver 102. In other words, the framework 103 cannot receive a plurality of input event information from the device driver 102 in parallel. That is, the input event information regarding the pen device 19 and the input event information regarding the touch device 20 cannot be received simultaneously. This means that the two events of the touch input by the pen device 19 and the touch input by the touch device 20 cannot be processed at the same time.

  Therefore, in this embodiment, when the touch input by the pen device 19 and the touch input by the touch device 20 are performed simultaneously, the coordinate data read by the device driver 102 from, for example, the pen device controller 16 is used. In combination with the coordinate data read from, it is transferred to the framework 103 as input event information related to the touch device 20. Each set of values of the X coordinate (X), Y coordinate (Y), and pressure (P) read from the pen device controller 16 has an ID number for identifying that it is related to touch input by the pen device 19. Since it is given, it is possible to distinguish from a set of values of X coordinate (X), Y coordinate (Y) and pressure (P) read from the touch device controller 17.

  Conversely, the coordinate data read from the touch device controller 17 may be combined with the coordinate data read from the pen device controller 16 and delivered to the framework 103 as input event information regarding the pen device 19. For example, when the touch input by the touch device 20 is performed first and the touch input by the pen device 19 is performed thereafter (in a state where the touch input by the touch device 20 is continued), the former touch device 20 Is input to the framework 103 as the input event information, and the touch input by the pen device 19 is performed first (in the state where the touch input by the pen device 19 continues), and then the touch input by the touch device 20 is performed. In this case, it is conceivable that the latter is handed over to the framework 103 as input event information related to the pen device 19.

  FIG. 3 shows a configuration example of coordinate data output from the device driver 102 to the framework 103 as input event information related to the touch device 20 when touch input by the pen device 19 and touch input by the touch device 20 are performed simultaneously. FIG.

  Here, touch input by the pen device 19 is performed on one point on the input area, and at the same time, touch input by the touch device 20 is performed on another point on the input area. Assume a case.

  In such a case, the device driver 102 delivers coordinate data as shown in FIG. 3 to the framework 103 as input event information regarding the pen device 19 or the touch device 20. In FIG. 3, the upper level (a1) is information regarding the touch device 20, and the lower level (a2) is information regarding the pen device 19.

  For example, it is assumed that the touch device 20 and the touch device controller 17 can accept a simultaneous touch input with ten fingers, and the ID number can have a value of “0-9”. In this case, a value other than “0-9”, that is, a value that cannot be a touch input by the touch device 20 (for example, “11” in the drawing) is identified to be related to the touch input by the pen device 19. ID number.

  Actually, two input events of the touch input by the pen device 19 and the touch input by the touch device 20 are generated. These two input events are input related to the pen device 19 or the touch device 20. Event information is collected as one event and is transferred from the device driver 102 to the framework 103 at a time. The UI application 104 that receives event information including coordinate data as shown in FIG. 3 from the framework 103 as a multi-touch event indicates that the ID number (11) relates to touch input by the pen device 19. By recognizing, for example, the following information processing can be performed. An example of information processing that enables the UI application 104 to be executed will be described with reference to FIG.

  It is assumed that the UI application 104 is a program that displays the handwriting on the LCD 15 in response to a touch input by the pen device 19. Upon receiving event information related to the pen device 19 from the framework 103, the UI application 104 displays a handwriting in “black” color at the position indicated by the coordinate data included in the event information. That is, the handwriting of touch input by the pen device 19 is basically displayed on the LCD 15 in “black” color.

  Here, it is assumed that the user performs a touch input with a finger on the object b3 for instructing to give a “red” color while inputting with the pen device 19. In this case, as described above, in the device driver 102, the coordinate data read from the pen device controller 16 and the coordinate data read from the touch device controller 17 are combined, and for example, input to the framework 103 as input event information regarding the pen device 19. Delivered. Accordingly, event information (multi-touch event) related to the pen device 19 is delivered from the framework 103 to the UI application 104.

  When the event information (multi-touch event) related to the pen device 19 is received from the framework 103, the UI application 104 determines that one of the coordinates indicated by the coordinate data included in the event information is a touch input by the pen device 19 based on the ID number. Recognize that The UI application 104 detects that the object b3 is selected from the coordinates relating to the touch input by the touch device 20, and displays the handwriting of the touch input by the pen device 19 in “red” color from the middle.

  That is, by the information processing executed by the UI application 104, the user uses green, blue, and red in addition to black by writing with the pen device 19 while touching the object b1, the object b2, and the object b3 with a finger. be able to.

  FIG. 5 is a diagram illustrating a flow of operation of the information processing apparatus when touch input by the pen device 19 and the touch device 20 is performed at the same time.

  When the touch input by the pen device 19 (A1a in FIG. 5) and the touch input by the touch device 20 (A1b in FIG. 5) are simultaneously performed, the device driver 102 reads the coordinate data from the pen device controller 16 (FIG. 5). 5A2a) and reading of coordinate data from the touch device controller 17 (A2b in FIG. 5) is executed.

  The device driver 102 combines the coordinate data read from the pen device controller 16 and the coordinate data read from the touch device controller 17 and outputs them as input event information related to the pen device 19 or the touch device 20 (A3 in FIG. 5).

  When the framework 103 receives input event information related to the pen device 19 or the touch device 20 from the device driver 102 (B1 in FIG. 5), the framework 103 delivers the multi-touch event information including the coordinate data received from the device driver 102 to the UI application 104. (B2 in FIG. 5).

  Upon receiving the multi-touch event information from the framework 103 (C1 in FIG. 5), the UI application 104 refers to the ID number in the coordinate data included in the multi-touch event information and identifies each touch input, for example, After performing a gesture recognition process (C2 in FIG. 5), information processing corresponding to each touch input is executed (C3 in FIG. 5).

  As described above, according to the information processing apparatus of the present embodiment, events relating to two input devices that occur simultaneously are processed as multi-events of one input device, so that only events relating to one input device can be processed at a time. Events relating to two input devices in a situation in which the framework 103 is mounted without changing the framework 103 and without changing the interface portion of the UI application 104 with the framework 103. It becomes possible to cope with one time.

  Note that the operation control processing of the present embodiment can be realized by software (program), and therefore by installing this software on a normal computer through a computer-readable storage medium storing this software, Effects similar to those of the present embodiment can be easily realized.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 11 ... CPU, 12 ... Memory, 13 ... IO controller, 14 ... Video controller, 15 ... LCD, 16 ... Pen device controller, 17 ... Touch device controller, 18 ... Storage device, 19 ... Pen device, 20 ... Touch device, 101 ... control program, 102 ... device driver, 103 ... framework, 104 ... user interface application.

Claims (7)

A first input device indicating one position on the input area;
A second input device that indicates one or more positions on the input area;
A device driver that outputs position information of one position designated by the first input device or position information of one or more positions designated by the second input device;
Comprising
When the instruction from the first device and the instruction from the second device are performed in parallel, the device driver and the position information of one position instructed by the first device and the second device Combining the position information of one or more positions indicated by the device, and outputting as event information related to one of the first device or the second device,
Information processing device. Identification information for identifying each is attached to the position information of one or more positions indicated by the second input device,
Identification information for identifying that it is an instruction by the first input device is attached to the position information of one position instructed by the first input device.
The information processing apparatus according to claim 1.   The identification information for identifying that it is an instruction by the first input device is defined as a value that the identification information attached to the position information of one or more positions indicated by the second input device cannot have. The information processing apparatus according to claim 2. A display device;
The input area is provided so as to overlap the display area of the display device.
The information processing apparatus according to claim 1.   The information processing apparatus according to claim 1, wherein the first input device is a stylus pen.   The information processing apparatus according to claim 1, wherein the second input device is a multi-touch compatible touch panel. A first input device;
A second input device;
When an event related to the first input device and an event related to the second input device occur at the same time, two events, an event related to the first input device and an event related to the second input device, are set as the second event. A device driver that handles multiple events for input devices
An information processing apparatus comprising:

Images