JP2011159089A - Information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a virtual keyboard which facilitates execution of touch typing. <P>SOLUTION: The information processor includes: a detection part 214 for detecting the size of a user's hand put on a touch screen display according to a detection signal from a touch screen display; a display part 215 which is a virtual keyboard display means for displaying a virtual keyboard including a plurality of virtual keys for inputting respective key codes on a touch screen display and displays a virtual keyboard having a size corresponding to the size detected by the detection means; and an output part 212 for outputting code data corresponding to an operated virtual key in the virtual keyboard. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention generally relates to an information processing apparatus such as a personal computer, and more particularly to an information processing apparatus that inputs key data using a virtual keyboard.

  A technology has been developed that displays a virtual keyboard (hereinafter referred to as a virtual keyboard) instead of a keyboard having a plurality of physical keys, detects a user's operation position, and outputs a key code corresponding to the detected position. Yes.

  Patent Document 1 discloses that the user sets the position of a reference key on a virtual keyboard.

JP 2005-531861 A

  By the way, since the virtual keyboard does not have physical keys, the positions of the keys are not intuitively understood, and it is difficult to perform touch typing.

  An object of the present invention is to provide an information processing apparatus capable of displaying a virtual keyboard that facilitates the execution of touch typing.

  An information processing apparatus according to an example of the present invention includes a touch screen display, and a detecting unit that detects a size of a user's hand placed on the touch screen display in response to a detection signal from the touch screen display. Virtual keyboard display means for displaying a virtual keyboard including a plurality of virtual keys for inputting key codes on the touch screen display, the virtual keyboard having a size corresponding to the size detected by the detection means It is characterized by comprising display means for displaying a keyboard, and output means for outputting code data corresponding to the operated virtual key in the virtual keyboard.

  According to the present invention, it is possible to display a virtual keyboard that facilitates execution of touch typing.

1 is a perspective view showing an appearance of an information processing apparatus according to an embodiment of the present invention. 2 is an exemplary diagram showing an example of a virtual keyboard displayed on the touch screen display of the information processing apparatus of the embodiment. FIG. 2 is an exemplary block diagram showing an example of the system configuration of the information processing apparatus of the embodiment. FIG. 2 is an exemplary block diagram illustrating a configuration example of a key input control program used by the information processing apparatus of the embodiment. FIG. The figure which shows the area | region detected when both hands are put on the touch screen display of the information processing apparatus of the embodiment. 2 is an exemplary diagram showing an example of a virtual keyboard displayed on the touch screen display of the information processing apparatus of the embodiment. FIG. 4 is a diagram showing an example of virtual buttons displayed on the touch screen display of the information processing apparatus of the embodiment. FIG. 4 is a diagram showing an example of virtual buttons displayed on the touch screen display of the information processing apparatus of the embodiment. FIG.

  Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the appearance of an information processing apparatus according to an embodiment of the present invention. This information processing apparatus is realized as, for example, a battery-driven portable personal computer 10.

  FIG. 1 is a perspective view of the computer 10 with the display unit opened. The computer 10 includes a computer main body 11 and a display unit 12. A display device composed of a liquid crystal display device (LCD) 13 is incorporated on the upper surface of the display unit 12, and the display screen of the LCD 13 is positioned substantially at the center of the display unit 12.

  The LCD 13 may be realized as a touch screen display capable of detecting a touch area (also referred to as a touch position) on the display screen of the LCD 13 touched with a pen or a finger. The user can select various objects (for example, icons representing folders and files, menus, buttons, and the like) displayed on the display screen of the LCD 13 using a fingertip or a pen. Coordinate data indicating the touch area on the display screen is input to the CPU in the computer 10 from the touch screen display.

  The display unit 12 has a thin box-shaped housing, and the display unit 12 is rotatably attached to the computer main body 11 via a hinge portion 14. The hinge portion 14 is a connecting portion that connects the display unit 12 to the computer main body 11. That is, the lower end portion of the display unit 12 is supported by the hinge portion 14 at the rear end portion of the computer main body 11. The display unit 12 is attached to the computer main body 11 so as to be rotatable between an open position where the upper surface of the computer main body 11 is exposed and a closed position where the upper surface of the computer main body 11 is covered by the display unit 12. A power button 16 for powering on or off the computer 10 is provided at a predetermined position on the upper surface of the display unit 12, for example, on the right side of the LCD 13.

  The computer main body 11 is a base unit having a thin box-shaped housing, and a liquid crystal display device (LCD) 15 that functions as a touch screen display is incorporated on the upper surface thereof. The display screen of the LCD 15 is the display screen of the computer main body 11. It is located at the center. A transparent touch panel is disposed on the upper surface of the LCD 15, and a touch screen display is realized by the LCD 15 and the transparent touch panel. This touch screen display can detect a touch area (also referred to as a touch position) on a display screen touched by a pen or a finger. The LCD 15 on the computer main body 11 is a display independent of the LCD 13 of the display unit 12. These LCDs 13 and 15 can be used as a multi-display for realizing a virtual screen environment. In this case, the virtual screen managed by the operating system of the computer 10 includes a first screen area displayed on the LCD 13 and a second screen area displayed on the LCD 15. Arbitrary application windows, arbitrary objects, etc. can be displayed in each of the first screen area and the second screen area.

  In the present embodiment, the LCD 15 (touch screen display) provided on the upper surface of the main body 11 is used to display a virtual keyboard (also referred to as a software keyboard) 151 as shown in FIG. For example, the virtual keyboard 151 can be displayed in the full screen mode on the entire display screen of the LCD 15. The virtual keyboard 151 includes a plurality of virtual keys (a plurality of numeric keys, a plurality of alphabet keys, a plurality of arrow keys, a plurality of auxiliary keys, a plurality of function keys, etc.) for inputting a plurality of key codes. More specifically, the virtual keyboard 151 includes a plurality of buttons (software buttons) corresponding to the plurality of virtual keys.

  On the other hand, the LCD 13 in the display unit 12 can be used as a main display for displaying various application windows and the like as shown in FIG. The user can input various code data (key code, character code, command, etc.) to the application window or the like displayed on the LCD 13 by touching the virtual keyboard displayed on the touch screen display 15. I can do it.

  Two button switches 17 and 18 are provided at predetermined positions on the upper surface of the computer main body 11, for example, on both sides of the LCD 15. An arbitrary function can be assigned to each of these button switches 17 and 18. For example, the button switch 17 can be used as a button switch for starting a key input control program which is an application program for controlling a key input operation using a virtual keyboard. When the button switch 17 is pressed by the user, the key input control program is activated. The key input control program displays a virtual keyboard on the touch screen display 15.

  Next, the system configuration of the computer 10 will be described with reference to FIG. Here, it is assumed that both LCDs 13 and 15 are realized as touch screen displays.

  The computer 10 includes a CPU 111, a north bridge 112, a main memory 113, a GPU (Graphics Processing Unit) 114, a south bridge 115, a BIOS-ROM 116, a hard disk drive (HDD) 117, an embedded controller 118, and the like.

  The CPU 111 is a processor provided to control the operation of the computer 10 and executes an operating system (OS), various application programs, and the like loaded from the HDD 117 to the main memory 113. The application program includes a key input control program 201. The key input control program 201 displays the virtual keyboard 151 on the touch screen display 15 and generates code data according to the touch operation of the virtual keyboard 151 by the user. The generated code data (for example, a key code corresponding to the touched virtual key) is passed to the active application via, for example, an operating system (OS). The CPU 111 also executes a system BIOS (Basic Input Output System) stored in the BIOS-ROM 116. The system BIOS is a program for hardware control.

  The north bridge 112 is a bridge device that connects the local bus of the CPU 111 and the south bridge 115. The north bridge 112 also includes a memory controller that controls access to the main memory 113. The GPU 114 is a display controller that controls the two LCDs 13 and 15 respectively used as a display monitor of the computer 10. The GPU 114 executes display processing (graphics calculation processing) for drawing display data in a video memory (VRAM) based on a drawing request received from the CPU 111 via the north bridge 112. A storage area for storing display data corresponding to the screen image displayed on the LCD 13 and a storage area for storing display data corresponding to the screen image displayed on the LCD 15 are allocated to the video memory. A transparent touch panel 13 </ b> A is disposed on the LCD 13. Similarly, a transparent touch panel 15 </ b> A is disposed on the LCD 15. Each of the touch panels 13A and 15A is configured to detect a touch area (touch position) on the touch panel (touch screen display) using, for example, a resistance film method or a capacitance method. Further, as each of the touch panels 13A and 15A, a multi-touch panel capable of simultaneously detecting a plurality of touch positions may be used.

  The south bridge 115 incorporates an IDE (Integrated Drive Electronics) controller and a Serial ATA controller for controlling the HDD 121. The embedded controller (EC) 118 has a function of powering on / off the computer 10 in accordance with the operation of the power button switch 16 by the user. The embedded controller (EC) 118 includes a touch panel controller 118A that controls each of the touch panels 13A and 15A.

Next, the configuration of the key input control program 201 will be described with reference to FIG.
The key input control program 201 receives touch area detection information from the touch panel 15A via the touch panel driver program, and based on the touch area detection information, a virtual key touched by a user from a plurality of virtual keys in the virtual keyboard 151 Select. The touch area detection information includes coordinate data indicating a touch area (touch position) on the touch panel display (LCD 15 and touch panel 15A) touched by an external member (for example, a user's finger or pen).

  The key input control program 201 includes a virtual key selection unit 211, a code output unit 212, a coordinate / operation signal output unit 213, a detection unit 214, and a display unit 215 as function execution modules. The virtual key selection unit 211 selects a virtual key located in the touch area on the touch screen display from a plurality of virtual keys included in the virtual keyboard.

  The selection of the virtual key by the virtual key selection unit 211 can be performed based on, for example, the above-described touch area detection information indicating the touch area and the key assignment information. The key assignment information indicates an area on the touch screen display where each of the plurality of virtual keys is displayed, that is, a display area of each virtual key.

  The code output unit 212 outputs code data corresponding to the virtual key selected by the virtual key selection unit 211 to an application program or the like.

  In addition, the coordinate / operation signal output unit 213 indicates that, when the touch panel 15A is used as a pointing device, the coordinate signal indicating the operation position and the button of the pointing device are operated according to the detection signal from the touch panel 15A. An operation signal indicating is output.

  The detection unit 214 receives the touch area detection information and detects the size of the user's hand placed on the touch screen display based on the touch area detection information. The detection unit 214 detects the position of the user's finger placed on the touch screen display based on the touch area detection information.

  The display unit 215 shown in FIG. 4 displays the virtual keyboard 151 on the display screen of the touch screen display via the display driver program. As described above, the virtual keyboard 151 includes a plurality of virtual keys for inputting a plurality of key codes. Further, the display unit 215 has a function of changing the size of the virtual keyboard 151 displayed on the display screen of the touch screen display according to the size of the hand detected by the touch screen display. The display unit 215 has a function of displaying a virtual button of the pointing device when one of the user's hands is placed on the touch screen display.

  The display unit 215 displays a virtual keyboard 151 having a size corresponding to the size of the user's hand. The display unit 215 enlarges or reduces the image showing the virtual keyboard 151 according to the detected hand size, and displays the enlarged or reduced image on the LCD 15.

  The detection of the hand size by the detection unit 214 will be described below. FIG. 5 shows an area detected when the user places both hands on the touch panel 15A. As shown in FIG. 5, regions 300 to 304 and 310 to 314 are detected. The region 300 is a region adjacent to the wrist in the palm of the right hand of the user and in contact with a part including the thumb ball and the child ball. A region 301 is a region where the index finger of the right hand contacts. A region 302 is a region where the middle finger of the right hand contacts. A region 303 is a region where the ring finger of the right hand contacts. A region 304 is a region where the little finger of the right hand contacts. Hereinafter, the regions 300 to 304 are referred to as a right hand palm region 300, a right hand index finger region 301, a right hand middle finger region 302, a right hand ring finger region 303, and a right hand little finger region 304, respectively.

  An area 310 is an area adjacent to the wrist in the palm of the left hand of the user and in contact with a part including the thumb ball and the child ball (hereinafter, the left palm lower part 310). A region 311 is a region where the index finger of the left hand contacts. A region 312 is a region where the middle finger of the left hand contacts. A region 313 is a region where the ring finger of the left hand contacts. A region 314 is a region where the little finger of the left hand contacts. Hereinafter, the areas 310 to 314 are referred to as a left hand palm area 310, a left hand index finger area 311, a left hand middle finger area 312, a left hand ring finger area 313, and a left hand little finger area 314, respectively.

The display unit 215 detects the position coordinates of the center of the right hand palm area 300. The center position coordinates are the leftmost position x-coordinate x _L ), the rightmost position x-coordinate x _R , the backmost position y-coordinate y _FS , and the foremost position y-coordinate y _NS. Are detected respectively. The coordinates (x _R −x _L , y _FS −y _NS ) are set as the center position. Similarly, the center position coordinate of the right index finger region 301 is detected. Then, the distance between the center position coordinate in the right palm area 300 and the center position coordinate in the right index finger area 301 is calculated.

  The display unit 215 detects the center position coordinate of the left hand palm area 310 and the center position coordinate of the left hand index finger area, and calculates the distance between the two center position coordinates. Then, the virtual keyboard 151 having a size based on the calculated average value of the two distances is displayed. More specifically, the display unit 215 generates an image of the virtual keyboard 151 by enlarging or reducing the original keyboard image representing the virtual keyboard based on the calculated average value, and displays the generated image on the LCD 15. To do.

  Therefore, the position and size of the virtual keyboard 151 on the display unit 215 depends on where the regions 300 to 304 that the parts of the right hand contact and the regions 310 to 314 that the parts of the left hand contact are located on the touch panel 15A. Be changed. The original keyboard image is preferably a vector format image that does not deteriorate even when enlarged or reduced. Note that the pitch of adjacent finger areas may be measured, and a virtual keyboard having a pitch based on the measured pitch may be displayed.

  By displaying a virtual keyboard having a size corresponding to the size of the user's hand, the user has a key pitch or the like, and the touch typing can be easily performed.

  When the eight fingers of the right index finger, right middle finger, right hand ring finger, right hand little finger, left hand index finger, left hand middle finger, left hand ring finger, and left hand little finger are not aligned, use the virtual keyboard for the right hand as shown in FIG. The virtual keyboard 151R and the left-hand virtual keyboard 151L may be divided into two so as to correspond to the habit of hands.

  The display unit 215 enlarges or reduces the image indicating the right hand virtual keyboard 151R according to the detected size of the right hand. Further, the image is rotated according to the arrangement of the fingers. The display unit displays on the LCD 15 an image that has been enlarged or reduced and rotated. Further, the display unit 215 enlarges or reduces the image showing the left hand virtual keyboard 151L according to the detected size of the left hand. Further, the image is rotated according to the arrangement of the fingers. The display unit displays on the LCD 15 an image that has been enlarged or reduced and rotated.

A linear function F _R (x) approximating a straight line passing through the center position coordinate of the right hand index finger region 301, the center position coordinate of the right hand middle finger region 302, the center position coordinate of the right hand ring finger region 303, and the center position coordinate of the right hand little finger region 304; , A linear function F _L (x) approximating a straight line passing through the center position coordinates of the left hand index finger area 311, the center position coordinates of the left hand middle finger area 312, the center position coordinates of the left hand ring finger area 313, and the center position coordinates of the left hand little finger area 314 For each. If the slopes of the two linear functions are greater than or equal to the set value, it is subsequently determined whether the eight fingers are aligned.

If the slopes of the two linear functions are greater than or equal to the set value and it is determined that the eight fingers are not aligned on a straight line, the rotation angle of the virtual keyboard on the right hand side according to the slope of the linear function F _R (x) To decide. Then, the center position coordinates of the right hand palm area 300 and the center position coordinates of the right hand index finger area 301 are detected, and the size of the right hand side virtual keyboard is determined according to the distance between the two center position coordinates. Further, the display position of the right-hand side virtual keyboard is determined so that the “J” key, which is the home key for the right hand, is positioned in the right-hand index finger area 301.

Further, the rotation angle of the left hand side virtual keyboard is determined according to the inclination of the linear function F _L (x). Then, the center position coordinate of the left hand palm area 310 and the center position coordinate of the left index finger area 311 are detected, and the size of the left hand side virtual keyboard is determined according to the distance between the two center position coordinates. Further, the display position of the left hand side virtual keyboard is determined so that the “F” key that is the home key of the left hand is located in the left hand index finger region 311.

  When the user's hands are separated from the touch screen display, the display unit 215 stops displaying the virtual keyboard 151 after a set time has elapsed since the user's hands were separated. For example, when it is set to 1 second, the display unit 215 continues to display the virtual keyboard 151 for 1 second even after the user's hands are separated. On the other hand, if the time is set to 0 seconds, the display unit 215 stops displaying the virtual keyboard 151 as soon as the user's hands are released.

  When one hand of the user is placed on the touch screen display, the virtual button of the pointing device is displayed on the LCD 15 without displaying the virtual keyboard 151. For example, as shown in FIG. 7, a virtual button 401 is displayed in a region between the right hand palm region 300, the right hand index finger region 301, the right hand middle finger region 302, the right hand ring finger region 303, and the right hand little finger region 304. Further, as shown in FIG. 8, a virtual button 401 may be displayed in a region 305 where the thumb of the user's right hand contacts. Further, a virtual button may be displayed in a direction away from the user from the right hand index finger region 301, right hand middle finger region 302, right hand ring finger region 303, and right hand little finger region 304.

  The area other than the area where the virtual button 401 is displayed on the touch screen display shown in FIGS. 7 and 8 is used as an area for operating the pointing cursor displayed on the LCD 13. That is, if the user touches the area where the virtual button 401 is displayed on the touch screen display, code data corresponding to a pointing device button (sometimes referred to as a right click button or a left click button) is input. Can do. On the other hand, if the user touches another area on the touch screen display, the pointing cursor displayed on the LCD 13 can be moved.

  As described above, by displaying the virtual keyboard corresponding to the size of the user's hand, the key pitch or the like is in the user's hand, and the touch typing can be easily performed.

  Although the computer 10 of the present embodiment includes the main body 11 and the display unit 12, almost all of the components constituting the system of the computer 10 are not necessarily provided in the main body 11. For example, one of the components A part or almost all may be provided in the display unit 12. In this sense, it can be said that the main body 11 and the display unit 12 are substantially equivalent units. Therefore, the main body 11 can also be considered as a display unit, and the display unit 12 can also be considered as a main body.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  DESCRIPTION OF SYMBOLS 10 ... Portable personal computer (information processing apparatus), 212 ... Code output part, 213 ... Coordinate / operation signal output part, 214 ... Detection part, 215 ... Display part.

Claims (10)

Touch screen display,
Detecting means for detecting a size of a user's hand placed on the touch screen display in response to a detection signal from the touch screen display;
Virtual keyboard display means for displaying a virtual keyboard including a plurality of virtual keys for inputting key codes on the touch screen display, the virtual keyboard having a size corresponding to the size detected by the detection means Display means for displaying a keyboard;
Output means for outputting code data corresponding to the operated virtual key of the virtual keyboard;
An information processing apparatus comprising:   The detection means is selected from the position of the part adjacent to the wrist of the palm of the user placed on the touch screen display and the index finger, middle finger, ring finger, and little finger to detect the size of the hand The information processing apparatus according to claim 1, wherein a distance from a finger position is detected.   2. The display unit according to claim 1, wherein the display unit enlarges or reduces an image showing the virtual keyboard according to a size of the hand, and displays the enlarged or reduced image on the touch screen display. Information processing device. The detection means includes
Detecting the size of the right hand of the user and the size of the left hand of the user;
Detecting the position of the finger of the right hand of the user and the position of the finger of the left hand of the user;
Calculating the slope of the first linear function from the detected position of the index finger, middle finger, ring finger, and little finger of the right hand of the user;
Calculating the slope of the second linear function from the detected position of the index finger, middle finger, ring finger, and little finger of the left hand of the user;
The display means includes
Displaying a right-hand side virtual keyboard operated by the user's right hand based on the position of the finger of the right hand, the size of the right hand, and the inclination of the first linear function;
The left-hand side virtual keyboard operated by the left hand of the user is displayed based on the position of the finger of the left hand, the size of the left hand, and the inclination of the second linear function. The information processing apparatus described in 1. The detection means detects the position of the index finger of the user's right hand and the position of the index finger of the user's left hand,
The display means displays the right hand side virtual keyboard so that the right hand side reference key is displayed at the position of the right hand index finger, and displays the left hand side reference key at the position of the left hand index finger. The information processing apparatus according to claim 3, wherein the left-hand side virtual keyboard is displayed. The display means includes
The first image indicating the right-hand side virtual keyboard is enlarged or reduced according to the size of the right hand, and the first image is rotated based on the inclination of the first linear function to be enlarged or reduced. And displaying the rotated first image on the touch screen display;
The second image indicating the left-hand side virtual keyboard is enlarged or reduced according to the size of the left hand, and the second image is rotated based on the inclination of the second linear function to be enlarged or reduced. The information processing apparatus according to claim 1, wherein the rotated first image is displayed on the touch screen display.   The information processing apparatus according to claim 1, wherein when the user's hand is released from the touch screen, the display unit stops displaying the virtual keyboard.   The information processing apparatus according to claim 7, wherein the display unit stops displaying the virtual keyboard after a set time has elapsed since the hand is released. When one hand is placed on the touch screen, the display means stops displaying the virtual keyboard, displays virtual buttons of a pointing device on the touch screen display,
And a second output unit configured to output a coordinate signal corresponding to an operation position of a user's finger and an operation signal indicating that the virtual button has been operated in response to a detection signal from the touch screen display. The information processing apparatus according to claim 1.   The display means is located between the position of the part adjacent to the wrist of the user's palm and the position of the user's finger, at the position opposite to the user side from the user's finger, or at the position of the user's thumb. The information processing apparatus according to claim 7, wherein a button is displayed.

Images