JPH09330175A - Information processor and its operating method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make the processor thin and lightweight and to efficiently input characters, etc., by detecting pressure applied to positions corresponding to respective keys of a virtual keyboard displayed on a display screen, and performing input processing for a character string, etc., according to the pressure signals of the detected keys. SOLUTION: A transparent plate type tablet device 240 having lattice-shaped grids arranged detects pressure applied to all the grating points at the same time and inputs pressure data on all pressure sensing points to an applied pressure distribution processing module 241 at constant time intervals. The pressure distribution processing module 241 detects information regarding whether respective keys including modification keys are pressed at constant time intervals, detects whether or not each key is pressed newly or continuously according to whether its data rise or fall, and sends the result to a central processor 231 and also informs the central processor 231 of information showing whether or not the position or kind of the display keyboard, etc., is updated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing device such as a computer and a word processor. In particular, the present invention relates to an information processing apparatus and an operating method for efficiently performing input work while achieving size reduction and weight reduction.

[0002]

2. Description of the Related Art Conventionally, in order to input a character string or the like into a computer or the like, it was general to use a built-in mechanical keyboard.

However, recently, there is a tendency that the device is reduced in size and weight for the convenience of carrying it or for reducing the occupied space. Therefore, an information processing device without a mechanical keyboard has been developed. In such a device, for example, a method of inputting a character string or the like by touching the screen with a pen or the like provided with a means for sensing that the upper surface and the lower surface of the screen have come into contact with each other. That is, a character string is input by touching and selecting a predetermined character in the character group displayed on the screen with a pen, or the device recognizes the character by handwriting the character with a pen on the screen. It is for inputting a character string.

Further, the above-mentioned Japanese Patent Laid-Open No. 06-083512 discloses that the movement of hands and fingers on a desk or the like is captured by a camera, and the captured signal is digitized to input characters or the like. ing.

[0005]

However, the input efficiency of the input using the pen or the like is significantly lower than that of the input using the keyboard. This is because it is necessary to look for the character to be input character by character from the character group displayed on the screen, move the pen tip there, and touch the screen. Further, handwriting a character with a pen on the screen to cause the device to recognize the character requires a lot of time for handwriting operation and character recognition by the device.

In the method of digitizing information captured as an image as disclosed in the above-mentioned Japanese Patent Laid-Open No. 06-083512, it is considered difficult to obtain high precision, and therefore characters and the like are efficiently processed. It is difficult to type well. It should be noted that the above-mentioned JP-A-6-083521 does not disclose inputting characters or the like by touching a keyboard displayed on the screen.

An object of the present invention is to provide an information processing apparatus which is small in size and thin and has high input efficiency, and an operating method thereof. That is, it is an object of the present invention to provide an information processing apparatus capable of inputting characters and the like with an efficiency equal to or higher than that even in an apparatus having no mechanical keyboard and an operating method thereof.

[0008]

The gist of the present invention is (1)
Pressure detection means for detecting the pressure applied to the display screen, and based on the detected pressure signal, a keyboard of a type corresponding to the pressure signal is provided at the position corresponding to the pressure signal on the display screen to the pressure signal. Means for displaying in a corresponding size, and pressure detection means for detecting a pressure applied to a position corresponding to each key of the keyboard on the display screen in a state where the keyboard is displayed, and the detected pressure An information processing device, characterized by having means for performing information processing based on the pressure signal of each key, (2) means for detecting pressure applied to a display screen in a plurality of stages according to its size, and detection When the magnitude of the pressure applied is within the first pressure range, the keyboard of the type corresponding to the detected pressure signal is placed at the position corresponding to the pressure signal on the display screen. Means for displaying at a size corresponding to a signal, and in the state where the keyboard is displayed, when a pressure within a second pressure range is detected at a position corresponding to any key of the keyboard, An information processing device, characterized in that it has means for recognizing that a key has been designated, (3) a step of detecting the pressure applied to the display screen, and a keyboard of a predetermined type based on the detected pressure signal Is displayed at a predetermined position on the display screen in a predetermined size, and a step of detecting a pressure applied to each key of the keyboard displayed on the display screen in a state where the keyboard is displayed, And a step of performing information processing based on the pressure signal of each key, and (4) the pressure applied to the display screen according to the size of the operation method. The step of detecting in a plurality of steps, and when the magnitude of the detected pressure is within the first pressure range, a keyboard of a type corresponding to the detected pressure signal is placed at a position corresponding to the pressure signal on the display screen. In the step of displaying in a size corresponding to the pressure signal, in the state where the keyboard is displayed, when a pressure within the second pressure range is detected at a position corresponding to any key of the keyboard, And a step of recognizing that the key has been designated.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, an embodiment of transmitting an electronic mail according to the present invention will be described in detail with reference to FIGS.

In the drawings, the same symbols represent the same objects.

FIG. 1 is a perspective view of an information processing apparatus according to this embodiment, FIG. 2 is a block diagram showing the configuration of the information processing apparatus according to this embodiment, and FIGS. 3 to 10 are for explaining the pressure distribution detection of this embodiment. 11 to 16 are diagrams showing an operation flow of the present embodiment, and FIGS. 17 to 32 are diagrams showing examples of display screens at respective stages of the operation of the present embodiment.

In FIG. 1, the present apparatus 200 has a structure in which the inside thereof is partially or entirely covered by a main body apparatus case 201. The size of the device is about B5 to A4, which is a size that can be easily carried. The display device 202 is fitted in the main body device case 201. A transparent tablet device 240 on the display device 202
Is placed. Since the tablet device 240 is transparent, the user can use the display device 202 from outside the device 200.
You can see the display contents of. Reference numeral 24C denotes a tablet device cutting portion, which is provided for explaining that the tablet device 240 is placed on the display device 202 and is not actually cut.

A keyboard displayed on the display screen (hereinafter,
The “virtual keyboard” 430 is, in the example of FIG.
This is for inserting a character string at the position of the cursor 415 displayed on the display device 202.

Reference numeral 220 is a mouse device, and 228 is a pen. 421 is the user's left hand and 422 is the right hand.

FIG. 2 is a block diagram showing the arrangement of a tablet-equipped information processing apparatus to which the virtual keyboard according to this embodiment is applied.

The configuration of this apparatus will be described with reference to FIG.

In FIG. 2, a power source 209 is for supplying electric power to various parts of the electronic component of the present apparatus, and a power button 210 is for controlling the power source 209. The clock 211 is an oscillating device for operating the respective parts of the device in synchronization. The control device 224 is the mouse device 22.
It has a function of performing necessary processing on an input signal to 0 and transmitting the signal to the central processing unit 231. The operator
Data can be input by pushing the input surface of the tablet device 240 with a finger, a pen, or the like. The tablet device pressure distribution processing module 241
It has a function of processing an input signal to 0 and sending it to the central processing unit 231 and receiving information such as a display state of a virtual keyboard on the display device 202 from the central processing unit 231. The secondary storage device 234 has a function of supplying necessary data to the central processing unit 231 from the outside. Display device 2
Reference numeral 02 is for displaying the data written in the display memory in the central processing unit 231 to the outside of the device.

Next, the detection of the key data in this apparatus will be described in detail with reference to FIGS.

FIG. 3 is a block diagram for explaining the tablet device pressure distribution processing module 241 of FIG. 2 in detail. Reference numeral 242 is a pressed key detection unit, 243 is a post-processing module, and 244 is a control device. Reference numeral 240 is a tablet device.

The tablet device 240 of the present device is a transparent plate-shaped input device and has a grid-like grid arranged therein. It is possible to detect the pressure applied to all grid points at the same time. The lattice points that detect this pressure are called pressure sensitive points.

From the tablet device 240, pressure data at all pressure-sensitive points are input to the pressed key detection unit 242 at regular time intervals.

The pressed key detection unit 242 passes to the post-processing module 243 information regarding whether each key including a modifier key or a function key can be regarded as pressed at a constant time interval. At the same time, the pressure key detection unit 242 provides the central processing unit 231 with information on whether or not to update the position or type of the display keyboard or the like, and if so, at which position of the display device the keyboard of which type should be displayed. Tell. The keyboard types include a QWERTY keyboard and a numeric keypad.

The post-processing module 243 rises the data of each key received from the pressed key detection unit 242,
See the falls. Thereby, for each key including the modifier key, it is detected whether it is newly pressed or continuously pressed, and the result is transmitted to the central processing unit 231. Modifier keys are control keys, shift keys, and the like.

When eight fingers excluding both thumbs are placed on the tablet device 240 as shown in FIG. 4, the pressure is actually measured at each point where there is a pressure-sensitive point on the tablet device 240. However, if the number of pressure-sensitive points per unit area of the tablet device 240 is sufficiently large, there is no practical problem even if the pressure between the pressure-sensitive points is approximated by interpolation. In this way, the isobar of the pressure that presses the tablet device 240 can be considered.

When the isobars of the same finger are combined into one set, this set can be regarded as a mountain in the topographic map when the pressure value is replaced with the height in the topographic map.
Hereinafter, a group of isobars formed by the same finger may be referred to as a “mountain”. In the case of FIG. 5, there are eight peaks. In addition, as a result of pressure interpolation, the center of the area with the highest pressure is called the "vertex".
I will call it. The density of pressure sensitive points of the tablet device shall be sufficiently large.

In FIG. 5, reference numeral 551 is an isobar of a pressure distribution formed by the index finger of the user's right hand. In this description, the rightward direction toward the tablet device 240 is the + x direction, and the upward direction is the + y direction. The xy coordinate axes 550 are for the purpose of this description and are not present in the actual device.

FIG. 6 is an explanatory view of the isobar of pressure with which the finger pushes the tablet device 240, and 552 is the isobar 55 of FIG.
An enlarged view of 1 and 553 are vertices.

FIG. 7 shows a tablet device 24 with each finger.
0 is "strongly pressed" and "weakly pressed"
It is a figure explaining a state. Reference numeral 554 is an enlarged view of the isobar 551 in FIG. 5, and 555 is a vertex.

Pressures P1 and P for pressing the tablet device 240
Let 2 be a constant and 0 <P1 <P2. The pressure at the top is p
Then, if P1 ≦ p ≦ P2, the key to which the vertex belongs is defined as “weakly pressed”, and if P2 <p, the key to which the vertex belongs is defined as “strongly pressed”. To do.

P1 and P2 are x at the top of the mountain when the template such as the virtual keyboard 430 is not displayed.
It can be set individually according to the coordinate values. Thus, the “strongly pressing” reference can be set individually for the middle finger of the right hand and the little finger of the right hand, for example.

When the templates are displayed, they can be set separately by the key area. Here, the "key area" represents an area delimited by the display of each key top of the template such as the QWERTY type virtual keyboard 430 and the numeric keypad.

The operation of the pressed key detection unit 242 will be described below with reference to FIGS. 8 to 10 which are state transition diagrams.

The screen 2001 represents a screen state in which the virtual keyboard 430 is not displayed.

The screen 2002 has a screen state 2001.
However, unlike the case where the internal state of the present device is on the screen 2001, it is a state in which it is measured whether the number of specific mountains continues for a certain period of time, so it is expressed as a node different from the state node of the screen 2001.

A screen 2003 shows a screen state in which a QWERTY type virtual keyboard 430 is displayed and a key code can be input.

The pressure key detection unit 242 is the tablet device 2.
It operates in synchronization with 40. That is, it is assumed that there is an input from the tablet device 240 to the pressed key detection unit 242 at fixed time intervals, and the state transitions of FIGS. 8 to 10 occur once over the fixed time.

The user can display at least two types of templates on this device. One is Q
It is a WERTY type virtual keyboard 430, and the other is a virtual numeric keypad.

When any one of the following two types of calling operations is input on the screen 2001, a QWERTY type virtual keyboard 430 or a virtual numeric keypad is displayed.

The calling operation for displaying the QWERTY type virtual keyboard 430 satisfies all the following conditions (1) to (4).

(1) The apparatus is in a state where neither the QWERTY type virtual keyboard 430 nor the virtual numeric keypad is displayed.

(2) The state of being weakly pressed by the eight fingers continues for a certain time or longer. The state of being weakly pressed by eight fingers here means that the number of peaks is eight, and the pressure p at the apex of all eight peaks satisfies P1 ≦ p ≦ P2.

(3) The four points from the smallest x-coordinate of the apex of the mountain are "almost on a straight line and at intervals of about finger intervals". Here, “almost on a straight line” means, for example, 4
It means that the distance from the straight line by the method of least squares is within 8 mm for all four points. Further, the “spacing of the finger spacing” means that the spacing between the vertices of adjacent mountains is 19 mm to 22 mm, for example.

(4) The four points from the largest x-coordinate of the vertices of the mountain are "almost on a straight line and at finger intervals".

Hereinafter, this is called a QWERTY type virtual keyboard calling operation.

The calling operation for displaying the virtual numeric keypad satisfies all the following conditions (1) to (3).

(1) The apparatus is in a state where neither the QWERTY type virtual keyboard 430 nor the virtual numeric keypad is displayed.

(2) The state of being weakly pressed by three fingers continues for a certain time or longer. The state of being weakly pressed by three fingers here means that the number of ridges is 3, and the pressure p at the apex of all three ridges satisfies P1 ≦ p ≦ P2.

(3) The vertices of the three peaks are "almost on a straight line and at intervals of about finger intervals". Here, “almost on a straight line” means that the distance from the straight line by the least square method from the three points is within 8 mm for all the three points, for example. In addition, the "finger spacing" means
For example, the distance between the peaks of adjacent mountains is 19 mm to 22 mm.

Hereinafter, this is called a ten-key calling operation.

The templates other than the QWERTY type virtual keyboard 430 and the virtual numeric keypad are prepared in advance in association with the calling operation for displaying them, so that the user can use these templates. is there.

The calling operation in that case may be any appropriate operation, and does not necessarily follow the above example.

For example, if five fingers have been touching the tablet device 240 for a certain period of time or longer, it is possible to display a template in the order of Japanese syllabary.

When a QWERTY type virtual keyboard calling operation is input on the screen 2001, the screen 20
QWERTY type virtual keyboard 430
Is displayed.

When a virtual ten-key call operation is input on the screen 2001, the screen transitions to the screen 2003 and the ten-key is displayed.

In the following, the operation of the pressure key detection unit 242 will be described.
It will be explained step by step according to the state transition.

First, the variable N used in the following description will be described.

When the number of fingers pressed strongly, that is, the pressure at the vertices is p, the number of vertices satisfying P2 <p is set as N.

When any template such as the QWERTY type virtual keyboard 430 is displayed on the display device and the number of fingers pressing the tablet device with pressure of P1 or more is set to 0, the display of the template is erased.

This state is the start state, and the screen 2 in FIG.
001 corresponds to this state.

An input 1601A is an input which is weakly pressed by three or eight fingers and which satisfies the software keyboard calling operations (3) and (4).

In the state of the screen 2001, input 160
When 1A is input, the present apparatus starts measurement for checking whether the number of peaks in input 1601A continues for a certain period of time, and transits to the state of screen 2002. The fixed time is, for example, 200 milliseconds.

In the state of the screen 2001, input 160
When an input other than 1A is input, the pressed key detection unit 242 informs the post-processing module 243 that there is no key that is strongly pressed by a finger, and the central processing unit 23
Instruct No. 1 not to change the display of the template without deleting it. Then, the state remains on the screen 2001.

The term "for convenience" is used here because there is a possibility that there is a key area that is actually strongly pressed.

For example, if the operation of “touching the tablet device with four fingers for a certain period of time or more” is not defined as an operation of calling any template, the device is pressed with four fingers at a pressure of P1 or more. It is assumed that at least one of the four pushes is pushing at a pressure of P2 or higher.

In this case, for the sake of convenience, the post-processing module 243 is informed that there is no such key even though there is a key pressed with a pressure of P2 or more.

In the state of the screen 2002, input 160
If the number of peaks in 1A is the same and all fingers continue to press the tablet device weakly for a certain period of time, if the number of peaks is 3, the numeric keypad is used, and if it is 8, QW
The ERTY-type virtual keyboard 430 is displayed to transition to the state of the screen display 2003, and at the same time, the post-processing module 243 is informed that there is no key area including a vertex where P2 <p, that is, strongly pressed.

In the state of the screen 2002, input 160
Input 1 if all the vertices are weakly pressed with the number of peaks in 1A remaining unchanged and the fixed time has not elapsed.
The measurement for checking whether or not the number of peaks in 601A continues for a certain time is continued, and the state of the screen 2002 remains.

In the state of the screen 2002, if the number of peaks that are weakly pressing the tablet device is different from the number of peaks in the input 1601A within a certain period of time, it is determined whether the number of peaks in the input 1601A continues for a certain period of time. The measurement for checking is discarded, and the state transitions to the screen 2001.

There are the following three types on the screen 2003.

(1) The number of peaks is 1 or more, and N = 0
At the time of inputting, the pressed key detection unit 242 notifies the post-processing module 243 that there is no key that is strongly pressed by the finger, and also notifies the central processing unit 231 not to change the display of the template. And the status is screen 2
It remains 003.

(2) The number of peaks is 1 or more, and N> 0
At the time of inputting, the pressed key detection unit 242 notifies the post-processing module 243 of all the key regions that are strongly pressed by the finger, and also notifies the central processing unit 231 not to change the display of the template. And the state is screen 20
It remains at 03.

(3) When the number of crests is 0, that is, in the case of an input in which none of the fingers presses the tablet device 240 with a pressure of P1 or more, the pressed key detection portion 242 is strongly pressed by the finger. Post-processing module 243 that there is no key
And the central processing unit 231 not to display the template. And the status is screen 2
Transition to 001.

Next, an operation example of the information processing apparatus according to the present invention will be described in detail with reference to FIGS.

11 to 16 are flowcharts showing the operation flow of the information processing apparatus according to this embodiment.

11 to 16, the portion following "A:" indicates the user's intention and the user's input to the device.
The part following "B:" represents the reaction of the device to the input by the user.

The steps 1000 to 1010 will be described.

The user presses the power button 210 shown in FIG. 2 so that the power of the power source 209 can be supplied to each part of the electronic parts of the apparatus and the apparatus is activated. Then, the apparatus displays the initial screen 4 as shown in FIG.
00 is displayed.

An example of reading an electronic mail will be described. The user uses the mouse device 220 or a similar pointing device (hereinafter referred to as the mouse device 220 or the like) to operate the mouse pointer 225 or an equivalent operation to select and execute the e-mail activation icon 401.

The step 1011 will be described. This device activates the e-mail software, displays a user name / password input window 410 as shown in FIG. 18 on the display device, and prompts the user to input the user name etc. at the position of the cursor 415. Encourage.

The step 1020 will be described. The user wants to enter a character string that represents the user name. As shown in FIG. 19, the user uses his or her left hand 4 as when placing his left and right hands on an ordinary keyboard that is currently commonly used.
Of the ten fingers of 21 and the user's right hand 422, eight fingers excluding the thumb are placed with the left and right index fingers sufficiently separated. Here, "sufficiently separates" means that QWER will be used later.
When the TY-type virtual keyboard is displayed, QWERT
This means that the left-hand portion and the right-hand portion of the Y-shaped virtual keyboard are separated by a distance such that they do not overlap each other. For the QWERTY type virtual keyboard,
It will be described later.

The step 1021 will be described. The device determines that eight fingers are placed on the tablet device 240, and as shown in FIG.
A QWERTY type virtual keyboard 430 having a key pitch suitable for the finger of the user is displayed at the position where the position where 1 and the left hand 422 are placed becomes the home position as it is.

The home position is a way of placing the finger such that the index finger of the left hand is the position of the F key and the index finger of the right hand is the position of the J key in the case of the QWERTY type keyboard.

According to this embodiment, the user can set the position where the hand is placed on the display device as the home position as it is. Therefore, it is not necessary to move the line of sight to the display portion of the QWERTY type virtual keyboard 430 and move the finger to adjust the position of the finger to the home position. That is, the user can keep the line of sight from the insertion point of the character string and the virtual keyboard 4 of the QWERTY type.
It is possible to use 30. Therefore, it is possible to reduce the visual fatigue of the user, further, to adjust the position of the fingers,
It is possible to shorten the operation time for moving the viewpoint.

The key pitch is the distance between the centers of any two keys arranged side by side in the lateral direction.

Since the shape of the finger including the thickness varies depending on the individual, it is desirable that the key pitch of the keyboard is variable.

In the case of the conventional virtual keyboard, (1) an appropriate key pitch is registered in advance for each user,
Alternatively, (2) it is necessary for the user to explicitly perform some operation using a control panel or the like for adjustment.

However, since this device determines the key pitch suitable for the user based on the distance between the fingers touching the tablet device 240, the key pitch can be designated directly and automatically.

Further, since the thickness of each finger is different for each finger, it is considered that the distance between adjacent fingers when the fingers are naturally opened is different for each finger. This device is a virtual keyboard 430 in which the key pitch is automatically adjusted to the interval between adjacent fingers.
It is also possible to provide As a result, it is possible to reduce unreasonable movements of fingers and to reduce the physical burden.

The QWERTY type virtual keyboard 430 in the state shown in FIG. 20 may be displayed in a semitransparent state so that the portion originally displayed is not hidden by the display of the QWERTY type virtual keyboard 430, or is opaque. You may display by a state. Displaying in a semi-transparent state has an advantage that the user can recognize the originally displayed portion and use the information.

The semi-transparent state in the display of the QWERTY type virtual keyboard 430 is to prevent the portion originally displayed from being hidden by the display of the QWERTY type virtual keyboard 430 as much as possible. Q from the part displayed on
The display of the WERTY type virtual keyboard 430 is transparent and can be seen overlapping each other. Among the display dots of the display device, only the dot value sufficient for the user to distinguish the key is changed so that the hidden portion is reduced in the originally displayed portion. By doing so, the user can recognize the part that was originally displayed to some extent and use the information.

Steps 1022 to 1056 will be described. The user inputs a correct user name and a correct password by typing with a finger using the QWERTY type virtual keyboard 430 displayed on the display device.

Then, the system displays a mail title list screen 400E as shown in FIG. 452
Indicates an email with the title "Invitation to a hot spring trip". Four
Reference numeral 53 is a rectangular area that represents an electronic mail whose title is "Thank you for the other day". Reference numeral 454 is a rectangular area representing an electronic mail having the title “Impression of seeing xxx”. Reference numeral 455 is a rectangular area that represents an electronic mail with the title "Feast". Reference numeral 456 is a rectangular area representing an electronic mail with the title “Recent Status”. 452, 453, 454, 455, 4
Reference numeral 56 indicates the sender and the title of the electronic mail that is currently received by the user.

The step 1060 will be described. The user uses the mouse device 220 or the like to move the mouse cursor 22.
5 and the like are operated as necessary to select and execute the rectangular area 452.

The step 1061 will be described. This device displays the content of the "invitation to a hot spring trip" on the display device. In this example, it is assumed that the device displays the text on the upper part of the display device and the pamphlet on the lower part.

FIG. 22 is a diagram for explaining the display of the "invitation to a hot spring trip" and its attached document, a pamphlet. 610 is an "invitation to a hot spring trip" received by the user
Is a window that displays the text of the email. 62
0 is a window for the contents of the brochure attached to the received mail. In this example, it is a hot spring pamphlet. Also, 60
Reference numeral 1 is a menu bar, and 602 is a menu item for displaying a submenu related to "message".

The steps 1070 to 1071 will be described. The user reads the content of the window 610 on the screen and the window 620 of the content of the attached brochure of the received mail and writes a reply.

FIG. 23 illustrates the state of the screen when the user selects and executes the menu item for starting the reply creation. Reference numeral 603 is a submenu related to “message”. 603A is one of the sub-menus, and is a menu item for selecting and executing a reply creating command. A sub-menu 603B is a menu item for selecting and executing a transfer command. 603C is one of the submenus, and is a menu item for selecting and executing a send command.

FIG. 24 illustrates the state of the screen when the window for the user to write a reply is displayed. The user uses the mouse device 220 or the like to select the menu item 603A for creating a reply on the screen of FIG. Then, the device displays a window 640 for the user to write a reply on the display device. Others are the same as the description of FIG.

The steps 1100 to 1103 will be described. In FIG. 25, 431 is a return key area,
Reference numeral 432 is a space key area. First, the user touches the tablet device 240 with eight fingers of the right hand 421 and the left hand 422 excluding the thumb. Then, this device displays QWE on the display device.
The RTY type virtual keyboard 430 is displayed so as to match the home position of each finger of the user.

If necessary, the user switches to the Japanese input mode by using the keys in the QWERTY type virtual keyboard 430. Here, it is assumed that the commonly known Romaji-kana conversion and kana-kanji conversion are used. And, "hiyouhadonokurakaikar
isoudesuka? By typing "How long is it going to take?" The conversion character type "" is input. Then, the user performs conversion by typing the space key area 432, which is a key area for conversion, with a finger in the same manner as a normal kana-kanji conversion operation.
Then, the device inserts the character string "How much does it cost?" At the position of the cursor 415 and displays it on the display device. In the kana-kanji conversion system of this embodiment, the space key is also used as the conversion key.

FIG. 26 illustrates the state of the screen when the user inputs the line feed code and the character string “brochure” following the input of the character string “How much does it cost?”. is there. The line feed code can be input by pressing the return key area 431. Reference numeral 651 is a character string inserted by input, and the other is the same as the description of FIG.

Steps 1104 to 1105 will be described. When the user separates the right hand 421 and the left hand 422 from the tablet device 240, the QWERTY type virtual keyboard 430 is not displayed on the display device as shown in FIG.

Generally, it is sufficient for the virtual keyboard to be displayed only when inputting. With this device, it is sufficient if the display is made only when the user is touching the tablet device 240 with a finger. When it is displayed at any other time, the display area of the display device is substantially reduced because the virtual keyboard is displayed.

With the completion of the input, this apparatus can perform an operation of automatically turning off the display of the QWERTY type virtual keyboard 430 by releasing all the fingers of both hands from the tablet apparatus 240. Therefore, it is possible to prevent a substantial reduction of the display area due to the display of the QWERTY type virtual keyboard 430 with a small number of operating procedures.

Steps 1110 to 1111 will be described. When the user puts his / her hand back on the tablet device 240 again, the state of FIG. 26, that is, the QWERTY type virtual keyboard 430 is displayed, and the state in which characters can be input is returned. After continuing the text input operation and completing the reply, the user uses the mouse device 220 or the like to select and execute the menu item 603C representing the transmission shown in FIG. 23 to transmit the mail.

The description of the subsequent operations will be omitted.

Hereinafter, the behavior of this apparatus will be described when the window in which the user is trying to input characters is displayed in a position where it is difficult to see because it overlaps with the user's finger placed on the display screen. .

For example, the state of the screen in step 1071 is as shown in FIG. 28 instead of FIG.
This is the case where the reply creation window 640 is displayed at the bottom of the screen where it is likely to overlap the user's finger. It is desirable to input a character string or the like while watching the input progress, but in such a case, it is difficult to input while watching the input progress unless special measures are taken.

Steps 1312 to 1313 shown in FIG.
Will be described. The user touches the tablet device 240 with eight fingers to input a sentence. Then, this device scrolls the display of the display device upward, and
The Y-shaped virtual keyboard 430 is displayed so that the position of the user's finger is the home position.

29 and 30 are diagrams for explaining the scrolling state. FIG. 29 shows both hands 44 of the user
It is a state when Nos. 1 and 442 have not touched the tablet device 240 yet. FIG. 30 shows a QWERTY virtual keyboard 43 so that the window 640 is not hidden by the fingers by both hands 441 and 442 that have touched the tablet device 240.
Just before 0 is displayed, the screen of the display device is automatically scrolled upward.

The user can set whether or not to perform this automatic scroll function.

Steps 1320 to 1333 will be described. It is assumed that the user wants to see the window 620 of the contents of the brochure attached to the received mail when he / she inputs "How much does it cost? (New line character) brochure" as shown in FIG. However, in this state, the window 620 for the contents of the attached brochure of the received mail is out of the visible portion due to the upward scrolling of the display device in step 1321. Here, FIG.
As shown in FIG. 2, when all the fingers are temporarily released from the tablet device 240, the upward scroll is released, and the window 620 of the content of the brochure attached to the received mail is displayed on the display device.

The user refers to the window 620 of the pamphlet contents attached to the received mail and then puts eight fingers again on the tablet device 240 to continue the input.
When the sentence is complete, select and execute the menu item that represents the send and send.

The description of the operation until the power button 210 is turned off will be omitted below.

As described above in detail, the information processing apparatus of this embodiment does not require a mechanical keyboard, so that it is possible to reduce the weight and thickness of the entire apparatus.

Further, since a template such as a keyboard is displayed on the display device and a character string or the like can be input by bringing a finger or the like into contact with the tablet device, an operation can be performed as if a mechanical keyboard were present. It is possible to prevent a reduction in input efficiency in a device without a keyboard.

Since the virtual keyboard method is used, various templates can be provided to the user. The various templates here include, for example, a variety of display positions, key pitches, shapes of key areas, and the like, and also include a variety of key labeling and array including multilingual support and functions. Specifically, in the case of a character input keyboard, the user can input by displaying a DVORAK arrangement in addition to the commonly known QWERTY type arrangement, or in Japanese alphabetical order. Can be provided to. It is possible to provide a user with an input by displaying a numeric keypad layout for numeric input and a keyboard layout of a piano for music. Also, the function key labeling can be changed and displayed for each application software.

[0118]

As described above, according to the present invention,
Since a virtual keyboard displayed on the display screen of the information processing device can be used to perform operations such as input of character strings, the device can be made thin and lightweight, and characters and the like can be efficiently input.

[Brief description of drawings]

FIG. 1 is a perspective view of an information processing apparatus showing an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an information processing apparatus according to an embodiment of the present invention.

FIG. 3 is a diagram showing in more detail a block diagram showing a configuration of an information processing apparatus according to an embodiment of the present invention.

FIG. 4 is a diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 5 is a diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 6 is a diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 7 is a diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 8 is a state transition diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 9 is a state transition diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 10 is a state transition diagram for explaining pressure distribution detection according to an embodiment of the present invention.

FIG. 11 is a diagram showing an operation flow according to an embodiment of the present invention.

FIG. 12 is a diagram showing an operation flow of an embodiment of the present invention.

FIG. 13 is a diagram showing an operation flow of an embodiment of the present invention.

FIG. 14 is a diagram showing an operation flow of an embodiment of the present invention.

FIG. 15 is a diagram showing an operation flow of an embodiment of the present invention.

FIG. 16 is a diagram showing an operation flow of an embodiment of the present invention.

FIG. 17 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 18 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 19 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 20 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 21 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 22 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 23 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 24 is a diagram showing an example of a display screen in the embodiment of the present invention.

FIG. 25 is a diagram showing an example of a display screen in one embodiment of the present invention.

FIG. 26 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 27 is a diagram showing an example of a display screen in the embodiment of the present invention.

FIG. 28 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 29 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 30 is a diagram showing an example of a display screen in an embodiment of the present invention.

FIG. 31 is a diagram showing an example of a display screen in one embodiment of the present invention.

FIG. 32 is a diagram showing an example of a display screen in the embodiment of the present invention.

[Explanation of symbols]

200 ... This device, 201 ... Main device case, 202 ... Display device, 209 ... Power supply, 210 ... Power button, 211 ...
Clock, 220 ... Mouse device, 224 ... Mouse device control device, 225 ... Mouse pointer, 228 ... Pen, 2
31 ... Central processing unit, 234 ... Secondary storage unit, 240 ...
Tablet device, 24C ... Tablet device cutting part, 2
41 ... Tablet device pressure distribution processing module, 242
... Pressurized key detector 243 ... Post-processing module 244
... Control device, 400 ... Initial screen, 400E ... Received mail subject list screen, 401 ... E-mail activation icon, 41
0 ... User name / password input window, 415 ... Cursor, 421 ... User's left hand, 422 ... User's right hand, 430 ... QWERTY type virtual keyboard, 431
... Return key area, 432 ... Space key area, 45
2 ... Rectangular area showing the email of "Invitation to a hot spring trip", 453 ... Rectangular area showing the email of "Thank you the other day", 454 ... A rectangular area representing 455, ...
A rectangular area representing an e-mail with the title "Goshiso-sama", 456 ... A rectangular area representing an e-mail with a "recent situation", 550 ... xy coordinate axes, 551 ... Isobars of pressure distribution formed by the index finger of the user's right hand, 552 ... Isobars 551 enlarged view (1), 553 ... vertex (1), 554 ...
Enlarged view (2), 555 ... vertex (2), 5 of the isobar 551
56 ... An isobar representing P1, 557 ... an isobar representing P2,
601 ... Menu bar, 602 ... Menu item for displaying submenu regarding “message”, 610
... window of body text of received mail, 620 ... window of brochure content attached to received mail, 225 ... mouse pointer, 603 ... submenu related to "message", 603A ... menu item representing reply creation, 60
3B ... Menu item indicating transfer, 603C ... Menu item indicating transmission, 640 ... Window for user to write reply, 651 ... Inserted character string, 1000
... operation step, 1001 ... operation step, 1010 ...
Operation step, 1011 ... Operation step, 1020 ... Operation step, 1021 ... Operation step, 1022 ... Operation step, 1051 ... Operation step, 1056 ... Operation step, 1060 ... Operation step, 1061 ... Operation step, 1070 ... Operation step, 1071 ... Operation step, 1100 ... Operation step, 1101 ... Operation step, 1102 ... Operation step, 1103 ... Operation step, 1104 ... Operation step, 1105 ... Operation step, 1110 ... Operation step, 1111 ... Operation step, 1312 ... Operation step, 1313 ... Operation step, 1320 ... Operation step, 1321 ... Operation step, 1322 ... Operation step, 1323 ... Operation step, 1330 ... Operation step, 1331 ... Operation step, 1332 ... Operation step, 13 3 ... operation step, 1601A ... input, 2001 ... screen, 2002 ... screen, 2003 ... screen.

Claims (13)

[Claims] 1. A pressure detecting means for detecting a pressure applied to a display screen, and a keyboard of a type corresponding to the pressure signal based on the detected pressure signal and a position corresponding to the pressure signal on the display screen. Means for displaying at a size corresponding to the pressure signal, and pressure detection means for detecting pressure applied to a position corresponding to each key of the keyboard on the display screen in a state where the keyboard is displayed. An information processing device, comprising: means for performing information processing based on the detected pressure signal of each key. 2. A means for detecting the pressure applied to the display screen in a plurality of stages according to the magnitude thereof, and a pressure signal detected when the magnitude of the detected pressure is within a first pressure range. Means for displaying a type of keyboard at a position corresponding to the pressure signal on the display screen in a size corresponding to the pressure signal, and corresponding to any key of the keyboard when the keyboard is displayed. And a means for recognizing that the key has been designated when a pressure within the second pressure range is detected at the position. 3. The second pressure range is a pressure range that does not overlap with the first pressure range and is larger than the first pressure range. The information processing device described. 4. When the cursor is displayed at a position overlapping with the keyboard while the keyboard is displayed,
4. The information processing apparatus according to claim 1, further comprising means for automatically scrolling a screen so that the cursor is displayed at a position that does not overlap with the keyboard. 5. The information processing apparatus according to claim 1, further comprising means for displaying the keyboard in a semitransparent state. 6. The apparatus according to claim 1, further comprising means for erasing the display of the keyboard when a pressure of a predetermined magnitude or more is not detected for a predetermined time or longer in a state where the keyboard is displayed. The information processing apparatus according to claim 5. 7. The keyboard is any one of a QWERTY type keyboard, a DVORAK type keyboard, a numeric keypad and a music keyboard.
The information processing apparatus according to claim 6. 8. When pressure is detected at eight locations on the display screen while the keyboard is not displayed,
The information processing apparatus according to claim 1, wherein a QWERTY type keyboard is displayed on the display screen. 9. When pressure is detected at three locations on the display screen in a state where the keyboard is not displayed,
The information processing apparatus according to claim 1, wherein a ten-key pad is displayed on the display screen. 10. The Japanese syllabary keyboard is displayed when pressure is detected at five points on the display screen in a state where the keyboard is not displayed. Or the information processing device described. 11. A step of detecting a pressure applied to a display screen, a step of displaying a keyboard of a predetermined type at a predetermined position on the display screen in a predetermined size based on the detected pressure signal, and the keyboard displaying In a state of being operated, the method further comprises a step of detecting a pressure applied to each key of the keyboard displayed on the display screen, and a step of performing information processing based on the detected pressure signal of each key. Method of operating information processing device. 12. A step of detecting a pressure applied to a display screen in a plurality of steps according to the magnitude of the pressure, and responding to the detected pressure signal when the magnitude of the detected pressure is within a first pressure range. A step of displaying a type of keyboard at a position corresponding to the pressure signal on the display screen in a size corresponding to the pressure signal, and in a state where the keyboard is displayed,
When a pressure within the second pressure range is detected at a position corresponding to any key of the keyboard, the step of recognizing that the key is designated is included. Method of operation. 13. The second pressure range is a pressure range that does not overlap with the first pressure range and is larger than the first pressure range. A method for operating the described information processing apparatus.