JPH11345072A - Gesture processor and gesture processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processor capable of performing editing by handwriting by a pen or the like on a screen. SOLUTION: A display device 14 is provided, a tablet 19 capable of inputting images by performing plotting by an input pen 12 is provided and a gesture function capable of editing the images of the array or the like of characters and symbols by the images is provided. The gesture valid area of the input pen in the gesture function and a gesture invalid area for displaying the input function of an icon or the like given priority over the gesture function are set on the tablet 19. A gesture judgement part 22 for judging the validity and invalidity of the gesture function by a pen-down coordinate which is the coordinate where the input pen 12 is abutted on the tablet 19 for the first time at the time of the gesture function is provided. Thus, the validity and invalidity of the gesture function are discriminated at the time of pen-down and the gesture function is quickened.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus capable of performing handwriting editing on a screen with a pen or the like, and more particularly, to a gesture processing apparatus and a gesture processing method.

[0002]

2. Description of the Related Art In recent years, a display device in which a display portion such as a liquid crystal display and a tablet for detecting the coordinates of a contact portion with an input pen or the like are provided is provided, and is set so as to be directly editable with the input pen. Information devices, so-called pen information input type information devices, have appeared on the market. Therefore, the display device can realize, on the tablet, the same thing as proofreading on manuscript paper, as if reminiscent of a paper and a pen. Japanese word processor "WV- made by Sharp Corporation"
S200 "and the like are known.

[0003] In these pen information input devices, a gesture function has been considered as a function for making the most of the characteristics. Such a gesture function is to draw a text on a display device by drawing a character or the like at a desired position on a tablet with an input pen using an editing symbol set and stored in advance with an arbitrary editing symbol. It can be edited.

[0004]

However, in the prior art described above, it is determined whether the gesture function by the input pen is valid or invalid after the pen-down at the start of drawing and after the pen-up at the completion of drawing is completed. , It took time to operate the gesture function. Also, in the above-mentioned conventional,
If the input pen passes through an area such as an icon, which is an invalid area of the gesture function during drawing, the gesture function has been canceled, so at such cancellation, the setting is reset and the input pen is used to draw on the tablet Need to use the gesture function.

Further, in the above-mentioned conventional technique, for example, there is a case where a user inputs an erasure or erasure-related editing symbol without intention and edits it into an unintended text, and it takes time to restore such a text. The problem has arisen.

Furthermore, in the above-described conventional art, a locus of an input pen, a list of edit symbols, and the like can be displayed in order to effectively operate the gesture function. Such display is repeatedly erased and displayed. Then, there is also a problem that the processing time is increased and the processing speed is reduced.

[0007]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, a gesture processing apparatus according to the present invention has a display unit such as a liquid crystal display and a display surface of a display unit designated by an input unit such as an input pen. A coordinate extracting unit that outputs a stroke of continuous coordinates in step (a), a determining unit that determines whether a display position corresponding to the first coordinate output from the coordinate extracting unit is a predetermined effective area, A gesture recognition unit that compares a stroke that is continuous to the first coordinate determined to be an effective area with a basic stroke stored in advance and outputs an edit command corresponding to a basic stroke that is closest to the gesture, and a gesture recognition unit. Editing means for editing the information displayed in the effective area on the display means based on the output editing command. It is set to.

Further, in the gesture processing apparatus according to the present invention, in addition to the above-described configuration, the determination means does not perform the area determination on the coordinates continuous with the first coordinate in the effective area. I have.

Further, the gesture processing method of the present invention comprises:
A gesture processing method in a gesture processing device, comprising: a display unit such as a liquid crystal display; and coordinate extraction unit that outputs a stroke of continuous coordinates on a display surface of the display unit specified by an input unit such as an input pen. It is determined whether or not the display position corresponding to the first coordinate output from the coordinate extracting means is a predetermined effective area, and a stroke continuous to the first coordinate determined to be the effective area is stored in advance. And outputting an edit command corresponding to the closest basic stroke by comparing the basic stroke with the output basic command, and editing the information displayed in the effective area on the display means based on the output edit command. .

Further, the gesture processing method according to the present invention is characterized in that, in addition to the above-described configuration, the area determination is not performed on the coordinates continuous with the first coordinates in the effective area.

According to the present invention, the gesture function is enabled by the output of the first designated coordinates on the display surface by the coordinate extracting means instructed by the input means such as the input pen at the time of the gesture function. And a gesture recognizing means for outputting an editing command by a subsequent stroke, so that it is possible to accurately determine and extract valid / invalid of a gesture function and its intended command. Can be.

According to the present invention, valid / invalid area determination is not performed for the coordinates of a stroke input after the first coordinates are output by the input means, so that the first coordinates are valid. For example, even if a subsequent continuous stroke is drawn in the invalid area of the gesture function, the gesture function itself is not invalidated.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to FIGS.
In the gesture processing device as the information processing device, FIG.
As shown in the block diagram shown in FIG. 1, a central processing unit (hereinafter, referred to as a CPU) 1 for executing a program is provided, and a bus line 18 of the CPU 1 is provided.
A RAM 2 for storing and reading various data, a ROM 3 for storing a control program, a printer controller 4 for controlling a printer 5, and an IC card interface 7 for reading from and writing to an IC card 6 as external storage means. And an FD controller 9 for controlling a floppy disk drive (hereinafter referred to as FDD) 8.

Further, the bus line 18 controls a key interface 11 for inputting a signal from a keyboard 10 as input means to the bus line 18 and a display device 14 (display means) such as a liquid crystal display. Display control 15 is connected.

In addition, a signal from the input pen 12 for inputting or selecting information is converted into pen coordinates by contacting the bus line 18 with an image on the display device 14 to convert the signal into pen coordinates. A pen interface 13 for inputting to the line 18 is connected. Therefore, the input pen 12 is provided with a so-called pointing function for displaying on the image or a predetermined image on the image to draw a predetermined image on the image. Input means for a gesture function that allows the user to select information and input an edit symbol.

Further, a handwriting editing interface 16 is connected to the bus line 18, and the handwriting editing interface 16, as described later in detail,
Based on a signal of each coordinate data indicating the pen coordinates from the input pen 12, a recognition result of each coordinate data is output to the bus line 18 based on a handwriting recognition dictionary (storage means) 17 which is a memory. It has become.

In the handwriting recognition dictionary 17, each basic stroke indicating a character or a symbol is described as (1) a horizontal writing gesture command table, (2) a handwriting character table, and (3) a vertical writing gesture command table. Classified according to the contents and stored. As an example of such a basic stroke, an inverted L character which is an edit symbol indicating a line feed is, as shown in FIG.
Are buried in the positions corresponding to the respective squares as indicated by the triangle.

On the other hand, as shown in FIG.
4, a transparent tablet (coordinate extraction means) 19 for detecting each coordinate of the screen is integrally formed, and the tablet 19 has a character arrangement of the display device 14, for example, FIG. 4B, each XY (640 × 640) is arranged in a grid pattern as shown in FIG.
400) The coordinates are set.

Thus, the input pen 12 shown in FIG.
When the tip of the tablet abuts on the tablet 19, the pen coordinates on the tablet 19 are detected by the pen interface 13, converted into a coordinate code, and output to the bus line.

The tablet system described above includes:
There is no particular limitation as long as the coordinates of the trajectory can be detected when drawing with the input pen 12 in contact with the input pen 12. For example, an electrostatic coupling type tablet, an electrostriction and magnetostriction type tablet, a pressure-sensitive type or an ultrasonic type Formula and electromechanical tablets can also be used.

With the above-described configuration, the display device 1
When a text is created on the keyboard 4, characters and symbols input from the keyboard 10 are converted into predetermined character codes and symbol codes by the key interface 11, and those codes are converted into the CPU 1 (the text editing unit in FIG. 1). 1
According to a), the data is temporarily stored in the RAM 2 sequentially (text memory 2a in FIG. 1).

Thereafter, the CPU 1 transmits the code stored in the RAM 2 to the display device 1 based on an instruction from the keyboard 10 and a program stored in the ROM 3.
4 so that they can be displayed on the RAM 2 (see FIG. 1).
Bit map memory 2b). Then, the RA
Each code sequence on M2 is displayed on the display device 14 via the display control 15.

In the above configuration, the sentence once displayed on the display device 14 can be edited by the keyboard 10, the icon set on the display device 14, or the instruction by the input pen 12 described later. The CPU (editing means) 1 can rearrange, that is, edit, each code array of the bitmap memory 2b in the RAM 2 according to a predetermined program and the above-mentioned instruction.

Next, a gesture function for facilitating editing of a sentence displayed on the display device 14 will be described. First, as shown in FIG. 1, a display device 14 such as a CRT or a liquid crystal and a transparent tablet 19 provided on a display surface thereof are integrally formed on a display tablet 20. At a desired position of a displayed image such as a sentence, for example, an image showing an edit symbol is drawn by the input pen 12.

Each coordinate data of such drawing is sequentially detected by the locus coordinate extracting unit 21 connected to the tablet 19. At this time, of the coordinate data, the XY where the input pen 12 and the tablet 19 first contacted each other.
The tablet 1 shown in FIG.
9 to determine whether the gesture function is within the effective area 19a or the invalid area 19b of the gesture function.
The gesture determination unit (determination means) 22 to which the coordinate data from 1 is input is determined.

Further, when the gesture determination section 22 determines that the pen-down coordinates are valid, the pen-down coordinates are set according to each display screen on the display device 14 and selected by the input pen 12. Is determined to be a pen button area to be executed.

Thereafter, when it is determined that the pen-down coordinates are within the effective area and not in the pen button area, the trajectory determination unit 23 to which the coordinate data is input from the gesture determination unit 22 is used. A gesture function ON flag indicating ON is set in the flag indicating section (stop means) 24.

When the gesture function ON flag is set in the flag indicating section 24, the coordinate data indicating the drawing locus includes a coordinate data buffer 25 connected to the locus judging section 23, including the pen down coordinates. Are sequentially stored.

In addition, when the gesture function ON flag is set in the flag instruction section 24, the gesture determination section 22 determines whether or not the input coordinate data is within the valid area and determines whether or not the input coordinate data is in the pen button area. The determination as to whether or not there is any is stopped, and all the input coordinate data is output as valid to the trajectory determination unit 23.

When the drawing with the input pen 12 is completed and the input pen 12 is in a pen-up state in which the input pen 12 is separated from the tablet 19, the trajectory determination unit 23 confirms that the above-mentioned gesture function ON flag is set. Then, all of the trajectory coordinate data is output from the data buffer 25 to the gesture command determination unit (gesture recognition means) 26.

As described above, when all the coordinate data are output to the gesture command determination unit 26, the gesture function ON flag by the flag instruction unit 24 is reset.

Each of the trajectory coordinate data is normalized by the gesture command determination unit 26 and collated with the basic stroke data stored in the gesture command table 27 included in the handwriting recognition dictionary 17 shown in FIG. Is done. Note that the gesture command determination unit 27 is included in the handwriting editing interface 16, and the trajectory coordinate extraction unit 21, the gesture determination unit 22, the trajectory determination unit 23, and the flag instruction unit 24
Is included in the pen interface 13 described above,
The storage area of the coordinate data buffer 25 is set in the RAM 2.

The above gesture command table 27
In addition to the basic strokes corresponding to the one-character deletion command 27a, the space insertion command 27b, and the handwritten character input command 27c shown in FIG.
Are stored together with the corresponding commands.

A command selection unit (selection means) 3 which can select and set each basic stroke in the gesture command table 27 to be accessible or inaccessible from the gesture command determination unit 26.
The selection by the command selection unit 32 can be set by input means such as the input pen 12 and the keyboard 10 via the text editing unit 1a.

In the command selection section 32, each basic stroke is individually determined by the gesture command determination section 26.
It is also possible to select and set accessible and inaccessible from the group, and to group and set each of the basic strokes so that accessible and inaccessible can be selectively set (setting means) for each group. Has become.

A registration buffer 33 for storing, in a table, the accessibility and inaccessibility of each command corresponding to each of the above basic strokes, arbitrarily calling up the table, and displaying the table on the display device 14 is provided. Is provided. Further, in the registration buffer 33, when grouping is performed in the command selection unit 32, a table of grouping can be stored so that, for example, all invalid, partially valid, and all valid can be selected.

The above-mentioned gesture command determination unit 27
Outputs a command corresponding to the basic stroke closest to the trajectory coordinates to the text editing unit 1a as a result of the comparison with each of the basic strokes, and is processed from the supplied trajectory coordinates by the command. The coordinate information of a character or a character string, that is, position information on the display device 14 is output to the text editing unit 1a.

Thus, the text editing unit 1a
The characters and symbols of the position information in the character and symbol array in the text memory 2a can be edited by deleting or inserting them at that position. At this time, the text editor 2a displays the text memory 2a displayed on the display device 14 when the above-described gesture function is activated, that is, immediately before the input operation with the input pen 12 is started.
A screen information buffer (storage area) 31, which is a memory for recalling and temporarily storing the screen information in the area, is set. In the screen information buffer 31, when the next screen information is stored, the previously stored screen information is cleared.

As a result, the previous screen information is stored in advance, such as when the above editing is canceled. For example, as shown in FIG. 17, the trajectory display of the input pen 12, the rectangular display of the recognition area, the recognition information Even if a display, a list of recognition displays, a message asking whether to cancel the processing result, or the like is displayed, execution of redisplay of the editing screen, execution of the recognition function, cancellation of the editing processing, and the like can be speeded up.

The text editing unit 1a receives not only a command from the gesture command determination unit 27 but also a command from the keyboard 10 as described above, and performs editing by the command. Is available.

On the other hand, independently of the above-described processing of the locus coordinates, the locus coordinate data extracted by the locus coordinate extracting unit 21 is output to the locus drawing unit 28 to become a locus image connecting the locus coordinate data. .

The generated locus image is combined with a character string, symbol string or image currently formed in the bitmap memory 2b and stored in the bitmap memory 2b. Therefore, when the operator operates the input pen 12
Thus, the gesture image input by tracing the tablet 19 is displayed on the display device 14 in real time.

FIGS. 8 and 9 show operation examples of such gesture editing. First, in FIG. 8, when deleting one character “e” in the text displayed on the display tablet 20, a gesture such as “V” is performed on the character “e” with the input pen 12, That is, after drawing, the input pen 12 is
Release from above.

In the above-described drawing, it is important for the recognition of the above-mentioned characters that the start position thereof be within the range of the above-mentioned characters. In particular, in order to facilitate recognition of a designated character, it is desirable to start drawing from substantially the center of the character.

Subsequently, the trajectory coordinates are extracted, the trajectory image is displayed on the display device 14 of the display tablet 20, and the one-character erasure corresponding to the drawing of "V" obtained by collation with the basic stroke is performed. Along with the command, the position information of the character "e" is output to the text editing unit 1a.

Thereafter, based on the position information, the cursor 14a on the display device 14 is operated from the original position, for example, the position of the character "" by the input pen 12, that is, the character "e" in the above case. ”Position,
By deleting the "e" in the text memory 2a based on the command, the character of the "e" in the bitmap memory 2b is deleted, and the "e" in the display device 14 is deleted. Is deleted. In this way, the gesture editing is completed.

As shown in FIG. 9, each edit symbol is displayed on the display tablet 20 by an input pen (not shown).
By drawing on the top, gesture editing can be performed in the same manner as described above. The basic stroke, which is an image of the above-mentioned editing symbol, can be set arbitrarily.

Next, the pointing function according to the above configuration will be described. First, in the conventional word processor, the cursor is sequentially moved when a plurality of characters on the text are designated as an area. In the above configuration, the position of the cursor 14 a is directly moved by the direct input pen 12. It is possible.

The switching between the gesture and the pointing is determined based on whether or not the extracted coordinates are coordinates of one point having no trajectory, that is, whether or not each of the coordinates is continuous. Such a determination is made by the trajectory determination unit 23.

Therefore, for example, when it is determined that the pointing is performed, the obtained coordinates are not collated with the gesture command table 27, and only the position information of the coordinates is output to the text editing unit 1a. An operation example of such a pointing function is as shown in FIG.

Next, handwritten character input will be described using the above configuration. The handwritten character input can be executed by a processing method equivalent to the processing in which the above-described gesture editing compares the locus coordinates with a previously stored basic stroke.

That is, as shown in FIG. 1, the handwritten character recognition unit 2 is provided at the position equivalent to the processing of the gesture command determination unit 26, that is, in the handwriting editing interface 16.
9 is formed, and the handwritten character table 30 in which handwritten characters are stored in advance is a gesture command table 27.
2 is formed in the handwriting recognition dictionary 17 shown in FIG.

The handwritten character recognizing unit 29 compares the locus coordinates supplied from the locus coordinate extracting unit 21 with a plurality of basic character strokes stored in the handwritten character table 30, and finds the most similar basic character stroke. Is output from the handwritten character table 30 to the text editing unit 1a.

The difference between the handwritten character input function and the gesture editing function is that the handwritten character recognition unit 29 does not output position information. Note that the above-described handwritten character input function can be started from an instruction from the keyboard 10 or an instruction from an icon on the display device 14, and is displayed on the tablet 19 of the display tablet 20 as shown in FIG. In addition, by gesturing "△", it can be started from the gesture editing function.

An operation example of such a handwritten character input function will be described. As shown in FIG. 11, when the handwritten character input function is activated, the display tablet 2 is activated.
Window 1 for inputting handwritten characters at a predetermined position 0
4b is displayed, and a character, for example, "su" is handwritten with the input pen 12 in the first input area in the window 14a.

Thereafter, when the input pen 12 is brought into contact with the adjacent second area for the next input, the first pen
Character recognition of the locus coordinates handwritten in the area is executed by the handwritten character recognition unit 29 with reference to the handwritten character table 30 as described above, and the character of the recognition result is input to the position indicated by the cursor 14a. And display it.

Thus, for example, a font such as a cursive font is stored in the handwritten character table 30 in advance, and the cursive font is input by handwriting, and the handwritten character table 30 is input.
Can display and print cursive fonts.

Next, an operation example of the above configuration in the gesture editing function will be described with reference to the flowchart of FIG. 12. First, the tip of the input pen 12 is contacted or pressed at a desired position on the display tablet 20. That is, when the camera is down, the coordinates at that time are indicated via the pen interface 13.

On the other hand, the function of the input pen 12 in, for example, the icon area or the pen button area on the display device 14 is a selection function. Therefore, the effective range of the gesture editing function on the display tablet 20 as described above is the tablet 19 area excluding the above-described areas.

Accordingly, in the tablet 19 of the display tablet 20, for example, as shown in FIG. 13, the effective area 19a of the tablet 19 and the gesture function are previously set according to each screen used in the display tablet 20. An invalid area 19b indicating, for example, an icon area which is prioritized, and a pen button area 19c which is prioritized for the gesture function are set.

Therefore, as described above, when the pen down is executed and the coordinate data signal is output to the locus coordinate extracting unit 21, first, the pen down coordinate which is the contact coordinate of the first input pen 12 is detected ( Step 1, hereinafter, step is abbreviated as S), and compared with the effective area coordinates (S
2) As a result, if the pen-down coordinates are outside the effective area coordinates, the gesture function is invalidated and the gesture function ends.

If the pen-down coordinates are within the effective area coordinates, the next step is to determine whether or not a pen button area, which is one of the other factors for inhibiting the gesture function, is at the pen-down coordinates. If the pen down coordinate is on the coordinates of the pen button area (S3), the routine proceeds to the pen button processing routine (S3).
4).

On the other hand, if the pen-down coordinates are outside the coordinates of the pen button area, the pen-down coordinates are stored, and when the next coordinate data is input by the input pen 12, the coordinate data and the pen-down coordinates are stored. Comparing the input coordinates with the pen-down coordinates,
That is, it is determined whether or not at least one of the X coordinate and the Y coordinate is continuous for eight coordinates (S5).

At this time, if the coordinate data is a continuous locus, the gesture function is turned on, the gesture ON flag is set (S6), and the input pen 1 is turned on.
Even if 2 passes outside the effective area of the gesture function, that is, through the icon area, the icon indicating function of the input pen 12 in the icon area is stopped. If the determination in S5 is not a continuous trajectory, the process proceeds to a pointing process routine (S7).

The steps S5 and S6 are reversed.
That is, the gesture ON flag may be set first. In this case, when the coordinate data is not continuous, the processing shifts to a pointing processing routine. At this time, the gesture ON flag is reset.

Thereafter, a gesture operation, that is, an operation of inputting a predetermined image on the tablet 19 by the input pen 12 is performed, and the contact between the input pen 12 and the tablet 19 is released, that is, the input pen 12 is raised. When,
As shown in the flowchart of FIG. 14, first, it is determined whether or not the flag indicating that the gesture function is ON is set as described above (S11). Is determined to be an operation unrelated to the gesture function, and the gesture function ends.

If the above flag is set,
The gesture command determination unit 26 first normalizes and stores the coordinate data of the input pen 12 from the pen down coordinate to the pen up coordinate in a dedicated coordinate data memory for recognition processing (S12).

Next, the above-described coordinate data recognition processing is executed by the gesture command determination unit 26 (S13).
Subsequently, if the recognition is OK based on the comparison with the gesture command table 27 (S14), each editing process according to the recognition result is executed (S15), and the gesture editing is completed. If the recognition processing is not possible in the above recognition processing, the processing shifts to error processing (S16), and the gesture editing ends.

As described above, in the configuration of the above-described embodiment, when pen-up is performed, even if the pen coordinates at that time are outside the effective area of the gesture function, if the gesture function is ON at that time, the gesture recognition is performed. If the pen down coordinates are within the effective area and outside the pen button area, the coordinates of the drawing locus following the pen down coordinates are outside the effective area and the pen button area. In this case, the coordinate data of the drawing trajectory is validated, and the process can proceed to the recognition process.

Therefore, in the above configuration, for example, as shown in FIG. 15, when the input pen (not shown) is lowered at the point A, since the point A is within the gesture valid area 19a, the input pen Is turned on, and after the drawing with the input pen is completed, the up coordinate of the input pen becomes point B, for example, even if the point B is outside the gesture valid area 19a, that is, the invalid area 19b. , That is, the gesture is valid. The same applies to the case where each coordinate in the middle of the drawing trajectory becomes the invalid area 19b.

However, as shown in FIG. 16, for example, as shown in FIG.
When an input pen (not shown) is lowered at a point C within 9a, the operation by the input pen is performed by the gesture function O.
It becomes FF, for example, and it shifts to icon processing etc.

By the way, in the conventional gesture function, after the drawing is completed, it is determined whether or not the drawing is valid. Therefore, it takes a long time to perform the determination. If the input pen touches an invalid area outside the valid area of the gesture or a tablet area that is the pen button area while drawing with the pen, the processing of the gesture function stops and the processing shifts to error processing. Was.

For this reason, in the related art, it is necessary to pay attention to the operation of the input pen not only at the start of the operation but also at the end of the operation. .

However, in the configuration of the above embodiment, if the pen down coordinate of the input pen 12 is within the effective area 19a of the gesture function, the gesture function is started and the input pen 12 The drawing is valid even if it touches the outside of the gesture valid area 19a, that is, the invalid area 19b or the pen button area 19c.

With this configuration, the gesture function can be started by the pen-down coordinates, so that the gesture function, such as calling the gesture command table 27, can be prepared in advance, so that the gesture function can be speeded up. Possible, and, on top of that,
During drawing with the input pen 12, it is not necessary to pay particular attention to the effective area, and the operability of the gesture function can be improved.

Next, each operation at the time of the command selection function in the gesture function will be described with reference to the flowchart of FIG. When the above-described processing of the command selection function is activated by an instruction signal from an input means such as the keyboard 10, first, the processing function names of all recognizable commands are called (S21), and the processing function names are changed. It is displayed on the display device 14 as a table item (S
22).

Then, after selecting valid / invalid for each item using the execution key on the keyboard 10 (S2
3) Whether or not the valid / invalid command is updated by key operation is set, for example, as shown in FIG. 19 (S24). At this time, when updating to the newly selected valid / invalid command, the execution key is turned ON, the above selection data is fetched (S25), updated (S26), and the screen on the display device 14 is cleared ( S27) If the above-mentioned processing is completed, and if the command is not updated to the newly selected valid / invalid command, the screen on the display device 14 is cleared without turning on the release key and updating (S27). The process ends.

Next, each operation in the registration / call processing in the command selection function will be described with reference to the flowchart of FIG. First, when the above-described registration / call processing is activated by an instruction signal from input means such as the input pen 12 or the keyboard 10, first, it is determined whether the processing is registration processing or calling processing (S31). After calling all of the current valid / invalid command data to the display device 14 (S32) and confirming it, a registered name is entered from the keyboard 10 or the like (S33), and the valid / invalid command data is registered with the registered mates. Buffer 3
3 (S34), and the registration process is completed.

On the other hand, in the case of the calling process in S31, after the registered name is inputted and selected (S35), the registered name is searched in the registered buffer 33 (S36), and the valid / invalid corresponding to the registered name is searched. The command data is called and displayed on the display device 14 (S37), and the valid / invalid command data up to that time is updated to the valid / invalid command data of the registered name (S38), and the calling process is completed.

As described above, in the configuration of the above-described embodiment, the validity and invalidity can be set for each command, so that the gesture function required by the user can be validated. This can prevent a malfunction due to unnecessary gesture function recognition, and can reduce the number of gesture commands recognized by the user, thereby increasing the user recognition rate of commands.

Therefore, the above configuration can reduce erroneous recognition of each command used for the gesture function, and can perform more accurate editing processing according to the user's intention.

Further, in the above configuration, when a plurality of users use the same information processing apparatus, the valid / invalid of the command is selected for each user, and the selected contents are stored, that is, registered. Can be. This allows
Gesture functions individually required by a plurality of users can be arbitrarily selected at any time, and editing processing suitable for each user's editing method and the like can be easily performed.

In addition, the above-described configuration is capable of generating texts or the like caused by carelessly performing erroneous editing by unnecessary gesture functions when a gesture function necessary for each user is changed every time a user changes. Time can be reduced.

Next, a description will be given, with reference to FIG. 21, of a mode selection process based on the valid / invalid grouping in the command selection function. In the above-described configuration, when an interrupt by the mode selection key is generated by the keyboard 10, the input pen 12, or the like during the gesture function, first, processing function names of all recognizable commands are called, and then the processing function names are displayed. It is displayed on the display device 14 as a table item (S41).

Then, after selecting valid / invalid for each item by using the execution key on the keyboard 10, a valid command group and an invalid command group are set (S
42) It is set by a key operation whether to update the valid / invalid command so far to the command selected as described above (S43).

At this time, when updating to the newly selected valid / invalid command, the execution key is turned ON, the above selection data is fetched and updated (S44), and the display device 1 is updated.
4 (S45), and the mode selection process is completed. On the other hand, if the command is not updated to the newly selected valid / invalid command, the release key is turned ON and the command is not updated. Clear the screen (S4
5), the mode selection processing ends.

As described above, in the above configuration, since the invalid command in the gesture function can be set, it is possible to prevent erroneous recognition of the gesture command in the gesture function as described above.

In addition, in the above-described configuration, for example, the gesture editing can be enabled by setting to invalidate an erasure / deletion command and enable other commands, but the gesture operation can be performed on the display device 14 by the gesture operation. The possibility that an image such as a sentence is mistakenly erased is avoided. FIG. 2 shows an example of such a partially effective gesture function selection screen.
It is shown in FIG.

Since the partial validity can be selected in this way, the effect of preventing the image such as a sentence on the display device 14 from being accidentally erased is particularly great for a beginner who handles the gesture function, and the above configuration is operated. It is possible to save the trouble of re-creating a sentence or the like due to inadvertent mistakes, thereby improving operability.

Further, the above configuration is different from the screen information buffer 31 shown in FIG.
As shown in the flowchart of FIG. 23, when the gesture function is activated, the image information on the display device 14 at that time is stored in the screen information buffer 31, that is, saved (S51).

Subsequently, the input pen 12 is lowered (S
52), a command instructing an editing function is input by the input pen 12 (S53), the recognition of the editing range is determined (S54, S55), and when the input pen 12 is raised (S52), the command and the command The editing range is output to the gesture command determination unit 26 (S56).

Thereafter, the display screen of the display device 14 is returned to the display screen with the screen information saved in the screen information buffer 31 (S57). Subsequently, a function such as an editing operation is performed on the screen information stored in the text memory 2a, and the result is displayed on the display device 14 (S
58).

Next, it is determined whether or not the display screen of the display device 14 is to be canceled (S59). If not, the above-mentioned gesture function is completed, and if it is to be canceled, it is saved in the screen information buffer 31. The display screen of the display device 14 is returned to the displayed screen information, and the screen information in the text memory 2a is returned to the original (S60).

As described above, in the above configuration, the image information on the display device 14 immediately before the operation by the input pen 12 for the gesture function is stored in the screen information buffer 31. Therefore, as described above, when the gesture function is canceled or the like. Thus, it is possible to quickly return to the original image.

[0095]

According to the gesture processing apparatus and the gesture processing method of the present invention, the instruction is started on the display surface by the input means, that is, when the first coordinate is specified, it is determined whether the gesture function is valid or invalid. Since the editing command is extracted based on the stroke following, the gesture function of editing the information displayed in the effective area when the gesture function is valid can be performed accurately and promptly. Further, in the gesture processing device and the gesture processing method of the present invention, the gesture function is not disabled even if the input means passes through the gesture invalid area while instructing a stroke to an arbitrary position on the display screen by the input means. As a result, the effect of disabling the gesture function as in the related art and eliminating the trouble of instructing again with the input means can be achieved.

[Brief description of the drawings]

FIG. 1 is a main block diagram of a gesture processing apparatus of the present invention.

FIG. 2 is a block diagram of the gesture processing device of the present invention.

FIG. 3 is an explanatory diagram showing an example of recognition of a basic stroke in the gesture processing device of the present invention.

FIG. 4 is an explanatory diagram showing position information on a display tablet of the gesture processing device of the present invention,
(A) is a tablet, (b) is a display surface of the display device.

FIG. 5 is an explanatory diagram showing a gesture function effective area and a gesture function invalid area in the display tablet of the present invention.

FIG. 6 is an explanatory diagram showing examples of basic strokes of the present invention and commands corresponding to the respective examples.

FIG. 7 is an explanatory diagram showing examples of basic strokes of the present invention and commands corresponding to the respective examples.

FIG. 8 is an explanatory diagram showing an example of an editing operation of a gesture function in the gesture processing device of the present invention.

FIG. 9 is an explanatory diagram showing each editing operation example of the gesture function in the gesture processing device of the present invention.

FIG. 10 is an explanatory diagram showing an operation example of a pointing function in the gesture processing device of the present invention.

FIG. 11 is an explanatory diagram showing an operation example of a handwritten character input function in the gesture processing device of the present invention.

FIG. 12 is a flowchart at the time of pen down in the gesture function of the present invention.

FIG. 13 is an explanatory diagram showing a pen button area in the display tablet of the present invention.

FIG. 14 is a flowchart at the time of pen-up in the gesture function of the present invention.

FIG. 15 is an explanatory diagram showing an example of a trajectory on a display tablet in the gesture function of the present invention.

FIG. 16 is an explanatory diagram showing an example of another trajectory on the display tablet in the gesture function of the present invention.

FIG. 17 is an explanatory diagram showing a display example on the display tablet in the gesture function of the present invention.

FIG. 18 is a flowchart for setting valid / invalid of calling in the gesture function of the present invention.

FIG. 19 is an explanatory diagram showing a display example on a display device when setting validity / invalidity of a call in the gesture function of the present invention.

FIG. 20 is a flowchart for setting registration / call in the gesture function of the present invention.

FIG. 21 is a flowchart for setting a mode in the gesture function of the present invention.

FIG. 22 is an explanatory diagram showing a display example on the display device when selecting a mode of the present invention.

FIG. 23 is a flowchart showing a method for saving image information in the gesture function of the present invention.

[Explanation of symbols]

 12 input pen 14 display device 19 tablet 22 gesture determination unit (determination means)

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiroyuki Nagasawa 22-22 Nagaike-cho, Abeno-ku, Osaka-shi, Osaka Inside (72) Inventor Hiroshi Nakao 22-22 Nagaike-cho, Abeno-ku, Osaka-shi, Osaka Inside (72) Inventor Yasuhiro Nakajima 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Inside Sharp Corporation (72) Inventor Masaaki Kurata 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Sharp Corporation

Claims (4)

[Claims] 1. A display means such as a liquid crystal display, a coordinate extraction means for outputting a stroke of continuous coordinates on a display surface of a display means specified by an input means such as an input pen, and an output from the coordinate extraction means. Determining means for determining whether or not the display position corresponding to the first coordinate is a predetermined effective area; and a stroke which is stored in advance as a stroke continuous to the first coordinate determined to be an effective area by the determining means. A gesture recognition unit that outputs an editing command corresponding to the closest basic stroke by comparing the basic stroke with the extracted basic stroke, and edits information displayed in the effective area on the display unit based on the editing command output from the gesture recognition unit. A gesture processing device comprising: 2. The gesture processing apparatus according to claim 1, wherein said determination means does not perform an area determination on coordinates continuous with a first coordinate in the effective area. 3. A gesture processing apparatus comprising: a display means such as a liquid crystal display; and a coordinate extracting means for outputting a continuous coordinate stroke on a display surface of the display means specified by an input means such as an input pen. In the processing method, it is determined whether or not a display position corresponding to the first coordinate output from the coordinate extracting means is a predetermined effective area, and is continuously connected to the first coordinate determined to be an effective area. The editing command corresponding to the closest basic stroke is output by comparing the stroke to be performed with the previously stored basic stroke, and the information displayed in the effective area on the display means is edited based on the output editing command. Gesture processing method. 4. The gesture processing method according to claim 3, wherein region determination is not performed for coordinates that are continuous with the first coordinates in the effective region.