JP2000172428A - Information processor and information system using same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain environment wherein user's intension is recognized and implemented by receiving external information from the outside and inputting information of a voice, characters, numbers, etc., displaying the inputted information or received external information, and transmitting the displayed information. SOLUTION: Coordinate data inputted from handwriting input hardware H211 as the external information from outside are stored temporarily in a stroke buffer T100 through a stroke input part P110. The coordinate data in the stroke buffer T100 are read out of a format definition part P120 and a stroke interpretation execution part P140 in order and processed. When the operation mode is changed, the stroke interpretation execution part P140 is actuated, the coordinate data in the stroke buffer T100 and format definition data T200 are read out, and a handwriting stroke is interpreted. Consequently, the result is displayed on the handwriting input hardware H211 and the displayed information is transmitted to an antenna 9 and a speaker 6.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a handwriting input information processing apparatus capable of inputting paper and a pen, and a handwriting input information processing system using the same, and more particularly to a handwriting input information processing apparatus suitable for a format document. .

[0002]

2. Description of the Related Art As a handwriting input information processing apparatus capable of inputting with the feel of a paper and a pen, conventionally, a British publication number GB2193827A, a German publication number DE3511353A1, a Japanese Patent Publication No. 63-184130,
Further, SIG CHICHI I VOL. 18, No. 2 (1989), pp. 73-74 (SIGCH
I, Volume18, Number2, (1989), PP73-7
4) and so on.

On the other hand, Japanese Patent Application Laid-Open No. 63-158623 (US priority claim: US940,408) discloses a device for inputting format data in a touch screen manner.

The above prior art is characterized in that a direct or touch input can be made on the screen.

[0005] Furthermore, it is assumed that the operation description is performed in a specific area (icon or menu) of the CRT screen, and that the operation description is executed when the mouse is clicked (switch on / off). (1987) (The Complete Hyrer Card Handboo
k, Bantom Books Inc., Good man, D (1987)).

As a recent user interface system of a workstation, there is an X window tool kit as disclosed in, for example, Nikkei Computer, April 24, 1989, pp. 81-92. . It features an object-oriented environment.

[0007]

The prior art described above does not take into account a handwriting environment in which a user can input with the sensation of a paper and a pen, and understands and executes a user's intention. There was a problem that it did not fit. This problem will be further described with reference to FIGS.

Now, consider a case where a mail format document shown in FIG. 2 is processed by a computer. The mail format document G100 in FIG. 2 may be divided into areas G110 to G210. Here, in the receiver input area G130, the sender input area G150, the message input area G170, and the content input area G190, it is necessary to grasp the contents inside the computer. Need to recognize strokes. The recipient input area G13
At 0, when the input is completed, this document is used as information for mail transfer to the recipient.

On the other hand, the comment input area G210 is information that does not need to be processed as information in the computer in particular, and is simply information returned by the receiver to the sender. In this case, the text is not handwritten and may be a handwritten comment on the image.

To implement the above mail format document on a computer by a conventional method, as shown in FIG. 3, it is necessary to manually input while pointing at the menu areas G220 and G230. For example, if the user manually writes in the sender area G150 and the comment area G210, and then tries to transfer this mail document, the user's operation is as follows.

(1) The "character" in the menu area G220 is designated by the pen 2.

(2) Input a handwritten character with the pen 2 in the input area G150 of the sender. (3) Specify an "image" in the menu area G220 with the pen 2.

(4) A pen 2 is used to input a handwritten comment in the comment input area G210.

(5) Grasp the contents of the screen, and if OK, instruct “Transfer” in the menu area G230 with the pen 2.

As described above, menu operations (1),
(3) and (5) have a problem that the user must be conscious and interrupt human thinking.

Further, in the conventional touch screen format input device, menus have a hierarchical structure, and there is an advantage that data can be input only by instructing menus one after another.
The format is limited to a specific format, and the data must be selected from a menu. Therefore, there is a problem that various formats cannot be easily handled.

On the other hand, in the Xwindow toolkit and the hyper card, as shown in FIG. 4, the user interacts while clicking on the CRT screen with the mouse of the pointing input device. Characters and figures cannot be entered. Therefore, there is a problem that it is not possible to talk with the processing device as if using paper and a pen.

Further, as another problem of the above-mentioned prior art, since the conventional information processing apparatus is of a desk (disk top) or a knee (lap top) size, it can be carried and used anywhere like a notebook. There was a problem that did not reach.

It is an object of the present invention to provide an environment in which an input can be made as if a note is made of a paper and a pen, and the information processing apparatus understands and executes the user's intention.

Further, as another object, the information processing apparatus is thinned to a notebook or sheet size, and can be used anywhere.
The aim is to provide a device that is portable and can be used anywhere.

[0021]

SUMMARY OF THE INVENTION To achieve the above object, the present invention provides a receiving means for receiving external information from outside, a microphone for inputting voice, and a keyboard for inputting information such as characters and numerals. , A display unit for displaying the input information or the received external information, and a transmission unit for transmitting the displayed information.

A handwriting input unit for inputting data by handwriting; a screen definition unit for defining input information for describing a program for defining processing of a handwriting stroke input by the handwriting input unit; A format definition unit having an operation definition unit that displays a screen for describing the program on a display unit for each input area defined by the screen definition unit, and a handwriting stroke input by the handwriting input unit. The apparatus includes a stroke interpretation execution unit that processes according to a program defined in the format definition unit, and a transmission unit that transmits the data or the data processed by the stroke interpretation execution unit.

Further, in an information processing system for transmitting data between a plurality of information processing apparatuses via a relay station, the information processing apparatus includes a handwriting input unit that inputs data by hand. A screen definition unit that defines input information for describing a program that defines processing of a writing stroke, and an operation of displaying, on a display unit, a screen for describing the program for each input area defined by the screen definition unit A format definition unit having a definition unit, a stroke interpretation execution unit that processes a handwritten stroke input by the handwriting input unit according to a program defined in the format definition unit, and the data or the stroke interpretation execution unit. And a transmission unit for transmitting the processed data to the relay station.

[0024]

The receiving means receives external information from the outside,
Information that can be freely used in offices, on business trips, in trains, or at home by inputting information such as voices, characters, and numbers from a microphone or keyboard, displaying them on display means, and transmitting them by transmission means. A processing device can be provided.

Since the handwriting input unit has an integrated structure of the input and the screen, the screen operation also means the input operation, and the input can be performed as if using paper and a pen.

For the screen of the handwriting input section, the format definition section first defines the screen layout of a slip, a mail document and the like. Next, a plurality of meaningful areas are generated by automatically setting an area or resetting the area from these screen layouts. Next, an operation description of what processing operation is desired when a stroke is input for each area is defined.

For example, in a certain area, when a handwriting stroke is input, it is recognized as a character, and it is desired to display a correct copy in the place and display the same character in a predetermined position in another area. Also, in that area, you want to recognize handwritten strokes as editing symbols and edit the characters below the editing symbols, and in some areas you want to use handwritten strokes as image data for handwritten comments. I do. This operation description is defined in a handwriting-specific description language suitable for a handwriting processing operation.

When a handwritten stroke is input to the document defined by the format definition unit, the stroke interpretation execution unit calls a predefined operation description in the corresponding area, and interprets the operation according to the operation description. To do.

In general, once a format definition is made, it is defined only once unless the content or layout changes. Therefore, the user calls the required form document,
All that is required is to directly enter the necessary information (area) by handwriting. Then, since the processing device interprets and executes what is meant in the area, an environment in which the user's intention is understood and executed can be realized.

Next, in the handwriting input information processing apparatus, the input / output device and the logic circuit are all mounted on the glass in an integrated structure, and the file is stored in a removable memory card. It is also equipped with a battery and a wireless interface so that it can access the network so that it can be used anywhere. This makes it possible for offices, business travel destinations,
It can be used freely on the train or at home.

[0031]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will now be described with reference to FIGS.
6 will be described. Here, the way of assigning the leader line number name is as shown below.

(I) 1 to 22; device or system name (ii) Hxxx ~; hardware component name (xxx is a number) (iii) Pxxx ~; system configuration name (iv) Txxx ~; table configuration Name (v) Gxxx ~; Screen area name (vi) Fxxx ~; Program flowchart name Fig. 1 shows the system configuration of a handwriting input information processing apparatus according to an embodiment of the present invention.

P100 is an internal block diagram of the system configuration of the present invention.
Be composed. H211 is a handwriting input hardware with an integrated structure of a handwriting coordinate input device and a flat display, P1
10, a stroke input unit for reading coordinate data of the handwriting input hardware H211; T100, a stroke buffer for buffering data read by the stroke input unit; P120, a format definition unit; T200, generated format definition data; Execution unit, P130
Is a wireless input / output unit for cordless operation, 9 is an antenna, and P150 is a voice input / output unit connected to the microphone 7 or the speaker 6 to realize a telephone or the like.

The outline of the operation of the embodiment of the present invention will be described below with reference to FIG. The input coordinate data is temporarily stored in the stroke buffer T100 via the stroke input unit P110 from the handwriting input hardware H211.
In the stroke input unit P110, the handwriting input data is read, and at the same time, the handwriting is displayed on the handwriting input hardware H211. The coordinate data of the stroke buffer T100 is sequentially read from the format definition unit P120 and the stroke interpretation execution unit P140 and processed.

First, when the entire processing mode is set to the definition mode, the format definition section is started. The format definition unit P120 creates a format document using an editor function, and cuts out and sets an area of the format document. When a handwriting stroke is input to this area, what processing operation is desired is defined by a handwriting-specific description language (referred to as UI / HAND-TALK). The result of this operation definition is filed as format definition data T200. This format definition data T200 can be used any number of times by the stroke interpretation execution unit P140 as long as the format is not changed.

Next, when the entire processing mode is changed from the definition mode to the operation mode, the stroke interpretation execution unit P140 is activated, and the coordinate data of the stroke buffer T100 and the format definition data T defined for each of the areas are set.
200 is read and the handwriting stroke is interpreted and executed. The result of the interpretation execution is displayed on the handwriting input hardware H211 or exchanges information with another processing device via the wireless input / output unit P130, or makes a call via the voice input / output unit P150. be able to.

FIG. 5 shows an image of one embodiment of the present invention, taking the mail format document of FIG. 2 as an example. FIG. 5G1
10 to G250 are the contents of the operation description for each area, and this part is not displayed on the normal screen. Here, for example, “OBJ. Menu” of G220 is the name of the program of the operation description. Each program operation description will be described later. In short, when a handwritten stroke is input to a given area, the handwritten stroke is processed according to the description programmed in that area. For example,
In the message area of G170 and the content area of G190, it indicates that handwritten characters are recognized and displayed in a fair copy.
In the comment area of G210, a handwritten memo is input with the pen 2, and the handwriting is displayed as it is.

As described above, according to the embodiment of the present invention, it is possible to directly input a predetermined area as if using paper and a pen.

Hereinafter, details of one embodiment of the present invention will be described according to the following items.

(1) Overall system configuration using handwritten input information processing device (FIG. 6) (2) Hardware configuration of handwritten input information processing device (FIG. 7)
9) (3) Detailed system configuration of handwritten input information processing apparatus and description of its operation (FIGS. 10 to 13) (4) Detailed program and description of its operation (FIGS. 14 to 3)
6) (1) Overall System Configuration Using Handwritten Input Information Processing Device FIG. 6 shows the overall system configuration. 1, 10 is a handwriting input information processing device main body, 2 is a pen, 3 is a handwriting input hardware H having an integrated structure of a handwriting coordinate input device and a flat display.
211 screen, 4-5 memory cards for storing files and programs, 6 speaker, 7 microphone, 8 handwriting data with pen, 9 wireless terminal (WT)
An antenna for communicating with the wireless communication device 12 and the wireless device 20 is built in the housing of the handwritten input information processing device 1. Reference numerals 11 and 15 denote networks, 11 denotes a network connected to an integrated services digital network (ISDN) terminal 19, 15 denotes a workstation (WS) 16, and a print server (P).
S) 17, L connected to the file server (FS) 18
In an AN (Local Area Network), for example, Ethernet or token ring is used.

Wireless terminals (WT) 12, 13
Are connected to both networks 11 and 15, so that both networks can be accessed from the handwritten input information processing apparatuses 1 and 10. In addition, ISDN
A facsimile (FAX) 14 is connected to the network.

In the above system configuration, the handwritten input information processing apparatus 1 can operate alone, but it is also possible to share data and disperse processing with other information processing apparatuses via a network.

On the other hand, since the devices 1 and 10 can be wirelessly connected to the wireless terminals (WTs) 12 and 13, the devices 1 and 10 can be freely carried and the usability is improved. In addition, since it can be connected to a facsimile, the input data of the handwritten input information processing apparatus 1 can be directly output to another facsimile via the ISDN network 22 as it is, or the handwritten input information processing apparatus can be connected via the network. , E-mail exchange and communication conference are possible. Particularly, in a teleconference, an online teleconference using voice and handwritten data can be performed, so that the teleconference is suitable for meetings between remote parties.

(2) Hardware Configuration of Handwritten Input Information Processing Apparatus FIG. 7 is a cross-sectional view of hardware of the handwritten input information processing apparatus. FIG. 9 is a sectional view configured for a presentation as shown in FIG.

In FIG. 7A, reference numeral H10 denotes a glass for inputting coordinates and a transparent electrode H20 for detecting X-axis coordinates is coated. On the other hand, the transparent electrode H6 for detecting the Y-axis coordinates
Reference numeral 0 is used in common with glass H50 for liquid crystal (LCD) display, and is coated on the back side. Therefore, the glass for liquid crystal display is H30 and H50, and the liquid crystal H40 is between them. H80 and H100 are logic circuits for performing coordinate input and processing of the processing device 1, and the coordinate input glass H
10 are mounted on the back side.

This is LS by COG (Chip On Grass) technology.
In this method, the bare chip I is directly bonded to a peripheral portion having no coordinate detecting or display segment. Recently, LS, which is applied to mounting of a liquid crystal driver IC,
In order to bond the bare chip of I directly to the glass,
It can be smaller and thinner than the current mainstream surface mounting technology.

Although the logic circuits H80 and H100 are bonded to the coordinate input glass H10 in the embodiment of the present invention, they may be bonded to the liquid crystal display glass H30 or H50.

H70 is a backlight EL (Electro Luminescence), H90 is a connector of the memory cards 4 and 5, and H110 is a battery of the processing apparatus 1. 6,7
Are ultra-thin speakers and microphones, respectively, used for hands-free phones.

As described above, the device for inputting coordinates, the device for display, and the logic device are characterized in that they are made thin as an integrated structure.

FIG. 7 (b) shows a case where the back side of the housing of the present processing apparatus is cut open, and the transmission type is removed except for the EL of the backlight H70. This is as shown in FIG.
(Overhead projector) The light sources 27 of the light source lamps 23 to 27 pass through the processing apparatus via the lens 24, and the screen of the processing apparatus is displayed on the screen via the mirror 26. This is called LCDOHP, and has recently been commercialized as a screen display for personal computers.

Therefore, in this processing apparatus, the LCDO
It can be used as an HP, or it can be handwritten input with the pen 2 directly, as if writing with a magic pen on a transparent sheet. In particular, an effective presentation can be made using a color LCD.

Although FIGS. 7 (a) and 7 (b) show the individual processing devices, they may have a hand-held configuration so that the processing can be switched from (a) to (d).

FIG. 9 shows a hardware block configuration of a logic circuit of the handwritten input information processing apparatus. H200
Is an entire block configuration, H201 is a microprocessor M
PU and H202 are bus signal lines of this MPU, H203 is a main memory, H204 is a display memory for LCD display, H2
05 is a card interface (I / F) for the memory cards 4 and 5, H206 is a wireless I / F, H207 is a TBI / F for coordinate input (TB), H208 is an image developed on the display memory H204, and is displayed on the LCD. A rendering processor H209 is a voice I / F for interfacing with the speaker 6 and the microphone 7, and a H210 is a KBI / F of an optional keyboard (KB) H212. The keyboard H212 is normally unnecessary, and is connected and used as needed for data input and program development. The antenna 9 is built in the housing 1 of the processing apparatus as described above with reference to FIG. In particular, as shown in FIG. 6, it is arranged at the upper part so as not to be affected by a hand holding the pen 2 at the time of handwriting input.

Most of the above logic circuits are LSI or IC
Can be bonded on the glass as described above. In particular, components that cannot be bonded are dealt with by temporarily mounting them on another substrate and wiring the substrate and the logic IC.

(3) Detailed System Configuration of Handwritten Input Information Processing Apparatus and Explanation of Its Operation FIG. 10 is a detailed system configuration diagram of the format definition unit 120 of the system configuration of FIG. 1, and FIG. 11 is a diagram of the stroke interpretation execution processing unit P140. It is a detailed system configuration diagram.

In FIG. 10, P121 is a stroke management unit, P122 is a screen definition unit, P123 is an operation definition unit, and P1 is
Reference numeral 24 denotes a UI / HAND-TAL for translating the source data T120 of the handwriting-specific description language UI / HAND-TALK.
It is a K language translator. T110 is the screen definition part P
Screen data for storing the data processed in step 122, T1
Reference numeral 30 denotes a module file of a program used in P122, P123, and P124, which is a UI / HAND-TA.
LK basic functions and other application (AP) functions.

On the other hand, in FIG.
This is a stroke management unit similar to 1. P142 is UI /
A UI / HAND-TALK interpreter that reads the HAND-TALK definition data T200 and interprets and executes the data.

As shown in FIGS. 10 and 11, in one embodiment of the present invention, a handwriting-only description language (UI / HAND-TA) is used.
LK). As will be described later, this language is an interpreter dedicated to handwriting using the concept of object-oriented programming, which has recently attracted attention as a man-machine interface, and is particularly characterized in management and recognition of handwriting strokes.

The operation and functions of the system configuration shown in FIGS. 10 and 11 will be described below with reference to FIG. First, FIG.
The description starts with the format definition part P120 of 0. Format definition part P1
20 is roughly divided into a screen definition and an operation definition. In this case, one of the programs is started according to the mode of the menu area MENU2 shown in FIGS. FIG. 12A shows a screen in the screen definition mode, which includes a submenu MENU for defining a form document screen G100.
3 is displayed. For example, when "character input" is instructed with the pen 2, a character input sheet G320 is displayed. When a handwriting input stroke is input in the rectangular input area of the sheet, the character is recognized as a character, and characters are sequentially placed at the cursor position on the sheet G100. A fair copy can be displayed. This may be performed by connecting an optional keyboard (KB) and using kana-kanji conversion software.

Similarly, when "drawing" of the menu MENU 3 is designated, a drawing sheet is displayed and the drawing can be performed.

On the other hand, in the “template” menu, screens that are frequently used, such as a receiver and a sender, are stored in a file including their operation description. It is.

When the screen definition is completed, the menu MENU 3
Is completed, and this process ends.

As described above, this screen definition unit 120
This is equivalent to the function of a so-called screen editor.

Next, the operation definition unit P123 of FIG.
As shown in FIG. 2B, when the “operation definition” of the menu MENU 2 is instructed, the operation is started by the stroke management unit P121. Then, a submenu MENU3 is displayed corresponding to this mode. When an area for which an operation is to be defined is designated, an operation description sheet G410 appears as shown in FIG. 12B, and the operation of the corresponding area is described on this sheet. For example, the screen G10 to be defined
In the message area of 0, an operation description for character input is defined using the “character input” or “description language” menu of the menu MENU3.

For example, it is defined that "when an event of an input stroke is detected, cut out the stroke as a character and recognize the stroke as a character" (G4).
10). A frequently used description can be easily defined simply by calling the description language sheet G420 and instructing a corresponding menu. Instead of using a description language sheet, a description command for calling a file having predefined contents may be provided.

The description data thus defined is stored in the UI / HAND-TALK source file T120 of FIG. Then, the menu MENU 3 of FIG.
Is instructed, the UI / H of P124 in FIG.
The AND-TALK language translator is activated, and T1
20 and T130 files, refer to the T200 UI
/ HAND-TALK definition data (intermediate data) is created.

Here, the screen G10 to be defined
Each area of 0 is automatically extracted with reference to the screen data T110 when the screen definition unit P122 determines “complete” of the menu MENU3 of FIG. 12A. However, when editing an area such as resetting an area or grouping (combining two or more areas into one), the “area” in the menu MENU 3 in FIG.
And instruct.

When the above two definitions of the screen definition and the operation definition are completed, as shown in FIG.
By instructing "operation" of U2, an actual form document can be input. The screen G500 of FIG. 12C shows that the handwriting stroke is directly input to the message area G170.

That is, when a handwritten stroke is input to the message input area G170, the stroke management unit P141 of FIG. 11 transmits the “Ob” defined in G410 of FIG.
In this program, the operation is defined so as to recognize the handwriting input stroke as a character as described above, so that the handwriting strokes are recognized as characters one after another as shown in FIG. And will be displayed.

This processing is performed by the UI / H shown in FIG.
The AND-TALK interpreter refers to the files T200 and T130.

In the "operation" mode, there may be a case where the user wants to know the outline of the operation description in the form document. In this case, as shown in FIG. 13, when "definition display" of the menu MENU 3 is instructed with a pen, the operation description content defined in each area is displayed as an icon (pictogram).
Is schematically displayed. For example, it is displayed that “character recognition” and “mail” are defined in the recipient area. This means that the handwriting stroke is first recognized as a character, and then the document is mailed to the address of the character.

By displaying the outline as described above, it is possible to understand what meaning is defined in the area. This is a convenient function especially when a new format is used or when the definition of the area is forgotten.

If you want to know more detailed information, for example, the song of the character recognition function, what kind of character is recognized,
When the pictograph of “character recognition” is further designated, the content may be displayed. However, if it is desired to change or add an operation content, the operation cannot be performed without shifting to the above-described definition mode.

FIG. 33 shows the types of definition display as described above. The types of definition display other than FIG. 13 include "recognition of figures" such as circles and rectangles, "recognition of line segments" such as underlines, and "recognition of editing symbols" when editing a document with a red pen. . Further, there are "telephone", "tellighting" used as a communication conference, and "FAX" in consideration of using the functions of the network in FIG.

(4) Detailed Program and Explanation of Its Operation Lastly, the detailed program of the system configuration described above is described with reference to the flowcharts of FIGS. 14 to 36 and an operation description example using a handwriting-specific description language (UI / HAND-TALK). It will be described using FIG.

FIG. 14 shows a main program of the present processing apparatus. First, initialization is performed in step F100. In this process, initialization of the table and UI / HAND-TAL
Initialization necessary for the K-interprinter and initialization display of the screen are performed.

Next, data input processing of the stroke buffer T100 (step F200) is performed, and the processing mode of the menu MENU 2 shown in FIG. 12 is determined (step F300). By this determination, the screen definition processing (step F400) or the operation definition processing (step F50)
0), one of the operation processes (step F600) is started.

Hereinafter, each specific flowchart will be described.

FIG. 15 shows a screen definition process (step F40).
It is a detailed flowchart of 0). In this process, a corresponding process is started according to the submenu MENU3 (FIG. 12A).

First, data is input from the stroke buffer (step F410), the mode of the submenu is determined from the data (step F420), character input processing (step F430), and drawing processing (step F44).
0), one of the template processes (step F450) is started. When the screen definition mode is determined to be “completed” in the determination of the screen definition mode (step F420), the screen definition processing ends with file of the screen data (step F4).
60).

FIG. 16 shows an operation definition process (step F50).
It is a detailed flowchart of 0). First, the screen data T1 defined in the screen definition processing (step F400)
Based on 00, an area is automatically extracted (step F5).
10).

In the case of a format document written on paper, each area is separated by a boundary mainly composed of a straight line written for separating the format. However,
Not all areas follow the above rules. This example and the area separating method at that time will be described next with reference to FIG.

In the example of FIG. 17A, areas a to g exist, and in the example of FIG. 17B, areas h to s exist. The solid line is a line written on the drawing, and indicates a region where a broken line is extracted. Hereinafter, the region separation method will be described.

(I) Closed area search method FIGS. 17 (a) to 17 (c) show closed areas formed by elliptical figures often used for addressing and the like. Here, the rectangular frame of the elliptical figure is defined as an area. Also, each of FIGS. 6D to 6G is a closed region.

Therefore, the area can be separated by searching for the closed area. As a method of searching for a closed area, a method of scanning line image data drawn in the display memory H204 in FIG. 9 line by line and a method of sequentially tracing line images and separating the smallest loop as one region. There is.

(Ii) U-shaped rectangular area search method An area where one side of the rectangular area is missing, such as k, n, q, m, p, and s in FIG. It is called an area. Such a format is often used for tables.

As a method of separating the area, the line drawing is traced from an arbitrary point, and when the end point is reached, the line drawing is traced in the opposite direction. For example, there is a method of separating a U-shaped rectangular area based on four-way quantum codes).

(Iii) Blank character search method The blank character areas h, i, and j of the date in FIG. 17B are the target areas.

When a character screen is defined, mapping can be performed by storing mapping coordinates of the character string and searching for a blank code of the mapping coordinates and the character string.

The automatic region extraction method has been described above.
Other methods may be used.

Now, returning to the flowchart of FIG.
After the area automatic extraction processing, the stroke buffer data input processing (step F520) is performed, and FIG.
The operation definition mode of the submenu MENU3 shown in (b) is determined (step F530), and the corresponding process is started by this operation definition mode (steps F540 to F540).
F570).

Here, the area processing (step F560)
Performs deletion, change, addition of a new area, and grouping processing of a plurality of areas extracted in step F510. The grouping process is, for example, a process in which the two regions n and q in FIG.

If the set area is highlighted, for example, by filling the area, coloring the area, or displaying the area with a dotted line, the description can be easily defined. If the “complete” menu is determined in step F530, the UI / HAND-TALK language processing (step F57)
0) is performed, and the UI / HAND-TALK definition data T
200 is generated. This data is an intermediate code, which is interpreted and executed by the interpreter shown in FIG.

The detailed flowchart of the definition processing has been described above. Next, the operation processing of FIG. 14 (step F600)
This will be described below with reference to FIGS.

FIG. 18 is a flowchart of the operation process when the already defined form document is actually used. This process is performed using the above-described object-oriented programming technique. First, the basic concept of the process according to this method will be described, and then the content of each operation description will be described.

In FIG. 18, step F610 relates to event processing, and includes stroke buffer input processing (step F611) and event processing (step F61).
2) There is an event occurrence determination process (step F613).

There are five types of events in this event processing, as shown in FIG. That is, (i) stroke on (): an event in which the handwriting stroke shifts from off to on; (ii) stroke off (): an event in which the handwriting stroke shifts from on to off; (iii) stroke timeout (): An event when a predetermined time has elapsed since the handwriting stroke was turned off. (Iv) Pen button on (): An event when the button on the pen tip changed from off to on. (V) Pen button off (): On the pen tip. This is the event that the button that has changed from on to off. Other events include a case in which the stroke-on coordinate has moved from the own area to another area and a case in which a stroke has entered the opposite self-area.

In the physical device shown in FIG.
Each book is of one type, but the concept of a logical device is introduced so that a physical device can be extended to a plurality of logical devices.

FIG. 35 shows that a physical device is extended to two logical devices, an input device and an editing device.

Thus, for example, one pen can be used for multiple purposes, sometimes as an input pen and sometimes as an editing pen. It can also be extended to pens that can be used only by specific people.

In order to perform this extension, a function for associating a physical device with a logical device is required (how to establish correspondence will be described later). FIG. 34 shows five types of physical devices corresponding to input logical devices. Of course, it is also possible to assign only the pen button to the editing pen button.

In the event shown in FIG. 34, () describes the argument of the event. If nothing is written, the default is used.

The following are the arguments of the event.

(A) Argument of stroke ON / OFF stroke The argument of this event describes the feature number of the coordinate at the time of occurrence of the event. The feature numbers include the start point, end point, center of gravity, and center point of the stroke. For example, selecting a center of gravity is suitable for inputting a check mark. Note that, by default, the stroke on is the start point and the stroke off is the end point.

(B) Argument of stroke timeout Timeout time is described. Default is 2 seconds.

(C) Argument of pen button off / on Describes the pen button type number. This button is usually attached to the pen tip, corresponds to a mouse button, and has one (left) or two (left and right) buttons. The default is (left).

Returning to FIG. 18, the event processing (step F)
In step 610), when the occurrence of the event described in FIG. 34 is detected, the coordinates at which the event occurred (coordinates previously operated with the stroke timeout and the pen button) are known, and the object in the area on the coordinates is activated. multiply.

The received object checks the type of the event, and if there is a description in its own object, performs the processing. If there is no object, the process is to pass the process to the object in the overlapped lower area including the user, in the example of FIG. 5, the sheet object G110.

In FIG. 18, an object A (objectA
Suppose ()) is activated. In object A,
Perform the event processing defined for you, but want to request other objects to do some processing. At this time, the object-oriented programming is performed by activating a message. Message activation is performed in the following format.

SEND Object Name Message Name (Argument) Here, the object name is the name of the destination, and the message name is the message name of the object.

That is, the object A requests the other objects B and C for processing, and the object C requests the other object D for processing, so that send B message 1 () send C message 2 () send D message 3 ( ) Is issued. Here, messages 1 to 3 are message names.

As described above, in object-oriented programming, objects are sequentially processed by event activation and message activation.

When all the object processes are completed, the process returns to the event process in step F610.

The basic processing concept of the operation processing has been described above. Next, specific processing operations will be described with reference to FIGS.

In the following description, an operation description example and processing contents of the mail format document shown in FIG. 5 will be described. FIG.
In the following figures, the numbers at the right end are line numbers for explanation.

FIG. 19 shows an example of the operation description of the menu object shown in FIG. 5G220. FIG. 19A shows an input menu object and FIG. 19B shows an edit menu object. This operation description is a description for a menu for switching between an input mode for inputting objects such as characters and figures for a handwritten stroke and an edit mode for editing these objects.

Here, “Object” is a reserved word representing an object, “Input menu” is the name of the object, and “(menu)” is
This indicates that the prototype of the “input menu” object, that is, the function and data structure of the object “menu” are inherited. Using this inheritance function has the advantage that a basic object can be created, functions can be added on top of it, and a new object can be created to create a modular structure. “$” On the first line and “$” on the eighth line indicate the start and end of the object.

In this “input menu” object,
The correspondence between the physical device and the logical device described in FIG. 34 is defined, and the operation description for highlighting the own menu is provided. That is, when an editing stroke off () event occurs (this event occurs first because a physical device is assigned to the editing stroke), the device assignment of Nos. 1 and 2 in FIG. 34 is first changed. (3rd and 4th rows in FIG. 19A). The device allocation change is performed by the event_map function.

Next, the input stroke is erased, and menu highlight processing is performed, and the processing of this object is completed (5th to 6th lines).

FIG. 19B shows the operation description of the “edit menu” object. Since the description is almost the same as that of the “input menu” object, the description is omitted.

FIG. 20 is an operation description of the “sheet” object in FIG. 5G110. Note that the prototype of this object is window.

In this object, in order to have the same meaning as a blank sheet, it is described that even if a handwritten stroke is input, only the stroke is erased, and otherwise, nothing is done.

More specifically, when an event for an input stroke or an editing stroke occurs, only stroke erasure is performed, and the process ends. As a result, the handwriting of the handwritten stroke is displayed once, but the handwriting is erased each time the stroke is turned off, so that the input to the meaningless area can be processed as if nothing.

FIG. 21 is an operation description example of the object in the recipient area in FIG. 5G130.

The operation description of this area is a description in which a handwritten stroke is recognized as a character, and then the recognized character code is transferred as a mail address of a document.

The first to third lines in this figure are descriptions for defining the type of handwriting stroke. The concept and purpose of this handwritten stroke type will be described below.

FIG. 36 shows the types of the handwritten stroke type. There are the following five types.

(I) Character stroke type (CHA_ST) A stroke recognized as a character.

(Ii) Graphic stroke type (GRAPH_S
T) A stroke recognized as a figure.

(Iii) Line segment stroke type (LINE_S
T) A stroke type recognized as a line segment other than a figure.

(Iv) Image stroke type (IMAGE_S)
T) A stroke recognized as an image.

(V) Edit stroke type (PROOF_S
T) A stroke recognized as an edit symbol.

Each stroke type further includes:
There is a small classification type. In particular, there are as many as 10 character stroke types.

As described above, the handwritten stroke is used for many purposes, but in a format document, the type of the stroke is often limited depending on the area. For example, FIG.
In the h, i, j area of the format document (b), only numbers are allowed. On the other hand, in the recipient area of FIG. 5G130, only the characters of the first and second kanji, hiragana, and katakana are allowed to input the name.

For this reason, when what type of stroke is input is limited by each region, by defining it, the recognition performance can be improved, or if a stroke other than that type is input, An alarm may be displayed to the user as an erroneous input.

Further, since the stroke type can be defined in advance, there is no need to provide a menu for designating the type of stroke as shown in G220 of the conventional example in FIG.
Contributes to modelessness, greatly improving usability.

Further, by expanding the stroke type so that the recognition range can be specified as follows,
Can be used for many applications. For example, if sttypedef ALPHA_L {H, I, T, A, C}: HITACHI attributes = pass_name: HITACHI, the static variable pass_name is interpreted as a HITACHI stroke type. Then, this type is interpreted as being the uppercase alphabetic ALPHA_L type and allowing only the character codes of H, I, T, A, and C. Therefore, if the HITACHI stroke type is defined in a certain area, only a specific character can be recognized, so that it can be used like a password.

It is preferable to use the method of defining the stroke type and the range designation for inputting the slip shown in FIG. For example, the specification area G720 often has only specific characters depending on the product name. For this reason, if a specific code of this product name is declared, if other characters are input, an alarm can be displayed to the operator, and erroneous input can be prevented.

In the example of FIG. 21, the variable name_result is set to NA
This is an example in which a variable proof_result is declared as a PROF stroke type for the ME stroke type.

Note that "!" Indicates a logical sum, and attributes indicates that the variable is static (variable is secured in the resident memory).
ables means that the variable is dynamic (allocates the variable on the stack).

FIG. 21 has the following three event descriptions.

(I) Stroke ON for event input () (ii) Stroke time-out for event input () (iii) event Stroke time-out for editing () When the event (i) occurs, first, the stroke is cut out as a character. The ninth line, that is, the send stroke cutout frame (cutout OK) is executed. This means that the message "frame" of the "stroke cutout" object is processed and the result is desired to be reflected in the "cutout OK" flag.

Next, on the tenth line, the description of if cutout OK {send recognition recognition execution (name_result)} is executed. In this description, if the character extraction flag (extraction OK) is true, the “recognition execution” message of the “recognition” object shown in FIG.
It means that you want to store it in esult.

However, if the character extraction flag (extraction OK) is false, the while stroke off ｛目 の on the eleventh line is executed, and the process loops until the handwriting stroke is changed from on to off.

Next, when the input stroke time-out event of (ii) occurs, first, a "recognition execution" message of the "recognition" object is transferred (line 14). Next, a message for registering the code of the recognition result in the recipient name is transferred (line 15). Finally, in order to transmit this mail document, the message “mail” of the “transfer” object is transferred and the processing ends. (Line 16).

Finally, the event (iii) is a description in the case where the receiver name is corrected by the edit symbol.

Although the operation description of the "receiver" object has been described above, other objects will be briefly described below.

FIG. 22 is a diagram showing the operation description of the "recognition" object. There are six message descriptions.

This object does not correspond to an area, but is an object activated by a message from another object.

The first message description in the third to eleventh lines, message recognition execution (result: STTYPE) {...}, Is the object main program. Here, "STTYPE" on the third line is a dummy stroke type.

In a normal programming language such as PASCAL, if the types of the actual argument and the formal argument do not match, it is recognized as a syntax error. Think of it as attribute data attached to variables. That is, the "ST
"TYPE" is a dummy stroke type having no attribute, meaning that the attribute of the actual argument is inherited at the time of execution.
For example, as shown in the tenth and fourteenth lines in FIG. 21, send recognition recognition execution (name result) is executed, name The stroke type of result is shown in Fig. 2.
“NAME” is defined in advance as shown in the fourth line of FIG. Because the type is inherited at runtime,
“STTYPE” is the same stroke type as “NAME”.

On the other hand, as shown in the 19th line of FIG. 21, send recognition recognition execution (proof When "result" is executed, "STTYPE" inherits the "PROOF" type.

Therefore, as shown in the fourth line of FIG. 22, the type of stroke can be determined at the time of execution by transmitting a message of send stroke type (result).

If the stroke type can be determined,
As shown in the 0th line, a message recognized by each stroke type is transmitted, and the processing operation of “recognition execution” ends.

The individual recognition processing is described in the first part of FIG.
The operation description is described in the second and subsequent lines.

Here, there is a send edit type (kind) in the "edit symbol recognition execution" message on the 31st line, and attention will be paid to this. This is based on the coordinates (proof_result) coordinates where the edit symbol was entered)
This is the processing of a message that searches for the code below it and thereby determines the type of object to be edited.

For example, if there is a character code under the coordinates of proof_result, character editing is performed, and if there is a graphic code, graphic editing is performed. This is executed by send edit execution (kind, proof_result) on the 35th line.

As another method of judging the object to be edited, there is a method of defining an editing variable for each area, and judging by looking at the value and attribute of this variable.

Next, FIG. 23 is a description of a “display” object activated mainly from the object of FIG. In this object, pay particular attention to the following description on the fourth line.

The result of the send range check is (cha result,
ok) This description means that the recognized code is checked for a character code in the range defined by sttypedef, and the result is to be reflected in the “ok” flag. Where cha As shown in FIG. 24, the data structure of the result is a recognition code (2 bytes), a stroke type attribute (3 bytes), and a pointer to a range table (2 bytes). The same applies to other stroke type data structures.

Therefore, since the character range defined in the operation description is stored in this range table, the range can be checked by determining this code and the recognition code.

In the fifth line of FIG. 23, if the result of the range check is false, "alarm display" is executed, and a warning message can be issued to the operator.

FIG. 25 is a description of the "sender" object in the area of FIG. 5G150.

This object is a description for recognizing a handwritten stroke as a character and registering the code as a handwritten sender.

In this description, in the “Recipient” object in FIG. 21, there is no description of the mail transfer on the 16th line, and the description on the 15th line in FIG.
Others are the same except that it is changed to "caller registration".

FIG. 26 is a diagram showing the “message” in the area of FIG. 5G170.
Description of the object.

This object is for inputting a title of a business message, and recognizes and displays a handwritten stroke as a character, or is a description for character editing. The description is omitted because it can be easily inferred from the description of the “recipient” object.

FIG. 27 is a description of the "content" object in the area G190 in FIG. 5G.

This object is a description for inputting the specific contents of the message. Like the “message” object,
There are descriptions for character recognition and character editing. However,
In this “content” object, the stroke type of the static variable c_result is changed so that all characters can be input.
(See the first to fifth lines of FIG. 27).

FIG. 28 is a description of the "comment" object in the area G210 in FIG.

As described above, this area is an area for recognizing a handwritten stroke as a code and using the stroke as it is. Therefore, in order to make the handwriting stroke an image, in this object, the static variable image_resulf is declared to be a DOT stroke type (first line). This declaration is reflected in the description on the fifth line, and a handwritten comment can be actually input. Note that there is also a description of image editing (8th to 10th lines).

FIG. 29 is a description of the "vertical scroll" object in the region of FIG. 5G240. This "vertical scroll" object designates a prototype "screen operation", and performs actual scroll processing using this function. That is, when a stroke is input to the area, scroll processing is performed upward or downward until the stroke is turned off.

The "screen operation" object is shown in FIG. Here, get in the fourth line scroll_co (x, y) is a function for examining the current scroll position coordinates (x, y), and scroll_exec (x, y, o, delta_y) on the fifth line is a function for actually executing scrolling.

Further, delta_y is obtained by the get_delta_co (delta_x, delta_y) function on the fourth line in FIG.

The "vertical scroll" object is defined so as to accept events in both the input and edit modes.

FIG. 31 is another description example of the “recipient” object in the area of FIG. 5G130. This description example is
This is a description for dynamically changing the attribute of a specific area when a mail document is transferred to a recipient.

Attention is paid to attributes = c_result: DOT on the sixth line. c_result is a static variable, and it is assumed that the attribute change in the subsequent object processing has priority. Therefore, since the attribute has priority over attributes = c_result: ALL_CHA on the fifth line in FIG. 27, when the recipient actually receives the mail, the attribute of the “content” area of G190 in FIG. It will be changed to a stroke type.

For example, a document created by the person in charge is converted to a handwritten stroke when it is input, but when the document is transferred to a superior, it is not converted, and is converted to a stroke as it is. This is an example in which a memo can be freely entered. This is common in real organizations.

The embodiment of the present invention has been described above. The effects will be described below.

First, as a first effect, a handwriting input unit having an integrated structure of a handwriting coordinate input device and a flat screen display (LCD) has a format definition unit for preliminarily defining a description of a handwriting stroke for each area. Since a stroke interpretation execution unit is provided for interpreting and executing the operation description defined in this area when a handwritten stroke is input, the machine can automatically interpret the meaning when the handwritten stroke is input. Was. As a result, the form becomes closer to the form of inputting with a pen on paper, and the input can be performed without interrupting the human thinking form, and the man-machine interface is greatly improved.

As a second effect, since a handwriting recognition function for recognizing a handwriting stroke for each area and converting it into a code can be described, a system function using a network such as a mail, a telephone, a facsimile, and a conference can be handwritten. Can be operated by stroke.
Therefore, compared to the conventional point designation, the office work can be easily simulated, and the role of the personal secretary can be supported by the handwritten input information system.

As a third effect, since the format definition unit and the stroke interpretation execution unit can define the type of handwritten stroke and the recognition range thereof, it is possible to improve the performance of the recognition rate and cope with erroneous input. Therefore, it is particularly suitable for inputting a slip form document.

A fourth effect is that the processing device automatically extracts an area for which an operation is to be defined at the stage when the screen definition of the format has been completed, so that the user does not need to set the area and the efficiency of the format definition can be improved. Become.

As a fifth effect, a mechanism is provided for easily changing the processing operation content of a handwriting stroke in a specific area or in another area only by changing the attribute, so that the flexibility and ease of use of the format definition can be improved. effective.

As a sixth effect, the input system of the handwriting input unit and the functional system of the format definition unit or the stroke interpretation execution unit are:
Since the interface is provided by the stroke buffer, the input system and the function system can be separated, and handwriting strokes can be input one after another without depending on the speed of the function system. For this reason, there is an effect that the irritability due to the response delay of the operator can be reduced.

As a seventh effect, the present processing apparatus has portability since it interfaces wirelessly with a network and has a built-in battery. Therefore, it can be used freely in an office, on a business trip, or at home.

As an eighth effect, since the logic circuit of the processing device is also mounted on the glass of the handwriting input portion, there is an effect that the thickness can be reduced to a notebook or sheet shape.

As a ninth effect, since the logic circuit of the present processing device is mounted on the periphery of the handwriting input unit, it can be used for presentations on an OHP.

Next, another embodiment of the present invention will be described with reference to FIGS.
2 will be described.

FIGS. 37 to 38 are system configuration diagrams in which processing systems of a handwriting input unit, a format definition unit, and a stroke interpretation execution unit are distributed on a network.

FIG. 37 shows two processing systems, a format definition unit P120 and a stroke interpretation execution unit P140, distributed to workstations WS on a network, and a handwriting input unit.
FIG. 2 is a system configuration diagram in which only the processing system of P110 is dispersed in the present processing apparatus.

In this block diagram, the handwriting input information processing apparatus has only a handwriting input unit and a man-machine unit such as a voice.
There is an effect that the processing device can be reduced in size and cost.

FIG. 38 shows a processing system of the handwriting input unit P110-1 and the format definition unit P120 which is a handwriting input information processing device P30.
0 is a system configuration diagram in which the processing system of the handwriting input unit P110-2 and the interpretation execution unit P140 is distributed to another handwriting input information processing device P400. The idea of this configuration is to separate the department that defines the format and the department that operates the format and exchange them with each other using UI / HAND-TALK definition data. The exchange of the definition data may use a memory card without passing through a network.

In this configuration, since the definition data of the format document can be managed at one place, the management of the document is easy. Further, the operating department does not need an editor for defining the format of the handwritten document, and the system scale is reduced. This has the effect of reducing the size and the cost.

FIG. 39 is a system configuration diagram provided with a plurality of stroke input units. That is, the handwriting input unit includes the first stroke input unit P110 and the second stroke input unit P
As shown in FIG. 40, the first stroke input unit is used for handwriting input using a pen as in the embodiment of the present invention, and the second stroke input unit is used for screen operation strokes using the other hand or foot. Make the input.

In general, for a right-handed person, holding the pen in the right hand does nothing with the left hand. Therefore, if a scroll operation or a basic menu operation is performed with the left hand, a suitable man-machine interface is obtained. For example, forty
As shown in the figure, when dragging the second stroke input area to the lower right with the left hand, the screen G100 is scrolled to the lower right.

FIG. 41 is an example of the operation description of the "scroll" object in the second stroke input area, and FIG. 42 is an event map for this input area. That is, in the operation description of FIG. 41, when the event of the screen operation stroke on occurs, the operation description is made to execute the two-dimensional scroll until the screen operation stroke is turned off. Here, the send screen operation on the fifth line, two-dimensional scroll (delta_x, (delta_x
y) is a message described in the “screen operation” object in FIG. 39 to 40,
Although the second stroke input area is arranged so as not to overlap with the first stroke input area, it may be overlapped. In this case, the screen can be scrolled directly up, down, left, and right by hand, so that the user feels like dragging a paper, and is closer to the real world.

In the other embodiments described above with reference to FIGS. 39 to 41, since the interface with the handwritten input information processing device can be made using both hands, there is an effect that the man-machine property can be greatly improved.

Next, in the operation explanatory diagram of FIG. 5 which is an embodiment of the present invention, the input and edit mode changes are designated by a menu. There is a method using stroke length which is characteristic information of a writing stroke. The method of switching based on the stroke length is described below. This operation description is as follows.

[0200] By describing the above content in, for example, the stroke-on event for inputting the “content” object in FIG. 27,
The menu for switching between input and edit can be eliminated. Thereby, modelessness can be further achieved and usability can be improved.

Further, in one embodiment of the present invention, P1 in FIG.
20 and P140, it is assumed that an interpreter is used. However, the interpreter may be configured by a compiler as in a normal programming language. However, when changing the operation description,
You need to compile the program each time.

Further, in one embodiment of the present invention, the physical device for handwriting strokes is extended to two logical devices for input and for editing. However, if the display can display color, the physical device depends on the type of the logical device. You can change the color,
Further, usability is improved. For example, if black is used for input and red is used for editing, the image is close to the actual paper and pen image.

Further, by using a color display, an area to be recognized and an unrecognized area (image) can be distinguished by color, thereby making it easier for the user to input.

Further, in one embodiment of the present invention, the handwriting coordinate input device and the flat display are formed as an integrated structure, but may be separated from each other. However, in this case, there is a disadvantage that the hand and the display surface must be repeated.

Further, the battery of FIG. 9, which is an embodiment of the present invention, may be changed from a battery to a solar battery.

Further, the glass for inputting coordinates and the glass for display shown in FIG. 7 which is an embodiment of the present invention may be made of a plastic material. In this case, the image becomes closer to the original image of paper and becomes a sheet computer or a paper computer, and the office environment can be made more paperless.

[0207]

According to the present invention, in a handwriting input information processing apparatus having a handwriting input section in which a handwriting coordinate input apparatus and a flat display are integrated, a region for the handwriting input section is set in advance for each area. Because we defined the behavior of what process to do and interpreted and executed when there was a handwritten input,
It is possible to input with the sense of a paper and a pen and without interrupting human thinking, and there is an effect that a man-machine interface is greatly improved.

Further, since the logic circuit of the handwriting input information processing device is mounted on the glass of the handwriting input section to make it thinner in the form of a notebook or a sheet, and to achieve cordless operation by wireless and battery, somewhere. Has the effect of being portable.

[0209] Also, since a mechanism for describing a recognition function for converting a handwriting stroke into a code and a processing function utilizing a network is provided, document creation, mail, telephone, FA, and the like are provided.
There is an effect that an electronic secretary system that supports most office operations such as X and meetings can be constructed.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of an embodiment of the present invention.

FIG. 2 is an operation explanatory view of a conventional example.

FIG. 3 is an operation explanatory view of a conventional example.

FIG. 4 is an operation explanatory view of a conventional example.

FIG. 5 is an operation explanatory diagram of FIG. 1;

FIG. 6 is an overall configuration diagram of a system using one embodiment of the present invention.

FIG. 7 is a hardware configuration diagram of FIG. 1;

FIG. 8 is a hardware configuration diagram of FIG. 1;

FIG. 9 is a hardware configuration diagram of FIG. 1;

10 is a detailed system configuration diagram of FIG. 1 and its operation explanatory diagram.

11 is a detailed system configuration diagram of FIG. 1 and an operation explanatory diagram thereof.

FIG. 12 is a detailed system configuration diagram of FIG. 1 and an operation explanatory diagram thereof.

FIG. 13 is a detailed system configuration diagram of FIG. 1 and an operation explanatory diagram thereof.

14 is a flowchart of FIG. 1, a program and an explanatory diagram of its operation.

15 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

FIG. 16 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

17 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

18 is a flowchart of FIG. 1, a program and an explanatory diagram of its operation.

19 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

20 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

21 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

FIG. 22 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

FIG. 23 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

24 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

25 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

26 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

27 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

28 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

29 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

30 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

31 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

FIG. 32 is a flowchart, a program, and an operation explanatory diagram of FIG. 1;

FIG. 33 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

FIG. 34 is a flowchart, a program, and an operation explanatory diagram of FIG. 1;

FIG. 35 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

36 is a flowchart of FIG. 1, a program, and an explanatory diagram of its operation.

FIG. 37 is a diagram showing a system configuration for explaining another embodiment of the present invention and an explanatory diagram of its operation.

FIG. 38 is a system configuration diagram for explaining another embodiment of the present invention and its operation explanatory diagram.

FIG. 39 is a system configuration diagram for explaining another embodiment of the present invention and its operation explanatory diagram.

FIG. 40 is a system configuration diagram for explaining another embodiment of the present invention and its operation explanatory diagram.

FIG. 41 is a system configuration diagram for explaining another embodiment of the present invention and its operation explanatory diagram.

FIG. 42 is a system configuration diagram for explaining another embodiment of the present invention and its operation explanatory diagram.

[Explanation of symbols]

H211: Handwriting input hardware, P110: Stroke input unit, T100: Stroke buffer, P120 ...
Format definition part, P140: stroke interpretation execution part, T20
0 ... UI / HAND-TALK definition data, P130 ...
Wireless input / output unit, P150: voice input / output unit, 6: speaker, 7: microphone, 9: antenna.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Hiroshi Masashima 4026 Kuji-cho, Hitachi City, Ibaraki Prefecture Inside Hitachi Research Laboratory, Hitachi, Ltd. (72) Inventor Toshifumi Arai 4026, Kuji-cho, Hitachi City, Ibaraki Prefecture Hitachi Inside Hitachi, Ltd. (72) Inventor Masaki Miura 4026, Kuji-cho, Hitachi, Ibaraki Prefecture Inside Hitachi Research Lab., Hitachi, Ltd. (72) Inventor Tosumi Yokota 4026, Kuji-cho, Hitachi, Hitachi, Ltd.

Claims (3)

[Claims] 1. A receiving means for receiving external information from outside, a microphone for inputting voice, a keyboard for inputting information such as characters and numerals, and displaying the input information or the received external information. An information processing apparatus, comprising: a display unit that performs transmission; and a transmission unit that transmits the displayed information. 2. A handwriting input unit for inputting data by handwriting, a screen definition unit for defining input information for describing a program defining processing of a handwriting stroke input by the handwriting input unit, A format definition unit having an operation definition unit that displays a screen for describing the program on a display unit for each input area defined by the screen definition unit, and a handwriting stroke input by the handwriting input unit. An information processing apparatus comprising: a stroke interpretation execution unit that processes according to a program defined in the format definition unit; and a transmission unit that transmits the data or the data processed by the stroke interpretation execution unit. 3. An information processing system for transmitting data between a plurality of information processing apparatuses via a relay station, wherein the information processing apparatus includes a handwriting input by a handwriting input unit for inputting data by hand. A screen definition unit for defining input information for describing a program defining stroke processing, and an operation definition for displaying, on a display unit, a screen for describing the program for each input area defined by the screen definition unit A format definition unit having a unit, a stroke interpretation execution unit that processes a handwritten stroke input by the handwriting input unit according to a program defined in the format definition unit, and processing by the data or the stroke interpretation execution unit. A transmission unit for transmitting the obtained data to the relay station.