JP2005173807A - Voice input/output system and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To hear again a voice when a note has been taken by taking the note and then tracing the note. <P>SOLUTION: The voice in the circumference of a handwriting input pen 26 is received as voice data which are sent as pen information data to a pen information server 16 while the data are made to correspond to the position of a note on a memo pad. When a user points the position of the note on the memo pad in the vicinity of a PC 18 and performs an operation for outputting a voice, the PC 18 requests the pen information data including the voice data made to correspond to the position of the note from the pen information server 16. An input device 20 sends the pen information data complying with the request to the PC 18. The PC 18 outputs the voice data included in the pen information data as a voice. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a voice input / output system and method for storing and outputting input voice.

For example, Patent Document 1 discloses an apparatus that uses a writing instrument having an image input function and a voice input function.
JP 2001-105786 A

However, Patent Document 1 does not particularly disclose how to use the voice obtained with the writing instrument.
The present invention has been made from such a background, and an audio input / output system in which an audio signal obtained by a writing instrument having an image input function and an audio input function can be used conveniently. And an object thereof.

  In order to achieve the above object, a first audio input / output system according to the present invention includes audio input means for receiving an input of an audio signal, pointing means for pointing an arbitrary position on a predetermined member, and pointing on the member. Detecting means for detecting the position, information storage means for storing the position pointed at the member and the received audio signal in association with each other, and pointing to the position on the stored member is detected. In some cases, audio output means for outputting an audio signal stored in association with the position of the member is provided.

  Preferably, the information storage means stores an audio signal received before the pointing by the pointing means is performed in association with the position pointed at the member.

  Preferably, the pointing means includes voice receiving means for receiving voice and outputting it as a voice signal, and the voice input means receives the output voice signal.

  Preferably, the member is provided with a mark for identifying a position on the member, and the pointing means has an image input means for receiving an image of the mark attached to the member and outputting it as an image signal. Then, the detection means detects a position on the pointed member based on the output image signal.

  Preferably, the mark given to each of the members is unique to each of the members, and the detection means detects the position of the pointed member and the unique mark, and stores the information storage The means stores the detected unique mark in association with the position and voice signal in the associated member, and the voice output means stores the unique mark and the position in the stored member. When pointing is detected, an audio signal stored in association with the detected unique mark and the detected position on the member is output.

  Preferably, the pointing means further includes writing means for writing on the pointed portion.

A second voice input / output system according to the present invention includes a voice input device having voice input means for receiving an input of a voice signal and first detection means for detecting a position pointed at a predetermined member; Information storage means for associating and storing the detected position on the member and the received audio signal, and the pointing on the position on the stored member is associated with the position on the member when detected. An audio server device having first audio output means for outputting the stored audio signal, second detection means for detecting a position pointed at the member, and depending on the detected position on the member And an audio output device having second audio output means for outputting the audio signal output as audio.

  Further, the voice input device according to the present invention stores the detected position in the predetermined member and the received voice signal in association with each other, and when pointing to the position in the stored member is detected, A voice server device that outputs a voice signal stored in association with a position in the member, a pointed position in the member is detected, and a voice signal that is output according to the detected position in the member, A voice input device in a voice input / output system having a voice output device for outputting as voice, comprising voice input means for receiving an input of a voice signal, and detection means for detecting a pointed position on the member.

  The voice server device according to the present invention receives a voice signal input, detects a position pointed at a predetermined member, detects a position pointed at the member, and detects the detected member. A voice server device in a voice input / output system having voice output means for outputting a voice signal output in accordance with the position in the voice as a voice, the position in the detected member, and the received voice signal; Information storage means for storing the information in association with each other and sound output means for outputting a sound signal stored in association with the position in the member when the pointing to the position in the stored member is detected. .

  An audio output device according to the present invention receives an input of an audio signal and detects an audio input device that detects a position pointed at a predetermined member; a position at the detected member; and the received audio signal. And a voice server device that outputs a voice signal stored in association with the position on the member when pointing to the position on the stored member is detected. And a sound output means for outputting a sound signal output in accordance with the detected position on the member as a sound.

  In the first voice input / output method according to the present invention, an input of a voice signal is received, an arbitrary position on a predetermined member is pointed, a position pointed on the member is detected, and the detected member is detected. The position and the received audio signal are stored in association with each other, and when the pointing to the position in the stored member is detected, the stored audio signal is output in association with the position of the member. .

  Further, the second audio input / output method according to the present invention receives an input of an audio signal, detects a pointed position on a predetermined member, and detects the position on the detected member, and the received audio signal. Are stored in association with each other, and when pointing to the position of the stored member is detected, an audio signal stored in association with the position of the member is output, and the pointing position of the member is detected. And the audio | voice signal output according to the position in the said detected member is output as an audio | voice.

  In addition, a first program according to the present invention includes a voice input step for receiving an input of a voice signal, a pointing step for pointing an arbitrary position on a predetermined member, and a detection step for detecting a position pointed on the member. A position / speech storage step for storing the detected position in the member and the received sound signal in association with each other, and a position in the member when the pointing to the position in the stored member is detected. And a sound output step of outputting a sound signal stored in association with the computer.

  According to a second program of the present invention, in a voice input device including a computer, a voice input step of receiving a voice signal and a first detection step of detecting a pointed position on a predetermined member are described above. In a voice server device including a computer, a position / speech storage step for storing the detected position in the member and the received voice signal in association with each other, and a position in the stored member In the audio output device including the computer, the first audio output step of outputting the audio signal stored in association with the position in the member when pointing is detected. A second detection zone for detecting the pointed position And-up, the audio signal output in accordance with the position of the detected member, to execute a second sound output step of outputting as a voice on the computer.

  The third program according to the present invention stores the detected position on the predetermined member and the received audio signal in association with each other, and when pointing to the position on the stored member is detected. A voice server device that outputs a voice signal stored in association with a position on the member; a position pointed at the member; a voice signal output according to the detected position on the member; In a voice input device including a computer of a voice input / output system having a voice output device for outputting as voice, a voice input step for receiving an input of a voice signal and a detection step for detecting a position pointed at the member Let the computer run.

  Further, a fourth program according to the present invention receives an input of a voice signal, detects a pointed position on a predetermined member, detects a pointed position on the member, and detects the detected point. In a voice server apparatus including a computer of a voice input / output system having voice output means for outputting a voice signal output in accordance with a position on a member as a voice, the position on the detected member and the received voice A position / speech storage step for storing the signal in association with each other, and a sound output for outputting a sound signal stored in association with the position in the member when the pointing to the position in the stored member is detected And causing the computer to execute steps.

  Further, a fifth program according to the present invention includes an audio input device that receives an input of an audio signal and detects a position pointed at a predetermined member, the position on the detected member, and the received audio signal. And a voice server device that outputs a voice signal stored in association with the position on the member when pointing to the position on the stored member is detected. In a sound output device including a computer of the system, a detection step for detecting a position pointed at the member, and a sound output step for outputting a sound signal output according to the detected position on the member as sound. Let the computer run.

  According to the present invention, for example, by taking a note and tracing the note later, it is possible to listen back to the voice when the note was taken.

[Information management system 1]
FIG. 1 is a diagram showing an outline of an information management system 1 according to the present invention.
First, an information management system 1 to which the present invention is applied will be described.
As shown in FIG. 1, the information management system 1 is certified by the handwriting input systems 2-1 and 2-2, the background management system 3, the certified printer 122 certified by the background management system 3, and the background management system 3. A non-certified printer 124, an information server 16, a personal computer (PC) 18, and an input device 20 (the PC 18 and the input device 20 may be shared) are connected via a network 120. take.

The handwriting input systems 2-1 and 2-2 include an authorized printer 122, an input device 20, an electronic authentication stamp 24, a first handwriting input pen 26 (pointing means), and the like.
The background management system 3 includes a background management device 32 and a charging device 30.

Hereinafter, a plurality of components such as the handwriting input systems 2-1 and 2-2 may be simply abbreviated as the handwriting input system 2.
In addition, each component of the information management system 1 may be collectively referred to as a node.
Further, the number of each component part of the information management system 1 shown in FIG. 1 is an exemplification, and the information management system 1 can include any number of each component part.

Further, the function sharing of the information management system 1 and its nodes shown in FIG. 1 is an example, and a plurality of functions distributed via the network 120 in the information management system 1 are collected in one node. May be.
Conversely, a plurality of functions collected in one node may be distributed to a plurality of nodes, or may be distributed to a plurality of nodes via the network 120.
As described above, whether the plurality of components are grouped in one device or distributed to the plurality of devices via a network or the like does not affect the technical scope of the present invention. Absent.

FIG. 2 is a diagram illustrating a hardware configuration of the background management device 32, the billing device 30, the information server 16, the PC 18, and the input device 20 illustrated in FIG.
As shown in FIG. 2, these components of the information management system 1 include a main body 100 including a CPU 102, a memory 104, and peripheral circuits thereof, an input / output device 106 including a keyboard, a display device, a speaker (SP), and the like. Consists of a communication device 110 that communicates with other nodes via the network 120, a recording device 112 such as a CD device / HDD device, and a function realization device 38 that is used to realize functions unique to each device such as a scanner. Is done.
That is, the background management device 32, the billing device 30, the information server 16, the information management system 18, and the input device 20 include components as a computer having a communication function.

Further, for the input device 20 and the PC 18, wireless communication (communication using radio waves or light) with the electronic authentication stamp 24 and the handwriting input pen 26 is added to the configuration shown in FIG. 2 as necessary. Wireless interface (IF) 116 used for communication).
Note that the wireless interface (IF) 116 can be replaced with a wired interface (IF) having the same function depending on the configurations and applications of the input device 20 and the PC 18.
In addition, a printer engine (not shown) for realizing a function as a printer is added to the configuration shown in FIG. 2 for the certified printer 122 and the non-certified printer 124.

FIG. 3 is a diagram illustrating a background attached to the sheets 128 and 130 shown in FIG.
The paper 128 is plain paper, and the paper 130 is a synthetic paper that can be printed and erased (reused) a plurality of times, and a background as shown in FIG. 3 is given by printing.
That is, on the plain paper 128, 130, for example, as assumed by a dotted line in FIG. 3, on a lattice point 132 of a lattice composed of a large number of straight lines perpendicular to each other at intervals of about 0.3 mm, or A background pattern 136 (mark) with dots 134 is printed on the left, right, top, bottom, top right, bottom right, top left, and bottom left of these grid points 132 at positions shifted by, for example, about 20 μm. Paper 128 and 130 are obtained.
As described above, there are cases where the paper 128, 130 (members) is provided with the background pattern 136 and cases where the background pattern 136 is not provided.

The background pattern 136 becomes an enormous number of types by combining the positional relationship between the lattice points 132 and the numerous dots 134.
Therefore, by detecting which part of the background pattern 136 is hit by the handwriting input pen 26, the handwriting input pen 26 is pointing at which position on the paper 128, 130, or You can also detect if you are writing.
In addition, since a unique background pattern 136 can be attached to each of the sheets 128 and 130, if one background pattern 136 is printed only on one plain sheet 128 and 130. The background pattern 136 can also be used to specify the paper 128, 130 with the background pattern 136 attached thereto.

Note that the background pattern 136 (FIG. 3) is exemplified here as a mark attached to the paper 128, 130 and used for position identification of the paper 128, 130. In addition to the optically readable material, an electrically, magnetically or mechanically readable material can be used.
In addition, here, the case where the background pattern 136 is attached to the two-dimensional paper 128, 130 is illustrated, but the background pattern 136 is attached to the three-dimensional member, and the position on the member is detected. You may make it.
Further, instead of the background pattern 136 as shown in FIG. 3, a specific figure, pattern or dot array may be attached to each of the sheets 128 and 130 and used as this mark.

Alternatively, a three-dimensional member may be marked with a mark that can identify the three-dimensional coordinates (address) of the member so that the three-dimensional coordinates of the member can be detected.
A three-dimensional background pattern may be printed on the surface or inside of the two-dimensional plain paper 128, 130 using a special toner (three-dimensional toner).
In addition, a light-transmitting resin or glass may be coated or pasted on the surface of the paper 128 or 130 with a background.

The background pattern 136 attached to the sheets 128 and 130 is managed by the background management apparatus 32 of the background management system 3 and printed on the sheets 128 and 130 by the authorized printer 122.
For the management and use of the background pattern 136, the charging device 30 can charge the user.

The electronic authentication mark 24 is used together with the sheets 128 and 130, and communicates with the input device 20 or the PC 18 to perform electronic authentication.
The handwriting input pen 26 is used together with the sheets 128 and 130, communicates with the input device 20 or the PC 18, and is used for inputting handwritten characters, inputting voice, and other operations.

With these components, for example, the following can be realized in the information management system 1.
(1) Printing and accounting for background paper 136 (FIG. 3) on plain paper or synthetic paper:
(2) Reuse of the background pattern 136 printed on the synthetic paper (When the background pattern 136 printed on the synthetic paper is erased, the background pattern 136 can be printed on other plain paper or synthetic paper. To manage):

(3) Input / output of voice: When a memo and a voice written using the handwriting input pen 26 are stored in association with each other on the paper 128, 130, and the memo is traced with the handwriting input pen 26, Play the associated audio.
(4) User authentication with the handwriting input pen 26:
(5) User authentication using paper 128, 130 and electronic authentication stamp 24:

  (6) Various services using the paper 128, 130, the electronic authentication stamp 24 and the handwriting input pen 26: an education service to be described later, or a user writing or correcting a comment created by a word processor by handwriting After that, when you go to the personal computer, the part of the document with the handwritten comment / correction displayed on the screen will be displayed. The convenience of re-entering the handwritten comment / correction with the word processor Services that are designed

[Handwriting pen 26]
FIG. 4 is a diagram showing a configuration of the first handwriting input pen 26 shown in FIG.
As shown in FIG. 4, the handwriting input pen 26 includes a writing member 260 that can be used for writing and pointing such as a mechanical pencil mechanism and a ballpoint pen tip, a lens system 264, a CCD (Charge Coupled Device) element 266 (image input means). , Pressure sensor 268, image capturing circuit 270, microphone (MIC) 272 (speech receiving means; may be externally attached to handwriting input pen 26 and placed on the desk and used for voice input), voice processing circuit 274, power supply 278, transmission circuit 280, buffer memory 282, transmission memory 284, control unit 286, and switch 288 are integrally housed in a handwriting input pen body 262 formed in the form of a general writing instrument. Is done.

The handwriting input pen 26 has, for example, a hardware configuration as a one-chip microprocessor application device, and each component of the handwriting input pen 26 is appropriately configured with dedicated hardware or software operating on a one-chip microprocessor. As realized.
Due to these components, the handwriting input pen 26 uses characters or pointing operations written on paper 128, 130 (FIG. 1) with a background as handwriting data to the input device 20. Output.
Moreover, the handwriting input pen 26 receives surrounding sounds and outputs them to the input device 20 as sound data.

The handwriting input pen 26 detects the user's writing pressure and outputs it as writing pressure data to the input device 20.
Note that the handwriting input pen 26 is operated in accordance with a predetermined operation by the user, for example, an operation on a button (not shown) of the handwriting input pen 26, or pointing at a predetermined timing with respect to the paper 128, 130. Perform these functions.
That is, the handwriting input pen 26 is used to input handwritten characters, pointing operations, and voices to the input device 20.

In the handwriting input pen 26, the pressure sensor 268 detects the pressure applied to the writing member 260 when the user presses the writing member 260 against the paper 128, 130, etc., and performs writing and pointing. Output to the input circuit 270.
If the pressure sensor 268 can further detect the user's gripping force applied to the handwriting input pen body 262, user authentication based on the user's gripping force is possible.
In the handwriting input pen 26, the lens system 264 and the handwriting input pen 266 are, for example, the paper 128, the portion of the paper 128, to which the writing member 260 is pressed when the pressure sensor 268 detects the pressure applied to the writing member 260. Images such as 130 background patterns 136 (FIG. 3) are received and output to the image capture circuit 270.

The image capturing circuit 270 captures an image received by the lens system 264 and the CCD element 266, generates handwriting data indicating the handwriting of the user and the background pattern 136 (FIG. 3) attached to the paper 128, 130, and the buffer. The data is output to the transmission circuit 280 via the memory 282.
Further, the image capturing circuit 270 captures the pressure detected by the pressure sensor 268, generates pen pressure data indicating the user's pen pressure or pointing strength, and sends it to the transmission circuit 280 via the buffer memory 282. Output.

The MIC 272 and the sound processing circuit 274 always accept the surrounding sound when the pressure sensor 268 detects the pressure applied to the writing member 260, generate sound data, and output the sound data to the transmission circuit 280.
The control unit 286 controls the operation of each component of the handwriting input pen 26 in accordance with a user operation on the switch 288.
As an example of operation control by the control unit 286, audio data is stored in the buffer memory 282 or the transmission memory 284, the transmission circuit 280 is controlled in accordance with a user operation on the switch 288, and the stored audio data is transmitted. Control.

The transmission circuit 280 receives handwriting data, writing pressure data, and voice data from the image capturing circuit 270 and the handwriting input system 274 via the buffer memory 282 and stores them in the transmission memory 284 under the control of the control unit 286. .
Further, the transmission circuit 280 transmits the handwriting data, the writing pressure data, and the voice data, which are stored in the buffer memory 282 by radio waves or light, to the input device 20 (FIG. 1) to the wireless IF 116 (FIG. 2).
The control unit 286 stores an encrypted identification code / identification number unique to each handwriting input pen 26, and the transmission circuit 280 transmits the identification code / identification number together with the above-described data. You may do it.

[Other aspects of handwriting input pen]
Here, various aspects of the handwriting input pen that can be used in place of the handwriting input pen 26 in the information management system 1 will be described.
FIGS. 5 and 6 respectively show the external appearances of the second and third handwriting input pens 40 and 42 used in place of the first handwriting input pen 26 shown in FIG. 4 and the like in the information management system 1 (FIG. 1). FIG.
5 and 6, substantially common components are denoted by common reference numerals (the same applies to the following drawings).

As shown in FIG. 5, the second handwriting input pen 40 includes a contact member 400, a magnifying glass 402, a CCD camera 412, a function switching unit 416, a function display LED 418, a sheet-like display 420 including an organic EL display, a cap 422, and the like. Consists of
The thickness of the thickest part of the main body of the handwriting input pen 40 is, for example, a diameter of 4 cm or less (a diameter smaller than a ring made of an adult thumb and a forefinger) in consideration of ease of gripping. .
Furthermore, in addition to the function of the first handwriting input pen 26 shown in FIG. 4, the second handwriting input pen 40 has functions as described below with reference to FIG. 5 according to the control of the control unit 288, for example. (The same applies to the third handwriting input pen 42 (FIG. 6)).
The contact member 400 corresponds to the writing member 260 of the first handwriting input pen 26, and is composed of a ballpoint pen, a pointing stick, a needle, a pencil, a marker pen, felt, or a resin having a round tip, and is a member such as paper 128, 130. To touch.
Note that the contact member 400 can be used not only for character input and pointing, but also for selecting a function according to writing pressure.

The magnifying glass 402 includes, for example, a magnifying lens 406 fitted into the lens frame 404 and provided with a sighting scale 408, and an illumination light 410 that illuminates the user's hand.
The lens frame 404 is detachable from the main body, and is fixed to the main body by a joint 412 that flexibly fixes the lens frame 404 to the main body.
The CCD camera 414 corresponds to the lens system 264 and the CCD element 266 in the first handwriting input pen 26 (FIG. 4), and is for inputting handwritten characters and for taking images similar to a general digital camera. Can also be used.

The function switching unit 416 corresponds to, for example, the switch 288 (FIG. 4) of the first handwriting input pen 26 and is used for selecting the function (function) of the handwriting input pen 40 as indicated by an arrow in FIG. It is done.
Examples of functions of the handwriting input pen 40 include functions such as voice input / sound reproduction, voice buffering time setting, and function lock.
This function selection is performed by the user rotating the joint of the main body and matching the scale indicating the function.
The function display LED 418 is composed of light emitting diodes having different colors, and displays the selected function.

In the main body of the handwriting input pen 40, a power switch (not shown) is provided in a portion where the cap 422 is attached, and when the cap 422 covering the contact member 400 is removed and attached to a predetermined portion, The power of the handwriting input pen 40 is turned on.
Further, in the main body of the handwriting input pen 40, a display 420 formed in a cylindrical shape is provided in a portion where the cap 422 is mounted, and is used for various displays.

As shown in FIG. 6, the third handwriting input pen 42 is provided with a contact member 424 instead of the contact member 400, and instead of the display 420, a knock function switching unit 426 and a directional microphone 430 are provided. And the directivity adjustment part 434 is provided.
In the handwriting input pen 42, the knock function switching unit 426 used for turning on / off the voice buffering function may be used in combination with the function switching unit 416 and the function display LED 418. Each time the switching unit 426 is pressed, the function of the handwriting input pen 42 is changed, and the changed function is displayed on the function display unit 428 with a color or the like.
When the user turns the directivity adjusting unit 434 provided at the joint of the main body, the directivity of the directional microphone 430 is adjusted, and the directivity of the directional microphone 430 is displayed by a mark 432 indicating directivity. The

[Electronic authentication stamp 24]
FIG. 7 is a diagram showing the configuration of the electronic authentication stamp 24 shown in FIG.
As shown in FIG. 7, the electronic authentication mark 24 includes a lens system 264, a CCD element 266, a pressure sensor 268, an image capture circuit 270, an MIC 272, an audio processing circuit 274, a power supply 278, a buffer memory 282, a transmission memory 284, and a control. The unit 286, the switch 288, the seal surface 240, the seal data storage unit 244, and the transmission circuit 246 are housed in an electronic authentication seal body 242 formed in a general seal form.
Similar to the handwriting input pen 26, for example, the electronic authentication stamp 24 has a hardware configuration as a one-chip microprocessor application device. Each component of the electronic authentication stamp 24 is appropriately configured with dedicated hardware or one chip. Realized as software running on a microprocessor.

The electronic authentication stamp 24 performs user authentication with the information server 16 or the like by using these components.
The electronic authentication stamp 24 outputs electronic seal stamp information data to the input device 20 and the information server 16.

In the electronic authentication stamp 24, the stamp surface 240 is brought into contact with the paper 128, 130, etc. to give a seal impression and transmit the received pressure to the pressure sensor 268.
As with the handwriting input pen 26, if the pressure sensor 268 can detect the user's gripping force applied to the electronic authentication seal main body 242, user authentication using the electronic authentication seal 24 can be performed.
The seal data storage unit 244 holds seal data (which may include an identification code / identification number similar to the handwritten input pen 26) to be output from the electronic authentication seal 24 to the input device 20 or the like, and transmits the seal data. Output to the circuit 246.

The transmission circuit 246 receives the image of the background pattern 136 (FIG. 3) on the paper 128, 130 input from the image capture circuit 270, and image / pressure data (handwriting input) indicating the pressure detected by the pressure sensor 268. Seal information data including seal data input from the seal data storage unit 244 and voice data input from the voice processing circuit 274 is generated.
Similar to the transmission circuit 280 in the handwriting input pen 26, the transmission circuit 246 outputs the generated seal information data to the input device 20 or the PC 18 under the control of the control unit 288.

[Input Device Program 200]
FIG. 8 is a diagram showing an input device program 200 operating on the input device 20 shown in FIG.
As illustrated in FIG. 8, the input device program 200 includes a reception processing unit 202, a handwriting analysis unit 204, a character identification unit 206, a voice processing unit 208 (speech input unit), an association unit 210, a background detection unit 212, a position. It comprises a detection unit 214 (detection means), a database (DB) management unit 216, a pen information DB 220, a communication processing unit 222, and a seal information processing unit 224.

The input device program 200 is supplied to the input device 20 via, for example, the recording medium 114 (FIG. 2), loaded into the memory 104, and executed (the same applies to each program shown below).
The input device program 200 processes handwriting data, writing pressure data, and voice data input from the handwriting input pen 26 using these components, and associates them with each other to the information server 16 (FIG. 1). To send.

In the input device program 200, the reception processing unit 202 outputs handwriting data and writing pressure data received by the wireless IF 116 (FIG. 2) of the input device 20 to the handwriting analysis unit 204.
The reception processing unit 202 also outputs the audio data received by the wireless IF 116 to the audio processing unit 208.
In addition, the reception processing unit 202 outputs handwriting data to the background detection unit 212.
Further, the reception processing unit 202 sends handwriting data, voice data, writing pressure data, image / pressure data, and seal data obtained by the electronic authentication seal 24 and the handwriting input pen 26 to the communication processing unit 222 as necessary. Output.

For example, the handwriting analysis unit 204 obtains information indicating the background pattern 136 from the background management device 32, analyzes the pen pressure data and the pen pressure data input from the reception processing unit 202, and allows the user to use the sheets 128 and 130. Is detected, and the moving direction and moving speed of the dot 134 of the background pattern 136 (FIG. 3) are calculated.
The handwriting analysis unit 204 outputs the analysis result thus obtained to the character identification unit 206.

Based on the analysis result by the handwriting analysis unit 204, the character identification unit 206 identifies the character written on the paper 128, 130, etc. by the user, and outputs it to the association unit 210 as character data.
In addition, the character identification unit 206 identifies an operation such as pointing performed by the user on the paper 128, 130 based on the analysis result by the handwriting analysis unit 204, and the operation data is transmitted to the association unit 210 as operation data. Output.
Note that the character recognizing unit 206 may output the handwriting data and the writing pressure data subjected to handwriting analysis to the associating unit 210 together with the character data and the operation data.
Moreover, the character recognition part 206 transmits character data with respect to the communication process part 206, when the input of a handwritten character is permitted as a result of user authentication.

The audio processing unit 208 performs processing such as compression / encoding and buffering on the audio data input from the reception processing unit 202 and outputs the processed data to the associating unit 210.
The background detection unit 212 detects the background pattern 136 (FIG. 3) attached to the sheets 128 and 130 from the handwriting data (image) input from the reception processing unit 202 and outputs the background pattern 136 to the position detection unit 214.

The position detection unit 214 receives information indicating the background pattern 136 attached to the sheets 128 and 130 (hereinafter also referred to as background pattern data) from the background management device 32, and the sheet on which the handwriting input pen 26 is writing or pointing. The positions on 128 and 130 are detected, and are output to the associating unit 210 as position data indicating the positions on the sheets 128 and 130.
The associating unit 210 includes character data and operation data input from the character identifying unit 206 (to which handwriting data and writing pressure data may be added as described above) and a voice processing unit 208. The input voice data and the position data input from the position detection unit 214 are associated with each other, and are output to the DB management unit 216 as pen information data.

The seal stamp information processing unit 224 receives seal stamp information data from the electronic authentication stamp 24 and outputs the stamp information data to the information server 16 or the like via the communication processing unit 222 at the timing when the pressure applied to the stamp surface 240 is greatly changed.
Further, the seal stamp information processing unit 224 outputs the seal stamp information data received from the electronic authentication stamp 24 to the DB management unit 216.
In addition, when the user authentication using the electronic authentication seal 24 is performed, the seal stamp information processing unit 224 receives the seal information data received from the electronic authentication seal 24 only when the user of the electronic authentication seal 24 is authenticated. Output to the server 16 or the DB management unit 216.

The communication processing unit 222 performs processing necessary for communication with other nodes via the network 120 (FIG. 1).
The DB management unit 216 stores the pen information data input from the association unit 210 and the seal information data input from the seal information processing unit 224 in the pen information DB 220 for management.

Further, the DB management unit 216 transmits pen information data and seal stamp information data stored in the pen information DB 220 to the information server 16 or the like via the communication processing unit 222 as necessary.
Further, the DB management unit 216 outputs the stored pen information data and seal stamp information data from the input / output device 106 or the like as character data or voice data in accordance with a user operation.

When the information server 16 authenticates the user using the handwriting input pen 26, the DB management unit 216 uses the communication processing unit 222 only for the pen information data input by the user authenticated by the information server 16. To the other node or stored in the pen information DB 220 for management.
In addition, when the information server 16 authenticates the electronic authentication seal 24, the DB management unit 216 receives only the seal information input from the electronic authentication seal 24 authenticated by the information server 16 via the communication processing unit 222. Or stored in the pen information DB 220 for management.

[Information Server Program 160]
FIG. 9 is a diagram showing an information server program 160 operating on the information server 16 shown in FIG.
As shown in FIG. 9, the information server program 160 includes a communication processing unit 222, an operation analysis unit 164, a DB management unit 166, a pen information DB 168, a service providing unit 170 (voice output means, first voice output means), and an addition. The service DB 172 (information storage means) is configured.

With these components, the information server program 160 stores and manages pen information data and seal stamp information data input from each input device 20, and provides the stored pen information data to the PC 18 or the like upon request. To do.
The information server program 160 provides additional services to the PC 18 and the like.

Similarly to the input device program 200 (FIG. 8), the communication processing unit 222 performs processing necessary for communication with other nodes via the network 120 (FIG. 1).
The DB management unit 166 receives pen information data and seal stamp information data from the input device 20 and stores them in the pen information DB 168 for management.
Also, the DB management unit 166 provides pen information data and seal stamp information data stored in the pen information DB 168 in response to requests from other nodes such as the PC 18 and the service providing unit 170.

The operation analysis unit 164 analyzes operation data, pen information data, and seal stamp information data from the input device 20 or the like, and outputs them to the service providing unit 170.
The service providing unit 170 uses the pen information data stored in the pen information DB 168, the seal information data, and the background pattern data input from the background management device 32 as appropriate according to the analysis result by the operation analysis unit 164. Additional services (such as education services) stored in the service DB 172 are provided to the PC 18 and the input device 20.
The additional services provided by the service providing unit 170 will be further described later with reference to the first and subsequent embodiments.

[Information utilization program 180]
FIG. 10 is a diagram showing an information use program 180 operating on the PC 18 shown in FIG.
As shown in FIG. 10, the information utilization program 180 includes a reception processing unit 202, a background / operation detection unit 184, a position detection unit 214, a communication processing unit 222, an information output processing unit 182 (second audio output means), a background An operation detection unit 184 and an application 186 are included.
When the information utilization program 180 further handles the seal information data, the seal information processing unit 224 shown in FIG. 8 is added.
With these components, the information output processing unit 182 provides various services according to an operation in which the user points the paper 128 or 130 using the handwriting input pen 26.
In the following description, in order to make the description more specific and clear, the information use program 180 outputs a voice (voice input / output service; FIG. 12 and FIG. 13) according to the user's pointing operation. A case will be described as a specific example where reference is made to the following.
The reception processing unit 202 receives handwriting data and writing pressure data from the handwriting input pen 26 via the wireless IF 116 (FIG. 2) and outputs them to the background / operation detection unit 184.

The background / operation detection unit 184 detects the background pattern 136 from the handwriting data (image of the background pattern 136 (FIG. 3)) input from the reception processing unit 202.
Further, the background / operation detection unit 184 detects a user operation from the pen pressure data input from the reception processing unit 202 and sets it as operation data.
The background / operation detection unit 184 outputs background pattern data and operation data to the position detection unit 214.

[Background pattern 136 addition processing by application 186]
The application 186 includes a function such as a word processor, for example, and creates image data to be printed on the paper 128 and 130 and additional data associated with each position on the paper 128 and 130.
The creation of image data by the application 186 can further include, for example, data copying, clipping, and insertion of documents created by other applications (not shown) and images taken by a digital camera. .
Further, the application 186 requests the background management device 32 (FIG. 1) to add the background pattern 136 (FIG. 3) to the plain paper 128, 130 in response to a user operation. In response, the background pattern 136 is received from the background management device 32.

The application 186 transmits the background pattern 136 received from the background management device 32 to the certified printer 122 together with the created image data, and prints these on the plain paper 128 and 130.
Further, the application 186 sends information indicating the background pattern 136 attached to the sheets 128 and 138 to the information server 16, one or more positions on the sheets 128 and 13, and additional data associated with each of these positions. And register.

[Reuse of Background Pattern 136 by Application 186]
Further, the application 186 implements a function for reusing the background pattern 136.
In other words, for example, the application 186 reads the background pattern 136 from the paper 128 to be discarded or the paper 130 from which the background pattern 136 is erased by using a scanner (function realization device 38; FIG. 2) or the like. It transmits to the background management device 32 (FIG. 1 etc.) of the management device 3.
Further, the application 186 notifies the background management apparatus 32 that the background pattern 136 can be reused together with the transmission of the background pattern 136.

  The position detector 214 uses the background pattern data received from the background management device 32 to detect the position of the paper 128 or 130 that the user is pointing to, and outputs it to the communication processor 222 together with the operation data.

The communication processing unit 222 notifies the information server 16 of the position data input from the position detection unit 214 and requests pen information data corresponding to the positions on the paper 128 and 130 detected by the position detection unit 214.
The communication processing unit 222 receives pen information data in response to the request from the information server 16 and outputs the pen information data to the information output processing unit 182.

When the information server 16 authenticates the user using the handwriting input pen 26, the communication processing unit 222 authenticates the information server 16 in the same manner as the DB management unit 216 of the information server program 160 (FIG. 8). Only the pen information data input by the user is output to the information output processing unit 182 or other nodes.
The information output processing unit 182 outputs voice data included in the pen information data input from the communication processing unit 222 from the input / output device 106 (FIG. 9).

[Background management program 34]
FIG. 11 is a diagram showing a background management program 34 that operates on the background management apparatus 32 shown in FIG.
As illustrated in FIG. 11, the background management program 34 includes an unused background pattern DB 340, a used background pattern DB 342, a DB management unit 344, and a communication processing unit 222.
The background management program 34 manages which background pattern 136 is attached to each of the sheets 128 and 130 by these components.
The background management program 34 distributes and reuses the background pattern 136 for each node of the information management system 1.

The communication processing unit 222 performs processing necessary for communication with other nodes via the network 120 (FIG. 1) as in the input device program 200 (FIG. 8).
The unused background pattern DB 340 stores background pattern data indicating the background pattern 136 that is not attached to any of the sheets 128 and 130.
The used background pattern DB 342 stores background pattern data indicating the background pattern 136 attached to one of the sheets 128 and 130.

When an unused background pattern 136 is requested from another node (such as PC 18), the DB management unit 344 selects and requests an unused background pattern 136 stored in the unused background pattern DB 340. Deliver to the node.
The DB management unit 344 further deletes the background pattern data of the distributed background pattern 136 from the unused background pattern DB 340 and stores and manages the background pattern data in the used background pattern DB 342.
Further, when receiving the background pattern 136 from another node together with the fact that it is no longer used, the DB management unit 344 deletes the background pattern data of the background pattern 136 received from the used background pattern DB 342, The background pattern data of the received background pattern 136 is stored in the unused background pattern DB 340.

[Voice input / output by information management system 1]
FIG. 12 is a diagram illustrating a voice input / output service by the information management system 1 (FIG. 1).
As shown in FIG. 12, the voice input / output service in the information management system 1 is realized by the user performing the following operations.

(1) The user operates the handwriting input pen 26 (FIGS. 1 and 4) to accept voice (“the target for this year is a 10% increase in sales” illustrated in FIG. 12).
(2) The user takes a note (“the story of the president this morning” illustrated in FIG. 12) on the paper 128, 130 with the background by using the handwriting input pen 26 near the input device 20.
(3) The user performs an operation of pointing a memo on the paper 128, 130 and outputting a sound near the PC 18.

Through the above-described user operation, the input device 20 accepts the voice around the handwriting input pen 26 as voice data, associates it with the position data of the memo on the paper 128, 130 with the background, and the pen information. The data is sent to the information server 16 as data.
When the user performs an operation of pointing the position of the memo on the paper 128, 130 with the background and outputting a sound by the PC 18, the PC 18 includes the audio data associated with the position of the memo. Pen information data is requested to the information server 16.
The input device 20 sends pen information data corresponding to this request to the PC 18.
The PC 18 outputs the voice data included in the pen information data as voice from the input / output device 106 (FIG. 2).

With reference to FIG. 13, the voice input / output service in the information management system 1 will be further described.
FIG. 13 is a diagram showing a communication sequence (S10) for realizing the voice input / output service shown in FIG. 12 in the information management system 1 (FIG. 1).
As shown in FIG. 13, in step 100 (S <b> 100), the handwriting input pen 26 accepts surrounding voice and sequentially outputs voice data to the input device 20.

In step 102 (S102), the user takes a note by using the handwriting input pen 26 on the paper 128, 130 with the background near the input device 20, and thereby the position on the paper 128, 130. And perform a predetermined operation for voice input (simply an operation of simply moving the handwriting input pen 26 away from the paper 128, 130).
The handwriting input pen 26 transmits these operations to the input device 20.

In step 104 (S104), the input device 20 associates the position data indicating the position of the memo on the paper 128, 130 with the background and the voice data obtained by the process of S100 into pen information data. To the information server 16.
The information server 16 receives, stores and manages the pen information data.

In step 106 (S106), the user moves near the PC 18 and uses the handwriting input pen 26 to point the memo on the paper 128, 130 with the background, and to perform a predetermined operation for voice output. (For example, pointing the position of the memo twice in succession).
The handwriting input pen 26 transmits handwriting data and writing pressure data corresponding to pointing and operation to the PC 18.

In step 108 (S108), the PC 18 requests the information server 16 for pen information data including position data indicating the position of the memo.
In step 110 (S110), the information server 16 transmits pen information data including position data notified from the PC 18 to the PC 18.
The PC 18 receives the pen information data from the information server 16 and outputs the sound data included in the pen information data as sound from the input / output device 106 (FIG. 2).

[Education service]
FIG. 14 is a diagram illustrating an education service by the information management system 1 (FIG. 1).
Here, for example, a case where an English education service is provided by the information management system 1 is taken as a specific example.
As shown in FIG. 14, English words (Patents) are printed on papers 128 and 130 with a background, and the pronunciation of the English words is heard at predetermined positions on the papers 128 and 130, respectively. A button and a button for listening to the meaning of the English word are printed.

When the user points the “pronunciation” button of the paper 128 with the background with the handwriting input pen 26 (FIGS. 1 and 4), the English corresponding to the background pattern 136 (FIG. 3) of the paper 128 and 130 is displayed. The pronunciation of a word (here, “Patent”) is output as a voice.
When the “meaning” button of the paper 128 with the background is pointed, the meaning (patent) of the English word corresponding to the background pattern 136 of the paper 128, 130 is output as a voice.
When the user points to a position on the paper 128, 130 other than these buttons, the fact that there is no information corresponding to that portion is displayed by voice or image.

FIG. 15 is a diagram exemplifying data stored in the additional service DB 172 (FIG. 9) of the information server 16 in order to realize the educational service shown in FIG. (B) indicates audio data referred to by the index.
For realizing the education service shown in FIG. 14, for example, data as shown in FIG. 15 is stored in the additional service DB 172.

First, when the user performs a pointing operation on the background paper 128, 130 as shown in FIG. 14, the background pattern data of the paper 128, 130 (FIG. 14) and the user on the paper 128, 130 are displayed from the PC 18. Position data indicating the position where the character is pointed is sent to the information server 16.
The service providing unit 170 (FIG. 9) of the information server 16 associates the background pattern data from the PC 18 with the identifier (ID; background pattern ID) of the background pattern 136.

Further, the service providing unit 170 searches the index shown in FIG. 15A by using the background pattern ID and the position data indicating the pointing position from the PC 18, and the identifier of the voice data corresponding to these pieces of information ( Audio data ID) is obtained.
The service providing unit 170 returns the audio data (FIG. 15B) corresponding to the audio data ID obtained in this way to the PC 18.

Alternatively, when the voice data ID corresponding to the background pattern ID and the position data cannot be obtained, the service providing unit 170 notifies the PC 18 of the absence of the voice data.
The PC 18 outputs the audio data thus obtained from the input / output device 106 (FIG. 2), or displays the absence of the audio data on the input / output device 106 by sound or image.

FIG. 16 is a diagram showing a communication sequence (S12) for realizing the education service shown in FIG. 14 in the information management system 1 (FIG. 1).
As shown in FIG. 16, in step 120 (S120), the user uses the handwriting input pen 26 near the PC 18 (FIGS. 1, 2 and 10) to provide paper 128 and 130 with a background. Point anywhere above.
The PC 18 detects the background pattern data of the papers 128 and 130 and the pointed position on the papers 128 and 130.

In step 122 (S122), the PC 18 transmits the background pattern data of the pointing sheets 128 and 130 and the position data indicating the pointed position to the information server 16.
In step 124 (S124), the information server 16 transmits audio data (or data indicating the absence of audio data) corresponding to the background pattern data and position data from the information server 16 to the PC 18.
The PC 18 outputs the voice data (or absence thereof) from the information server 16 by voice or the like.

[User authentication using handwriting input pen]
FIG. 17 is a diagram illustrating user authentication using the handwriting input pen 26 (FIGS. 1 and 4).
As shown in FIG. 17, when user authentication is performed using the handwriting input pen 26, for example, first, the user uses the handwriting input pen 26 to the paper 128 and 130 with a background attached thereto. Then, handwritten the characters “authentication request”, and then handwritten the identifier (user ID) given to the user or the user's name (hereinafter referred to as user ID / name).

FIG. 18 is a diagram illustrating user authentication data stored in the additional service DB 172 illustrated in FIG. 9 and used for user authentication using the handwriting input pen 26.
When the input device 20 recognizes the character of “authentication request” inputted first, it recognizes the handwritten user ID / user name, and recognizes the recognized user ID / name and handwritten user ID / name. Handwriting data and handwriting pressure data (which may be either; handwriting data and handwriting data) are transmitted to the information server 16.
In the additional service DB 172 of the information server program 160 operating on the information server 16, as shown in FIG. 18, handwriting data / writing pressure data indicating the handwriting of the user ID / name by the user, user ID / name, , Information (email address or the like) indicating the contact address (for example, PC 18) is stored in association with each other.

The information server 16 compares the handwriting data / writing pressure data sent from the input device 20 with the handwriting data / writing pressure data associated with the user ID / name sent from the input device 20. Are recognized as the handwriting of the same person, the user who wrote the handwritten character is authenticated as a valid user and the input of information is permitted.
Further, the information server 16 notifies the contact registered as the contact of the user of the handwriting input pen 26 that there is an authentication request and the result.

FIG. 19 is a diagram showing a communication sequence (S14) for realizing user authentication using the handwriting input pen 26 shown in FIG. 17 in the information management system 1.
As shown in FIG. 19, in step 140 (S140), the user uses the handwriting input pen 26 to hand-write the characters “authentication request” on the paper 128, 130 with the background, When the user ID / name is handwritten, the handwriting input pen 26 outputs handwriting data / writing pressure data indicating these characters to the input device 20.

In step 142 (S142), the input device 20 recognizes the “authentication request” handwritten by the user and the user ID / name, and the handwritten data indicating the recognized user ID / name and the handwriting of the user ID / name. The pen pressure data is transmitted to the information server 16.
In step 144 (S144), the information server 16 writes the handwriting data / writing pressure data sent from the input device 20 and the handwriting data / name corresponding to the recognized user ID / name in the additional service DB 172 (FIG. 9). The writing pressure data (FIG. 18) is compared.

When they are in the same range, the information server 16 authenticates the user using the handwriting input pen 26 and informs the input device 20 that the input of information is permitted.
Alternatively, when they are not in the same range, the user using the handwriting input pen 26 is not authenticated, and the input device 20 is notified that information input is not permitted.

In step 146 (S146), upon receiving notification from the information server 16 that the user has been authenticated, the input device 20 accepts information from the handwriting input pen 26.
At this time, as illustrated in FIG. 17, the input device 20 may notify the user of the handwriting input pen 26 of the authentication result by voice.
In step 148 (S148), the information server 16 notifies the PC 18 that there is an authentication request and the result.

Although the user authentication by handwriting is shown here, for example, by providing a pressure sensor in the grip portion of the handwriting input pen body 262 of the handwriting input pen 26, the strength of the user's grip is taken into account. It is also possible to authenticate the user.
Alternatively, the user's fingerprint image is obtained by the lens system 264 of the handwriting input pen 26 and the CCD element 266 (FIG. 4) and used for authentication, or each of the handwriting input pens 26 described above uses a unique identification number and identification code. It is also possible to perform authentication, or to perform authentication using the user's voice obtained by the MIC 272 of the handwriting input pen 26.

[User authentication using electronic authentication stamp]
FIG. 20 is a diagram illustrating user authentication using the electronic authentication stamp 24 (FIGS. 1 and 7).
As shown in FIG. 20, when using the user authentication using the electronic authentication mark 24 in the information management system 1 (FIG. 1), for example, the user first places an image of his / her fingerprint on the electronic authentication mark 24 or Enter information that can be used for user authentication, such as your voice.
In addition to these, information that can be used for user authentication includes the strength with which the electronic authentication mark 24 is brought into contact with the paper 128, 130, and pressure on the side surface of the electronic authentication mark main body 242 (FIG. 7). The grip strength that can be detected by the sensor can be exemplified.

The electronic authentication stamp 24 sends the input authentication data to the information server 16 via the input device 20, and the information server 16 authenticates the user using the authentication data.
Further, the information server 16 obtains the background pattern 136 (FIG. 3) from the image of the paper 128, 130 with which the electronic authentication mark 24 is in contact, and identifies the paper 128, 130 that is about to be stamped. The position where the user is going to impress is detected.

Based on the background pattern 136 obtained as a result of the authentication and the position where the electronic authentication mark 24 comes into contact, the information server 16 confirms that the electronic authentication mark 24 is correct on the correct sheets 128 and 130 by a valid user. It is contacted with the position (department mark; FIG. 20), it is determined whether it is used for the seal, and the determination result is transmitted to the input device 20 and notified to the user's contact information.
The input device 20 indicates the above determination result to the user by voice (authenticated. Please stamp the garden position; FIG. 20) or an image, etc. Seal data is received from the electronic authentication stamp 24 that is in contact with the correct position of the correct paper 128 or 130.
As shown in FIG. 20, user authentication using the electronic authentication stamp 24 and user authentication using the handwriting input pen 26 shown in FIG. 17 can be used in combination.

FIG. 21 is a diagram showing data stored in the additional service DB 172 of the information server program 160 (FIG. 9) for user authentication using the electronic authentication stamp 24 shown in FIG.
As shown in FIG. 20, for user authentication using the electronic authentication mark 24, authentication data used for user authentication, such as user ID / name, user fingerprint, voice print, stamp / grip strength, etc. The background pattern data for identifying the correct sheets 128 and 130, the position data for identifying the correct stamp position, and the contact information such as the user's mail address of the electronic authentication stamp 24 are stored in the additional service DB 172. The
Further, when the seal stamp information data is accepted from the electronic authentication stamp 24, the accepted seal stamp information data is stored in the additional service DB 172 in association with these data.

FIG. 22 is a diagram showing a communication sequence (S16) for realizing user authentication using the electronic authentication stamp 24 shown in FIG.
As shown in FIG. 22, in step 160 (S160), the user of the electronic authentication stamp 24 uses the handwriting input pen 26 or performs a predetermined operation (input of voice or background) on the electronic authentication stamp 24. Pointing to the paper 128, 130, etc.), and information required for authentication such as a fingerprint image is input to the electronic authentication mark 24 to request authentication.

In step 162 (S162), the input device 20 accepts the authentication request and transmits authentication data to the information server 16.
In step 164 (S164), the information server 16 compares the authentication data from the input device 20 with the authentication data stored in advance (FIG. 21). The user of the mark 24 is authenticated as valid, and otherwise, it is not authenticated as the user of the electronic authentication mark 24 is not valid.

In step 164 (S164), the information server 16 returns an authentication result to the input device 20.
The input device 20 indicates this authentication result to the user by voice or the like.
In step 166 (S166), the information server 16 notifies the user's contact information (for example, the PC 18; FIG. 1) of the electronic authentication seal 24 that there is an authentication request and the result.

In step 168 (S168), when the user who has been authenticated as legitimate touches and stamps the electronic authentication mark 24 on a predetermined position on the background-added paper 128, 130 (denoted by the section head mark; FIG. 20), the electronic The authentication stamp 24 transmits seal stamp information data, background pattern data, and position data indicating the contacted position to the input device 20.
In step 170 (S170), the input device 20 transmits background pattern data and position data to the information server 16.
The information server 16 compares the background pattern data and position data from the input device 20 with the background pattern data and position data stored in advance, and the electronic authentication mark 24 comes into contact with the correct position of the correct sheets 128 and 130. Judge whether it has been.

In step 172 (S172), the information server 16 notifies the input device 20 of the determination result in the process of S170.
The input device 20 accepts the seal information data received from the electronic authentication mark 24 when the determination result from the information server 16 indicates that the electronic authentication mark 24 is in contact with the correct position of the correct paper 128, 130. If necessary, send to other nodes.

It should be noted that by processing the images of the paper 128 and 130 with background obtained by the electronic authentication mark 24, the impression angle of the electronic authentication mark 24 can be detected, so that the user authentication based on the impression angle of the electronic authentication mark 24 is performed. May be performed.
Further, since the image of the user's fingerprint can be obtained by the electronic authentication mark 24, the user may be authenticated by the fingerprint.
Moreover, since the grip strength of the user holding the electronic authentication mark 24 can be detected, the user authentication by the grip strength may be performed.

[Selective addition of background pattern, etc.]
FIG. 23 shows selective addition of the background pattern 136 (FIG. 3) to the plain paper 128, 130 in the back information management system 1 (FIG. 1), management of additional data using the background pattern 136, and output thereof. It is a figure illustrated.
The selective addition of the background pattern 136 and the management / output of additional data using the background pattern (hereinafter also simply referred to as “selective addition of the background pattern 136”, etc.) described below are performed in the information management system 1 (FIG. 1). In FIG. 14, if necessary, a background pattern 136 is selectively attached to the plain paper 128, 130, and data management similar to the education service shown in FIG. 14 is performed using the paper 128 with the background pattern 136 attached. And it is devised to be able to output.

As shown in FIG. 23, the selective addition of the background pattern 136 in the information management system 1 is realized as follows.
First, image data ("Patent", "[pronunciation") for printing on plain paper 128, 130 by the application 186 (FIG. 10) of the information use program 180 operating on the PC 18 (FIG. 1) of the information management system 1. ] ”,“ [Meaning] ”), position data indicating the position of this image data on the paper 128, 130, and additional data that is added to the image on the paper 128 and is output when these images are pointed. (For example, “[pronunciation]” image and sound data “patent” associated with the position, or “[meaning]” image and character data “patent” associated with the position) Is created.

Next, when the application 186 (FIG. 10) inputs / outputs information as shown in FIG. 14 using the papers 128 and 130, the application 186 (FIG. 10) inputs to the background management device 32 (FIGS. 1 and 11). A background pattern 136 to be attached to the plain paper 128, 130 is requested, and the background pattern 136 sent from the background management device 32 is received in response to this request.
Next, the application 186 outputs the created image data and the background pattern 136 received from the background management device 32 to the certified printer 122, and outputs the image data and the background pattern 136 to the plain paper 128, 130 is printed.

FIG. 24 is a diagram illustrating data stored in the additional service DB 172 (FIG. 9) of the information server 16 in order to realize the selective addition of the background pattern 136 shown in FIG. Indicates an index used to output additional data, and (B) indicates additional data referenced by the index.
Next, the application 186 determines the position of the image on the paper 128, 130 associated with the background pattern data indicating the background pattern 136 printed on the paper 128, 130 together with the image and the additional data, with respect to the information server 16. The position data shown and the additional data associated with the images on the sheets 128 and 130 are transmitted and registered.

As a result, in the additional service DB 172 of the information server program 160 (FIG. 9) operating on the information server 16, the background pattern 136 attached to the sheets 128 and 130 is identified as shown in FIG. Therefore, a background pattern ID, position data indicating the position of each image, and additional data ID indicating additional data associated with this position are associated with each other and stored as an index.
Further, as shown in FIG. 24B, the additional service DB 172 stores additional data corresponding to the additional data ID shown in FIG.

After this registration, for example, in the vicinity of the PC 18 (FIG. 1), the user uses the handwriting input pen 26 to display each image (FIG. 23) on the paper 128, 130 with the image and the background pattern 136 attached thereto. Pointing with the handwriting input pen 26.
In response to this pointing, the additional data associated with the pointed image is sent from the information server 16 to the PC 18 and displayed / output from the PC 18 to the user.

FIG. 25 is a diagram showing a communication sequence (S18) for realizing the selective addition of the background pattern 136 shown in FIG.
As shown in FIG. 25, in step 180 (S180), image data, position data, additional data, and the like are created in the PC 18 in accordance with a user operation.

In step 182 (S182), the PC 18 requests the background pattern 136 from the background management apparatus 32 (such as FIG. 1) in accordance with the user operation.
In step 184 (S 184), the background management device 32 selects an unused background pattern 136 that has not been printed on any of the papers 128 and 130 so far, and transmits it to the PC 18.

In step 186 (S186), the PC 18 transmits the background pattern 136 and image data received from the background management device 32 to the authorized printer 122 (FIG. 1).
In step 188 (S188), the authorized printer 122 prints the image data from the PC 18 and the background pattern 136 on the plain paper 128,130.

In step 190 (S190), the PC 18 registers position data, additional data, and a background pattern in the information server 16 (FIG. 1) as shown in FIG.
In step 192 (S192), when the user points an image printed on the paper 128, 130 using the handwriting input pen 26 near the PC 18, the background pattern attached to the pointed paper 128, 130 is displayed. 136 and handwriting data indicating the pointing position are transmitted to the PC 18.

In step 194 (S194), the PC 18 determines, based on the handwriting data from the handwriting input pen 26, the background pattern data indicating the background pattern 136 attached to the paper 128, 130 and the position on the pointing paper 128, 130. The position data shown is generated and transmitted to the information server 16.
In step 196 (S196), the information server 16 returns additional data corresponding to the background pattern data and position data from the PC 18 to the PC 18.
The PC 18 receives the additional data from the information server 16 and displays / outputs it as a sound or an image to the user.

[Reuse Background Pattern]
FIG. 26 is a diagram illustrating reuse of the background pattern 136 in the information management system 1 (FIG. 1).
There are an enormous variety of background patterns 136 (FIG. 3), and they can be used uniquely for the sheets 128 and 130 in practice.

However, since the types of the background pattern 136 are finite, it is desirable that the background pattern 136 be reusable.
That is, as shown in FIG. 26, when the paper 128 on which the printed image cannot be erased and the paper 130 on which the printed image can be erased are discarded, or when the background pattern 136 is erased from the paper 130 In addition, the background pattern 136 attached to the sheets 128 and 130 is managed by returning the used pattern from the used state to the unused state, and can be newly printed on the plain sheets 128 and 130. It is desirable.

FIG. 27 is a diagram illustrating a function implementing device 38 used for reusing the background pattern 136.
For the reuse of the background pattern 136, as the function realization device 38, for example, as shown in FIG. 27, as the function realization device 38 of the PC 18 (FIGS. 1 and 2), the background pattern 136 printed on the paper 130, etc. An erasing device 382 that erases the image of the image, or a combination of a shredder 384 that discards the paper 128 and 130 with the background and a scanner 380 that reads the background pattern 136 attached to the paper 128 and 130. It is desirable.

In the function realizing device 38 shown in FIG. 27, when an image such as the background pattern 136 printed on the paper 130 is erased, the scanner 380 reads the background pattern 136 attached to the paper 130 and then erases the device 382. However, the image printed on the paper 130 and the background pattern 136 are erased.
Alternatively, in the function realization device 38, when the paper 128, 130 with the background is discarded, the scanner 380 reads the background pattern 136 printed on the paper 128, 130, and then the shredder 384 Shear 130 and discard.
The background pattern 136 read by the function realization device 38 is used for reusing the background pattern 136 by the application 186 (FIG. 10) of the information use program 180 operating on the PC 18.

FIG. 28 shows the background pattern 136 (FIG. 3) attached to the paper 128, 130 when the paper 128, 130 with the background is discarded in the information management system 1 (FIG. 1). It is a figure which shows the communication sequence (S22) for setting it into a use state.
As shown in FIG. 28, in step 220 (S220), when the user inserts the used paper 128, 130 into the function realizing device 38 (shredder 384; FIG. 27) of the PC 18 (FIG. 1), the scanner 380 The background pattern 136 attached to the sheets 128 and 130 is read and output to the information utilization program 180 (FIG. 10).
The shredder 384 shears and discards the sheets 128 and 130 inserted in the function realizing device 38.

In step 222 (S222), the PC 18 transmits the background pattern 136 attached to the discarded paper 128, 130 to the background management device 32 (FIG. 1), and further notifies the discarding.
In step 224 (S224), the background management apparatus 32 manages the background pattern 136 sent from the PC 18 as an unused state so that it can be reused.

FIG. 29 shows the background pattern 136 (FIG. 29) attached to the paper 130 when the image and the background pattern 136 attached to the reusable paper 130 are deleted in the information management system 1 (FIG. 1). FIG. 3 is a diagram showing a communication sequence (S24) for setting 3) to an unused state and further reusing it.
In step 240 (S240), in step 240 (S240), when the user inserts the used paper 130 into the function realizing device 38 (erasing device 382; FIG. 27) of the PC 18 (FIG. 1), the scanner 380 The background pattern 136 attached to the paper 130 is read and output to the information utilization program 180 (FIG. 10).
The erasing device 382 erases the image and the background pattern 136 attached to the paper 130 inserted into the function realizing device 38 so that it can be reused.

In step 242 (S242), the PC 18 transmits the background pattern 136 attached to the paper 130 from which the image has been deleted to the background management device 32 (FIG. 1), and further, the background attached. The deletion of the pattern 136 is notified.
In step 244 (S244), the background management apparatus 32 sets the background pattern 136 sent from the PC 18 to an unused state and manages it so that it can be reused.

In step 246 (S246), the PC 18 requests the background pattern 136 from the background management apparatus 32.
In step 248 (S248), the background management device 32 sends an unused background pattern 136 (whether or not it is the background pattern 136 erased in the processing of S240) to the PC 18 that issued the request. To send.

In step 250 (S250), the background management apparatus 32 manages the background pattern 136 transmitted to the PC 18 in a used state.
In step 252 (S252), the PC 18 causes the background pattern 136 to be transmitted to the authorized printer 122 (FIG. 1) and printed on the paper 130.

  The present invention can be used for voice storage and input / output thereof.

It is a figure which shows the outline | summary of the information management system concerning this invention. It is a figure which illustrates hardware configurations, such as a background management apparatus shown in FIG. It is a figure which illustrates the background attached | subjected to the paper shown in FIG. It is a figure which shows the structure of the handwriting input pen shown in FIG. It is a figure which shows the external appearance of the 2nd handwriting input pen used instead of the 1st handwriting input pen shown in FIG. 4 etc. in an information management system (FIG. 1). It is a figure which shows the external appearance of the 3rd handwriting input pen used instead of the 1st handwriting input pen shown in FIG. 4 etc. in an information management system (FIG. 1). It is a figure which shows the structure of the electronic authentication stamp 24 shown in FIG. It is a figure which shows the program for input devices which operate | moves on the input device shown in FIG. It is a figure which shows the information server program which operate | moves on the information server shown in FIG. It is a figure which shows the information utilization program which operate | moves on PC shown in FIG. It is a figure which shows the background management program which operate | moves on the background management apparatus shown in FIG. It is a figure which illustrates the voice input / output service by an information management system (FIG. 1). FIG. 13 is a diagram showing a communication sequence (S10) for realizing the voice input / output service shown in FIG. 12 in the information management system (FIG. 1). It is a figure which illustrates the education service by the information management system 1 (FIG. 1). FIG. 15 is a diagram exemplifying data stored in the additional service DB (FIG. 9) of the information server in order to realize the education service shown in FIG. 14, wherein (A) shows an index used for outputting audio data. (B) shows the audio data referred to by the index. It is a figure which shows the communication sequence (S12) for implement | achieving the educational service shown in FIG. 14 in an information management system (FIG. 1). It is a figure which illustrates user authentication using a handwriting input pen (Drawing 1, Drawing 7). It is a figure which illustrates the data for user authentication which are memorize | stored in additional service DB shown in FIG. 9, and are used for the user authentication using a handwriting input pen. It is a figure which shows the communication sequence (S14) for implement | achieving user authentication using the handwriting input pen shown in FIG. 17 in an information management system. It is a figure which illustrates user authentication using an electronic authentication seal | sticker (FIG. 1, FIG. 7). It is a figure which shows the data memorize | stored in the additional service DB of an information server program (FIG. 9) for the authentication of the user using the electronic authentication seal | sticker shown in FIG. It is a figure which shows the communication sequence (S16) for implement | achieving user authentication using the electronic authentication seal | sticker shown in FIG. It is a figure which illustrates selective addition of the background pattern (FIG. 3) with respect to the paper in an information management system, management of the additional data using a background pattern, and its output. FIG. 24 is a diagram illustrating data stored in the additional service DB (FIG. 9) of the information server in order to realize the selective addition of the background pattern shown in FIG. 23, and FIG. (B) indicates additional data referred to by the index. It is a figure which shows the communication sequence (S18) for implement | achieving the selective addition of the background pattern shown in FIG. It is a figure which illustrates reuse of the background pattern in an information management system (Drawing 1). It is a figure which illustrates the function implementation apparatus used for reuse of a background pattern. FIG. 10 is a diagram showing a communication sequence (S22) for setting a background pattern (FIG. 3) attached to a sheet to an unused state when the sheet is discarded in the information management system (FIG. 1). In the information management system (FIG. 1), when the image and the background pattern attached to the reusable paper are deleted, the background pattern (FIG. 3) attached to the paper is set to an unused state. Furthermore, it is a figure which shows the communication sequence (S24) for reuse.

Explanation of symbols

1 Information management system,
120 ... Network,
122 ... Certified printer,
124... Non-certified printer,
128, 130 ... paper,
136 ... Background pattern,
132 ... lattice points,
134 ... dots,
3 ... Background management device,
32 ... Background management device,
100 ... body,
102 ... CPU,
104: Memory,
106: I / O device,
110: Communication device,
112... Recording device,
114... Recording medium,
116: Wireless IF,
38 ... function realization device,
34 ... Background management program,
340 ... Unused background pattern DB,
342 ... Used background pattern DB,
344 ... DB management unit,
30 ... Billing device,
16 ... information server,
160... Information server program,
164 ... Operation analysis unit,
166... DB management unit,
168 ... Pen information DB,
170: Service providing unit,
172 ... Additional service DB,
18 ... PC,
180 ... Information utilization program,
182 ... Information output processing unit,
184 ... Background / operation detector,
186 Application,
2 ... Handwriting input system,
20 ... Input device,
200 ... Program for input device,
202... Reception processing unit,
204: Handwriting analysis unit,
206 ... Character identification part,
208: Audio processing unit,
210 ... associating unit,
212 ... Background detection unit,
214 ... position detector,
216: DB management unit,
220 ... pen information DB,
222... Communication processing unit,
224 ... seal information processing part,
24 ... Electronic authentication seal,
26 ... Handwriting input pen,
260 ... writing member,
262 ... handwriting input pen body,
264 ... Lens system,
266 ... CCD element,
268 ... Pressure sensor,
270 ... Image capture circuit,
272: MIC,
274: Audio processing circuit,
278 ... Power supply,
280 ... Transmission circuit,

Claims (18)

A voice input means for receiving a voice signal;
Pointing means for pointing at an arbitrary position on the predetermined member;
Detecting means for detecting a pointed position on the member;
Information storage means for storing the position pointed at the member and the received audio signal in association with each other;
An audio input / output system comprising: audio output means for outputting an audio signal stored in association with the position on the member when pointing to the position on the stored member is detected. The voice input / output system according to claim 1, wherein the information storage means stores a voice signal received before the pointing by the pointing means is performed in association with a position pointed at the member. The pointing means has voice receiving means for receiving voice and outputting it as a voice signal;
The voice input / output system according to claim 1, wherein the voice input unit receives the output voice signal. The member is provided with a mark for identifying a position in the member,
The pointing means has an image input means for receiving an image of a mark attached to the member and outputting it as an image signal,
The voice input / output system according to claim 3, wherein the detection unit detects a position of the pointed member based on the output image signal. The mark given to each member is unique to each member,
The detecting means detects a position in the pointed member and the unique mark;
The information storage means stores the detected unique mark in association with the position and audio signal in the associated member,
The voice output means is associated with the detected unique mark and the detected position on the member when the unique mark and pointing with respect to the position on the stored member are detected. The voice input / output system according to claim 4, wherein the voice signal stored in the step is output. The voice input / output system according to claim 1, wherein the pointing means further includes writing means for writing on the pointed portion. A voice input means for receiving a voice signal;
A voice input device having first detection means for detecting a position pointed at a predetermined member;
Information storage means for storing the detected position in the member and the received audio signal in association with each other;
A voice server device comprising: a first voice output means for outputting a voice signal stored in association with a position in the member when pointing to a position in the stored member is detected;
Second detection means for detecting a position pointed at the member;
An audio input / output system comprising: an audio output device having a second audio output means for outputting an audio signal output in accordance with a position on the detected member as audio. Each member has a unique mark,
In the voice input device,
The first detection means detects a position pointed at the member and a mark unique to each member;
In the voice server device,
The information storage means stores the detected unique mark in association with the detected position on the member and the received voice,
The first sound output means detects the detected unique mark and the detected unique mark when a pointing with respect to a position on the stored member and a unique mark attached to the member are detected. The voice input / output system according to claim 7, wherein a voice signal stored in association with a position on a member is output. The detected position on the predetermined member and the received audio signal are stored in association with each other, and when pointing to the position on the stored member is detected, the position is stored in association with the position on the member. A voice server device that outputs a voice signal, and a voice output device that detects a pointed position on the member and outputs a voice signal output according to the detected position on the member as a voice. An audio input device in an input / output system,
An audio input means for receiving an input of an audio signal;
A voice input device comprising: detecting means for detecting a pointed position on the member. An audio input device that receives an input of an audio signal and detects a position pointed at a predetermined member; an audio signal output according to the detected position at the member; A voice server device in a voice input / output system having voice output means for outputting as voice,
Information storage means for storing the detected position in the member and the received audio signal in association with each other;
A voice server device comprising: voice output means for outputting a voice signal stored in association with a position on the member when a pointing to a position on the stored member is detected. An audio input device that receives an input of an audio signal and detects a position pointed at a predetermined member, a position on the detected member, and the received audio signal are stored in association with each other, and the stored A voice output device in a voice input / output system having a voice server device that outputs a voice signal stored in association with a position in the member when pointing to a position in the member is detected,
Detecting means for detecting a pointed position on the member;
An audio output device comprising: audio output means for outputting an audio signal output according to the detected position on the member as audio. Receives audio signal input,
Point at any position on a given member,
Detecting a pointed position on the member;
Storing the detected position on the member and the received audio signal in association with each other;
A voice input / output method for outputting a voice signal stored in association with the position of the member when pointing to the position of the stored member is detected. Receives audio signal input,
Detect the pointed position on a given member,
Storing the detected position on the member and the received audio signal in association with each other;
When pointing to a position in the stored member is detected, an audio signal stored in association with the position in the member is output,
Detecting the pointed position of the member;
An audio input / output method for outputting an audio signal output according to a position on the detected member as audio. A voice input step for receiving a voice signal;
A pointing step for pointing an arbitrary position on a predetermined member;
A detecting step of detecting a pointed position on the member;
A position / speech storage step of storing the detected position of the member in association with the received sound signal;
A program that causes a computer to execute an audio output step of outputting an audio signal stored in association with a position on the member when pointing to the position on the stored member is detected. In a voice input device including a computer,
A voice input step for receiving a voice signal;
Causing the computer to execute a first detection step of detecting a pointed position on a predetermined member;
In a voice server device including a computer,
A position / speech storage step of storing the detected position of the member in association with the received sound signal;
Causing the computer to execute a first audio output step of outputting an audio signal stored in association with a position in the member when pointing to a position in the stored member is detected;
In an audio output device including a computer,
A second detection step of detecting a pointed position on the member;
A program for causing the computer to execute a second sound output step of outputting a sound signal output according to a position on the detected member as sound. The detected position on the predetermined member and the received audio signal are stored in association with each other, and when pointing to the position on the stored member is detected, the position is stored in association with the position on the member. A voice server device that outputs a voice signal, and a voice output device that detects a position pointed at the member and outputs a voice signal output according to the detected position on the member as a voice. In a voice input device including a computer of an input / output system,
A voice input step for accepting input of a voice signal;
A program for causing the computer to execute a detection step of detecting a pointed position on the member. An audio input device that receives an input of an audio signal and detects a position pointed at a predetermined member; an audio signal output according to the detected position at the member; In a voice server device including a voice input / output system computer having voice output means for outputting as voice,
A position / speech storage step of storing the detected position of the member in association with the received sound signal;
A program that causes the computer to execute an audio output step of outputting an audio signal stored in association with a position on the member when pointing to a position on the stored member is detected. An audio input device that receives an input of an audio signal and detects a position pointed at a predetermined member, a position on the detected member, and the received audio signal are stored in association with each other, and the stored In a voice output device including a computer of a voice input / output system having a voice server device that outputs a voice signal stored in association with a position in a member when pointing to a position in the member is detected,
A detecting step of detecting a pointed position on the member;
A program for causing a computer to execute an audio output step of outputting an audio signal output according to the position of the detected member as audio.

Images