JP2008077218A - Writing information processing system, pen device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To handle characters or character strings written on a medium as universal electronic information representing the characters or character strings. <P>SOLUTION: An electronic pen for reading information related to a writing operation from a medium has a character code information generation part 613 for generating character code information about characters written on the medium. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a writing information processing system that processes writing information obtained by digitizing writing on a medium such as paper.

In recent years, attention has been paid to a technique in which characters, figures, and the like written on paper are converted into electronic data, which is transferred to a personal computer, a mobile phone, etc., and the written contents are stored as they are or transmitted as an e-mail.
In such a technique, special paper on which fine dots are printed is used. That is, on this special paper, fine dots are formed with different patterns for each grid of a predetermined size, for example. Therefore, for example, when writing on this special paper using a dedicated pen with a built-in image sensor, the dot pattern at the position where characters, figures, etc. are written is read into the image sensor and written on this special paper. The coordinates of the position (drawing trace) are specified. As a result, it is possible to generate an electronic document composed of such written characters, figures, etc., or to add written characters, figures, etc. to a predetermined electronic document (see, for example, Patent Document 1).

Japanese Patent Laying-Open No. 2004-94907 (pages 12-14)

  Here, in general, the information processing apparatus cannot universally recognize the written character or character string in the electronic document as electronic information indicating the character or character string. For this reason, there is a disadvantage that information such as conversion from hiragana to kanji, translation into a foreign language such as English, retrieval of related information, etc. cannot be performed on written characters and character strings. .

  The present invention has been made in order to solve the technical problems as described above, and an object of the present invention is to express a character or character string written on a medium as a universal electronic character indicating the character or character string. It is to enable handling as information.

  For this purpose, the writing information processing system according to the present invention corresponds to a writing operation on a medium by a user, a pen device for reading information related to the writing operation from the medium, and information read by the pen device. An information processing apparatus that stores the information written on the medium in association with the electronic document printed on the medium or the medium based on the acquired information, and the pen device has character code information on characters written on the medium It is characterized by having a character code information generation unit for generating.

Here, the pen device may further include a switching unit that switches whether the character code information generation unit generates character code information. The character code information generation unit may generate character code information for katakana and hiragana in written characters. Furthermore, the character code information generation unit may generate character code information for characters written in a predetermined area on the medium. In addition, the character code information generation unit can generate character code information for a written character when a predetermined check box formed on the medium is marked.
The pen device may further include a display unit that displays written characters based on the character code information generated by the character code information generation unit.

Further, the present invention is regarded as a pen device, and the pen device according to the present invention includes a position information reading unit that reads position information on a medium of characters written on the medium, and position information read by the position information reading unit. A character code information generating unit that generates character code information of characters written on the medium is provided.
Here, the character code information generation unit may further include a switching unit that switches whether character code information is generated. In addition, the display device may further include a display unit that displays written characters based on the character code information generated by the character code information generation unit.

  Further, the present invention is regarded as a program, and the program of the present invention is written on a medium based on the function of reading the position information on the medium of characters written on the medium and the read position information. And a function of generating character code information of characters.

  This program may be executed by loading a program stored in a ROM or the like into the RAM, for example. In addition, there is a form that is executed by the CPU in a state stored in the ROM in advance. Further, when a rewritable ROM such as an EEPROM is provided, only a program may be provided and installed in the ROM after the device is assembled. In providing this program, it is also conceivable that the program is transmitted to a computer having a data recording device via a network such as the Internet and installed in a ROM of the data recording device.

According to the first aspect of the present invention, it is possible to handle a character or character string written on a medium as universal electronic information indicating the character or character string.
According to claim 2 of the present invention, it is possible to easily set whether or not to convert a written character or character string into character code information.
Moreover, according to Claim 3 of this invention, it becomes possible to form a pen device small.

According to the fourth aspect of the present invention, it is possible to easily select on the medium whether or not to convert a written character or character string into character code information.
According to claim 5 of the present invention, it is possible to easily select on the medium whether or not to convert a written character or character string into character code information, and convert it into character code information. It is possible to set a large area for writing a character or a character string to be written.
According to the sixth aspect of the present invention, the user can confirm the character or character string converted into the character code information at hand.

According to claim 7 of the present invention, it is possible to handle a character or character string written on a medium as universal electronic information indicating the character or character string.
According to claim 8 of the present invention, it is possible to easily set whether or not to convert written characters and character strings into character code information.
According to claim 9 of the present invention, the user can confirm the character or character string converted into the character code information at hand.
According to claim 10 of the present invention, it is possible to handle a character or character string written on a medium as universal electronic information indicating the character or character string.

The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described below in detail with reference to the accompanying drawings.
In the writing information processing system of the present embodiment, a code pattern image is printed on a medium such as paper in addition to a document image of an electronic document. The code pattern image is an image of an identification code and a position code obtained by encoding identification information and position information.
Here, as the identification information, either identification information for uniquely identifying a medium or identification information for uniquely identifying an electronic document printed on the medium is employed. When the former identification information is adopted, when a plurality of copies of the same electronic document are printed, different identification information is given to different media. On the other hand, when the latter identification information is adopted, when the same electronic document is printed, the same identification information is given even on different media.
The position information is information representing the coordinate position on the medium.

In the writing information processing system of the present embodiment, writing is performed with an electronic pen (pen device) on a medium on which such an image is printed. Thereby, writing information and character code information are produced | generated based on the positional information contained in a code pattern image. Further, the electronic document is specified based on the identification information included in the code pattern image. If the identification information included in the code pattern image is the identification information of the medium, the electronic document can be specified by managing the correspondence between the identification information and the electronic document printed on the medium. .
Then, writing information and character code information are associated with the specified electronic document.

  In this specification, the term “electronic document” is used, but this does not mean only data obtained by digitizing a “document” including text. For example, “electronic document” includes image data such as pictures, photographs, figures, etc. (regardless of raster data or vector data) and other printable electronic data.

Hereinafter, the writing information processing system according to the present embodiment that performs such a schematic operation will be specifically described.
As described above, in the writing information processing system according to the present embodiment, identification information that uniquely identifies a medium and identification information that uniquely identifies an electronic document printed on the medium are used as identification information. However, here, a case where identification information for uniquely identifying a medium is employed will be described as an example.

In the writing information processing system according to the present embodiment, as already described, the identification information of the medium is embedded in the code pattern image. Therefore, in this embodiment, “identification information” refers to identification information of a medium. For example, when each person writes a memo or the like separately in a print document created from the same electronic document, it is convenient to embed different identification information for each medium. This is because writing information and character code information for the medium can be easily managed separately for each medium.
In addition, if different identification information is embedded for each medium, it is useful for information tracking. For example, by recording the correspondence between the identification information of the medium and the user who has instructed printing, it becomes possible to easily grasp who has output a specific print document.
In the following description, the identification information and the position information are clearly distinguished from each other for the sake of simplicity. However, there is also a technique in which different position information for each medium is embedded in a code pattern image, and the medium is identified by the difference in the position information. Therefore, when such a method is adopted, it is assumed that the position information has a function of identifying the medium, and this is considered as identification information.

First, the configuration of the writing information processing system of the present embodiment will be described.
FIG. 1 is a diagram showing an overall configuration of a writing information processing system to which the present embodiment is applied. In this writing information processing system, a terminal device 10 as an example of an information processing device, a document server 20 as an example of an information processing device, an identification information server 30 as an example of an information processing device, and an image forming device 40 are connected to a network 90. It is configured by being connected. The writing information processing system includes a printed document 50 and an electronic pen 60 as an example of a pen device.

The terminal device 10 is used for instructing printing of an electronic document or generating writing information. For example, a personal computer (PC) is used as the terminal device 10.
The document server 20 stores an electronic document. When there is an instruction to print the electronic document, an instruction to form a superimposed image in which the image of the electronic document and the code pattern image are superimposed is output. In the present embodiment, the document server 20 also has a function of copying an electronic document. The document server 20 can be realized by a general-purpose server computer, for example.

The identification information server 30 issues identification information to be given to the medium. The issued identification information is managed in association with the electronic document printed on the medium. In the present embodiment, the identification information server 30 also has a function of specifying an electronic document that associates writing information and character code information. The identification information server 30 can be realized by a general-purpose server computer, for example.
The image forming apparatus 40 forms an image on a medium. Here, as an image forming method in the image forming apparatus 40, for example, an electrophotographic method can be used, but any other method such as an ink jet method may be used.

The print document 50 is a medium on which a superimposed image obtained by superimposing an image of an electronic document and a code pattern image is printed.
The electronic pen 60 is a pen device having a function of recording (writing) characters or figures on the printed document 50. In the present embodiment, the electronic pen 60 also has a mechanism for transmitting information acquired from the code pattern image to another device.

Next, an operation when the print document 50 is generated in the writing information processing system of the present embodiment will be described.
FIG. 2 is a sequence diagram showing an operation in the writing information processing system when the print document 50 is generated.
First, the user operates the terminal device 10 to designate an electronic document to be printed from among the electronic documents stored in the document server 20. As a result, the terminal device 10 transmits an instruction to print the electronic document to the document server 20 (step 101). At this time, the terminal device 10 also transmits print parameters designated by the user. Here, the print parameters include page, number of copies, paper size, N-up (printing in which N pages of an electronic document are allocated within one page of paper), margins, and the like.

  The document server 20 receives an electronic document print instruction from the terminal device 10 (step 201). Then, the storage location of the electronic document instructed to be printed and the print parameters received from the terminal device 10 are transmitted to the identification information server 30 (step 202).

Thereby, the identification information server 30 receives the storage location of the electronic document and the print parameters (step 301). Then, unused identification information is extracted from the database that manages the identification information (step 302). Here, the number of pieces of identification information to be extracted is determined according to the printing parameters. In other words, basically, the number of pieces of identification information obtained by multiplying the number of pages to be printed by the number of copies is extracted. However, if there is an N-up designation or the like in the print parameters, this is also taken into consideration. For example, when printing 5 copies of a 10-page electronic document in 2-up, 25 (= 10 ÷ 2 × 5) pieces of identification information are extracted.
Next, the identification information server 30 associates the identification information, the storage location of the electronic document, and the print parameters and registers them in the database (step 303). Then, the identification information server 30 transmits the identification information to the document server 20 (step 304).

Thereby, the document server 20 receives the identification information (step 203). Then, a code pattern image in which identification information and position information are embedded is generated (step 204). Details of the code pattern image generation process will be described later.
Thereafter, the document server 20 transmits the document image and code pattern image of the electronic document to the image forming apparatus 40, and instructs image formation (step 205). This instruction can be performed, for example, by generating a page description language (PDL: Print Description Language) from the electronic document, identification information, and position information, and transmitting it.

  The image forming apparatus 40 receives the document image and code pattern image of the electronic document from the document server 20 (step 401). The image forming apparatus 40 first develops the document image into C (cyan), M (magenta), and Y (yellow) images (step 402). Next, the document image is formed using C, M, and Y toners, and the code pattern image is formed using K (black) toner (step 403).

In the above-described example, the identification information server 30 simply issues the identification information, and the document server 20 generates a code pattern image including the identification information and instructs the image forming apparatus 40 to form an image. However, the identification information server 30 may generate a code pattern image and instruct the image forming apparatus 40 to form an image.
The code pattern image can also be generated by the image forming apparatus 40. In this case, the document server 20 or the identification information server 30 adds the identification information to the PDL generated from the electronic document and transmits it to the image forming apparatus 40, and the image forming apparatus 40 generates a code pattern image including the identification information. It will be.

  In the above description, the configuration in which the database configured by associating the identification information, the storage location of the electronic document, and the print parameters is placed in the identification information management server 30 has been described. This is because by placing such a database on a sharable device (identification information server 30), it is possible to deal with a plurality of users and to ensure the security of electronic documents using server access control technology. However, such a configuration is not necessarily employed, and a configuration in which such a database is placed in the terminal device 10 or the document server 20 may be employed.

In the image forming apparatus 40 described above, the code pattern image is formed using K toner. This is because the K toner has a larger amount of infrared light absorption than the C, M, and Y toners, and the code pattern image can be read by the electronic pen 60. However, the code pattern image can also be formed using special toner.
Here, as the special toner, an invisible toner having a maximum absorption rate of 7% or less in the visible light region (400 nm to 700 nm) and an absorption rate of 30% or more in the near infrared region (800 nm to 1000 nm) is exemplified. . Here, “visible” and “invisible” are not related to whether they can be recognized visually. “Visible” and “invisible” are distinguished depending on whether or not an image formed on a printed medium can be recognized by the presence or absence of color development due to absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes.
In addition, the invisible toner preferably has an average dispersion diameter in the range of 100 nm to 600 nm in order to increase the near-infrared light absorption capability necessary for machine reading of an image.

Next, the code pattern that is the basis of the code pattern image generated in the present embodiment will be described.
FIG. 3 is a diagram for explaining a code pattern.
First, the bit pattern constituting the code pattern will be described.
FIG. 3A shows an example of bit pattern arrangement.
A bit pattern is the minimum unit of information embedding. Here, as shown in FIG. 3A, bits are arranged at two locations selected from nine locations. In the figure, black squares indicate positions where bits are arranged, and hatched squares indicate positions where bits are not arranged. There are 36 (= ₉ C ₂ ) combinations for selecting 2 locations out of 9 locations. Therefore, 36 types (about 5.2 bits) of information can be expressed by such an arrangement method.
However, the identification information and the position information are expressed using 32 (5 bits) of these 36 patterns.

By the way, the minimum unit square shown in FIG. 3A has a size of 2 dots × 2 dots at 600 dpi. Since the size of one dot at 600 dpi is 0.0423 mm, one side of the minimum unit square is 84.6 μm (= 0.0423 mm × 2). The larger the dots that make up the code pattern, the more likely it is to be noticeable. However, if it is too small, printing with a printer becomes impossible. Therefore, the above value is adopted as the dot size, which is larger than 50 μm and smaller than 100 μm. Thereby, it is possible to form dots of an optimum size that can be printed by the printer. That is, 84.6 μm × 84.6 μm is the minimum size that can be stably formed by the printer (image forming apparatus 40).
In addition, by making the dot in such a size, one side of one bit pattern becomes about 0.5 mm (= 0.0423 mm × 2 × 6).

A code pattern composed of such bit patterns will be described.
FIG. 3B shows an example of the arrangement of code patterns.
Here, the minimum unit square shown in FIG. 3B corresponds to the bit pattern shown in FIG. That is, the identification code obtained by encoding the identification information is embedded using 16 (= 4 × 4) bit patterns. Also, the X position code obtained by encoding the position information in the X direction and the Y position code obtained by encoding the position information in the Y direction are each embedded using four bit patterns. Further, a synchronization code for detecting the position and rotation of the code pattern is embedded in the upper left corner using one bit pattern.
Since the size of one code pattern is equal to the width of five bit patterns, it is about 2.5 mm. In the present embodiment, a code pattern image obtained by imaging the code pattern generated in this way is arranged on the entire sheet surface.

Next, reflection of writing information and character code information in the electronic document in the writing information processing system of the present embodiment will be described. However, in the following description, it is assumed that writing information and character code information are directly added to the electronic document.
First, the electronic pen 60 used to read the writing on the printed document 50 will be described.
FIG. 4 is a block diagram illustrating a functional configuration of the electronic pen 60.
As shown in FIG. 4, the electronic pen 60 includes a control unit 61 that controls the operation of the entire pen, a pressure sensor 62 that detects a writing operation by the electronic pen 60 based on pressure applied to the pen tip 69, and infrared light on the medium. An infrared LED 63 for irradiating and an infrared CMOS sensor 64 for inputting an image are provided. Further, information memory 65 for storing identification information and position information, as well as character code information, a communication unit 66 for controlling communication with an external device, a battery 67 for driving the electronic pen 60, and identification information for the electronic pen 60 ( A pen ID memory 68 for storing (pen ID), a display unit 71 for displaying written characters and the like, and a changeover switch 72 as an example of a switching unit for switching whether to convert written characters into character mode information. ing. These units are connected to the control unit 61. Whether each unit is directly connected to the control unit 61 is appropriately determined according to the control method.
Here, as the display unit 71, a display device that displays characters or the like such as a liquid crystal display or the like, a display device that displays characters or the like by projecting light emitted by itself onto a paper surface, or the like is used. be able to.

FIG. 5 is a block diagram showing a functional configuration of the control unit 61. As shown in FIG. 5, the control unit 61 processes the code pattern image detected from the input image, and the data processing unit 612 extracts the identification information and the position information from the processing result in the image processing unit 611. A character code information generation unit 613 that generates character code information from the position information extracted by the data processing unit 612 is included.
The character code information generation unit 613 includes a feature amount extraction unit 613a, a character feature amount storage unit 613b, and a character identification unit 613c. The feature amount extraction unit 613a extracts a circumscribed rectangle for each written character from the position information extracted by the data processing unit 612, and extracts a feature amount of the extracted character image. The character feature amount storage unit 613b stores a feature amount for a standard character pattern of each character. The character identification unit 613c performs character recognition by comparing the feature amount of the character image extracted by the feature extraction unit 613a with the feature amount of the character pattern stored in the character feature amount storage unit 613b. The character code information corresponding to the recognized character is generated. Here, as the character code information, for example, a versatile ASCII code, a shift JIS code, or the like can be used.
Further, the character code information generation unit 613 is configured to function when, for example, a changeover switch is turned on.

Here, an outline of the operation of the electronic pen 60 will be described.
When writing with the electronic pen 60 is performed, the pressure sensor 62 connected to the pen tip 69 detects the writing operation. Thereby, the infrared LED 63 is turned on, and the infrared CMOS sensor 64 captures an image on the medium by the CMOS sensor.
The infrared LED 63 is pulsed in synchronization with the shutter timing of the CMOS sensor in order to reduce power consumption.
The infrared CMOS sensor 64 is a CMOS sensor having sensitivity in the infrared region, and a global shutter type CMOS sensor that can simultaneously transfer captured images is used. In order to reduce the influence of disturbance, a visible light cut filter is disposed on the entire surface of the CMOS sensor. The infrared CMOS sensor 64 captures an image with a period of about 70 fps to 100 fps (frame per second). The image sensor is not limited to a CMOS sensor, and other image sensors such as a CCD may be used.

When the captured image is input to the control unit 61, the control unit 61 acquires a code pattern image from the captured image. Then, it is decoded to obtain identification information and position information embedded in the code pattern image.
Hereinafter, operations of the image processing unit 611 and the data processing unit 612 in the control unit 61 at that time will be described.
FIG. 6 is a flowchart showing operations in the image processing unit 611 and the data processing unit 612 of the electronic pen 60.
First, the image processing unit 611 inputs an image on the print document 50 (step 601). Then, processing for removing noise included in the image is performed (step 602). Here, the noise includes variations in CMOS sensitivity, noise generated by an electronic circuit, and the like. What processing is performed to remove noise is determined according to the characteristics of the imaging system of the electronic pen 60. For example, sharpening processing such as blurring processing or unsharp masking can be applied.

Next, the image processing unit 611 detects a dot pattern (a position of the dot image) from the image (step 603). For example, the dot pattern portion and the background portion are separated by binarization processing, and the dot pattern can be detected from each binarized image position. When a binarized image contains a lot of noise components, for example, it is necessary to combine a filter process for determining a dot pattern based on the area and shape of the binarized image.
Also, the image processing unit 611 converts the detected dot pattern into digital data on a two-dimensional array (step 604). For example, on a two-dimensional array, a position where there is a dot is converted to “1”, and a position where there is no dot is converted to “0”. The digital data on the two-dimensional array is transferred from the image processing unit 611 to the data processing unit 612.

Next, the data processing unit 612 detects a bit pattern including a combination of two dots shown in FIG. 3A from the received digital data (step 605). For example, the bit pattern can be detected by moving the boundary position of the block corresponding to the bit pattern on the two-dimensional array and detecting the boundary position so that the number of dots included in the block is two. .
When the bit pattern is detected in this way, the data processing unit 612 detects the synchronization code by referring to the type of the bit pattern (step 606). Then, the identification code and the position code are detected based on the positional relationship from the synchronization code (step 607).
Thereafter, the data processing unit 612 acquires the identification information by decoding the identification code, and acquires the position information by decoding the position code (step 608).

Here, in the electronic pen 60 of the present embodiment, when the changeover switch 72 is set to ON, the character code information generation unit 613 of the control unit 61 acquires position information from the data processing unit 612, and Based on the acquired position information, character code information about the written character is generated.
Next, the operation of the character code information generation unit 613 in the control unit 61 at that time will be described.
FIG. 7 is a flowchart showing the operation of the character code information generation unit 613 of the electronic pen 60.
When the changeover switch 72 of the electronic pen 60 is set to ON, the character code information generation unit 613 acquires position information from the data processing unit 612 (S701). Here, the data processing unit 612 functions as a position information reading unit that reads position information of characters written on the medium on the medium.
When the character code information generation unit 613 acquires position information, the feature amount extraction unit 613a extracts a circumscribed rectangle for each character written from the acquired position information, and extracts a feature amount of the extracted character image (S702). . And the feature-value of the extracted character image is sent to the character identification part 613c.

The character identification unit 613c performs character recognition from the feature amount of the character image acquired from the feature amount extraction unit 613a (S703). Specifically, the character identification unit 613c performs the feature amount of the character image acquired from the feature amount extraction unit 613a and the feature amount of each character stored in the character feature amount storage unit 613b. Measure the logical distance between and. Based on the measurement result, it is determined that the character having the smallest feature distance is drawn (written).
Then, character code information (for example, ASCII code, shift JIS code, etc.) corresponding to the character determined to be drawn in step 703 is generated (S704).

  Here, the character identifying unit 613c can be configured to perform character recognition only for, for example, hiragana and katakana whose font structure is relatively simple. In other words, the character feature amount storage unit 613b stores only the feature amounts of standard hiragana and katakana character patterns, and compares the feature amounts with the character image feature amounts acquired from the feature amount extraction unit 613a. It can also be configured. With this configuration, the storage capacity of the character feature amount storage unit 613b and the processing load on the character identification unit 613c can be reduced, so that the character code information generation unit in the electronic pen 60 with limited internal space 613 can be formed in a small size.

After that, the character identification unit 613c stores the character code information generated in step 704 in the information memory 65 in association with the identification information and position information generated in the data processing unit 612 (S705). Then, the character code information is sent to the terminal device 10 together with the identification information and the position information via the communication unit 66.
At the same time, the character identification unit 613c sends the character code information generated in step 704 to the display control unit 614 (S706). Then, the display control unit 614 displays the characters written on the display unit 71 in a character code format (S707).
Here, the electronic pen 60 may include a sound source unit that emits a predetermined sound or a light emitting unit that emits predetermined light, instead of or in addition to the display unit 71. Then, instead of the display on the display unit 71 or together with the display on the display unit 71, the character written by the character code information generation unit 613 is recognized by a predetermined sound or a predetermined light, and character code information is generated. It is also possible to inform the user that this has been done.

By the way, in the above description, when the changeover switch 72 of the electronic pen 60 is set to ON, the configuration has been described in which the character code information generation unit 613 generates character code information for a written character.
In the electronic pen 60 according to the present embodiment, for example, when a character is written in a predetermined area on the print document 50, the character code information is generated by the electronic pen 60 for the character written in the area, and the print document For the characters written in other areas on 50, the terminal device 10 can also be configured to generate writing information.
Here, FIG. 8 is a diagram illustrating an example of the print document 50 that is a medium. As shown in FIG. 8, an area G for generating character code information for a written character is set in advance on the print document 50. And when writing in the area | region G, the electronic pen 60 produces | generates character code information about the written character. In addition, for characters written in other areas on the print document 50, the terminal device 10 to which position information is sent generates writing information.
Further, as shown in FIG. 8, a frame for designating the size of a written character can be provided in the region G. The size of the frame is set to a reference size when the character code information is reflected on the electronic document. Therefore, by setting the characters so as to fit within this frame, it is possible to set so that the characters of a certain size are always reflected on the electronic document.

  At that time, the ON setting by the changeover switch 72 can be linked. That is, when the writing of characters in the region G is completed in units of one character or in units of a predetermined character string, the character code information is changed in units of one character or in units of predetermined character strings by turning on the changeover switch 72. It can also be configured to generate. Thereby, the user can easily select whether or not to convert the written characters into character code information in units of one character or in units of predetermined character strings. Further, for example, it is possible to set so that the generated character code information is erased by continuously turning on the changeover switch 72 twice. In this way, by linking the region G for generating the character code information for the written character and the ON setting by the changeover switch 72, the conversion to the character code information is executed / not executed, and the generated character code information is displayed. Erasing can be performed freely in units of one character or a predetermined character string.

In the electronic pen 60 according to the present embodiment, for example, a predetermined check box is formed on the print document 50, and a character written on the print document 50 when, for example, a check mark is written in the check box. It can also be configured to generate character code information for.
Here, FIG. 9 is a diagram showing an example of another print document 50. As shown in FIG. 9, when a check mark is described in a check box H formed in advance on the print document 50, the electronic pen 60 displays character code information on characters written on the print document 50. Generate.
As described above, the generation instruction of the character code information for the character written on the print document 50 can use various modes.

Next, a process of generating writing information from information acquired by the electronic pen 60 and adding it to the electronic document, and adding character code information generated by the electronic pen 60 to the electronic document will be described. This process is realized by the terminal device 10, the document server 20, and the identification information server 30 exchanging information.
FIG. 10 is a diagram illustrating functional configurations of the terminal device 10, the document server 20, and the identification information server 30. The terminal device 10, the document server 20, and the identification information server 30 also have a function for generating the print document 50, but here a function for adding writing information and character code information to the electronic document. Show only.

As illustrated in FIG. 10, the terminal device 10 includes an input unit 11, an output unit 12, a writing generation unit 13, a transmission unit 14, and a reception unit 15.
The input unit 11 receives an input such as a response from the user in response to an inquiry to the user.
The output unit 12 outputs an inquiry to the user.
The writing generation unit 13 generates writing information based on the position information obtained from the electronic pen 60. Accordingly, the writing generation unit 13 functions as writing information generation means. The writing generation unit 13 sends the character code information obtained from the electronic pen 60 to the transmission unit 14 as it is.
The transmission unit 14 transmits identification information, writing information, character code information, a response to an inquiry to the user, and the like.
The receiving unit 15 receives identification information, position information, and character code information from the electronic pen 60, and receives an inquiry to the user from the identification information server 30.

  These functional parts are realized by cooperation of software and hardware resources. Specifically, a CPU (not shown) of the terminal device 10 transfers a program for realizing the functions of the input unit 11, the output unit 12, the writing generation unit 13, the transmission unit 14, and the reception unit 15 from the external storage device to the main storage device. Read and process.

The document server 20 includes a document storage unit 21, a document generation unit 22, a writing addition unit 23, a transmission unit 24, and a reception unit 25.
The document storage unit 21 is a storage unit and stores an electronic document.
The document generation unit 22 generates a new electronic document (hereinafter referred to as a “replicated document”) obtained by copying the electronic document stored in the document storage unit 21, and stores the copied document in the document storage unit 21.
The writing adding unit 23 adds writing information and character code information to the duplicate document stored in the document storage unit 21.
The transmission unit 24 transmits a result of generating a duplicate document and adding writing information and character code information to the duplicate document.
The receiving unit 25 receives an instruction for generating a duplicate document and adding writing information and character code information to the duplicate document.

  These functional parts are realized by cooperation of software and hardware resources. Specifically, a CPU (not shown) of the document server 20 reads a program that realizes the functions of the document generation unit 22, the writing addition unit 23, the transmission unit 24, and the reception unit 25 from the external storage device to the main storage device and performs processing. I do. The document storage unit 21 is realized by, for example, a magnetic disk.

The identification information server 30 includes a DB (database) storage unit 31, a control unit 32, a registration unit 33, a transmission unit 34, and a reception unit 35.
The DB storage unit 31 stores a database in which identification information, a storage location of an electronic document, a print parameter, and a flag indicating whether a duplicate document exists are associated with each other.
The control unit 32 controls the overall operation of the identification information server 30.
The registration unit 33 registers information in the database stored in the DB storage unit 31.
The transmission unit 34 transmits an inquiry or the like to the user to the terminal device 10, and transmits to the document server 20 an instruction for generating a duplicate document or adding writing information or character code information to the duplicate document.
The receiving unit 35 receives identification information, writing information, character code information, a response to an inquiry to the user, and the like from the terminal device 10, and generates a duplicate document, writing information about the duplicate document, Receives the result of adding character code information.

  These functional parts are realized by cooperation of software and hardware resources. Specifically, a CPU (not shown) of the identification information server 30 reads a program for realizing the functions of the control unit 32, the registration unit 33, the transmission unit 34, and the reception unit 35 from the external storage device to the main storage device, and performs processing. Do. The DB storage unit 31 is realized by a magnetic disk, for example.

Next, the contents of the database stored in the DB storage unit 31 in the identification information server 30 will be specifically described.
FIG. 11 shows an example of data in this database.
Among these, FIG. 11A shows a table for managing electronic documents printed on each medium.
As shown in FIG. 11A, the table includes identification information, a storage location of the electronic document, a print parameter, and a flag as items.

In this embodiment, the identification information is identification information of each medium as described above.
The storage location of the electronic document is information (address information) of the location where the electronic document printed on each medium is stored.
The print parameter is a print parameter set when an electronic document is printed on each medium. In FIG. 11A, print parameters are shown in the form of “page (P), margins (A, B, C, D)”. P is a page number, and A, B, C, and D are left, right, upper, and lower margins (unit: mm), respectively. Here, the page and the margin are shown as print parameters, but the present invention is not limited to this. In addition to this, it is possible to manage various printing parameters used in normal printing.

  The flag indicates whether a duplicate document has been generated for the electronic document printed on each medium. In the present embodiment, the flag “ON” indicates that a duplicate document has been generated, and the flag “OFF” indicates that a duplicate document has not been generated. Immediately after the data is registered in step 303 of FIG. 2, the corresponding flag is set to “OFF” which is an initial value. However, after that, when a duplicate document is generated by writing on the print document 50, the flag is set to “ON”.

The database in FIG. 11A will be described more specifically.
First, the first to fourth lines indicate that two copies of the first page to the second page of the electronic document “aaa.doc” have been printed. In that case, one part is printed with left and right and top and bottom margins of 1 mm, and the other part is printed with left and right and top and bottom margins of 2 mm.
The fifth line indicates that one copy of the first page of the electronic document “bbb.doc” has been printed. At that time, the left and right margins are printed at 1 mm, and the upper and lower margins are printed at 2 mm.

  Since the flags in the 3rd to 5th lines out of the 1st to 5th lines are “OFF”, the original electronic document is not duplicated. On the other hand, the flags in the first and second lines are “ON”. This indicates that writing was performed on the medium managed in the first line or the medium managed in the second line, and the original electronic document “aaa.doc” was copied.

When a duplicate document for the original electronic document is generated in this way, the identification information server 30 of the present embodiment manages this with a separate table.
FIG. 11B shows a table for managing such a duplicate document.
That is, the first page of “aaa.xdw”, which is a duplicate document for “aaa.doc”, is associated with the medium managed in the first line of FIG. Further, the second page of “aaa.xdw”, which is a duplicate document for “aaa.doc”, is associated with the medium managed in the second line of FIG.

  Here, an example is shown in which a duplicate document of an electronic document managed in the first and second lines in FIG. 11A is managed in FIG. 11B. However, this does not mean that writing was actually made on both the media managed in the first and second lines in FIG. When writing is made on a certain medium, a duplicate document may be generated for the entire original electronic document, or a duplicate document may be generated only for a page printed on the medium of the original electronic document. is there.

Next, an operation when writing information or character code information is added to an electronic document will be described.
FIG. 12 is a sequence diagram showing the operation at this time.
First, the electronic pen 60 uses the identification information and position information acquired by the process of FIG. 6 and, further, for example, when the changeover switch 72 is set to ON, the identification information and position information acquired by the process of FIG. The associated character code information is transmitted to the terminal device 10 (step 611).

In the terminal device 10, the receiving unit 15 receives identification information and position information from the electronic pen 60, and further character code information associated with the identification information and position information, and passes these pieces of information to the writing generation unit 13. (Step 111).
Next, the writing production | generation part 13 produces | generates writing information based on position information among the delivered information, and delivers it to the transmission part 14 (step 112). At this time, the writing information is generated based on position information indicating a plurality of positions on the medium.

When there is an error in the position information received from the electronic pen 60, the writing generation unit 13 may correct the position information by performing the following process.
The first is a filtering process that compares a plurality of pieces of position information and removes singularities from them. That is, the continuity of a plurality of pieces of position information acquired by one writing operation (stroke) is inspected, and points greatly deviating from the continuity are excluded as points having a decoding error. Thereby, smooth writing information can be obtained.
The second is correction processing for detecting position information that has failed in decoding and correcting the position information based on the peripheral position information. That is, a decoding error is detected by checking the continuity of a plurality of pieces of position information acquired by one writing operation (stroke). Then, position information that has failed in decoding is corrected using position information that has been successfully decoded in the vicinity thereof. Here, as a correction method, a known method such as linear interpolation or spline interpolation can be used.

The writing generation unit 13 passes the identification information transferred from the reception unit 15 to the transmission unit 14 as it is. Similarly, the character code information transferred from the receiving unit 15 is transferred to the transmitting unit 14 as it is in a form associated with the identification information and the position information.
Thereafter, the transmission unit 14 transmits the identification information, the position information, and the character code information to the identification information server 30 (step 113).

  Thereby, in the identification information server 30, the receiving unit 35 receives the identification information, the position information, and the character code information, and passes them to the control unit 32 (step 311). Then, the control unit 32 refers to the database stored in the DB storage unit 31 and determines whether or not the flag corresponding to the received identification information is “ON” (step 312). That is, it is determined whether or not a duplicate document for an electronic document printed on the medium has been generated by writing on the same medium in the past.

Here, when it is determined that the flag is “ON”, that is, when it is determined that there is a duplicate document, the control unit 32 instructs the transmission unit 34 to transmit an inquiry to the terminal device 10. And the transmission part 34 transmits an inquiry to the terminal device 10 (step 313).
If there is a duplicate document, writing information or character code information is added to the duplicate document, and a new duplicate document is generated separately from the duplicate document, and writing information or character code information is added to the new duplicate document. It may be the case. Therefore, in this embodiment, it is left to the user to decide which one to select. For example, when writing is performed on the medium managed in the first line of FIG. 11A, writing information or character code information is added to the duplicate document managed in the first line of FIG. An inquiry is made as to whether writing information or character code information is added to a new duplicate document generated from the original electronic document managed in the first line of FIG.

  Thereby, in the terminal device 10, the receiving part 15 receives an inquiry (step 114). The receiving unit 15 passes the inquiry to the output unit 12, the output unit 12 outputs an inquiry message, and the user inputs a response to the inquiry. Here, the content of the response is selection information on whether to add writing information and character code information to an already existing duplicate document, or to add writing information and character code information to a newly generated duplicate document. When such a response is input, the input unit 11 receives the response and passes it to the transmission unit 14, and the transmission unit 14 transmits the response to the identification information server 30 (step 115).

As a result, in the identification information server 30, the receiving unit 35 receives the response and passes it to the control unit 32 (step 314). Then, the control unit 32 refers to the response content, and determines whether to add writing information or character code information to an existing duplicate document, or to generate a new duplicate document and add writing information or character code information to it. (Step 315).
Here, when it is determined that writing information or character code information is to be added to an already copied document, an instruction to that effect is given to the transmission unit 34. Then, the transmission unit 34 instructs the document server 20 to add writing information and character code information to the duplicate document identified in Step 312 (Step 319).

As a result, in the document server 20, the receiving unit 25 receives an instruction to add writing information or character code information and passes it to the writing adding unit 23. Then, the writing adding unit 23 adds writing information and character code information to the designated duplicate document among the duplicate documents stored in the document storage unit 21 (step 213).
Here, the writing adding unit 23 adds the character code information to the position of the duplicate document and the duplicate document corresponding to the identification information and the position information when adding the character code information to the designated duplicate document. Is set to a predetermined size (default value). Moreover, it is also possible to arbitrarily set the character size set in the writing addition unit 23 by the user's instruction input from the input unit 11 of the terminal device 10.

On the other hand, when it is determined that a new duplicate document is generated and writing information or character code information is added thereto, the process proceeds to step 316. In the present embodiment, writing information or character code information is not added to the electronic document itself (original electronic document) printed on the medium. Accordingly, if it is determined in step 312 that the flag is “OFF”, that is, if it is determined that there is no duplicate document, the process similarly proceeds to step 316.
That is, the control unit 32 instructs the transmission unit 34 to transmit a copy document generation instruction for the original electronic document identified in step 312 to the document server 20. Then, the transmission unit 34 instructs the document server 20 to generate a duplicate document (step 316). For example, when writing is performed on the medium managed in the third line of FIG. 11A, generation of a duplicate document for “aaa.doc” is instructed.

  As a result, in the document server 20, the receiving unit 25 receives this instruction and passes it to the document generating unit 22. Then, the document generation unit 22 generates a duplicate document for the designated electronic document among the electronic documents stored in the document storage unit 21 and stores it in the document storage unit 21 (step 211).

At this time, the control unit 32 of the identification information server 30 acquires the print parameter associated with the medium in the database in the DB storage unit 31. Then, the transmission unit 34 transmits this print parameter to the document server 20. By doing so, the document generation unit 22 of the document server 20 can generate a duplicate document that looks the same as a printed document. As a result, the writing information and the character code information can be reflected on the electronic document with higher accuracy.
If the electronic document is edited, the correspondence between the electronic document and the writing information or character code information may be lost, resulting in a contradiction. Furthermore, some electronic documents, such as Web pages, do not have a page structure corresponding to a print document, and the layout of the print document may vary greatly depending on settings at the time of printing. Even in such a case, writing information and character code information can be reflected at an appropriate position by generating an electronic document having the same layout as the printed document.

  At this time, the duplicate document can be in a format in which a portion corresponding to the original electronic document cannot be edited. As a format of such a duplicate document, there are “XDW format” in “DocuWorks” of Fuji Xerox Co., “PDF format” in “Acrobat” of Adobe Systems Incorporated. When a duplicate document in a format in which the portion corresponding to the original electronic document cannot be edited in this way is used, for example, writing information or text on the duplicate document using the “annotation” function in “DocuWorks” of Fuji Xerox Co., Ltd. Code information can be pasted.

  When the duplicate document is generated in this way, the document generation unit 22 acquires information on the storage location (including the file name) of the duplicate document and passes it to the transmission unit 24. Then, the transmission unit 24 transmits this information to the identification information server 30 (step 212).

  Thereby, in the identification information server 30, the receiving unit 35 receives the information on the storage location of the duplicate document, and passes it to the registration unit 33 via the control unit 32 (step 317). Then, the registration unit 33 stores the correspondence between the identification information and the storage location in the DB storage unit 31. Further, the flag corresponding to the original electronic document to be copied is changed to “ON” (step 318). For example, if writing is performed on the medium immediately after the first line in FIG. 11A is registered, the first line in FIG. 11B is registered here. Then, the flag on the first line in FIG. 11A is changed from “OFF” to “ON”.

  Thereafter, the control unit 32 instructs the transmission unit 34 to add writing information or character code information to the duplicate document. Then, the transmission unit 34 instructs the document server 20 to add writing information and character code information to the duplicate document identified in step 317 (step 319). For example, if writing is performed on the medium immediately after the first line in FIG. 11A is registered, writing to the first page of “aaa.xdw” registered in the first line in FIG. The addition of information and character code information is instructed.

  As a result, in the document server 20, the receiving unit 25 receives an instruction to add writing information or character code information and passes it to the writing adding unit 23. Then, the writing adding unit 23 adds writing information and character code information to the designated duplicate document among the duplicate documents stored in the document storage unit 21 (step 213).

In the above description, the user is inquired whether to add writing information or character code information to an existing duplicate document or a newly created duplicate document. However, various variations are conceivable. .
First, the system determines an electronic document to which writing information or character code information is added without making an inquiry to the user.
Alternatively, if the selection method of the electronic document to which writing information or character code information is to be added is already determined at the time of printing, specification information for specifying the selection method may be embedded in the medium as part of the code pattern image. Good. In this case, when the electronic pen 60 acquires identification information and position information from the code pattern image, this designation information is also acquired and transmitted to the identification information server 30. Then, the identification information server 30 performs the determination in step 315 based on this designation information.

As described above, in the writing information processing system of the present embodiment, the character or character string written by the electronic pen 60 is converted into character code information that is electronic information that universally indicates the character or character string. It is configured.
Thereby, not only the writing information but also the character code information can be reflected in the electronic document. Therefore, written characters and character strings can be electronically processed on the electronic document. Specifically, for a written character or character string, for example, display with the electronic pen 60, conversion from hiragana to kanji, translation into a foreign language such as English, the meaning of the character or character string It is possible to perform electronic processing such as searching for information and searching for related information.
In addition, since the electronic pen 60 has a function of converting to character code information, it is easy to specify characters to be converted.

It is the figure which showed the whole structure of the writing information processing system of this invention. It is the sequence diagram which showed the operation | movement in the writing information processing system at the time of producing | generating a printed document. It is a figure for demonstrating a code pattern. It is the block diagram which showed the function structure of the electronic pen. It is the block diagram which showed the function structure of the control part. It is the flowchart which showed the operation | movement in the image processing part and data processing part of an electronic pen. It is the flowchart which showed the operation | movement in the character code information generation part of an electronic pen. It is the figure which showed an example of the print document. It is the figure which showed another example of the printed document. It is the figure which showed the function structure of the terminal device, the document server, and the identification information server. It is the figure which showed an example of the data in the database which DB memory | storage part memorize | stores. It is the sequence figure which showed the operation | movement at the time of adding the writing information and character code information in a writing information processing system to an electronic document, and editing an electronic document.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Terminal device, 20 ... Document server, 30 ... Identification information server, 40 ... Image forming apparatus, 50 ... Print document, 60 ... Electronic pen

Claims (10)

In response to a user's writing operation on the medium, a pen device that reads information related to the writing operation from the medium;
An information processing apparatus that acquires the information read by the pen device and stores the writing content on the medium in association with the medium or an electronic document printed on the medium based on the acquired information. ,
The pen information processing system, wherein the pen device includes a character code information generation unit that generates character code information of characters written on the medium.   The writing information processing system according to claim 1, wherein the pen device further includes a switching unit that switches whether the character code information generation unit generates the character code information.   2. The written information processing system according to claim 1, wherein the character code information generation unit generates the character code information for katakana and hiragana in the written character.   The writing information processing system according to claim 1, wherein the character code information generation unit generates the character code information for the character written in a predetermined area on the medium.   2. The character code information generation unit according to claim 1, wherein the character code information generation unit generates the character code information for the written character when a predetermined check box formed on the medium is marked. Written information processing system.   The writing information processing system according to claim 1, wherein the pen device further includes a display unit that displays the written character based on the character code information generated by the character code information generation unit. . A position information reading unit that reads position information of characters written on the medium on the medium;
A pen device comprising: a character code information generation unit configured to generate character code information of a character written on the medium based on the position information read by the position information reading unit.   The pen device according to claim 7, further comprising a switching unit that switches whether the character code information is generated by the character code information generation unit.   The pen device according to claim 7, further comprising a display unit that displays the written character based on the character code information generated by the character code information generation unit. On the computer,
A function of reading the position information of the characters written on the medium on the medium;
A program for realizing a function of generating character code information of a character written on the medium based on the read position information.

Images