JP4729879B2 - Information management method and information management apparatus - Google Patents

Description

  The present invention relates to an information management method and information management that enable retrieval and presentation of related information based on a record of access to a paper document or electronic document, that is, a memo or annotation written on paper or a history of changes or browsing of an electronic document. Relates to the device.

  With the progress of IT (Information Technology), public offices and companies are increasingly using electronic documents for business. In other words, a device that creates an electronic document using a personal computer (PC) or the like, converts a paper document into an electronic document using a scanner or a copying machine, and connects it to a computer network by means such as e-mail Communicate between. The person who receives the electronic file browses it, modifies its contents electronically if necessary, and exchanges it on the computer network. The electronic document created and corrected in this way is stored in the database, and other users can acquire it by means such as search when necessary. Non-patent literature 1 and non-patent literature 2 are well-known methods for searching for such electronic documents.

On the other hand, paper documents are still distributed in the current government offices and corporate offices. At meetings, electronic documents are printed on paper and distributed as materials. Participants take notes or underline the materials as they proceed with the discussion. In the application reception work, etc., after the application is submitted, it is circulated between departments, and processing such as approval is performed, resulting in settlement. During this time, information regarding the process or result of processing in the department is added to the document by handwriting or stamping.
Here, information such as memos, underlines, and seals written on the paper document exists independently of the electronic document before printing the paper document. Japanese Patent Laid-Open No. 2003-67363 (Patent Document 1) and the like propose a document digitization technique that digitizes the original electronic document together with the writing time and the writer, and stores the handwritten information in association with the information related to the original document. ing.

JP 2003-67363 A

Japanese Patent Laid-Open No. 2003-258792 Ricardo Beaza-Yates, Berthier Ribeiro-Neto, “Modern Information Retrieval”, Addison Wesley, 1999 Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd, 'The PageRank Citation Ranking: Bringing Order to the Web', 1998 H.Bunke, P.S.P.Wang, “Handbook of Character Recognition and Document Image Analysis”, World Scientific, 1997 David Bargeron, Tomer Moscovich, “Reflowing Digital Ink Annotations,” Proceedings of the conference on Human factors in computing systems, pp. 385 393, 2003. Akihiro Takasu, Kimizumi Hayakawa, Norio Katayama, Keizo Oyama, Jun Adachi, “Extraction and matching of bibliographic information from bibliographic image data,” Information Processing Society of Japan, “Information Science Fundamentals”, No. 045-6, 1997

  As mentioned above, in government offices and corporate offices, a considerable amount of information exists as paper documents. In addition, personal memos and other information that is important for the person who wrote them or who has a business relationship Exists only on paper documents. For example, a memo such as “A that is highly relevant to the content previously reported by A” or a memo such as “Here is a point” that is added by hand on paper materials distributed during the meeting is valuable for business. Information. In addition to searching for electronic documents, it is required to improve the efficiency of business to quickly search for information directly pointed to by such handwriting. However, with a technique such as Japanese Unexamined Patent Publication No. 2003-67363 (Patent Document 1), it is possible to store the memo and the corresponding electronic document, but what is “contents previously reported by A”? It is not clear whether there is or "here".

  A first object of the present invention is an information management method for simultaneously handling documents expressed in both paper media and electronic media, and a link to a document or information representing an annotation handwritten on the paper media. It is to provide an information management method that holds information and can refer to related information by tracing it. Here, the paper medium refers to paper that physically exists. Electronic media refers to a certain area on a computer memory or magnetic disk that can be used exclusively.

  A second object of the present invention is an information management method for simultaneously handling documents expressed in both paper media and electronic media, from handwritten information on paper media to a specific part of other electronic media such as video and audio. It is to provide an information management method that enables creation of links, that is, indexing.

  A third object of the present invention is an information management method for simultaneously handling documents expressed in both paper media and electronic media, and matches related information obtained from annotations handwritten on paper media to the user's intention. It is an object of the present invention to provide an information management method that makes it possible to present or support this.

  In the present invention, in order to achieve the first object described above, (1) all information regardless of paper media / electronic media, including attribute information such as creator and creation time, (2) For annotations handwritten on paper media, in addition to attribute information such as the writer, writing time, and writing position, and using written contents digitized by character recognition technology, (3) Create links to information highly relevant to the annotation. This realizes an information management method that enables reference of related information from handwritten annotations.

  In order to achieve the second object of the present invention, (1) information that the creation of other media such as video and audio is performed at the same place and time, and (2) written on the paper media. Using the attribute information such as the writer / writing time concerning handwriting, (3) an index is assigned to the corresponding part of the other media information such as the video and audio. Thus, for example, the user can acquire video / audio of a scene that occurred at approximately the same time as the memo in the meeting from a handwritten memo written by the user.

  In order to achieve the third object of the present invention, (1) attribute information such as a writer, writing time or a combination thereof is prepared, and (2) handwritten information written on paper media, or information related thereto. Are classified and presented for the attribute information. As a result, it is possible to extract a paper document in which a specific person has made a note, extract a memo written on a specific paper document, and extract a document viewed by a specific person.

  Conventionally, in a situation where both a paper document and an electronic document exist, the information handwritten on the paper document has remained on the paper document without being digitized. However, such handwritten information is often important in public offices and business operations. According to the present invention, it is possible to link to related information from information such as a memo handwritten on a paper document. In addition, it is possible to extract a portion of a video of interest from a user's memo written without re-indexing video information of a meeting or the like and without being aware of a later search. Furthermore, it becomes possible to classify and organize information based on writing on paper documents. This makes it possible for each individual to search and acquire information and documents necessary for business with higher accuracy.

The configuration of the information management apparatus according to the embodiment of the present invention will be described with reference to FIG.
An information management apparatus as an embodiment of the present invention includes an information management server 101 and an information management client 102. The information management server unit manages and stores electronic documents and handwritten annotation information added thereto. In the same meeting as the storage means of the electronic document DB 103 for storing electronic documents, the paper / electronic document correspondence information DB 104 for storing correspondence between printed paper documents and electronic documents, and the annotation information DB 105 for storing handwritten annotation information An attribute information management unit 106 that performs processing related to attribute information between documents such as associating distributed materials; an attribute information DB 116 that stores attribute information between documents extracted by software such as groupware that performs schedule management; The document database management unit 107 executes search processing for various accesses such as a search for electronic document content and a search for annotation information, and returns an answer. The information management client 102 uses this information to digitize handwritten annotations on paper documents, start information retrieval including annotation information written on paper, and browse the obtained information. It is a terminal for doing.

  The paper document printing unit 108 is used to print an electronic document to be viewed as a paper document. For handwritten input IF109 that receives signals sent from pen-type input devices such as digital pens and tablets, and for annotations written on paper, attribute information such as writing time and writer is generated, and the annotations are converted into text. Handwritten annotation digitizing unit 110 that converts the data into a form that can be processed in an easy manner, an electronic information input IF 111 that makes changes to the electronic document, an execution mode that selects what processing should be activated for the annotation, or an execution mode that supports the selection Control unit 112, related information link generation unit 113 that associates related documents with computerized annotations, information search unit 114 that searches for related information using computerized annotations or related information links, and information that presents search results The presentation unit 115 and the information retrieval result temporary storage DB 117 for temporarily storing the results retrieved by the system. Thereby, the link to the related document using the handwritten annotation is realized. Both the information management server 101 and the information management client 102 are realized by a personal computer or a workstation. The four DBs of the information management server 101, that is, the attribute information DB 116, the electronic document DB 103, the paper / electronic document correspondence information DB 104, and the annotation information DB 105 are magnetic disk devices, and the attribute information management unit 106 and the document DB management unit 107 are in memory. Realized as a loaded program. Eight functions of the information management client 102, namely, a paper document printing unit 108, a handwritten input IF 109, a handwritten annotation electronic stock 110, an electronic information input IF, an execution mode control unit 112, a related information link generation unit 113, an information search unit 114, The information presentation unit 115 is realized as a program loaded on the memory. The information search result temporary storage DB 117 is realized as a magnetic disk device.

Next, a situation in which a paper document on which an electronic document is printed is annotated by handwriting or an original electronic document is annotated electronically using a keyboard will be described with reference to FIG.
First, the electronic document 201 is generated as a document P1 (202) using paper media by a printer. In the following, paper documents to be filled in with an electronic pen include special papers suitable for converting the written contents to electronic data, such as paper disclosed in Japanese Patent Laid-Open No. 2003-67363 (Patent Document 1) described later. Paper is used. On the other hand, handwritten information h1 (204) is entered by an electronic pen as disclosed in Japanese Patent Laid-Open No. 2003-67363 (Patent Document 1) or an electronic pen 203 as described later. This handwritten information is digitized and stored together with the electronic document 201 as e1 (205). In the next step in the business flow, handwritten information h2 (206) is added to the paper P1, and it is digitized in the same manner and stored as e2 (207) together with the electronic document 208. The information management client 102 performs printing of a paper document and digitizing handwritten information with an electronic pen.

  In the next step, the correction e3 (210) is electronically added to the electronic document 208 using the PC 209, and the electronic document 211 is generated. This PC also corresponds to the information management client 102. At this time, the correspondence between the latest paper document 211 and the electronic document 212 is broken, but the paper document 212 is invalidated here. If it is desired to validate the paper document, a copy of the electronic document is created and the correspondence between the paper document and the electronic document is maintained when the electronic document 208 is changed by the PC 209 as will be described later. You can also.

In order to execute the business flow, the electronic document 211 is printed, and a new paper document P2 (213) is generated. On the other hand, handwritten information h4 (214) is further added in the next step in the business flow. By digitizing the paper document 213, an electronic document 215 is generated, and by following the same procedure, a paper document 216 and an electronic document 217 are generated at the end of the work flow 1.
The electronic pen 203 is assigned unique ID information, and handwritten information is digitized together with this ID information. The ID information of the electronic pen 203 may be different for each electronic pen, or a plurality of electronic pens may share the same ID information as different for each department that performs each procedure of the business flow. .

  An example of the data structure when the handwritten annotation 204 is digitized is shown in the lower part of FIG. The writer's ID 128 obtained from an input device such as an electronic pen, the time 219 when the annotation was written, the stroke data 220 corresponding to the annotation (this is a coordinate series of the pen trajectory sampled by the electronic pen), the annotation Position 221 on a paper document with a note, code information 222 that the annotation has been converted into text by character string recognition, ID 223 for the paper on which the annotation is written, the ID 224 of the corresponding electronic document, and the location where the annotation corresponds in the electronic document 226, a link 226 to an electronic document or the like referred to when the annotation is written on a paper document, a link 207 to the annotation associated with the user by a lead-in line or the like. In the paper David Bargeron, Tomer Moscovich, “Reflowing Digital Ink Annotations,” Proceedings of the conference on Human factors in computing systems, pp. 385 393, 2003. It is clear that the annotation can be associated with the text by inputting the annotation on the screen using a pointing device such as a mouse or stylus and connecting the original text with a lead-in line. Has been. Using this, it is possible to correlate both using the coordinates of the place written with the digital pen and the coordinates of the printed or previously written character string.

  Here, as an example of the electronic pen used in the embodiment of the present invention, a digital pen disclosed in International Publication No. 01/71473 (Patent Document 3) will be described with reference to FIG. Reference numeral 301 denotes a digital pen, and reference numeral 302 denotes a camera device that acquires an image of a pattern for position detection printed on paper. When the paper 310 is enlarged (311), the small dots 313 are scattered inhomogeneously and printed. These dots are always printed with a certain distance from the intersection of the virtual grid lines 312 shifted up, down, left, or right, and a plurality of these dots are simultaneously referred to, for example, in a 6 × 6 range. A combination of the vertical and horizontal shift values of each dot provides a mechanism for providing absolute position information of a vast plane area. That is, by collecting images of a range including the plurality of dot patterns at a certain time interval by the camera 302 provided in the digital pen, it is possible to specify the position of the deviation combination in the space.

  Details of this principle are disclosed in International Publication No. 00/73983 (Patent Document 4). If this digital pen is applied to the document management method of the present invention, it is immediately extracted in which area of which paper media the user is filling in. Therefore, the user explicitly indicates the type of paper media. There is a merit that it is not necessary to send to. Therefore, this digital pen method can be said to be a method suitable for the document management method. Henceforth, it will be implemented when the digital pen method in which a paper document is printed on electronic pen paper and written by the user with the digital pen is applied. An example will be described. That is, in this embodiment, the handwritten annotation input using a digital pen as an input device connected to the handwritten input IF 109 of FIG.

  Another embodiment of the present invention will be described with reference to FIG. Here, a function for acquiring video and audio information other than documents using handwritten annotation information is realized. As described above, it is assumed that after participating in a conference and adding annotations to a material or taking notes in a notebook, video data recorded during the conference is searched from the handwritten information.

  As a configuration for that purpose, the information management server 401 and the information management client 402 are configured as in the first embodiment. The information management server 401 has a video / audio DB 403 for storing video and audio data in addition to document data. In the video / audio DB, for example, a video recording a conference pattern is stored using a photographing device such as a video camera installed in a conference room. By saving in digital video format such as Mpeg, it is possible to save together with the time when it was shot for each frame constituting the video. At the same time that such video / audio data is stored in the video / audio DB, the data is registered in the attribute information management unit 404, and the result is stored in the attribute information DB 405. The data format of the attribute DB will be described later. In addition, the attribute information management unit 404 determines whether to link video / audio data from the writer and writing time of the annotation information using, for example, a list and schedule of participants in the meeting for the meeting video. Also do.

Both the first embodiment and the second embodiment are composed of an information management client and an information management server. By taking a server / client configuration, the correspondence between paper documents and electronic documents, and history information of handwritten annotations on paper documents. Centrally manage. However, when realizing a small-scale system, the information management client and the information management server may be realized integrally.
It is also possible to distribute the document DB and the annotation DB on both the server side and the client side.
The above is the concept of the information management method using handwritten annotations in an environment where both paper media and electronic media exist according to the present invention.

Next, input of handwritten annotation information in the information management method according to the present invention will be described with reference to FIG.
The electronic document n502 stored in the electronic document DB 501 included in the information management server 101 is targeted. On the information management client 102 side, a user prints a paper document P1 503 using the paper document printing function 108. At this time, the correspondence relationship between the electronic document and the paper document is held in the paper / electronic document correspondence information DB 104 as will be described later. The user enters annotations 504 and 505 on the printed paper document 503 using the digital pen 301. In any case, writer 506, 507, writing time (or input time) 508, 509, stroke 510, 511, paper position 512, 513, code information 514, 515, paper ID 516, 517, electronic document ID 518, Each 519 holds an appropriate value. Here, in order to obtain the code information 514, the handwritten annotation digitizing 110 converts the written stroke information into a text code sequence using a character recognition technique. For character recognition, H.Bunke, PSPWang, “Handbook of Character Recognition and Document Image Analysis”, World Scientific, 1997 (Non-Patent Document 3), Chapter 7. “Overview and Synthesis of On-Line Cursive handwriting” It can be realized by the technique disclosed in “Recognition Techniques” and the like. When writing the second annotation 505, the two annotations are related by, for example, touching the area of the annotation 504 with a pen before writing and then writing the annotation. At this time, the order in which the annotations were written is identified from the value of the writing time of the annotation information, and the relationship between the annotations is expressed based on this order. A link 521 is made.
Alternatively, the user can electronically insert an annotation into the electronic document using the electronic information input IF111.

Next, the processing for linking from handwritten annotation information to related documents / information in the information management method according to the present invention will be described with reference to FIGS.
A paper document 602 is generated by printing the electronic document 601 stored in the electronic document DB 103 in the information management server 101. In response to this, the user writes the annotation 603. The handwritten annotation input on the computer through the handwriting input IF 109 of the information management client 102 is subjected to a handwriting annotation digitizing function 110 and a handwriting shape and position on the paper are detected 610, and annotation information such as 604 is obtained. Generated. Furthermore, the electronic document n601 is printed on the paper document from the information 605 on the correspondence relationship between the paper document and the electronic document registered in the paper / electronic document correspondence information DB 104 of the information management server 101. Recognize. As a result, the electronic document ID is recorded in the electronic document ID field 701 of FIG.

  Also, from the page information item 621 of the electronic document in the paper / electronic document correspondence DB 104, when the electronic document is printed on paper, refer to information that holds the position of the text or image that is a component of the electronic document on the paper. Is possible. Here, the on-paper coordinates 622 of the character “machine” printed on the paper document are obtained by referring to the on-paper position items. That is, by referring to the paper / electronic document correspondence DB from the on-paper position 623 of the annotation 603 written on the paper document 602, the character string printed at the position where the annotation is written is extracted as the corresponding text information. (702). This is performed by the electronic document / annotation corresponding part search processing 630.

Next, the execution mode control function 112 in the information management client 102 selects what action should be taken for the annotation. If symbol recognition is performed on the stroke information 703 and the symbol is recognized as a pre-defined symbol such as an ellipse or a wavy line, it can be used for an external database such as the Internet or a document stored in the electronic document DB 103 using the corresponding text information as a key. Then, text search is started, character string recognition is performed on the stroke information 703, and if it is recognized as a meaningful character string, processing such as storing in the recognition result in the code information 704 column as an annotation is performed. Interpreting the annotation according to its shape and code information obtained by character recognition, interpreting the meaning of the annotation, that is, whether it is a pointer that points to the text of the text or the character string itself, 710. Execution mode control can be realized by automatically switching these processes according to the results of symbol recognition and character string recognition in the execution mode control unit 112, and the user can select how annotations are handled each time. It is also possible to make it. The above handwriting shape / position detection processing 610, electronic document / annotation corresponding portion search 630, and annotation information interpretation processing 710 are executed by the handwritten annotation electronicization portion 710 in FIG.
The digitized information 705 related to the annotation generated in this way is stored in the annotation information DB 706 in the information management server 101.

Next, in the related information link generation in the information management client 102, the attribute information management function 106 of the information management server 101 is used to create a link between documents and data referred to at the same time, such as materials of the same conference. For example, if there is an electronic file that the writer was browsing on his or her PC at the time when the annotation was written, such as “meeting document_B.ppt”, the attribute information for registering it in the item 802 of the reference document A method for setting a link between documents by the management function will be described later. A related annotation is, as described above, if, for example, after writing this annotation, if someone wrote the annotation in another area on the paper after touching the position on the paper where the annotation exists with a digital pen, the related annotation information As a pointer to the newly written annotation information.
The annotation information created in this way is as shown by 801 in FIG.

Next, the information search function 114 of the information management client 102 will be described. For each item of the handwritten annotation information 801, a search question sentence is generated using the item as a query. That is,
(1) Documents where the writer of the annotation or the creator of the document matches
(2) Documents with the same writing time or containing annotations within a certain time difference, or the document itself
(3) Documents containing annotations whose stroke shape similarity is above the threshold
(4) Documents containing annotations whose deviation on the paper is within the threshold
(5) Documents with the same paper ID
(6) Paper document printed for the same electronic document
(7) Documents that include the text written in the annotation in the annotation or text
(8) Generate a search query such as a document having the same reference document. A part thereof is shown at 803.

  A new search question may be created by combining the above search questions by a logical operation. It is also possible to start the information search by presenting the search question created in this way to the user and the user explicitly selecting it. Alternatively, the search process for these search questions is voluntarily performed by the system, and the answer is stored in the document search result temporary storage DB 804. Alternatively, the information can be displayed on a PC display and the user can select necessary information. This eliminates the need for the user to enter a search question when he wants to obtain related information. A series of search result storage and result display are performed by the information presentation function 115 of the information management client 102.

  The attribute information management function 106 of the information management server 101 will be described with reference to FIG. The attribute information DB 116 of the information management server 101 stores information on various business spaces and places. For example, information related to a meeting place is represented by a table such as 901. That is, the conference name 902, start time 903, end time 904, venue 905, participants 906, distribution material 907, slide 908, recording video 909, recording audio 910, reference material 911, and the like. Each of these items is a pointer to that entity. For example, the distribution material document 920 has a pointer to the electronic document 913 stored in the electronic document DB 912. The electronic document 913 also has a pointer to information about the annotation 930. By following this pointer, a document distributed at the same meeting or the time at which the annotation was written is displayed for a certain annotation. You can create links to other slides.

In the recorded video 909, a pointer to a video file with a time index is recorded, which is taken by a video camera or the like. The recorded voice 910 is provided with a pointer to a voice file with a time index recorded by a recording device such as a voice recorder. Further, in the reference material 911, for example, a reference document described in a distribution material 907, a slide 908, or the like is automatically extracted and a pointer to the file is stored. For this purpose, Takahiro Akihiro, Hayakawa Kimizumi, Katayama Norio, Oyama Keizo, Adachi Akira, “Extraction and matching of bibliographic information from bibliographic image data,” Information Processing Society of Japan, “Information Science Fundamentals”, No. 045 -6, 1997 (Non-Patent Document 5) is known.
The above is the first embodiment of the present invention in which information acquisition from handwritten annotations is taken as an example.

The second embodiment of the present invention, in which information other than documents such as conference video and audio is also stored and searched and accessed from handwritten annotations including the information, is also implemented in the same manner as described above. That is, after the annotation is written with the digital pen, until the annotation information is generated from the attribute information such as the positional information and writing time of the annotation, the method shown in FIGS. Information is generated. Videos recorded in the same conference are registered in the recorded video column 921 of the attribute management DB. This video entity is stored in the video / audio DB 403 of the information management server. On the contrary,
(1) Search for the part of the video (sound) recorded at the same time that the annotation was written, and play a certain amount of time from there.
(2) Search and play back the portion of the video (audio) that was recorded at the same time that the annotation was written,
The search sentence 1002 is generated according to the above. On the other hand, a search sentence may be presented to the user and the user may select the search sentence. Alternatively, information search may be performed in advance and the result may be stored in the document search result temporary storage DB 1003. Having written notes and annotations during the meeting means that there was some interest, and by just specifying the notes later, the video and audio of the meeting can be reproduced only for that scene.

  As a hardware configuration for realizing the information management method described above, a configuration as shown in FIG. 11 is conceivable. The information management client 1101 and the information management server 1102 are connected through a network 1103. The information management client is connected to a computing device 1110, a storage device 1111, a printing IF 1112, a handwriting input IF 1113, and an electronic information input IF 1114 through a bus 1115. A printer 1116 is connected to the print IF, an electronic pen 1117 is connected to the handwriting input IF, and a PC 1118 is connected to the electronic information input IF. The PC 1118 may be a document electronic client itself. The storage device stores an execution mode control program 1120, a related information link creation program 1121, a handwritten annotation digitization program 1122, an information search program 1123, and an information presentation program 1124, which are executed by the arithmetic device 1110. Further, there is a document search result temporary storage DB 1125 as an external storage.

The information management server 1102 stores an attribute information management program 1132 and a document DB management program 1133 in the storage device 1131 and is executed by the arithmetic device 1130. External storage includes an electronic document DB 1134, a paper / electronic document correspondence information DB 1135, an annotation information DB 1136, an attribute information DB 1137, and a video / audio DB 1139.
The above is the description of the embodiment according to the present invention.

  In a business environment where electronic documents and paper documents coexist, and in situations where not only documents but also other media information such as conference video and audio are mixed, it is subject to management including writing on conventional paper media. An information management system can be realized.

It is the figure which showed the function structure of the document information management system from the handwritten annotation which is an Example of this invention. It is the figure which showed the mode of the correlation of the paper document and electronic document in embodiment of this invention according to the business flow. It is a figure explaining the principle of the electronic pen used by embodiment of this invention. It is the figure which showed the function structure of the document / video / audio information management system from the handwritten annotation which is an Example of this invention. It is a figure which shows the annotation information produced | generated from the handwritten annotation used in embodiment of this invention. It is a figure which shows the outline | summary of the process until the place which acquires a corresponding | compatible electronic document from a handwritten annotation regarding the method of performing an information search and acquisition from the handwritten annotation used in embodiment of this invention. It is a figure which shows the outline | summary of the process until the point which produces | generates the annotation information regarding a handwritten annotation regarding the method of performing an information search and acquisition from the handwritten annotation used in embodiment of this invention. It is a figure which shows the outline | summary of the production | generation of a search sentence and information acquisition from handwritten annotation information regarding the method of performing information retrieval and acquisition from the handwritten annotation used in the embodiment of the present invention. It is the figure which showed the structure of the attribute management DB used by embodiment of this invention. It is a figure which shows the outline | summary of the production | generation of a search sentence and information acquisition from handwritten annotation information regarding the method of performing the information search and acquisition including other than a document from the handwritten annotation used by embodiment of this invention. It is a figure which shows the structure of the information management system which concerns on this invention.

Explanation of symbols

101: Information management server, 102: Information management client, 103: Electronic document DB, 104: Paper / electronic document correspondence information DB, 105: Annotation information DB, 106: Attribute information management function, 107: Document DB management function, 108: Paper document printing function, 109: Handwritten input IF function, 110: Handwritten annotation digitizing function, 111: Electronic information input IF function, 112: Execution mode control function, 113: Related information link generation function, 114: Information search function, 115 : Information presentation function, 116: Attribute information DB, 117: Information search result temporary storage DB,
120: Copy of electronic document, 121: Paper document P3, 122: Electronic pen, 123: Handwritten information h6, 124: Paper document, 125: Handwritten information h7, 126: Electronic document, 127-132: Time stamp, 133: Original Storage DB,
201: electronic document, 202: paper document, 203: digital pen, 204: handwritten annotation, 205: digitized handwritten annotation, 206: handwritten annotation, 207: digitized handwritten annotation, 208: electronic document, 209: personal computer , 210: electronically entered annotation, 211: electronic document, 212: paper document, 213: paper document, 214: handwritten annotation, 215: electronic document, 216: paper document, 217: electronic document, 218: writer of annotation information Item, 219: writing time item of annotation information, 220: stroke item of annotation information, 221: position item on paper of annotation information, 222: code information item of annotation information, 223: paper ID item of annotation information, 224: annotation information Electronic document ID item, 225: Corresponding text item of annotation information, 226: Reference document item of annotation information, 227: Related annotation item of annotation information,
301: Digital pen, 302: Camera device, 310: Digital paper, 311: Digital paper and its enlargement, 312: Virtual grid lines, 313: Small dots,
401: Information management server 402: Information management client 403: Video / audio DB 404: Attribute information management unit 405: Attribute information DB
501: electronic document DB, 502: electronic document, 503: paper document, 504: handwritten annotation, 505: handwritten annotation, 506, 507: writer item of annotation information, 508, 509: writing time item of annotation information, 510, 511: Annotation information stroke item, 512, 513: Annotation information position item on paper, 514, 515: Annotation information code information item, 516, 517: Annotation information paper ID, 518, 519: Annotation information electronic Document ID, 520: Annotation information related annotation item, 521: Link expressing degree of association between annotation information,
601: Electronic document, 602: Paper document, 603: Handwritten annotation, 604: Annotation information, 605: Electronic document ID in paper / electronic document correspondence DB, 610: Handwriting shape / position detection processing, 621: Paper / document correspondence data Pointer to page information, 622: position where text is printed on paper, 623: position on annotation paper, 630: electronic document / annotation corresponding part search processing,
701: electronic document ID, 702: corresponding text, 703: annotation stroke, 704: code information of lunch information, 705: annotation information, 706: annotation information DB, 710: annotation information interpretation processing,
801: Annotation information, 802: Reference document of annotation information, 803: Retrieval text of related information, 804: Document search result temporary storage DB, 810: Related annotation information item of annotation information,
901: Example of attribute information, 902: Meeting name item of attribute information, 903: Start time item of attribute information, 904: End time item of attribute information, 905: Location item of attribute information, 906: Participant item of attribute information , 907: Distributing material item of attribute information, 908: Slide item of attribute information, 909: Recorded video item of attribute information, 910: Recorded audio item of attribute information, 911: Reference material item of attribute information, 912: Electronic document DB 913: Electronic document, 920: Electronic document ID of distribution material, 921: Video file name of recorded video, 930: Annotation information,
1001: Annotation information, 1002: Search query for related information, 1003: Document search result temporary storage DB,
1101: Information management client, 1102: Information management server: 1103: Network, 1110: Computing device, 1111: Storage device, 1112: Print IF, 1113: Handwriting input IF, 1114: Electronic information input IF, 1115: Bus, 1116: Printer, 1117: Digital pen, 1118: Personal computer, 1120: Execution mode control program, 1121: Related information link generation program, 1122: Handwritten annotation digitization program, 1123: Information search program, 1124: Information presentation program, 1125: Document Search result temporary storage DB, 1130: arithmetic device, 1131: storage device, 1132: document DB management program, 1133: attribute information management program, 1134: electronic document DB, 1135: paper / electronic document correspondence information DB, 1136: annotation information DB, 1137: attribute information DB, 1138: video / audio DB.

Claims (3)

A digitized character or image added on a paper medium on which the content of the electronic document is printed as digitized data, and at least attribute data including the writing time at which the additional writing is performed is added to the digitized data ; ,
A database that holds the electronic document and the digitized data in association with each other;
By searching the database using the writing time added to the digitized data as a search key, the digitized data to which attribute data including the writing time related to the search key is added is obtained, and the digitized data And a link generation unit that forms a link between the acquired digitized data ,
An information management apparatus for presenting an electronic document associated with the acquired digitized data by following the link. The information management device according to claim 1,
The information management apparatus according to claim 1, wherein the attribute data includes audio or video information. The information management device according to claim 1,
As the search, at least one of data referenced at the same time as the writing time, data recorded at the same time as the writing time, and data recorded after the writing time is searched. .

Images