JP2007011683A - Document management support device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document management support device supporting management of a document to which handwritten annotation is added. <P>SOLUTION: In receipt of image data representing a document in which a print character and a handwritten character are mixed, a control part 21 stores the received image data in a storage part 25. Document image data are generated from the image data. The document image data are divided into image data for a print character area, in which a print character is written, and image data for a handwritten character area, in which a handwritten character is written. In each area, character recognition processing is carried out for generating text data, and an identifier, which shows a storage place of the generated text data and the image data including the text data, and an identifier, which shows whether the area is the print character area or the handwritten character area, are stored in an index table. When a keyword is inputted in an operation part 24, the index table is searched for displaying matching image data in a display part 23. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for supporting management of a document to which handwritten annotation is added.

  The full text search performed on the digitized document is performed based on an index table created in advance. At this time, an annotation written by hand may be added to the document to be digitized. This annotation often indicates an important point of a sentence, a content that calls attention, or an important matter.

Japanese Patent Application Laid-Open No. 2004-133260 discloses a technique for performing character recognition processing on a marked portion and registering it in an index when a paper document in which an important part is marked with a highlighter pen is accumulated.
JP-A-5-233705

  However, in the technique disclosed in Patent Document 1, since only the characters at the location marked with the highlighter are registered as the index, no index is created for the characters at the location not marked with the highlighter. . Therefore, in order to search for a desired character, it is necessary to mark it with a highlighter. On the other hand, a commonly used annotation adding method is a method of writing characters in the margin of a document by handwriting. However, in the invention described in Patent Document 1, even if an annotation is added, it is troublesome because a mark must be additionally provided thereon.

  The present invention has been made in view of such circumstances, and its purpose is to recognize printed characters included in paper documents and handwritten characters added as annotations and create an index for each character. Another object of the present invention is to provide a document management support apparatus that can perform a search based on a created index.

  In order to solve the above problems, the present invention scans a document and obtains document image data representing the contents of the document, and the document image data generated by the document image data generation unit. From the storage means for storing and the document image data generated by the document image data generation means, the image data of the type region where one or more type characters are written, and one or more handwritten characters are written. Region separating means for cutting out image data of a handwritten area, character recognition processing means for performing character recognition processing on each of the image data of the printed area and the image data of the handwritten area, and outputting a recognized character string; A storage location identifier indicating a storage location of each image data stored in the storage unit, a recognized character string output from the character recognition processing unit, and an image data in the handwriting area. Providing document management support apparatus comprising an index information storage means for respectively associating and storing a character identifier indicating whether a character string representing the image data if it were a string print region representing the data.

  According to this document management support apparatus, a print area in which printed characters are written and a handwritten area in which handwritten characters are written are separated from document image data, character recognition is performed for each character area, and an index is set. Since it is created, an annotation written by hand as a memo in the margin of a paper document can also be set as an index creation target. Therefore, an index table for type characters and handwritten characters can be created. The index table also stores addresses at which document image data containing each character is stored in association with printed characters and handwritten characters stored in the index table. Refer to the index table. Thus, the location of the predetermined character can be specified.

  In a preferred aspect of the present invention, the region separating unit determines, from the document image data, a unit for specifying a region of interest and a distance between two characters adjacent to each other in the region of interest for each set of the two character strings. When the degree of variation of each distance obtained does not exceed a predetermined range, it is determined that the region of interest is a type region, while when the amount of variation exceeds a predetermined range, the region of interest is And a means for determining that it is a handwritten area.

  In general, the interval between two adjacent characters in a printed character is constant, but the interval between two adjacent characters in a handwritten character is not constant. Therefore, by determining whether or not the character spacing of the character string in the region extracted by the region separation means is constant based on a predetermined value, the region can be any character string of type and handwriting. Whether the image data is represented can be determined.

  Further, a search means for searching the index information storage means by comparing a character string input as a search key with a recognized character string stored in the index information storage means, and the search means You may make it comprise the display means which displays a result. According to this, it is possible to search the index table based on the character string serving as the search key and display the search result.

  Furthermore, it comprises designation means for designating from which area of the handwriting area and the type area the character string that serves as the search key, and the search means for the area designated by the designation means, The search may be performed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing a configuration example of a document management support system 1 which is an embodiment of a document management support apparatus according to the present invention. An image reading apparatus 10 in FIG. 1 is a scanner apparatus having an automatic paper feed mechanism such as an ADF (Auto Document Feeder), for example, and optically reads a paper document set in the ADF page by page, and converts the read image into an image. Corresponding image data is transferred to the document management support apparatus 20 via the communication line 12. As the communication line 12, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like can be used. In this embodiment, a LAN is used as the communication line 12.

FIG. 2 is a block diagram illustrating a hardware configuration of the document management support apparatus 20.
The control unit 21 is, for example, a CPU (Central Processing Unit), and controls each unit of the document management support apparatus 20 by executing various software. A communication interface (hereinafter referred to as “IF”) unit 22 is connected to the image reading apparatus 10 via the communication line 12, receives image data transmitted from the image reading apparatus 10 via the communication line 12, and Transfer to the control unit 21.

  The display unit 23 is, for example, a liquid crystal display and its drive circuit, and displays an image corresponding to the data transferred from the control unit 21. The operation unit 24 is, for example, a keyboard and a mouse provided with a plurality of operation elements (not shown), and outputs data (hereinafter, operation content data) corresponding to the operation contents of these operation elements to the control unit 21.

  The storage unit 25 includes a volatile storage unit 25a and a nonvolatile storage unit 25b. The volatile storage unit 25 a is, for example, a RAM (Random Access Memory), and is used as a work area of the control unit 21. The nonvolatile storage unit 25b is, for example, a hard disk, and stores a program such as an analysis processing program P1 and an index table T1.

Hereinafter, an operation in which the control unit 21 performs image data analysis processing on the documents 30 to 32 illustrated in FIGS. 3A to 3C will be described.
When the power (not shown) of the document management support apparatus 20 is turned on, the control unit 21 reads the analysis processing program P1 from the nonvolatile storage unit 25b and executes the operation of the flowchart shown in FIG.
First, when the user sets the documents 30 to 32 in the ADF of the image reading apparatus 10 and performs a predetermined operation, images representing the documents 30 to 32 are sequentially read by the image reading apparatus 10 and image data corresponding to the documents is read. Are sequentially transmitted from the image reading apparatus 10 to the document management support apparatus 20 via the communication line 12.

  When the control unit 21 receives the image data sent from the image reading device 10 via the communication IF unit 22 (step SA10), the control unit 21 stores the received image data in the storage unit 25 (step SA12). Further, the control unit 21 generates document image data from the image data of the documents 30 to 32 (step SA14). Next, the control unit 21 cuts out from the document image data, the image data of the type region in which type characters are written, and the image data of the handwritten region in which handwritten characters are written (Step SA16).

  The type region and the handwriting region are cut out as follows. First, each pixel represented by the document image data is scanned in the horizontal direction, and when the distance between two adjacent characters, that is, the width of the arrangement of consecutive white pixels is smaller than a predetermined value X, Are replaced with black pixels. The predetermined value X is approximately matched with a value assumed as a distance from the adjacent character. Similarly, each pixel is also scanned in the vertical direction, and when the width of the arrangement of consecutive white pixels is smaller than a predetermined value Y, those consecutive white pixels are replaced with black pixels. This predetermined value Y is generally matched with a value assumed as a character line interval. As a result, a region filled with black pixels is formed. FIG. 5 shows a state in which the above-described replacement processing is performed on the image of the document 30. In this figure, regions L1 to L6 filled with black pixels are formed. In addition, since the same operation | movement is performed also about the image of the documents 31 and 32, the following operation | movement description is performed on the image of the document 30 as object.

  When an area filled with black pixels is formed, it is now determined whether each area is a type area or a handwriting area. In this determination, first, an attention area to be processed is specified, black pixels replaced in the specified area are returned to white pixels, and the original drawing content is restored. Then, the pixels in the region are scanned in the horizontal direction, and it is determined whether or not the degree of variation in pitch of consecutive white pixels is smaller than a predetermined value. In general, since the interval between two adjacent characters is substantially constant in a region where printed characters are written, the degree of variation in the pitch of consecutive white pixels becomes smaller than a predetermined value. On the other hand, since an interval between two adjacent characters is not constant in a region where a handwritten character is written, the degree of variation in pitch between consecutive white pixels becomes larger than a predetermined value. In the example of the determination for the areas L1 to L6 shown in FIG. 5, the determination result that the areas L1 to L5 are type areas is given, and the determination result that the area L6 is a handwriting area is lower. Is done.

  Then, an OCR process is performed on each area to perform a character recognition process to generate type character text data from the type area and hand letter text data from the hand area (step SA18). Subsequently, the generated text data is subjected to morphological analysis, and the text data corresponding to the noun is extracted from each text data (step SA20) and stored in the index table T1 (step SA22).

  Here, the index table T1 will be described with reference to FIG. As shown in FIG. 6, the index table T1 includes “character string”, “image data address”, and “flag” fields. The noun extracted in step SA20 is stored in the character string field. In the image data address field, an identifier indicating the storage location of the image data of the document 30, that is, the storage location of the image data in the nonvolatile storage unit 25b is stored as an image data address. The flag field stores an identifier for identifying whether the extracted text data is printed character text data or handwritten character text data. In the present embodiment, the flag field stores “1” if it is printed character text data, and “0” if it is handwritten character text data.

As a result of the above, text data such as “Aiueo” and “Kakikukeko” is stored in the character string field. Further, in association with these text data, the image data address “01” of the image data of the document 30 is stored in the image data address field.
Furthermore, since the area including the text data of “Aiueo” and “Kakikukeko” is determined to be the type area (L1) in the above step SA16, “1” indicating the type text data is respectively set. Is stored in the flag field in association with the text data. Similarly, the text data “Iroha” is stored in association with the text data “Iroha” and the identifier “01” indicating the storage location of the image of the document 30. Since the text data is included in the area L6 and the area L6 is determined to be a handwriting area in the above-described step SA16, “0” is stored in the flag field in association with the text data.

Next, a search operation in which the control unit 21 searches the index table T1 based on the keyword will be described.
FIG. 7 is a flowchart showing the search operation.
First, the control unit 21 sets the corresponding type character string number x, the corresponding handwritten character string number y, and the corresponding image data number z as variables, and inputs initial values to the respective variables (step SB10).

  Subsequently, the keyword input screen 40 shown in FIG. 8 is displayed on the display unit 23, and the user is allowed to input keywords (step SB12). As shown in FIG. 8, the keyword input screen 40 includes an input box 41 and a “OK” button BT1. A user who visually recognizes the keyword input screen 40 can specify a keyword by operating the operation unit 24 and pressing the enter button BT1. At this time, it is assumed that the character string input by the user is “Aiueo”. The operation unit 24 supplies character data representing “Aiueo” to the control unit 21.

When receiving the character data, the control unit 21 searches the character string field of the index table T1 based on the content of the character data, that is, “Aiueo” (step SB14). At this time, there are three records in which the text data identical to the character data “Aiueo” is stored in the character string field of the index table T1.
The control unit 21 sequentially reads out these record groups to the volatile storage unit 25b (step SB16). FIG. 9 shows an example of the read record group.

  Subsequently, the control unit 21 sets the number of records in which “1” is stored in the flag field of the extracted record group as the number of type character strings x, and the number of records in which “0” is stored as the corresponding number. Substituted in the number of handwritten character strings y (step SB18). That is, since the number of records in which “1” is stored is two and the number of records in which “0” is stored is one in the flag field of the extracted record group, the control unit 21 determines the number of corresponding type character strings. It is assumed that x = 2 and the number of handwritten character strings y = 1.

  Next, the control unit 21 detects how many different addresses are stored in the address field of the extracted record group, and substitutes the detected number for the corresponding image data number z (step SB20). At this time, there are two different addresses, “01” and “03”. Therefore, the control unit 21 sets z = 2.

  Subsequently, the control unit 21 reads out the image data in ascending order of addresses in the extracted address field of the record group (step SB22). That is, since the youngest address is “01”, the control unit 21 first reads the image data stored at the address 01. At this time, since the image data stored at the address 01 is the image data of the document 30, the image data of the document 30 is read out by the control unit 21.

Then, the control unit 21 displays on the display unit 23 the search result screen 40 generated from the image data of the document 30, the above-mentioned corresponding type character string number x, the corresponding handwritten character string number y, and the corresponding image data number z ( Step SB24).
FIG. 10 shows the search result screen 40. As shown in the figure, the search result screen 40 includes a display area 42, a search result column, and a “next image display” button BT2. In the display area 42, an image representing the document 30 is displayed. In the search result column, the number x of the corresponding type character string, the number y of the corresponding handwritten character string, and the number z of the corresponding image data are displayed.

  When the user viewing the search result screen 40 presses the next image display button BT2, the control unit 21 stores the image data stored at the second youngest address in the address field of the record group extracted in step SB16. And the image of the read image data is displayed in the area 42. At this time, since the image stored at the second youngest address is the image of the document 32, the image of the document 32 is displayed in the area 42. As described above, the control unit 21 sequentially displays the corresponding images in the area 42.

  As described above, in the present embodiment, character recognition is also performed for annotations written in handwriting on a document and an index table is created, so that not only typed characters written in the document but also handwritten characters You can also search for. Therefore, such information can be presented to the user as a search result when important information is included in the handwritten character or when a comment desired to be communicated to others is included.

[Modification]
The present invention can be implemented in various forms other than the above-described embodiments.
(1) Since handwritten characters are wrinkled by people, they may be recognized as characters different from the original. Therefore, the non-volatile storage unit 25b stores a similar character dictionary table in which characters that are likely to be erroneously recognized, such as “tsu” and “shi” or “so” and “n”, are associated with each other. When text data is stored in the index table T1, text data created based on the similar character dictionary table may also be stored. Specifically, when a character string “page” is acquired as a result of character recognition in step SA18 (see FIG. 4), text data indicating the character string “page” is stored in the index table T1 in step SA22. At the same time, the text data indicating the character string of “page” converted based on the similar character dictionary table is stored in the index table T1. An index table T1 at this time is illustrated in FIG. In this way, even if text data indicating a character string different from the original character string is stored in the index table T1 due to human handwriting characters, the text data converted by the similar character dictionary table is also stored in the index table. Since it is stored in T1, it can be prevented that the image data that should be extracted in the search operation is not extracted due to the influence of handwriting characters.

(2) Further, a table (synonym table) in which synonyms are associated may be stored in the non-volatile storage unit 25b, and the index table T1 may be searched with keywords expanded based on the synonym table. . For example, when “monochrome” and “monochrome” are stored as synonyms in the synonym table and the keyword input by the user on the keyword input screen 40 (see FIG. 8) is “monochrome”, The control unit 21 searches the index table T1 based on keywords representing “monochrome” and “monochrome”. In this way, it is possible to search for image data under expanded conditions.

(3) In the present embodiment, the case where the control unit 21 performs a search operation using only character string data has been described. However, in addition to character string data, type character text data or handwritten characters You may make it search based on designation | designated from which of text data to search. As an example of this case, the control unit 21 displays the keyword input screen 50 shown in FIG. 12 in step SB12 described above. As shown in the figure, in the keyword input screen 50, a check box constituted by a printed character check box 51 and a handwritten character check box 52 is added to the keyword input screen 40 (see FIG. 8). The control unit 21 searches according to the contents of these check boxes. For example, when the character string input by the user in the input box 41 of the keyword input screen 50 is “Aiueo” and the type character check box 51 is checked, the control unit 21 displays the text “Aiueo” in the character string field. The data is searched for a record in which “1” is stored in the flag field.
By doing in this way, it is possible to perform a search under conditions that match the search purpose of the user.

(4) In the present embodiment, a case has been described in which type characters and handwritten characters are stored in the same table (index table T1), but each may be stored in a separate table.

(5) In the present embodiment, the case where the image reading device 10 and the document management support device 20 are configured as separate hardware has been described. However, both may be configured as integrated hardware. Good. At this time, the communication line 12 serves as an internal bus for connecting the image reading apparatus 10 and the document management support apparatus 20 within the hardware.

1 is a block diagram showing an overall configuration of a system according to an embodiment of the present invention. It is a block diagram which shows the hardware constitutions of the document management assistance apparatus which concerns on the same embodiment. FIG. 3 is a diagram illustrating a format of a document configuration according to the embodiment. It is the figure which showed the flowchart which showed the operation | movement which the control part which concerns on the same embodiment performs. It is the figure which showed the state which substituted to the black pixel for the document concerning the embodiment. It is the figure which showed the format of the index table which concerns on the same embodiment. It is the figure which showed the flowchart which showed the operation | movement which the control part which concerns on the same embodiment performs. It is the figure which showed the keyword input screen which concerns on the same embodiment. It is the figure which showed the record group which the control part which concerns on the same embodiment read. It is the figure which showed the search result screen which concerns on the same embodiment. It is the figure which showed the format of the index table which concerns on the modification of this invention. It is the figure which showed the keyword input screen which concerns on the modification of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Document management support system, 10 ... Image reading device, 12 ... Communication line, 20 ... Document management support device, 21 ... Control part, 22 ... Communication IF part, 23 ... Display unit, 24 ... Operation unit, 25 ... Storage unit, 25a ... Volatile storage unit, 25b ... Non-volatile storage unit, 26 ... Bus

Claims (4)

Document image data generation means for scanning a document and obtaining document image data representing the content of the document;
Storage means for storing the document image data generated by the document image data generation means;
From the document image data generated by the document image data generation means, image data of a type region where one or more type characters are written, and an image of a handwritten region where one or more type characters are written Area separation means for cutting out data;
Character recognition processing means for performing character recognition processing on each of the image data of the type region and the image data of the handwriting region, and outputting a recognized character string,
A storage location identifier indicating the storage location of each image data stored in the storage means, a recognized character string output from the character recognition processing means, and whether the recognized character string is a character string representing image data in a handwriting area A document management support apparatus comprising: index information storage means for storing a character identifier indicating whether it is a character string representing image data in a print area in association with each other. The region separating means specifies a region of interest from the document image data;
A distance between two characters adjacent to each other in the attention area is obtained for each set of the two character strings, and when the degree of variation in the obtained distances does not exceed a predetermined range, the attention area is a type area. The document management support apparatus according to claim 1, further comprising: a unit that determines that the attention area is a handwritten area when the degree of variation exceeds a predetermined range. Search means for searching index information by collating a character string input as a search key with a recognized character string stored in the index information storage means;
The document management support apparatus according to claim 1, further comprising: a display unit that displays a search result of the search unit. Comprising designation means for designating from which of the handwriting area and the type area the character string to be the search key is searched;
4. The document management support apparatus according to claim 3, wherein the search means performs the search for the area designated by the designation means.

Images