KR100655916B1 - Document image processing and verification system for digitalizing a large volume of data and method thereof - Google Patents

Abstract

The present invention analyzes and analyzes a series of document images obtained during a scanning process, and analyzes and corrects document image structure analysis and text area segmentation module for automatically batch segmenting a text area, and verifying and correcting the result of automatic segmentation of the document image through an image-based interface. A document image segmentation verification module, a character clustering module for determining a text code by applying intelligent character recognition to each character region of the document image, and generating a cluster image by collecting character regions of the same code; A character cluster unit verification and text conversion module for granting a batch text code by verifying the character recognition result in a cluster unit for the character recognition, and a character unit code input for individually inputting a small character area for which the text code is not determined in the character recognition. Module and the batch text code assignment or the individual input A cluster unit verification and correction module for first verifying and correcting each character region of the input document image through a cluster unit interface, and each character region of the document image input through the batch text code assignment or the individual input Text input unit verification and correction module for verifying and correcting the document secondly through the text line unit of the document, and inputting various tag information and additional information of the document through the interface representing the character area of the document image. The present invention relates to a document image processing and verification system for digitization of massive data including tagging and additional information input modules.

According to the present invention, it is possible to obtain high-quality digital text information from a large amount of document data at a high speed, and to increase accuracy and processing rate by enabling character cluster unit verification and batch text conversion, and extracting text information from document images. Data processing and visual verification process can be performed in clusters to reduce error detection time.

Digitizing Old Documents, Character Recognition, Clustering, Verification and Correction, Text Code, Unicode

Description

Document image processing and verification system for digitization of massive data and its method {DOCUMENT IMAGE PROCESSING AND VERIFICATION SYSTEM FOR DIGITALIZING A LARGE VOLUME OF DATA AND METHOD THEREOF}

1 is a view showing an example of a document image obtained through a scanner of a conventional manuscript, woodblock, letterpress document.

Figure 2 is a flow diagram of a manual hand-based digitization work process by a large-scale manpower in the prior art.

3 is a flowchart of a digitization work process using character recognition according to the prior art.

4 is a structural diagram of a document image processing and verification system for digitizing massive data according to an embodiment of the present invention.

5 is a diagram illustrating an example of an analysis and segmentation result of a document image including various miscellaneous and tilting in a document image processing and verification system according to an embodiment of the present invention.

6 is a diagram illustrating a hierarchical structure of a document image in a document image processing and verification system according to an embodiment of the present invention.

7 is a view showing an example of a structural analysis and character region segmentation results from woodblock, manuscript and letterpress images in a document image processing and verification system according to an embodiment of the present invention.

8 is a diagram illustrating an example of a toolbar used for verifying and correcting a document image segmentation result in a document image processing and verification system according to an embodiment of the present invention.

FIG. 9A is a diagram illustrating an application based on character recognition and recognition reliability of a text area in a document image processing and verification system according to an embodiment of the present invention. FIG.

FIG. 9B is a view illustrating rejection by application of character recognition and recognition reliability in a text area in a document image processing and verification system according to an embodiment of the present invention. FIG.

10 is a view showing an example of a character clustering module applied to various document data in a document image processing and verification system according to an embodiment of the present invention;

FIG. 11 is a diagram illustrating an example of a result of generating a cluster image and an error distribution for a Chinese character '之' class in a document image processing and verification system according to an embodiment of the present invention.

12 is a diagram illustrating an example of a character cluster image verification and text conversion interface in a document image processing and verification system according to an embodiment of the present invention.

FIG. 13A illustrates an example of an initial screen (one-to-one correspondence between a text area and a text code) of a manual input interface in a document image processing and verification system according to an embodiment of the present invention. FIG.

13B illustrates an example of an input mode of a manual input interface in a document image processing and verification system according to an embodiment of the present invention.

FIG. 14 is a diagram illustrating an example of a character cluster unit verification and correction interface (a text area listing screen of a Chinese character '中' class) in a document image processing and verification system according to an embodiment of the present invention; FIG.

15 is a view illustrating an example of a document image text line unit verification and correction interface of various data in a document image processing and verification system according to an embodiment of the present invention;

FIG. 16A illustrates an example of a document image-based interface for assigning structural tag information of a document in a document image processing and verification system according to an embodiment of the present invention; FIG.

16B is a diagram illustrating spacing information automatically extracted through structural analysis of a document image in a document image processing and verification system according to an embodiment of the present invention;

17A illustrates an example of a final text file to which an XML tag and additional information are attached in a document image processing and verification system according to an embodiment of the present invention;

17B is a diagram illustrating an example of an error file generated during Unicode and Shift + JIS conversion in a document image processing and verification system according to an embodiment of the present invention.

18 is a structural diagram of a document image processing and verification system for digitizing massive data according to another embodiment of the present invention.

19 is a flowchart of a document image processing and verification method for digitization of massive data according to the present invention.

<Description of the symbols for the main parts of the drawings>

410: Document image structure analysis and text area segmentation module

420: document image segmentation verification module

430: character clustering module

440: character cluster unit verification and text conversion module

450: character unit code input module

460: Cluster Validation and Calibration Module

470: Text line unit text verification and correction module

480: Content tagging and additional information input module

The present invention relates to a document image processing and verification system for the digitization of a large amount of data, and more particularly to high-performance document image processing to obtain high-quality digital text information at a high speed from various document data such as manuscript, woodblock, and printed text. The present invention relates to a document image processing and verification system for digitizing a large amount of data, which combines an automatic technology including intelligent character recognition with an image-based verification and correction interface.

1 is a view showing an example of a document image obtained through a scanner of a conventional manuscript, woodblock, and letterpress document, Figure 1 (a) is a seungwonwon diary (English-Japanese version) possessed by the National History Compilation Committee It is a document image of a manuscript, and FIG. 1 (b) is a document image of a woodblock copy of the Mungun Chonggan (文集) owned by the National Culture Promotion Committee, and (c) of FIG. I) is a document image of a manuscript of FIG. 1 (d) is a document image of a typeface of Lee Kwang-su's collection published in 1972.

Document data such as historical feeds belonging to the conventional manuscripts or woodblocks are generally digitized manually by specialists due to document damages due to long collection periods and the peculiarity of ancient Chinese characters. In other words, digitization is performed using a manual method that checks and inputs each character visually.In exceptional cases, if the preservation status is good and the reading is easy, the automation method that utilizes the character recognition (OCR) technology in the field of pattern recognition is considered. However, the accuracy is lower than the manual digitization work. Therefore, the conventional method of digitizing documents by manual has the problems of huge manpower cost, processing time, and limited data processing scale.

Figure 2 shows a flow diagram of a manual hand-based digitization work process by a large-scale manpower in the prior art.

As shown, the document to be digitized into document image through document copying and scanning (S210) by mobilizing a large manpower to manually input data (S220), also by mobilizing a large manpower to verify and correct every letter By (S230) it will generate a documented DB (S240).

In order to improve the shortcomings caused by manual data input, verification and correction, the digitalization of printed documents having a formal structure such as typefaces at home and abroad has been studied a lot of automation technologies and commercialized products have been released.

3 is a flowchart of a digitization work process using commercial character recognition according to the prior art. As shown, the document to be digitized into a document image through scanning (S310), convert it to text information using OCR (S320), verify and correct the converted information (S330) to generate a documented DB It becomes (S340).

In the case of mechanically generated typefaces, such conventional automatic character recognition technology can be applied. However, the technology for documents with atypical structures such as manuscripts and woodblocks does not solve various obstacles to automation. Insignificant In addition, the digitization by the conventional character recognition technology extracts text information at high speed by sequentially performing each element technology such as document preprocessing, document segmentation, character recognition, and post-processing, but the error between the element technologies is accumulated. Low quality text information will be extracted, and full manual verification and correction should be performed based on the extracted text information.

Therefore, in the conventional method of digitization by character recognition technology, the extraction of text information is very fast, but the quality of the extracted text information cannot be guaranteed, and the verification and correction process is very slow because it needs repeated full verification to improve the quality.

This is because, in the case of the digitization method by the conventional character recognition technology, very low quality text information is created by errors in various document layout analysis and quantitative and qualitative errors of the recognition engine. Therefore, in order to generate high quality information, it is necessary to manually verify and correct text information, and additional expenses are inevitable.

Therefore, in order to efficiently digitize document data that has long storage periods and include atypical structures such as manuscripts and woodblocks, it is necessary to solve the problem of relatively low recognition performance compared to the typeface. Develop.

An object of the present invention is to obtain high-quality digital text information from a large amount of document data at a high speed, and to increase the accuracy and throughput by enabling character cluster unit verification and batch text conversion, and extracting text information from document images. It is to provide a document image processing and verification system and method for digitizing a large amount of data that can reduce error search time by performing data processing and visual verification process in a cluster unit.

In order to achieve the above technical problem, the present invention uses a projection profile or a Connected Component Analysis of the document images to extract the hierarchical structure of the series of document images obtained in the scanning process. Document image structure analysis and character region segmentation module for automatically batch segmenting text areas by analyzing, Document image segmentation verification module for verifying and correcting the result of automatic segmentation of the document image through an image-based interface; A character clustering module for determining a text code by applying intelligent character recognition to each character region of the document image verified by the module, and generating a cluster image by collecting character regions of the same code, and targeting the cluster image for each code. Grant batch text code by verifying character recognition result by cluster unit A character group unit verification and text conversion module for inputting a character unit code input module for individually inputting a small character area for which a text code cannot be determined in the character recognition, and the batch text code is provided or inputted through the individual inputs A cluster unit verification and correction module for first verifying and correcting each character region of a document image through a cluster unit interface, and displaying each character region of the document image input through the batch text code assignment or the individual input. Content tagging and addition of text line-based original text verification and correction module for secondary verification and correction through text line interface and inputting various tag information and additional information of document through interface representing text area of the document image. Massive data digital with information input module Provides document image processing and verification system.

In the document image processing and verification system for the digitization of massive data according to the present invention, a document image database storing document images obtained through scanning, and a character storing image grouping image made of each character area by character recognition It is preferable to further include a cluster image database, a document image structure and segmentation database storing data by structure analysis and automatic segmentation of the document image, and a text database storing final text information obtained.

In addition, in the document image processing and verification system for the digitization of a large amount of data according to the present invention, the document image structure analysis and character region segmentation module, to automatically detect and correct various miscellaneous artifacts and skews generated during the document image acquisition process It is preferable.

In addition, in the document image processing and verification system for digitizing a large amount of data according to the present invention, the document image segmentation verification module includes: generation and deletion of text lines, generation and deletion of character regions, resizing, merging, horizontal division, It is preferable to include an interface having a function of vertical division.

delete

In addition, in the document image processing and verification system for digitization of a large amount of data according to the present invention, the character clustering module uses a Euclidean distance or Mahalanobis distance method to a minimum distance. It is desirable to determine the text code of the character area with the corresponding character class.

In addition, in the document image processing and verification system for digitization of the massive data according to the present invention, the character clustering module, using the Euclidean distance or Mahalanobis distance method using the distance value as the recognition reliability document It is desirable to determine the adoption or rejection of the clustering of.

In addition, in the document image processing and verification system for digitizing a large amount of data according to the present invention, the character group unit verification and text conversion module is collectively used for the character area in which the adoption of the clustering of documents is determined using the recognition reliability. It is desirable to give a text code.

In addition, in the document image processing and verification system for digitization of the massive data according to the present invention, the character unit code input module separately uses a text code for a character region in which rejection of grouping of documents is determined using the recognition reliability. It is preferable to give as.

In addition, in the document image processing and verification system for digitizing a large amount of data according to the present invention, it is preferable that the character unit code input module uses a kanji ultra-fast input method, a copy / stroke number-based input method, or a phonetic-based input method. .

In addition, in the document image processing and verification system for digitizing a large amount of data according to the present invention, the cluster unit verification and correction module, each text area of the document image input through the batch text code assignment or the individual input It is preferable to display in different colors based on the recognition reliability.

In addition, in the document image processing and verification system for digitizing a large amount of data according to the present invention, the content tagging and additional information input module converts text information supported by Unicode 4.0 into Shift + JIS text format. It is desirable to create an error log file that records the location of the document with the text code that does not match and the corresponding Unicode value.

In another aspect, the present invention is the step of automatically segmenting a text area by analyzing using a projection profile or a connection element analysis of the document images to extract the hierarchical structure of a series of document images obtained during the scanning process, and the document image Verifying and correcting the result of automatic segmentation through the image-based interface, and determining the text code by applying intelligent character recognition to each character region of the document image verified and corrected through the image-based interface. Generating a cluster image by collecting them, and verifying a character recognition result in a cluster unit for each cluster image for each code and assigning a batch text code to the cluster image, and a small character region in which the text code is not determined in the character recognition. Individually inputting, and assigning the batch text code Verifying and correcting each character area of the document image input through the individual input through a cluster unit interface; and assigning the batch text code or each character of the document image input through the individual input. Secondly verifying and correcting an area through a text line unit interface of the document, and inputting various tag information and additional information of the document through an interface representing a text area of the document image. Provides document image processing and verification method for digitization.

In the document image processing and verification method for digitization of massive data according to the present invention, the step of automatically dividing a text area by analyzing a series of document images obtained in the scanning process may occur in the document image acquisition process. It is preferable to include the step of automatically detecting and correcting various miscellaneous and tilting.

In addition, in the document image processing and verification method for the digitization of massive data according to the present invention, intelligent character recognition is applied to each character region of the document image to determine a text code, and generate a cluster image by collecting character regions of the same code. Preferably, the step of determining the text code of the character area by the character class corresponding to the minimum distance using the Euclidean distance or Mahalanobis distance method.

delete

In addition, in the document image processing and verification method for the digitization of massive data according to the present invention, intelligent character recognition is applied to each character region of the document image to determine a text code, and generate a cluster image by collecting character regions of the same code. Preferably, the step of using the Euclidean distance or Mahalanobis distance method using the distance value as the recognition reliability to determine the adoption or rejection of the clustering of the document.

Further, in the document image processing and verification method for digitizing the vast data according to the present invention, the step of verifying the character recognition result in a cluster unit for the clustered image for each code and assigning a batch text code to the recognition reliability It is preferable to include the step of assigning a batch text code to the character area is determined to adopt the clustering of the document by using.

In addition, in the document image processing and verification method for digitizing a large amount of data according to the present invention, the step of individually inputting a small character area that does not determine the text code in the character recognition, using the recognition reliability grouping of documents It is preferable to include the step of individually assigning a text code to the character area for which the rejection of the character is determined.

In addition, in the document image processing and verification method for digitizing a large amount of data according to the present invention, each character region of the document image input through the batch text code assignment or the individual input is primarily verified through a cluster unit interface. The step of calibrating may include displaying each text area of the document image input through the batch text code assignment or the individual input in different colors based on recognition reliability.

In addition, in the document image processing and verification method for digitizing a large amount of data according to the present invention, the step of inputting various tag information and additional information of the document through an interface representing the character region of the document image, Unicode 4.0 is Converting supported text information to Shift + JIS text form, and preferably, generating an error log file that records the location of the document and the corresponding Unicode value of the inconsistent text code.

In addition, the present invention provides a function of automatically batch-dividing a text area by analyzing using a projection profile or a connection element analysis of the document images to extract a hierarchical structure of a series of document images obtained during a scanning process, and the document image. The function of verifying and correcting the result of automatic segmentation through the image-based interface, and applying the intelligent character recognition to each character region of the document image verified and corrected through the image-based interface, determine the text code and determine the character region of the same code. A function of generating a clustered image by gathering them, a function of assigning a batch text code by verifying a character recognition result in a cluster unit for each clustered image of each code, and a small character area in which the text code is not determined in the character recognition. And the batch text code grant function Has a function of first verifying and correcting each character area of the document image input through the individual input through a cluster unit interface, and assigning the batch text code or each character of the document image input through the individual input. A program for secondly verifying and correcting an area through a text line interface of a document and inputting various tag information and additional information of a document through an interface representing a text area of the document image is provided. Provide a computer-readable recording medium for recording.

Hereinafter, a document image processing and verification system for digitizing a large amount of data of the present invention and a method thereof will be described in more detail with reference to the accompanying drawings.

4 is a structural diagram of a document image processing and verification system for digitizing massive data according to an embodiment of the present invention.

As shown, the document image processing and verification system 400 for digitization of massive data according to an embodiment of the present invention includes a document image structure analysis and character region segmentation module 410, a document image segmentation verification module 420, Character grouping module 430, character group unit verification and text conversion module 440, character unit code input module 450, group unit verification and correction module 460, text line unit text verification and correction Module 470 and content tagging and additional information input module 480.

The document image structure analysis and text area segmentation module 410 automatically divides the text area by analyzing a series of document images acquired during the scanning process.

Since the document image acquired through the scanner contains various noises and skews generated during the acquisition process, a preprocessing process to improve the quality before image analysis is necessary. Therefore, the document image structure analysis and character region segmentation module 410 includes a function of automatically detecting and correcting an inclination of an image before the structure analysis, and whether the connected component of pixels existing in the document image is miscellaneous. First, go through the process of checking.

5 is a diagram illustrating an example of an analysis and segmentation result of a document image including various miscellaneous and tilting in the document image processing and verification system according to an embodiment of the present invention.

As shown in (a) of FIG. 5, various miscellaneous images are included, and the scanned image is inputted, the document image is tilted, analyzed, divided into characters, and divided into images as shown in FIG. 5 (b). do.

In general, a document is a set of paragraphs, a paragraph is a set of text lines, and a text line is a set of characters. Thus, a document has a hierarchical structure of paragraphs, text lines, and characters. Therefore, the document image structure analysis and character region segmentation module 410 analyzes the document image as shown in FIG. 6 (a) in a hierarchical structure as shown in FIG. As shown in c), the document is divided into paragraphs, text lines, and characters to extract the hierarchical structure of the document. The character regions of the document image are assigned a logical order according to the hierarchical structure.

In the document image processing and verification system according to an embodiment of the present invention, for example, a projection profile of a document image, connection element analysis, and the like may be used in image processing and analysis.

The document image segmentation according to an embodiment of the present invention may be processed in a batch mode, for example, when a worker designates a folder of a document image to be processed.

FIG. 7 is a diagram illustrating an example of structural analysis and character region segmentation results from woodblock, manuscript, and letterpress images in a document image processing and verification system according to an embodiment of the present invention.

7 (a) shows the results of the structural analysis and the text area segmentation on the vertical woodblock document image, and FIG. 7 (b) shows the results of the structural analysis and the text area segmentation on the vertical manuscript document image. 7 (c) shows the results of structural analysis and character region segmentation on the horizontally printed document image, and FIG. 7 (d) shows structural analysis and character region segmentation on the vertically printed document image. Each result is shown.

The document image segmentation verification module 420 verifies and corrects the automatic segmentation result of the document image through an image-based interface.

The document image segmentation verification module 420 provides a function of verifying and correcting the text region, text line, character region, and logical order information of a document extracted in a segmentation process based on an interface representing an automatic segmentation result of a document image. .

FIG. 8 is a diagram illustrating an example of a toolbar used for verifying and correcting a document image segmentation result in a document image processing and verification system according to an embodiment of the present invention. As shown, character area creation 810, character area deletion 820, merging of character areas 830, vertical division of character area 840, horizontal division of character area 850, modification of character area ( 860, a document image repartition 870, an entire view 875, a text line generation display 880, a block designation 885, an order correction 890, and the like.

The character clustering module 430 determines a text code by applying intelligent character recognition to each character region of the document image, and generates a cluster image by collecting character regions of the same code.

The character clustering module 430 has a function of determining a text code by applying character recognition to a character region constituting a document image, extracting a stochastic recognition reliability of the determined text code, and having the same code according to the recognition reliability. It includes a cluster image function of gathering text areas into one cluster.

The character recognition function is a problem of extracting a feature by analyzing an input text area and classifying it into one of pre-trained character models. The classification of the text area is performed by the distance metric between the feature value of the text area and the representative value of the text model. In the present invention, the minimum distance is applied by using the Euclidean distance or the Mahalanobis distance method. The character class determines the code of the character field. In the learning process, the character model that is built in advance is the same as the criterion that maps to a concept through recognition, and the recognition performance is affected by the characteristics of the character model.

Character model generation through such adaptive learning is, for example, Korean Patent Application No. 10- entitled "Method and System for Digitization of Large Documents with Character Recognition Based on Adaptive Learning Module" filed February 25, 2004 by the present applicant. It is disclosed in detail in 2004-7621.

Extracting the recognition reliability for the character recognition defined by the function of arbitrarily matching a given character area with the appropriate character class is to determine whether to accept or reject the recognition result. In the Euclidean distance measurement method, the text code of the character area is determined as the character class that produces the minimum distance value, and the distance value is used as the recognition reliability.

In the recognition by Mahalanobis distance measurement method, linear discriminant analysis (LDA) was applied in consideration of statistical method using discriminant rule. It calculates all posterior probabilities for each character, which is the probability that a particular character class is shown, and assigns the code of the character area to the label of the class with the largest probability value among them. The probability value is used as the recognition reliability.

9A and 9B illustrate extraction of character recognition results and recognition reliability of a text area, an interface for accepting and rejecting a given threshold in a document image processing and verification system according to an embodiment of the present invention. It is a figure which shows an example.

9A is a screen for determining the adoption of clustering for batch verification because the determination of the text code for the input character area is '이' and the recognition reliability is raised to the reference value 0.99 given as 0.9999999934809.

On the other hand, Fig. 9B is a screen for determining the recognition and clustering rejection because the determination of the text code for the input character area is '改' and the recognition reliability is 0.5318746685161 that does not reach the reference value of 0.99. In this case, the individual text code input process is performed manually.

FIG. 10 is a diagram illustrating an example of a character clustering module 430 applied to various document data in a document image processing and verification system according to an embodiment of the present invention. FIG. 10A illustrates a vertical woodblock document image. 10 is a result of applying the character grouping module 430, FIG. 10 (b) is a result of applying the character grouping module 430 to the vertical manuscript document image, and (c) of FIG. The text clustering module 430 is applied to the image, and FIG. 10 (d) shows the results of applying the text clustering module 430 to the vertical letter document image.

The character clustering module 430 generates a clustered image for batch verification by adopting only a character region that raises a given threshold value according to text code determination and recognition reliability extraction of the character region. The clustered images for each character are composed of character areas classified with the same text code, and the character areas are arranged in a sorted form according to the recognition reliability. This is to allow selective attention in the verification process by the interface, and to reduce the time cost and increase the throughput due to error verification.

FIG. 11 is a diagram illustrating an example of a result of generating a cluster image and an error distribution for a Chinese character '之' class in a document image processing and verification system according to an embodiment of the present invention.

As shown, (a) of FIG. 11 shows the number of errors for each cluster image for each cluster image, that is, the cluster image (cluster image 1 to cluster image 100) for the Chinese character '之' class of 100 pages in total. 11 (b) shows an example of each cluster image. The cluster images are arranged in descending order based on the recognition reliability. As a result, the cluster image having a relatively high recognition reliability from the cluster image (for example, the cluster image 100) having a relatively low recognition reliability (for example, the cluster image 1). Many errors have been found. Therefore, it is possible to selectively focus on the part with low recognition reliability in the batch verification process on the recognition result, which can save time and double the processing rate.

The character cluster unit verification and text conversion module 440 verifies the character recognition result in a cluster unit for the cluster image for each code and gives a batch text code.

The character clustering unit verification and text conversion module 440 verifies an error by automatic character recognition for a clustered image composed of a character area having a raised recognition reliability threshold in a character clustering process, and assigns a batch text code to the clustering unit. .

As shown in (b) of FIG. 11, the clustered images for each character are composed of character areas classified as having the same text code by character recognition, and are arranged in a form aligned based on recognition reliability. Since it is composed of character areas having the same code value, it is very convenient to visually verify and reduce the error search time.

FIG. 12 is a diagram illustrating an example of an interface for verifying a text group image and converting the text group image. As shown, the interface for character cluster image verification and text conversion includes a character class 1210, the number of character regions clustered by character recognition 1220, a cluster image 1230, an original document image 1240, and a text code ( 1250), path information 1260 of the document image, and group image related information 1270 (eg, the total number of pages, the number of clustered character areas, the number of characters in a page, the current page), and the like of a specific character class. Through the interface for text group image verification and text conversion, when an error of character recognition is found, the operator displays 'X' in the corresponding text area so that it is excluded from the batch text conversion process.

The character code input module 450 individually inputs a small character area for which the text code is not determined in the character recognition.

The character unit code input module 450 directly inputs a text code to a small character region that is rejected due to a low recognition reliability in the clustering process by character recognition or is found to be a recognition error in the cluster image verification.

FIG. 13A is a diagram illustrating an example of an initial screen (one-to-one correspondence between a text area and a text code) of a manual input interface in a document image processing and verification system according to an embodiment of the present invention, and FIG. 13B is an embodiment of the present invention. 10 is a diagram illustrating an example of an input mode of a manual input interface in a document image processing and verification system according to an example.

As shown in FIG. 13A, in the manual input interface, a portion 1310 for arranging a character region and a text code portion of a document image in one-to-one (1: 1), and a portion 1320 indicating a corresponding character region of the original image for comparison. It includes. In the part 1310, which lists the text area and the text code part of the document image, the area in which the text code is assigned and the text code part are divided into the blank areas. It means the area to input directly.

Accordingly, as shown in FIG. 13B, the text code portion collects only the empty character area and proceeds with input. The input method supported by the manual input interface according to an embodiment of the present invention is a kanji ultra-fast input method, a copy / stroke number-based input method, Based typing. In particular, the Chinese character high speed input method is an input method that a skilled inputter can process about 15,000 characters per day (based on 8 hours). It analyzes the shape of Chinese characters and divides them into 248 kinds of root elements, collects the similarities of shapes and meanings of these root elements, forms 40 representative roots, and codes the representative root set to assign them to 40 keyboards. It is a principle.

Character unit code input module 450 according to an embodiment of the present invention basically supports the Unicode (unicode) 4.0, it is possible to input Korean, Chinese, Japanese, English by its own IME.

The cluster unit verification and correction module 460 primarily verifies and corrects each character region of the document image input through the batch text code assignment or the individual input through a cluster unit interface.

The cluster unit verification and correction module 460 first verifies and corrects the determined text code of the document image through the process of verifying the filtered data in the clustering using character recognition, converting the batch text, and inputting individual character unit text codes. It provides an interface to do this. The user interface of the cluster unit verification and correction module 460 is designed to correspond to the character area and the text code in one-to-one (1: 1) and verify the character areas having the same code value at a time.

FIG. 14 is a diagram illustrating an example of a character group unit verification and correction interface (a text area listing screen of a Chinese character '中' class) in a document image processing and verification system according to an embodiment of the present invention. As shown, information such as the number 1410 of the document image to be processed, the Chinese character class 1420, the corresponding character region 1430, the character region 1440, and the corresponding text code 1450 of the document image is displayed. By displaying the original document image for the text area, the operator can also perform verification through context.

In addition, the arrangement of character areas in the character cluster unit verification and correction interface is composed of part A of FIG. 14 in which a code is determined by manual input and part B and C in FIG. 14 in which a code is determined by character recognition and verification. Particularly, parts B of FIG. 14 and part C of FIG. 14 are arranged based on the recognition reliability of the character recognition process. For example, 5% and 95% of the total automation ratios are separately expressed.

In FIG. 14, the areas of the A, B, and C portions may be distinguished by displaying the colors in different colors, which may be determined by the operator who verifies whether the code of each character area is determined by hand or by character recognition. It is possible to reduce the error search time and improve the accuracy and throughput by determining the background of the decision such as whether it is determined by

The text line unit text verification and correction module 470 secondly verifies and corrects each character region of the document image input through the batch text code assignment or the individual input through the text line unit interface of the document. The text line unit original verification and correction module 470 performs secondary verification and correction on the text code determination of the character area.

FIG. 15 is a view illustrating an example of a document image text line unit verification and correction interface of various data in a document image processing and verification system according to an embodiment of the present invention, and FIG. 15A illustrates a vertical wood copy document. 15 (b) is an interface to a vertical manuscript document image, FIG. 15 (c) is an interface to a horizontal type document image image, and FIG. 15 (d) is a vertical interface. Each interface represents a print document document image.

As shown, based on the text line information of the document image, the character area and the text code are expressed one-to-one (1: 1). This interface verifies and corrects structural segmentation information errors such as text lines, text area segmentation, and logical order errors in document images.

It is necessary to input structural tags and additional information of the document for the final text information after the text code input and verification work for the document image is completed. The content tagging and additional information input module 480 inputs various tag information and additional information of the document through an interface representing a text area of the document image.

In the content tagging and additional information input module 480, a tag for generating an XML (or HTML) document and various additional information (such as spacing and paragraph division) may be provided through an interface in which a text area of a document image is expressed. In addition, it is possible to create a text file given the structural information of the final document.

The interface to the content tagging and additional information input module 480 may define various tags and additional information, and may also define information capable of batch automatic input. Therefore, the operator can assign the tag and the additional information defined through the mouse click on the interface where the text area of the document image is expressed.

16A is a diagram illustrating an example of a document image-based interface for providing structural tag information of a document in a document image processing and verification system according to an embodiment of the present invention, and FIG. 16B is a document according to an embodiment of the present invention. It is a diagram showing the spacing information extracted automatically through the structural analysis of the document image in the image processing and verification system.

After inputting various tags and additional information of the document, XML (or HTML) based text information can be extracted.

17A is a diagram illustrating an example of a final text file to which an XML tag and additional information are applied in a document image processing and verification system according to an embodiment of the present invention.

In addition, this interface can convert the text information supported by Unicode 4.0 into Shift + JIS text format. In this case, an unmatched text code generates an error log file that records the location and Unicode of the document. To allow for separate processing in the future.

17B is a diagram illustrating an example of an error file generated during Unicode and Shift + JIS conversion in a document image processing and verification system according to an embodiment of the present invention.

As shown, the error file includes data such as document number 1710, location 1720 on document, Unicode value 1730, and the like, so as to enable future correction.

18 is a structural diagram of a document image processing and verification system for digitizing massive data according to another embodiment of the present invention.

As shown, the document image processing and verification system 400 'for digitizing massive data according to another embodiment of the present invention includes a document image structure analysis and character region segmentation module 410 and a document image segmentation verification module 420. And, character grouping module 430, character group unit verification and text conversion module 440, character unit code input module 450, group unit verification and correction module 460, text line unit text verification and A calibration module 470, a content tagging and additional information input module 480, a document image database 1810, a document cluster image database 1820, a document image structure and segmentation database 1830, and a text database ( 1840).

Document image structure analysis and character region segmentation module 410, document image segmentation verification module 420, character grouping module 430, character group unit verification and text conversion module 440, character unit code input module 450, the cluster unit verification and correction module 460, the text line unit text verification and correction module 470, and the content tagging and additional information input module 480 are illustrated in FIG. 4. As it is the same as the example, the description is omitted.

The document image database 1810 stores a document image obtained through scanning.

The document cluster image database 1820 temporarily stores a character cluster image made of each character area by character recognition.

The document image structure and segmentation database 1830 stores data by structural analysis and automatic segmentation of document images. The document image structure and segmentation database 1830 stores a variety of information and processing results of each interface in the processing stage until final text information is extracted. It is a database.

The text database 1840 stores the final text information obtained by the document image processing and verification system according to the present invention.

The information stored in each of these databases can be read and updated by each module.

That is, the document image structure analysis and character region segmentation module 410 receives the scanned document image through the document image database 1810 and divides it, and stores the document image in the document image structure and the segmentation database 1830.

In addition, the document image segmentation verification module 420 receives the document image divided from the document image structure and the segmentation database 1830 and verifies it and stores the document image structure and the segmentation database 1830 again.

The character clustering module 430 receives the document image from the document image structure and partitioning database 1830, generates it as a cluster image, and stores the document image in the document cluster image database 1820.

The character group unit verification and text conversion module 440 receives the character group image from the document group image database 1820 and verifies it in the character group unit and converts the text into text in batches to the document image structure and the split database 1830. Save it.

The character unit code input module 450 individually inputs images that are not clustered by the character grouping module 430 among the document images input from the document image structure and division database 1830 to separately input the document image structure and division database 1830. ).

The cluster unit verification and correction module 460 performs cluster unit verification and correction based on the document image input from the document image structure and the segmentation database 1830 and stores the group image verification and correction in the document image structure and the segmentation database 1830.

The text line unit text verification and correction module 470 performs verification and correction on a text line basis based on the document image input from the document image structure and segmentation database 1830 and stores the text line unit in the document image structure and segmentation database 1830. .

The content tagging and additional information input module 480 performs content tagging and additional information input based on the document image input from the image structure and segmentation database 1830 and stores the content tagging and additional information in the document image structure and segmentation database 1830.

The text information generated through this process is finally stored in the text database 1840.

19 is a flowchart of a document image processing and verification method for digitizing massive data according to the present invention.

As shown, the document image processing and verification method for the digitization of the massive data according to the present invention comprises the steps of automatically dividing the character region by analyzing a series of document images obtained in the scanning process (S110), and the document image Verifying and correcting the result of the automatic segmentation through an image-based interface (S120), and applying intelligent character recognition to each character region of the document image to determine a text code, and collecting clustered character regions of the same code to generate a clustered image. (S130), verifying a character recognition result in a group unit for the clustered image for each code, and giving a batch text code (S140); and a small character area in which the text code is not determined in the character recognition. Step of individually inputting (S150) and the batch text code granting or the document input through the individual input First verifying and correcting each character area on the cluster unit interface (S160), and displaying each character area of the document image input through the batch text code assignment or the individual input unit in the text line unit interface of the document; Secondly verifying and correcting through S170 and inputting various tag information and additional information of the document through an interface representing a text area of the document image (S180).

The document image processing and verification method for digitizing the vast data according to the present invention has been described in detail through the document image processing and verification system for the digitization of the massive data according to the present invention described with reference to FIG.

Although the present invention has been described by way of example only, it is for the purpose of illustrating the invention only, and the protection scope of the present invention is not limited by these examples, the protection scope of the present invention is defined through the description of the claims .

As described above, according to the present invention, high-quality digital text information can be obtained at a high speed from a large amount of document data, and the accuracy and processing rate can be increased by enabling character group unit verification and batch text conversion, and the text of the document image. Data processing and visual verification process for information extraction are performed in groups to reduce error search time.

In particular, the present invention has a great ripple effect in that it is a combination of an automated technology and a verification interface that can promote efficient DBization in the digitization of historical and cultural feed, which has a great dependence on manual labor at home and abroad. This is because manual digitization requires vast periods of time, manpower, and costs, and the size of data processing is small.

In addition, the present invention increases the accuracy and processing rate by enabling character group unit verification and batch text conversion through the results of document image processing and recognition technology, and processing data and visual verification for extracting text information from document images. -inspection) can reduce the error detection time by clustering.

In addition, the present invention parameterizes the character model of the character recognition process in order to adapt to the actual data including various forms of variation, such as woodblock and manuscript, and is equipped with a periodic adaptive training module according to the data A separate character model could be provided.

In addition, the present invention 70% of the manuscript, 80% of the wood copy, 95% or more in the case of typeface to enable processing by the cluster unit by automation. And small data that could not be processed by automation is equipped with a super fast Unicode input. As a result, the time and economic cost is reduced by 60% to 80% compared to the existing manual-oriented digitization method, and the data processing size is increased by more than 500%.                     

The present invention also allows the operator to work while visually understanding the original document image by allowing the verification and correction of the text information extraction process of the document image to be performed through an image-based interface representing the text area of the document image. It can double the accuracy and throughput. In addition, the operator can know the information such as whether the text information determination background of the document image is by manual or automation, and in the case of automation, what is the reliability of character recognition, so that selective attention can be focused on the verification process. Efficient tracking and management is possible.

Claims (22)

In order to extract the hierarchical structure of a series of document images obtained in the scanning process, a document that automatically divides text areas by analyzing them using a projection profile or a connected component analysis. Image structure analysis and character region segmentation module, A document image segmentation verification module for verifying and correcting the automatic segmentation result of the document image through an image-based interface; A character clustering module for determining a text code by applying intelligent character recognition to each character area of the document image verified by the document image segmentation verification module, and generating a cluster image by collecting character areas of the same code; A character cluster unit verification and text conversion module for verifying a character recognition result in a cluster unit for each cluster image for each code and granting a batch text code;  A character unit code input module for individually inputting a small character area for which the text code has not been determined in the character recognition; A cluster unit verification and correction module for first verifying and correcting each character region of the document image input through the batch text code assignment or the individual input through a cluster unit interface; A text line unit original verification and correction module for secondly verifying and correcting each character area of the document image input through the batch text code assignment or the individual input through a text line unit interface of a document; Content tagging and additional information input module for inputting various tag information and additional information of a document through an interface representing a text area of the document image. Document image processing and verification system for digitization of massive data including a. The method of claim 1, A document image database for storing document images obtained through scanning; A character cluster image database for storing a character cluster image made of each character area by character recognition; A document image structure and segmentation database for storing data by structure analysis and automatic segmentation of document images, A text database that stores the final textual information obtained Document image processing and verification system for digitization of massive data further comprising. According to claim 1, wherein the document image structure analysis and character region segmentation module, Document image processing and verification system for the digitization of a large amount of data that automatically detects and corrects the various miscellaneous and skew generated in the document image acquisition process. (delete) The method of claim 1, wherein the document image segmentation verification module, A document image processing and verification system for digitizing vast data, including an interface having functions of creating and deleting text lines, creating and deleting text areas, resizing, merging, horizontal division, and vertical division. The method of claim 1, wherein the character clustering module, Document image processing and verification for digitization of massive data, which uses the Euclidean distance or Mahalanobis distance method to determine the text code of the text area with the character class corresponding to the minimum distance system. The method of claim 6, wherein the character clustering module, A document image processing and verification system for digitization of a vast amount of data, which uses the Euclidean distance or Mahalanobis distance method to determine the adoption or rejection of document clustering by using the distance value as recognition reliability. The method of claim 7, wherein the character group unit verification and text conversion module, The document image processing and verification system for digitizing a large amount of data for applying a collective text code to the character region is determined to adopt the clustering of documents by using the recognition reliability. The method of claim 7, wherein the character unit code input module, Document image processing and verification system for digitizing the vast amount of data to individually assign a text code to the character area is determined to reject the grouping of documents by using the recognition reliability. The method of claim 1, wherein the character unit code input module, A document image processing and verification system for digitizing a large amount of data, which uses a Chinese character high speed input method, an incident / stroke number based input method, or a sound value based input method. The method of claim 1, wherein the cluster unit verification and calibration module, And displaying each character area of the document image input through the batch text code assignment or the individual input in different colors based on recognition reliability. The method of claim 1, wherein the content tagging and additional information input module comprises: It converts text information supported by Unicode 4.0 to Shift + JIS text format and generates an error log file that records the location of the document with the unmatched text code and the corresponding Unicode value. Document image processing and verification system for digitization. Automatically dividing the text area by analyzing the projection profile or the connection element analysis of the document images to extract the hierarchical structure of the series of document images obtained during the scanning process; Verifying and correcting the automatic segmentation result of the document image through an image-based interface; Determining a text code by applying intelligent character recognition to each character area of the document image verified and corrected through the image-based interface, and generating a cluster image by collecting character areas of the same code; Verifying a character recognition result by a cluster unit for the clustered images for each code and assigning a batch text code;  Individually inputting a small character area for which the text code has not been determined in the character recognition; Firstly verifying and correcting each character region of the document image input through the batch text code assignment or the individual input through a cluster unit interface; Secondly verifying and correcting each character region of the document image input through the batch text code assignment or the individual input through a text line unit interface of the document; Inputting various tag information and additional information of the document through an interface representing a text area of the document image; Document image processing and verification method for digitization of massive data including a. The method of claim 13, wherein the step of automatically dividing a text area by analyzing a series of document images acquired in the scanning process, And automatically detecting and correcting various miscellaneous artifacts and inclinations generated during the document image acquisition process. (delete) The method of claim 13, wherein the determining of a text code by applying intelligent character recognition to each character area of the document image and generating a cluster image by collecting character areas of the same code comprises: A method of processing and verifying a document image for digitizing a large amount of data, comprising determining a text code of a character region using a Euclidean distance or Mahalanobis distance method. The method of claim 16, wherein the determining of a text code by applying intelligent character recognition to each character area of the document image and generating a cluster image by collecting character areas of the same code comprises: Using the Euclidean distance or Mahalanobis distance method to determine the adoption or rejection of the clustering of documents by using the distance value as the recognition reliability; and Verification method. 18. The method of claim 17, wherein the step of assigning a batch text code by verifying a character recognition result in a clustering unit for the clustered image for each code, And a batch text code is assigned to a character area in which adoption of a document clustering is determined using the recognition reliability. 18. The method of claim 17, wherein the step of individually inputting the small character area for which the text code is not determined in the character recognition, And individually assigning a text code to a character region in which rejection of clustering of documents is determined using the recognition reliability. The method of claim 13, wherein the verifying and correcting of each text area of the document image input through the batch text code assignment or the individual input through a cluster unit interface are performed. And displaying each text area of the document image input through the batch text code assignment or the individual input in different colors based on recognition reliability. . The method of claim 13, wherein inputting various tag information and additional information of a document through an interface representing a text area of the document image comprises: Converting the text information supported by Unicode 4.0 to Shift + JIS text form, and generating an error log file that records the location of the document of the inconsistent text code and the corresponding Unicode value. Document Image Processing and Verification Method for Digitalization In order to extract the hierarchical structure of a series of document images obtained in the scanning process, the function of automatically segmenting the text area by analyzing using the projection profile or the connection element analysis of the document images, Verifying and correcting the automatic segmentation result of the document image through an image-based interface; A function of determining a text code by applying intelligent character recognition to each character region of the document image verified and corrected through the image-based interface, and generating a cluster image by collecting character regions of the same code; A function of verifying a character recognition result by a cluster unit for the clustered image for each code and granting a batch text code;  A function of individually inputting a small character area for which the text code is not determined in the character recognition; A function of first verifying and correcting each character area of the document image input through the batch text code assignment or the individual input through a cluster unit interface; Secondly verifying and correcting each character area of the document image input through the batch text code assignment or the individual input through a text line unit interface of the document; A function of inputting various tag information and additional information of a document through an interface representing a text area of the document image. A computer-readable recording medium having recorded thereon a program for realizing this.

Images