KR101028670B1 - Method, system, and computer-readable recording medium for recognizing characters included in a document by using language model and ocr - Google Patents

Abstract

The present invention relates to a method, a system and a computer readable recording medium for recognizing a character string included in a document consisting of an image area and a text area. According to an aspect of the present invention, (a) analyzing the document structure of the document and classifying it into a text area and an image / noise area, (b) recognizing a character string included in the text area using a first OCR. (C) finding a character string included in a specific area that is incorrectly classified as a text area among the text areas through a language model, and referring to the location information of the specific area obtained from the first OCR, the specific area. Classifying the image / noise region into the image / noise region, and (d) using the second OCR for the image / noise region classified in the steps (a) and (c). A method is provided that includes recognizing a string.

OCR, Character Recognition, Language Model

Description

METHOOD, SYSTEM, AND COMPUTER-READABLE RECORDING MEDIUM FOR RECOGNIZING CHARACTERS INCLUDED IN A DOCUMENT BY USING LANGUAGE MODEL AND OCR}

The present invention relates to a method, a system and a computer-readable recording medium for recognizing character strings included in a document using a language model and an OCR. More particularly, the present invention provides a method of removing text noise included in an OCR result through a language model. The present invention relates to a method, a system, and a computer-readable recording medium for determining an image region through an OCR recognition result and a language model and performing recognition using an OCR engine specialized for an image.

Recently, with the rapid dissemination of digital storage media, digitalization of documents that have existed on the ground has been actively developed. Such a phenomenon is accelerating with the development of optical character recognition (OCR), which is a technology for automatically recognizing characters included in a document.

When images and text coexist in a document, it is necessary to distinguish between a text area and an image area for character recognition, but there has been a problem in that it is not easy to distinguish between the text area and the image area.

Here, there are several ways to recognize the character string included in the document, one of which is using a language model. The language model is the method that produces the most grammatical or probabilistic output for multiple input strings based on dictionaries, frequency of use, and probability of use. Regarding such a language model, it is disclosed in Korean Patent Application Publication No. 2006-46128, "Low Resolution OCR for Camera Input Documents", and the like, and is widely used in speech recognition methods and systems.

However, when a part of the image region is inserted into the text region, even if it is through the conventional language model as described above, the most grammar or probable probability is output, resulting in a very messy recognition result. Indeed, it is technically difficult to analyze a document structure, that is, to accurately classify a document into an image area and a text area, and thus the above problems may frequently occur.

Accordingly, an object of the present invention, to solve all the problems of the prior art as described above, in order to more accurately recognize the characters contained in the document consisting of the image / noise area and the text area to analyze through a language model and OCR device By referring to the information on where the input text is located throughout the document, it is possible to determine an image / noise area that is incorrectly incorporated into the text area.

In addition, another object of the present invention, in the document consisting of the image / noise area and the text area, for the characters contained in the area divided into the image / noise area with high accuracy, character recognition is successfully performed using image-specific OCR technology To make it work.

The characteristic structure of this invention for achieving the objective of this invention mentioned above, and realizing the characteristic effect of this invention mentioned later is as follows.

According to an aspect of the present invention, a method for recognizing a character string included in a document, the method comprising: (a) analyzing a document structure of the document and classifying it into a text area and an image / noise area; and (b) using a first OCR. Recognizing a character string included in the text region, (c) finding a character string included in a specific region misclassified as a text region among the text regions through a language model, and obtaining the specific region obtained from the first OCR. Reclassifying the specific region into the image / noise region with reference to the positional information on, and (d) a second method for the image / noise region classified in the steps (a) and (c). A method is provided that includes recognizing a character string included in the image / noise region using OCR.

According to another aspect of the present invention, there is provided a system for recognizing a character string included in a document consisting of a text region and an image / noise region, the system comprising: a first OCR portion for recognizing a character string included in the text region using a first OCR; A second OCR unit for recognizing a character string included in the image / noise area using a second OCR, and temporarily classifying the document structure of the document into a text area and an image / noise area, and then using a language model. Finding a character string included in a specific area that is incorrectly classified as a text area among the text areas, and referring to the positional information about the specific area obtained from the first OCR unit, redesigning the specific area as the image / noise area. A system is provided that includes a document structure analysis unit for sorting.

In addition to this, there is further provided a computer readable recording medium for recording another method, another system, and a computer program for executing the methods.

The effects of the present invention achieved due to the characteristic configuration of the present invention are as follows.

1. According to the present invention, it is possible to enjoy the effect of increasing the accuracy of the document consisting of the image area and the text area than performing character recognition using a conventional commercial OCR program.

2. According to the present invention, an image area and a text area included in an arbitrary document can be accurately distinguished, and text-only OCR and image-only OCR can be applied in place.

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

For reference, in the specification of the present invention, a normal text-specific OCR as an OCR that performs optical character recognition for a text area, and an image-specific OCR as an OCR that performs optical character recognition for a character included in an image area, Although described, the present invention is not necessarily limited thereto. An OCR that performs optical character recognition on a text area, which adopts OCR or another type of OCR that can be used for both text and images, or optical character recognition on a character included in an image area. In the case of adopting OCR or any other type of OCR that can be used for both images and text as an OCR for performing the above, it will be included in the scope of the present invention.

[Preferred Embodiments of the Invention]

1 is a diagram illustrating the configuration of an optical character recognition system 100 according to an embodiment of the present invention.

Referring to FIG. 1, the optical character recognition system 100 may include a document information input unit 110, a document structure analyzer 120, a text OCR unit 130, an image OCR unit 140, a controller 150, and a communication unit. 160 may be included. According to an embodiment of the present invention, the document information input unit 110, the document structure analysis unit 120, the text OCR unit 130, the image OCR unit 140, the control unit 150, and the communication unit 160 is At least some of the program modules may be in communication with an external terminal device or an external server. Such program modules may be included in the optical character recognition system 100 as an operating system, an application module, and other program modules, and may be physically stored on various known storage devices. In addition, these program modules may be stored in a remote storage device that can communicate with the optical character recognition system 100. Meanwhile, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, which will be described later, according to the present invention.

The optical character recognition system 100 according to an embodiment of the present invention may be included in or connected to an image data generating apparatus such as a scanner or a camera, and the optical character recognition system 100 according to another embodiment of the present invention may be a personal computer. (Eg, desktop computer, laptop computer, tablet computer, palmtop computer, etc.), workstations, PDAs, web pads, mobile phones, and the like, may be included or connected to. Here, the communication network may be configured without regard to communication modes such as wired and wireless, and may include a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). It may be configured with a variety of communication networks.

The document information input unit 110 according to an exemplary embodiment of the present invention may receive information on a document including text and / or images from a digital device, and transmit the information about the received document to the document structure analyzer 120. It can perform the function of delivering to.

The document structure analyzer 120 according to an embodiment of the present invention may classify the entire area of the document into a text area, an image area, and the like in identifying the structure of the document. The analysis of the document structure is performed in zones, which are generally similar to the paragraph units of strings. The document structure analysis unit 120 will be described later in more detail.

The text OCR unit 130 according to an embodiment of the present invention performs a function of recognizing a character included in a text area. As shown in FIG. 2, the text OCR unit 130 may include a segmentation unit 131, a character normalization unit 132, and a character recognition unit 133, but is not limited thereto.

The segmentation unit 131 according to an embodiment of the present invention may perform a function of dividing a string included in a text area for each character.

In detail, the segmentation unit 131 may perform a function of dividing the lines included in the text area through projection between the text areas and using the connected component labeling technique. It can be applied to recognize punctuation such as comma, period, exclamation point, colon, semicolon, parentheses, quotation marks, etc., and repartitions the word separated from the punctuation based on the space. Character division may be performed according to the characteristics of the language. Of course, the segmentation unit 131 of the present invention will not be limited to the above functions, and may be implemented assuming various modifications.

Meanwhile, the character normalization unit 132 according to an embodiment of the present invention may perform a function of normalizing the divided characters at a specific ratio, and the character recognition unit 133 may perform a function of recognizing normalized characters. Can be.

The image OCR unit 140 according to an embodiment of the present invention performs a function of recognizing a character included in an image / noise region.

The image OCR unit 140 according to an embodiment of the present invention may be implemented using an OCR specialized for a known image. For example, image-specific OCR technology is written by Chuang Li et al., Published in the IEEE in 2001, “Automatic Text Location in Natural Scene Images,” and Li Xu et al., Department of JiaoTong University, Shanghai, China. It may be carried out using at least one of known techniques such as "A Novel Method for Character Segmentation in Natural Scenes", which is a paper published by of Computer Science and Engineering. It is to be understood that the present invention is not to be construed as limited by the known techniques listed above.

In addition, the controller 150 according to an embodiment of the present invention includes a document information input unit 110, a document structure analysis unit 120, a text OCR unit 130, an image OC ““ R unit 140, and a communication unit ( 160) to control the flow of data between.

In addition, the communication unit 160 according to an embodiment of the present invention may perform a function to enable the optical character recognition system 100 according to the present invention to communicate with an external device such as a scanner, a camera, or the like.

Hereinafter, a process for recognizing a character string included in a document including an image area and a text area by the optical character recognition system 100 according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 and 4.

1. Distinction between text area and image / noise area

3 is a diagram specifically illustrating a process of recognizing a character string included in a document including a text area and an image area according to an embodiment of the present invention.

First, a step (S110) of analyzing a document structure of an input document and temporarily separating the document into a text area and an image / noise area is performed.

The document structure analyzer 120 may analyze the structure of the document according to the area of the binarized image information. For example, the binarized image information is classified into a text area by segmenting an area having high similarity with the text standard pattern in the storage device, and a region having a low similarity with the text standard pattern is classified into an image / noise area. The text standard pattern may be stored in the storage device in the form of a database as font information of various fonts, and may be referred to in the process of classifying the area by analyzing the structure of the document. In this case, information on the position occupied by the text area and the image / noise area in the entire document may be stored.

2. Recognize Strings in Text Areas

In operation S120, the text area classified in the step is analyzed to recognize the information about the character string included in the text area.

As described above, the text OCR unit 130 performs a function of splitting the strings included in the text area into respective constituent characters in the segmentation unit 131, and the character normalization unit 132 divides the divided characters into a specific ratio. The normalization operation may be performed, and the character recognition unit 133 may perform a function of recognizing normalized characters.

3. Exclude from the text area a specific area determined to be the text area

A step S130 of excluding a specific area from the text area determined as the text area is performed.

First, before describing step S130, the concept of a language model applied in this step will be described. The language model performs a function of correcting the OCR result. Specifically, the language model is included in a specific region of the image region to determine which specific region of the region temporarily determined as the image region is incorrectly classified as a text region. It determines whether the letters exceed a certain threshold distance and indicates that the specific area is an image / noise area that is incorrectly classified as a text area when it exceeds the threshold distance, and removes from the output data of the language model. Here, the method itself for calculating the distance of the letters included in any region may be obtained using a difference in the number of strokes or a difference in the position of the stroke, or may be implemented with reference to various conventional techniques, and known in the art. The detailed description will be shortened.

In this case, the document structure analyzer 120 determines that the specific area is an invalid area (an area that should not be classified as a text area) when the letters belonging to the specific area in the text area exceed a certain threshold distance through the language model described above. It can be removed from the output data of the language model, for this it can be helped by the text OCR unit 130. This means that the input value of the language model itself exists only as a pure text value. In order to remove the specific area from the output data, it is necessary to refer to the position information of the input character acquired by the text OCR unit 130. Because. In this way, it is possible to determine whether it is more suitable to separate the text area and the image / noise area by finding a specific area having a too high distance value through the language model and finding the location information of the specific area by OCR. It becomes possible.

4. Merge specific excluded areas into image / noise areas

Merging a specific area excluded from the text area by the document structure analyzing unit 120 and the text OCR unit 130 into the image / noise area (S140) is performed.

The control unit 160 merges the area classified into the image / noise area in step S110 by the document structure analysis unit 120 and the area which is excluded from the text area in step S130 and reclassified into the image / noise area in step S130. .

For example, referring to FIG. 4, it can be seen through step S130 and step S140 that any document 400 can be accurately classified into a text area 400a and an image / noise area 400b.

5. Recognize character strings in the image / noise area

The merged image / noise region is analyzed through an image OCR unit 140 including an optical character reader (OCR) specialized for an image, and a step (S150) of recognizing a character string that may exist in the image / noise region is performed. .

”이나“

”는 노이즈로 판단되어 OCR 결과에서 제거될 수 있을 것이다. Referring to FIG. 4, it can be seen that the character strings 420 and 440 existing in the image / noise area 400b are recognized through step S150. Here, the areas 410 and 430 existing in the image / noise area 400b may be misrecognized as including the characters in the areas 410 and 430, but noise may be determined in units of words by the language model. If these areas 410 and 430 are recognized by the OCR specialized for the image,

"or"

Is considered to be noise and can be removed from the OCR result.

6. Merge strings within text areas and image / noise areas

Merging the character string recognized in the text area and the character string recognized in the image / noise area (S160) is performed.

The controller 160 merges the character string recognized by the text OCR unit 130 and the character string recognized by the image OCR unit 140.

Referring to FIG. 4, a text string in the text area 400a recognized by the text OCR unit 130 and a text string 420 and 440 in the image area 400b recognized by the image OCR unit 140 are merged and provided. I can see that

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, etc., and magneto-optical media such as floptical disks. And hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine code, such as produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and limited embodiments and drawings, it is provided to help a more general understanding of the present invention, but the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from such descriptions.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

1 is a diagram showing the configuration of an optical character recognition system according to an embodiment of the present invention by way of example.

2 is a diagram exemplarily illustrating a detailed configuration of a text OCR unit according to an embodiment of the present invention.

3 is a diagram schematically illustrating a process of recognizing a character string included in a document including a text area and an image area according to an embodiment of the present invention.

4 is a diagram illustrating an example of recognizing a character string included in a document including a text area and an image area according to an embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

100: optical character recognition system

110: document information input unit

120: document structure analysis unit

130: OCR text

131 segmentation unit

132: character normalization unit

133: character recognition unit

140: image OCR unit

150: control unit

160: communication unit

Claims (12)

In a method of recognizing a string included in a document, (a) analyzing the document structure of the document and classifying it into a text area and an image / noise area; (b) recognizing a character string included in the text area using a first OCR, (c) finding a character string included in a specific area that is incorrectly classified as a text area among the text areas through a language model, and referring to the location information about the specific area obtained from the first OCR, Reclassifying into image / noise regions, and (d) recognizing a character string included in the image / noise region using a second OCR for the image / noise region classified in the steps (a) and (c) How to include. The method of claim 1, Wherein the first OCR is an OCR specific to plain text, and the second OCR is an OCR specialized to images. The method of claim 1, (e) merging and providing the results recognized in steps (b) and (d). The method of claim 1, In the step (c), The specific area, And obtaining a distance value with respect to a result of recognizing the letters included in the text area, corresponding to an area including letters that exceed a certain threshold distance. The method of claim 1, The step (d) Determining noise on a word basis to remove noise from the recognized character string. A computer-readable recording medium having recorded thereon a computer program for executing the method according to any one of claims 1 to 5. A system for recognizing character strings included in a document consisting of a text area and an image / noise area, A first OCR unit for recognizing a character string included in the text area using a first OCR, A second OCR unit for recognizing a character string included in the image / noise region using a second OCR, and After analyzing the document structure of the document and temporarily classifying it into a text area and an image / noise area, finding a string included in a specific area that is incorrectly classified as a text area among the text areas through a language model, and the first OCR unit. A document structure analyzer for reclassifying the specific area into the image / noise area by referring to the location information of the specific area obtained from the System comprising a. The method of claim 7, wherein Wherein the first OCR is an OCR specific to plain text, and the second OCR is an OCR specific to images. The method of claim 7, wherein And a controller configured to merge and provide a result recognized by the first OCR unit and a result recognized by the second OCR unit. The method of claim 7, wherein The specific area, And obtaining a distance value with respect to a result of recognizing the characters included in the text area, wherein the system corresponds to an area including letters that exceed a certain threshold distance. The method of claim 7, wherein The second OCR unit, And remove the noise from the recognized result using the second OCR. The method of claim 11, System for determining the noise in units of words to remove the noise.

Images