KR20200062806A - Method for recognizing characters on document images - Google Patents

Abstract

According to an embodiment of the present invention, disclosed is a computer program stored in a computer-readable storage medium. The computer program is configured to perform the following operations of: recognizing characters on a document image when executed in one or more processors of a computing device: determining a window at least partially based on a character block including one or more characters on a document image; recognizing a character by calculating the character block and the window through a character recognition model including one or more pre-trained network functions; readjusting the window based at least in part on the recognized character; and outputting a recognized character as a string by sliding the window in the character block.

Description

METHOD FOR RECOGNIZING CHARACTERS ON DOCUMENT IMAGES}

The present disclosure relates to a method for recognizing characters on a document image, and more particularly, to a method for recognizing characters on a document image by re-adjusting a window through the character recognition model.

Pharmacies, insurance companies, hospitals, etc., automatically recognize patient information, prescription details, and supply/receipt information on prescriptions using optical recognition modules such as OCR devices for prescriptions.

However, since the person does not directly check and work, the recognition through the OCR device is likely to cause a recognition error between numbers or characters with high similarity. In the case of a pharmacy, if a prescription OCR device recognition error occurs, the patient may be prescribed the wrong medicine, and the insurer may pay the wrong amount to the insured.

Accordingly, there is an increasing demand for a method for raising the recognition rate of OCR devices. In response to these demands, in particular, research is being conducted on ways to increase awareness of OCR solutions using deep learning.

Republic of Korea Patent Registration No. 10-0717734 discloses an input error prevention device for an optical mark or text reader.

The present disclosure has been devised in correspondence with the background art described above, and is intended to provide a method for recognizing characters on a document image.

Disclosed is a computer program stored in a computer readable storage medium in accordance with one embodiment of the present disclosure for realizing the above-described problems. The computer program, when executed on one or more processors of a computing device, performs the following operations for recognizing a character on a document image, the operations being based at least in part on a character block containing one or more characters on the document image an operation of determining a window, an operation of recognizing a character by calculating the character block and the window through a character recognition model including one or more pre-trained network functions, and the window based at least in part on the recognized character It may include the operation of re-adjusting and outputting the recognized character as a character string by sliding the window in the character block.

Alternatively, determining a window based at least in part on a character block containing one or more characters on the document image may include determining the character block based on a pixel having a color value on the document image The method may include determining a pixel line having a color value of a pixel line in one direction of the character block as a reference line, and determining the window based at least in part on the reference line.

Alternatively, determining a window based at least in part on a character block containing one or more characters on the document image may include determining the character block through extraction of connection elements on the document image, the character It may include an operation of determining whether a character is extracted on a block, and when the character is extracted on the character block, determining the window to correspond to the size of the character.

Alternatively, the operation of determining the text block through the extraction of the connection elements on the document image may include the operation of determining the unit character blocks respectively extracting the connection elements on the document image and the unit located within a predetermined distance among the unit character blocks. And grouping the text blocks into the one text block.

Alternatively, determining a window based at least in part on a character block containing one or more characters on the document image determines an attribute of the character located in the character block through context analysis of the document image And determining the window based on an attribute to be performed and the character.

Alternatively, the operation of determining the attribute of the character located in the character block through the context analysis of the document image performs semantic analysis on the recognized word recognized in another character block located vertically or horizontally in the character block. And based on a result of semantic analysis of the recognized word, determining an item of the character block and determining a type of character matching the item as an item and attribute of the character, and Determining the window based on the attribute of the letter may include determining the window based on the type of the letter matching the item.

Alternatively, the operation of recognizing a character by calculating the character block and the window through a character recognition model including one or more pre-trained network functions crops the character block based on the window to generate a character image. It may include an operation of generating and recognizing a character by calculating the character image through the character recognition model.

Alternatively, the operation of recognizing a character by calculating the character image through the character recognition model may include outputting a character recognized by calculating the character image through the character recognition model and recognition reliability of the character, the character And determining whether the characters of the character image are properly recognized based on the recognition reliability of the and the predetermined reference recognition reliability, and outputting the characters when it is determined that the characters of the character image are properly recognized. can do.

Alternatively, the operation of determining whether a character of the character image is properly recognized based on the recognition reliability of the character and a predetermined reference recognition reliability, when the recognition reliability of the character is less than a predetermined reference recognition reliability, the window It may include the operation of changing at least one of the size and shape of the.

Alternatively, sliding the window in the character block and outputting the recognized character as a character string determines the sliding direction for moving the window in the character block, the window and the sliding The method may include recognizing a character on one or more character images generated based on a direction, and generating the character string based on at least one of the recognized position and order of the characters in the character block.

Alternatively, the operation of determining the sliding direction for moving the window in the character block may include the operation of determining the sliding direction based on the shape of the character block.

Alternatively, the operation of generating the character string based on at least one of the recognized position and order of the characters in the character block extracts one or more recognized words by grouping one or more characters included in the character string into semantic units. Based on an operation and a result of comparing the recognized word with one or more reference words stored in memory, the method may include changing the window for one or more letters included in the character string, or changing the sliding direction. have.

Also disclosed is a method for recognizing text on a document image according to another embodiment of the present disclosure. The method includes determining a window based at least in part on a character block containing one or more characters on a document image, and a character recognition model comprising the character block and one or more network functions pre-trained with the window. Recognizing a character by calculating through, Re-adjusting the window based at least in part on the recognized character, and sliding the window in the character block to output the recognized character as a character string. can do.

Then, a computing device for recognizing characters on a document image according to another embodiment of the present disclosure is disclosed. The computing device includes a processor and memory comprising one or more cores, the processor determining a window based at least in part on a character block containing one or more characters on a document image, the character block and the Recognizing a character by calculating a window through a character recognition model including one or more pre-trained network functions, re-adjusting the window based at least in part on the recognized character, and sliding the window within the character block (sliding) the recognized characters can be output as a string.

The present disclosure can provide a method for recognizing text on a document image.

1 is a block diagram of a computing device performing an operation for recognizing a character on a document image according to an embodiment of the present disclosure.
2 is a diagram illustrating an example of a method for recognizing characters on a document image according to an embodiment of the present disclosure.
3 is a flowchart of a method for recognizing characters on a document image according to an embodiment of the present disclosure.
4 is a block diagram illustrating logic for implementing a method for recognizing characters on a document image according to an embodiment of the present disclosure.
5 is a block diagram of a computing device according to an embodiment of the present disclosure.

Various embodiments are now described with reference to the drawings, and like reference numbers throughout the drawings are used to denote similar elements. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is clear that these embodiments can be practiced without these specific details.

As used herein, the terms “component”, “module”, “system” and the like refer to computer-related entities, hardware, firmware, software, a combination of software and hardware, or execution of software. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, a thread of execution, a program, and/or a computer. For example, both the application and the computing device running on the computing device can be components. One or more components can reside within a processor and/or thread of execution, and one component can be localized within one computer, or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored therein. Components are, for example, signals having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or data via a network such as the Internet with other systems via signals and/or signals). ) Through local and/or remote processes.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise or unclear in context, "X uses A or B" is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or, if X uses both A and B, "X uses A or B" can be applied in either of these cases. It should also be understood that the term “and/or” as used herein refers to and includes all possible combinations of one or more of the listed related items.

Also, the terms “comprises” and/or “comprising” should be understood as meaning that the corresponding features and/or components are present. It should be understood, however, that the terms “comprises” and/or “comprising” do not exclude the presence or addition of one or more other features, components, and/or groups thereof. In addition, unless otherwise specified or contextually unclear as indicating a singular form, the singular in the specification and claims should generally be construed to mean "one or more."

Descriptions of the presented embodiments are provided to enable any person skilled in the art to use or practice the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art of the present disclosure, and the general principles defined herein can be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure should not be limited to the embodiments presented herein, but should be interpreted in the broadest scope consistent with the principles and novel features presented herein.

In one embodiment of the present disclosure, the server may include other configurations for performing the server environment of the server. The server can include any type of device. The server is a digital device, and may be a digital device equipped with a processor and having computing power, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone. The server may be a web server that processes services. The above-described types of servers are only examples, and the present disclosure is not limited thereto.

Throughout this specification, models, neural networks, network functions, and neural networks may be used in the same sense. A neural network may consist of a set of interconnected computational units, which may generally be referred to as "nodes." These “nodes” may also be referred to as “neurons”. The neural network comprises at least one node. Nodes (or neurons) constituting neural networks may be interconnected by one or more “links”.

In this specification, the network function may include one or more neural networks, and in this case, the output of the network function may be an ensemble of the output of the one or more neural networks. The model can perform data processing with other models. The models can be interconnected in series or in parallel with other models.

1 is a block diagram of a computing device 100 performing an operation for recognizing characters on a document image 200 according to an embodiment of the present disclosure.

The configuration of the computing device 100 illustrated in FIG. 1 is merely an example for simplicity. In one embodiment of the present disclosure, the computing device 100 may include other configurations for performing the computing environment of the computing device 100.

The computing device 100 for a method for recognizing a character on a document image 200 according to an embodiment of the present disclosure is configured to recognize a character through a character recognition model on the document image 200 including one or more characters Can be used.

The computing device 100 for a method for recognizing characters on a document image 200 according to an embodiment of the present disclosure may include a network unit 110, a processor 120, and a memory 130.

The network unit 110 may transmit/receive data, etc. for performing a method for recognizing characters on the document image 200 according to an embodiment of the present disclosure to other computing devices, servers, and the like. The network unit 110 may receive association information of items and attributes of recognized words from a server. In addition, the network unit 110 may enable communication between a plurality of computing devices so that learning of a model is distributed in each of the plurality of computing devices. The network unit 110 may enable communication between a plurality of computing devices so that character recognition of images using a model can be distributed.

The processor 120 may be composed of one or more cores, a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU). unit), and a processor for data analysis and deep learning. The processor 120 may read a computer program stored in the memory 130 and perform a method for recognizing characters on the document image 200 according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 120 may perform calculation for learning a neural network.

At least one of the CPU, GPGPU, and TPU of the processor 120 may process training of the model. Further, the computer program executed in the computing device according to an embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

In one embodiment of the present disclosure, the computing device 100 may distribute and process a model using at least one of a CPU, GPGPU, and TPU. In addition, in one embodiment of the present disclosure, the computing device 100 may distribute and process a model together with other computing devices.

Hereinafter, a method of determining a window used by the processor 120 for character recognition will be described.

The processor 120 may determine the window based at least in part on a character block including one or more letters on the document image 200. In the present disclosure, a window is a basic unit of image processing for recognizing characters in a document image, and may be a unit that designates at least a portion of a document image. In the present disclosure, a document image may be cropped in units of windows and input to a character recognition model. The document image 200 may be an image of a document including one or more characters to be recognized. Also, the document image 200 may include one or more character images. The letter image may be an image including one letter. For example, the letter image may be an image including one letter “A”. The processor 120 may receive the document image 200 through the network unit 110 or may receive the document image 200 through a camera unit (not shown). For example, when the character recognition model is used in a pharmacy, the document image 200 may be a prescription, or when used by an insurance company, the document image 200 may be a detailed statement of medical expenses. The descriptions of the document image 200 and the text image described above are only examples, and the present disclosure is not limited thereto.

The processor 120 may determine a window to be used for character recognition in the received document image 200. Methods such as pixel baseline recognition, connection element extraction, and context recognition may be used to determine the window.

First, in the present specification, the character block may be a group of pixels that are expected to have characters on the document image 200. In addition, the processor 120 may determine the text area searched through the global crowdedness analysis as a character block based on pixels having color values. Here, the global density analysis It may be performed by performing a Hough transform to measure and calculate the number of all segments passing a certain straight line. In addition, the processor 120 may determine a region having a density higher than or equal to a threshold as a character block by calculating the density in units of each segment.

More specifically, the processor 120 may determine a character block based on a pixel having a color value on the document image 200. Here, the document image 200 may be an image formed by arranging a plurality of pixels each having a single value in a grid structure. In addition, each pixel may include a value representing color or 0 and 1 values after binarization. The processor 120 may determine a pixel within a preset distance as a character block based on a pixel having a color value.

In addition, the processor 120 may determine, as a reference line, a pixel line in which a pixel having a color value among pixel lines in one direction in a character block is equal to or less than a predetermined ratio. The processor 120 may count pixels having a value in one pixel line arranged in the vertical direction of the character block. Then, the processor 120 may calculate a ratio of pixels having a color value through the number of pixels having a value among the number of pixels included in one pixel line. In addition, the processor 120 may determine a pixel line of a predetermined ratio (eg, 5%) or less stored in the memory 130 as a reference line. Accordingly, the processor 120 may determine one or more reference lines present on the character block.

The processor 120 according to an embodiment of the present disclosure may perform binarization processing on the document image 200. More specifically, the processor 120 may remove lines, pictures, tables, and the like of the document image 200. Also, the processor 120 may perform binarization processing on the document image 200 in order to remove noise from the document image 200. Here, the binarization process may mean processing as 1 when the sharpness is greater than or equal to the predetermined threshold, and processing as 0 when the sharpness is lower than the predetermined threshold. Accordingly, the processor 120 may determine that the pixel left as 1 in the binarization process on the document image 200 has a color value. In addition, the processor 120 may generate a reference line based on a ratio of pixels having a value of 1 among pixel lines in one direction (eg, a vertical direction of the document image 200).

Then, the processor 120 may determine the window based at least in part on the reference line. Also, the processor 120 may determine a window based on the reference line determined within the text block and the outline of the text block. More specifically, the processor 120 may determine a reference line of a vertical pixel line in which a ratio of pixels having color values in a character block is lower than a reference. In addition, the processor 120 may determine a window having a rectangular shape based on the reference line and the outer line of the character block (eg, a topmost pixel line and a bottommost pixel line). In addition, when only one reference line in the character block is determined, the processor 120 may determine a window of a rectangle based on a side outline (eg, a left pixel line), an upper/lower outline, and a reference line of the character block. The above-described outlines of the outline, reference line, and window are merely examples, and the present disclosure is not limited thereto.

For example, as illustrated in FIG. 2, the processor 120 may determine the reference line 230 in the character block 210. The processor 120 may determine a vertical pixel line in which a ratio of pixels having a color value is lower than a reference as the reference lines 231, 232, 233, and 234. Accordingly, the processor 120 may determine the first reference line 231, the second reference line 232, the third reference line 233, and the fourth reference line 234 based on the character block 210. Also, the processor 120 may determine the upper outer line 221 of the character block 210 and the lower outer line 222 of the character block 210. Then, the processor 120 may determine three windows 240 based on the outer lines 221 and 222 and the reference line 230 of the character block 210. The above-described window determination of the processor 120 is only an example, and the present disclosure is not limited thereto.

Additionally, the processor 120 may generate the window 240 based on the outline 220 of the character block. More specifically, when the outer line 220 of the character block has a square shape, the processor 120 may have a long length outer line based on the short length outer line 220 (eg, a vertical pixel line on the left side of the character block). The line 220 (eg, a horizontal pixel line on the top of a character block) may be cut. Accordingly, the processor 120 may determine the square window 240 having the short length outer line 220 of the character block as one side.

Also, the processor 120 may determine a character block through extraction of connection elements on the document image 200. The processor 120 may extract, as a connection element, pixels continuously arranged among pixels having color values on the document image 200. In addition, the processor 120 may merge elements that can be merged among the connection elements through various methods such as a hough transform, a run-length smoothing, and a morphological operation. In addition, the processor 120 may determine a character block by setting an area configured in an arbitrary shape including respective connection elements. For example, the processor 120 may determine an area of a rectangular shape that may include a pixel from which one connection element is extracted as a character block. The description of the character block determination of the above-described processor 120 is merely an example, and the present disclosure is not limited thereto.

Then, the processor 120 may determine whether a character is extracted on the character block. More specifically, the processor 120 may determine one or more character blocks on the document image 200. In addition, the processor 120 may determine whether a character is extracted by calculating each character block through a character recognition model. Here, the character recognition model may use each character block as an input value, and the recognition reliability of characters and characters extracted from the character block as an output value. More specifically, the character recognition model may use an image cropped for each character block in units of windows as an input value. Accordingly, the processor 120 determines whether the recognition reliability of the extracted characters is greater than or equal to a predetermined reference recognition reliability. Based on this, it is possible to determine which characters are to be extracted on the character block.

For example, as illustrated in FIG. 2, the processor 120 inputs and calculates a character block 210 called “examination fee” into the character recognition model, thereby recognizing reliability of “92” and “examination” for “true”. The recognition reliability for "88" and the recognition reliability for "table" "34" may be output. Then, the processor 120 may determine that “true” and “chalk” are extracted from the “examination table” based on the predetermined reference recognition reliability “85”. In addition, the processor 120 may determine that the letters for the “table” are incorrectly recognized based on that the recognition reliability for the “table” is less than or equal to the reference recognition reliability. Accordingly, the processor 120 may extract only the word “examination” with respect to the character block 210. The description of the above-described letters, numerical limits of recognition reliability and reference recognition reliability are merely examples, and the present disclosure is not limited thereto.

In addition, when the text is extracted on the text block 210, the processor 120 may determine the window 240 to correspond to the size of the text. More specifically, the processor 120 may recognize the size of characters extracted in the character block 210. Then, the processor 120 may determine the window 240 corresponding to the size of the text. Also, when two or more characters are recognized in the character block 210, the processor 120 may determine the window 240 based on the average size of the characters.

In addition, when two or more characters are recognized in the character block 210, the processor 120 according to another embodiment may generate a window 240 by analyzing the regularity of the character size. For example, the processor 120 divides the character block 210 when two characters in 80*80 pixel units and two characters in 80*40 pixel units are sequentially recognized in one character block 210. By doing so, two character blocks 210 may be generated. Then, the processor 120 determines a window 240 of 80*80 pixels for the character block 210 in which two 80*80 pixel units are recognized, and recognizes two 80*40 pixel units. It is possible to determine the window 240 of 80*40 pixels for the block. Recognition of the above-described letters and numerical limitation of the size of the window 240 are only examples, and the present disclosure is not limited thereto.

The processor 120 according to an embodiment of the present disclosure may determine a unit character block from which each connection element on the document image 200 is extracted. Also, the processor 120 may group unit character blocks located within a predetermined distance among unit character blocks into one character block. More specifically, the processor 120 may extract one or more connection elements within the character block, and determine a unit character block based on a pixel including each connection element. In addition, the processor 120 may determine unit blocks by merging unit blocks located within a predetermined distance from each other. Here, the preset distance may be determined based on the density of each connecting element. That is, the processor 120 may set the distance corresponding to the merging criterion of the unit character block as the connection elements in the character block are densely packed. Accordingly, the processor 120 according to the present disclosure generates a document image 200 by photographing a document located at a large number of letters on the document image 200 or at a distance from the camera unit (not shown). Even in this case, it is possible to maintain the merging criteria of unit character blocks.

For example, as illustrated in FIG. 2, the processor 120 extracts a connection element on the document image 200 through the first unit character block 211 "ㅈ", the second unit character block 212 " ㅣ" and the third unit character block 213 "b" may be output. Then, the processor 120 may determine to group the unit character blocks 211, 212, and 213 into one character block 210 based on the fact that the unit character blocks 211, 212, and 213 are disposed within a predetermined distance of 30 pixels. Accordingly, the processor 120 may combine unit character blocks 211, 212, and 213 to extract a single character block 210 "jin". The description of the grouping operation of the unit character block described above is only an example, and the present disclosure is not limited thereto.

Then, the processor 120 may determine the attribute of the character located in the character block through context analysis of the document image 200. Here, the attribute of the letter may include the type of the recognized letter (eg, number, alphabet, Hanja, Hangul, etc.). More specifically, the processor 120 may recognize characters located in a plurality of character blocks in one document image 200. Then, the processor 120 may determine the attribute of the character located in the character block based at least in part on the recognition word composed of the characters recognized in other character blocks. Here, the other character block is another character block existing in the document image 200 and may be a character block in which character recognition is performed before the character block 210. The processor 120 may recognize a recognized word in one or more other character blocks having a positional association with a character block in the document image 200.

The processor 120 according to another embodiment of the present disclosure may determine the attribute of a character located in the character block based on the meaning included in the recognized word recognized in the other character block. More specifically, the processor 120 may determine a corresponding item based on the meaning included in the recognized word recognized in another character block. Then, the processor 120 may determine the attribute of the character located in the character block based on the item determined for the recognized word. For example, as illustrated in FIG. 2, the processor 120 reads the meaning that the recognized word recognized in the text block 210 called “examination fee” is the cost for the examination, and then displays the item corresponding to the recognized word as “basic”. Item" or "cost". Then, the processor 120 may determine that the attribute of the character located in the character block 260 "3,285" is "number" based on the items such as "basic item" and "cost". The description of context analysis described above is merely an example, and the present disclosure is not limited thereto.

Then, the processor 120 may determine the window based on the attribute of the letter. More specifically, the processor 120 may determine the attribute of the character block based on the association information of the items and attributes stored in the memory 130. Here, the item may include one or more attributes. For example, the "cost" item may include "number" attributes, and the "pharmaceutical name" item may include "English" and "number" attributes. In this case, the processor 120 may determine the attribute of the character block as one of a plurality of attributes. The description of the above-mentioned items and attribute values is merely examples, and the present disclosure is not limited thereto. Accordingly, the processor 120 may determine at least one of the size and shape of the window based on the attribute of the letter. Also, according to an embodiment, the processor 120 may determine the window based on the type of the character matching the item.

The processor 120 according to an embodiment of the present disclosure may perform semantic analysis on a recognized word recognized in another character block positioned vertically or horizontally of the character block. Also, the processor 120 may determine an item of the character block based on the result of semantic analysis of the recognized word. Then, the processor 120 may determine the type of the character matching the item as the item and attribute of the letter. Then, the processor 120 may determine the window based on the type of the letter matching the item.

A method for recognizing a character on a document image 200 according to an embodiment of the present disclosure determines a window that is a reference for generating a character image using at least one of a pixel line criterion, a connection element extraction criterion, and a context criterion, It is possible to improve the character recognition rate through the initial window, and even when the window is readjusted according to the recognition result, it can be adjusted in the direction to increase the recognition rate, resulting in shorter work time.

Throughout this specification, neural networks, network functions, and neural networks may be used in the same sense. A deep neural network (DNN) may mean a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify potential latent structures in data. In other words, it is possible to grasp the potential structure of photos, text, video, voice, and music (for example, what objects are in the picture, what the content and emotions of the text are, what the content and emotions of the speech, etc.) are. . Deep neural networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), and deep belief networks (DBNs). , Q network, U network, Siam network, and the like.

The convolutional neural network is a kind of deep neural network, and includes a neural network including a convolutional layer. Convolutional neural networks are a type of multilayer perceptorns designed to use minimal preprocessing. The CNN may consist of one or several convolutional layers and artificial neural network layers combined with it. CNN may additionally utilize weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data of the two-dimensional structure. Convolutional neural networks can be used to recognize objects in an image. Convolutional neural networks can process image data as a matrix with dimensions. For example, in the case of red-green-blue (RGB)-encoded image data, each of R, G, and B colors may be represented as a two-dimensional (for example, two-dimensional image) matrix. That is, the color value of each pixel of the image data can be a component of the matrix, and the size of the matrix can be the same as the size of the image. Therefore, the image data can be represented by three two-dimensional matrices (three-dimensional data array).

A convolutional process (input/output of the convolutional layer) can be performed by moving the convolutional filter in the convolutional neural network and multiplying the convolutional filter and matrix components at each position of the image. The convolutional filter may be configured as an n*n matrix. The convolutional filter may generally consist of a fixed type filter smaller than the total number of pixels in the image. That is, when an m*m image is input to a convolutional layer (for example, a convolutional layer having a size of n*n of a convolutional filter), a matrix representing n*n pixels including each pixel of the image It can be multiplied by this convolutional filter (ie, the product of each component of the matrix). A component matching the convolutional filter in the image may be extracted by multiplying with the convolutional filter. For example, a 3*3 convolutional filter for extracting a vertical line component from an image may be configured as [[0,1,0], [0,1,0], [0,1,0]] Can be. When a 3*3 convolutional filter for extracting vertical components from the image is applied to the input image, vertical components that match the convolutional filter from the image may be extracted and output. The convolutional layer may apply a convolutional filter to each matrix for each channel representing an image (ie, R, G, and B colors in the case of R, G, and B coded images). The convolutional layer can apply a convolutional filter to the input image to extract features matching the convolutional filter from the input image. The filter value of the convolutional filter (that is, the value of each component of the matrix) may be updated by back propagation in the learning process of the convolutional neural network.

A subsampling layer is connected to the output of the convolutional layer, thereby simplifying the output of the convolutional layer to reduce memory usage and computation. For example, if the output of the convolutional layer is input to a pooling layer having a 2*2 max pooling filter, the maximum value included in each patch is output for each 2*2 patch in each pixel of the image to compress the image. You can. The aforementioned pooling may be a method of outputting a minimum value in a patch or outputting an average value of patches, and any pooling method may be included in the present disclosure.

The convolutional neural network may include one or more convolutional layers and sub-sampling layers. The convolutional neural network can extract features from an image by repeatedly performing a convolutional process and a subsampling process (for example, the Max pooling described above). Through the iterative convolutional and subsampling process, the neural network can extract global features of the image.

The output of the convolutional layer or subsampling layer may be input to a fully connected layer. A fully connected layer is a layer in which all neurons in one layer are connected to all neurons in a neighboring layer. The full connected layer may refer to a structure in which all nodes of each layer are connected to all nodes of another layer in a neural network.

In one embodiment of the present disclosure, in order to perform segmentation of image data, the neural network may include a deconvolutional neural network (DCNN). The deconvolutional neural network performs an operation similar to that of calculating the convolutional neural network in the reverse direction. The deconvolutional neural network may output features extracted from the convolutional neural network as a feature map related to original data. The description of the specific configuration for the convolutional neural network is discussed in more detail in US patent US9870768B2, which is incorporated herein by reference in its entirety.

Hereinafter, a method of recognizing a character and a method of re-adjusting a window will be described in which the processor 120 calculates a character block and a window through a character recognition model.

The processor 120 may recognize characters by calculating a character block and a window through a character recognition model including one or more pre-trained network functions. More specifically, the processor 120 may generate a character image by cropping a character block based on a window. In addition, the processor 120 may recognize a letter by calculating the letter image through a letter recognition model. In addition, the processor 120 may control at least one of the position and size of the window in the text block and generate one or more text images. Accordingly, the processor 120 may recognize one or more character images located in the character block by calculating one or more character images through a character recognition model.

In addition, the processor 120 according to an embodiment of the present disclosure may output the recognized letters and the recognition reliability of the letters by calculating the letter images through the letter recognition model. Then, the processor 120 may determine whether the characters of the character image are properly recognized based on the recognition reliability of the characters and the predetermined reference recognition reliability. Then, when it is determined that the characters of the character image are properly recognized, the processor 120 may output the characters. Also, according to an embodiment, the processor 120 may output a character having the highest recognition reliability among characters output based on the character image.

Then, the processor 120 may readjust the window based at least in part on the recognized character. The processor 120 according to an embodiment of the present disclosure may change at least one of a size and a shape of a window when the recognition reliability of characters is less than a predetermined reference recognition reliability. More specifically, the processor 120 may compare the recognition reliability of characters and characters output with respect to the character image with the reference recognition reliability. In addition, when the recognition reliability of the characters is greater than or equal to the reference recognition reliability, the processor 120 may determine that the window for generating the character image is suitable and maintain the window. In addition, when the recognition reliability of the characters is less than the reference recognition reliability, the processor 120 may determine that the window for generating the character image is not suitable and change the window. Here, the change of the window performed by the processor 120 may be one of a change in the shape (ie, shape) of the window or a change in the size of the window (for example, expansion/reduction in one or more of the 8 directions based on the midpoint). Can be.

For example, as illustrated in FIG. 2, the processor 120 calculates a character block 210 called “examination fee” through a character recognition model, and recognizes the character “true” and recognition reliability “90” and “chalk”. And the recognition confidence "95". In addition, the processor 120 may output two letters, "table" and "ryo", by calculating a character image of "Ryo" (ie, the correct answer) through a character recognition model. Further, among them, the recognition reliability output for the letter "table" may be "65", and the recognition reliability output for the letter "Ryo" may be "68". Then, the processor 120 may determine that the character recognition result of “true” is appropriate based on the predetermined reference recognition reliability “75”. Accordingly, the processor 120 may determine to maintain the window 240 used in the "jin". In addition, the processor 120 may move the window 240 used in the "jean" to generate a text image. In addition, the processor 120 may determine to maintain the window 240 used to generate the character image “Charge” based on the result of the character recognition “Charge” in the same manner. In addition, since the processor 120 outputs recognition reliability less than the reference recognition reliability, both “table” and “ryo” recognized in the text image “Ryo” may determine that the character recognition result is inappropriate. Accordingly, the processor 120 may determine to change the window 240 in which the text image “Ryo” is generated. Here, the processor 120 may perform a size change that extends in one direction among 8 directions (a combination of up, down, left, and right) while keeping the window 240 in the current position. Also, the processor 120 may change the shape of the window 240 from a rectangle to an arbitrary shape. Additionally, the processor 120 may determine the window 240 based on the attribute of the character to be located in the corresponding character image. Accordingly, the processor 120 may determine whether to keep the window 240 in use or change the window 240 according to the result of the character recognition of the character block 210. The above description of the character recognition and window re-adjustment operation is only an example, and the present disclosure is not limited thereto.

The processor 120 according to an embodiment of the present disclosure may determine a window, which is a reference for generating a text image, using at least one of a pixel line reference, a connection element extraction reference, and a context reference. Accordingly, the processor 120 may improve the character recognition rate through the initial window, and even when the window is readjusted according to the recognition result, the recognition rate may be adjusted in a direction to increase the recognition rate, resulting in shortening of work time.

Hereinafter, a method of outputting a recognized character as a character string by the processor 120 sliding the window 240 in the character block 210 will be described.

The processor 120 may output a recognized character as a character string by sliding a window in a character block. More specifically, the processor 120 may determine a sliding direction for moving the window within the character block. In addition, the processor 120 may recognize a letter on one or more letter images generated based on the window and sliding direction. Also, the processor 120 may generate a character string based on at least one of the position and order of recognized characters in the character block.

The processor 120 according to an embodiment of the present disclosure may determine the sliding direction based on the shape of the character block. The processor 120 may determine a character block based on the pixel where the character is located in the document image. Then, the processor 120 may determine the sliding direction based on the shape of the character block. The processor 120 according to an embodiment of the present disclosure may determine a direction in which the length is longer among the horizontal and vertical characters blocks as the slide movement direction. For example, as illustrated in FIG. 2, the processor 120 may determine the horizontal direction as the sliding direction when the horizontal length is longer than the vertical length on the character block. In addition, the processor 120 may determine the vertical direction as the sliding direction in the case of a character block in which the length in the vertical direction is longer than the length in the horizontal direction, such as "basic item" and "optional item". The above description of the determination of the sliding direction is only an example, and the present disclosure is not limited thereto.

Also, the processor 120 according to another embodiment may determine a direction in which the letters are recognized in the character block as a slide movement direction. More specifically, the processor 120 may generate one or more character images by cropping a character block through a window. Then, the processor 120 may recognize characters by calculating each character image through a character recognition model. Accordingly, the processor 120 may recognize one or more characters in the character block. Then, when a plurality of characters are recognized in the character block, the processor 120 may determine a sliding direction according to the recognized character positions. For example, as illustrated in FIG. 2, the processor 120 may recognize the text “medical statement receipt” through the character recognition model in the character block 270 of the “medical fee statement * receipt”. Then, the processor 120 may recognize that the letter arrangement in the "medical bill receipt" is horizontal. Accordingly, the processor 120 may determine a window sliding direction of the corresponding character block 270 in the horizontal direction. The above description of the determination of the sliding direction is only an example, and the present disclosure is not limited thereto.

The processor 120 according to an embodiment of the present disclosure may recognize a character on one or more character images generated based on a window and a sliding direction. The processor 120 may generate a character image by cropping a character block based on a window. Then, the processor 120 may slide the window in the character block based on the sliding direction. Here, the sliding may be performed by moving the window at a ratio of a predetermined window size (eg, 1/2, 1/4 of the window length). Accordingly, the processor 120 may recognize characters by generating one or more character images and calculating them through a character recognition model.

Then, the processor 120 may generate a character string based on at least one of the position and order of the recognized characters in the character block. More specifically, the processor 120 may generate one or more character images based on character blocks and windows. Then, the processor 120 may recognize characters by calculating each character image through a character recognition model. Also, when a character is recognized in the character image, the processor 120 may recognize the location of the character based on the location of the character image. In addition, the processor 120 may generate a synthesized string in the order of the characters located on the right from the characters located on the left among the one or more characters. In addition, the processor 120 may generate a character string synthesized in the order of the letters located on the upper side from the letters positioned on the upper side among the one or more letters. Also, the processor 120 may generate a character string based on the sequence of characters recognized in one character block. That is, the processor 120 may combine the recognized characters and the recognized characters based on the sliding direction, and output them as character strings.

In addition, the processor 120 according to an embodiment of the present disclosure may extract one or more recognition words by grouping one or more letters included in a character string into semantic units. Also, the processor 120 may change a window for one or more letters included in the character string, or change a sliding direction based on a result of comparing the recognized word with one or more reference words stored in the memory.

More specifically, the processor 120 may generate a string composed of one or more letters combined. Then, the processor 120 may extract the recognition word by grouping the characters included in the character string into semantic units. Here, the recognition word may be a word included in a letter recognition model document image. In addition, the recognition word may be a grouping of characters into semantic units. For example, the processor 120 may group characters on the basis of spacing, or may group characters on a noun basis. As another example, the recognition word may be a single word such as "examination fee" or a phrase composed of a plurality of words such as "prosthetic correction fee". The detailed description of the above-mentioned recognition words is only an example, and the present disclosure is not limited thereto.

In addition, the processor 120 may calculate a matching ratio of the recognition word with the letters included in each of the one or more reference words stored in the memory 130. The processor 120 may determine the reference word having the highest matching ratio based on the arrangement of one or more letters included in the recognized word and the arrangement of one or more letters included in the reference word. In addition, when the matching ratio of the recognized word to the reference word is equal to or greater than a predetermined threshold, the processor 120 may maintain the shape and size of the window or maintain the sliding direction. Further, the processor 120 may change at least one of the shape and size of the window or change the sliding direction when the matching ratio of the recognized word to the reference word is equal to or less than a predetermined threshold. Accordingly, the processor 120 may generate a character string different from the existing character string by changing a window (that is, a character image) or changing a moving direction of the character string generated in one character block.

Accordingly, the processor 120 according to an embodiment of the present disclosure may reduce the possibility of outputting a character string including a character that is incorrectly recognized by calculating the matching ratio between the extracted character string and the reference word in character recognition. .

The memory 130 may store a computer program for performing a method for recognizing characters on a document image according to an embodiment of the present disclosure, and the stored computer program may be read and driven by the processor 120.

The memory 130 according to embodiments of the present disclosure may store a program for the operation of the processor 120, and input/output data (eg, a document image, a text image, a text item, and an attribute of the text). Association information, etc.). The memory 130 may store data related to a display and sound. The memory 130 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), RAM (Random Access Memory, RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), magnetic memory, magnetic It may include at least one type of storage medium among disks and optical disks.

Accordingly, a method for recognizing characters on a document image according to an embodiment of the present disclosure correlates separation of characters by re-adjusting a window according to a result of character recognition and a character recognition model through a character recognition model in the document image. You can do it. In addition, through this, it is possible to accurately classify characters and recognize characters even in document images having different sizes and complicated structures.

In addition, the window, which is a standard for generating a text image, is determined using at least one of a pixel line criterion, a connection element extraction criterion, and a context criterion, thereby improving the character recognition rate through the initial window and re-adjusting the window according to the recognition result. It can also be adjusted in the direction of increasing the recognition rate, resulting in shorter working time.

In addition, by calculating the matching ratio between the extracted character string and the reference word and reflecting it in the character recognition, it is possible to reduce the possibility of outputting a character string containing the character that is not well recognized.

3 is a flowchart of a method for recognizing characters on a document image according to an embodiment of the present disclosure.

The computing device 100 may determine 310 a window based at least in part on a character block that includes one or more letters on the document image.

According to an embodiment of the present disclosure, the computing device 100 may determine the character block based on a pixel having a color value on the document image. Also, the computing device 100 may determine, as a reference line, a pixel line in which the pixel having a color value among pixel lines in one direction in the character block is equal to or less than a predetermined ratio. In addition, the computing device 100 may determine the window based at least in part on the reference line.

Further, according to another embodiment of the present disclosure, the computing device 100 may determine the text block through extraction of a connection element on a document image. In addition, the computing device 100 may determine whether a character is extracted on the character block. Then, when a character is extracted on a character block, the computing device 100 may determine the window to correspond to the size of the character. Further, the computing device 100 may determine a unit character block from which each connection element on the document image is extracted. In addition, the computing device 100 may group the unit text blocks located within a predetermined distance among the unit text blocks into the one text block.

In addition, according to another embodiment of the present disclosure, the computing device 100 may determine an attribute of a character positioned in the character block through context analysis of a document image. Then, the computing device 100 may determine the window based on the attribute of the letter. Also, the computing device 100 may perform semantic analysis on the recognized word recognized in another character block positioned vertically or horizontally of the character block. Then, the computing device 100 may determine the item of the character block based on the result of semantic analysis of the recognized word. Also, the computing device 100 may determine the type of the character matching the item as the item and attribute of the letter. Here, the computing device 100 may determine the window based on the type of the character matching the item.

The computing device 100 may recognize the character 320 by calculating the character block and the window through a character recognition model including one or more pre-trained network functions.

According to an embodiment of the present disclosure, the computing device 100 may generate a character image by cropping a character block based on the window. Then, the computing device 100 may recognize a character by calculating the character image through the character recognition model. In addition, the computing device 100 may output a recognized character and the recognition reliability of the character by calculating the character image through the character recognition model. In addition, the computing device 100 may determine whether the characters of the character image are properly recognized based on the recognition reliability of the characters and the predetermined reference recognition reliability. Then, when it is determined that the characters of the character image are properly recognized, the computing device 100 may output the characters. Here, the computing device 100 may change at least one of the size and shape of the window when the recognition reliability of the text is less than a predetermined reference recognition reliability.

The computing device 100 may readjust the window 330 based at least in part on the recognized character.

According to one embodiment of the present disclosure, the computing device 100 may readjust the window based at least in part on the recognized character. The computing device 100 according to an embodiment of the present disclosure may change at least one of a size and a shape of a window when the recognition reliability of characters is less than a predetermined reference recognition reliability. More specifically, the computing device 100 may compare the recognition reliability of characters and characters output with respect to the character image with the reference recognition reliability. The computing device 100 may maintain the window by determining that the window for generating the character image is suitable when the recognition reliability of the characters is higher than the reference recognition reliability. In addition, when the recognition reliability of the characters is less than the reference recognition reliability, the computing device 100 may change the window by determining that the window for generating the character image is not suitable. Here, the change of the window performed by the computing device 100 is one of a change in the shape (ie, shape) of the window, or a change in the size of the window (for example, expansion/reduction in one or more of the eight directions based on the midpoint). Can be

The computing device 100 may output 340 a character recognized as a character string by sliding the window in a character block.

According to an embodiment of the present disclosure, the computing device 100 may determine a sliding direction for moving the window within the text block. In addition, the computing device 100 may recognize a letter on one or more letter images generated based on a window and the sliding direction. Then, the computing device 100 may generate the character string based on at least one of the recognized position and order of the characters in the character block. Here, the computing device 100 may determine the sliding direction based on the shape of the character block. Also, the computing device 100 may extract one or more recognition words by grouping one or more letters included in the string into semantic units. In addition, the computing device 100 may change the window for one or more letters included in the string, or change the sliding direction based on a result of comparing the recognized word with one or more reference words stored in the memory. have.

Accordingly, a method for recognizing characters on a document image according to an embodiment of the present disclosure correlates separation of characters by re-adjusting a window according to a result of character recognition and a character recognition model through a character recognition model in the document image. You can do it. In addition, through this, it is possible to accurately classify characters and recognize characters even in document images having different sizes and complicated structures.

In addition, the window, which is a standard for generating a text image, is determined using at least one of a pixel line criterion, a connection element extraction criterion, and a context criterion, thereby improving the character recognition rate through the initial window and re-adjusting the window according to the recognition result. It can also be adjusted in the direction of increasing the recognition rate, resulting in shorter working time.

In addition, by calculating the matching ratio between the extracted character string and the reference word and reflecting it in the character recognition, it is possible to reduce the possibility of outputting a character string containing the character that is incorrectly recognized.

4 is a block diagram illustrating logic for implementing a method for recognizing characters on a document image according to an embodiment of the present disclosure.

A method for recognizing characters on a document image according to an embodiment of the present disclosure may be implemented by the following logic.

A method for recognizing a character on a document image according to an embodiment of the present disclosure includes: logic 410 for determining a window based at least in part on a character block including one or more characters on the document image; Logic 420 for recognizing characters by computing the character block and the window through a character recognition model including one or more pre-trained network functions; Logic 430 for re-adjusting the window based at least in part on the recognized character; And a logic 440 for outputting a recognized character as a character string by sliding the window in the character block.

In an alternative embodiment of a method for recognizing text on a document image, logic 410 for determining a window based at least in part on a text block containing one or more text on the document image includes: the document image Logic for determining the character block based on a pixel having a color value on the image, logic for determining a pixel line having a color value of a pixel value of a pixel line in one direction or less in the character block as a reference line, and the It can be implemented by logic for determining the window based at least in part on a baseline.

In an alternative embodiment of a method for recognizing text on a document image, logic 410 for determining a window based at least in part on a text block containing one or more text on the document image includes: the document image Logic for determining the text block through extraction of a connection element on the screen, logic for determining whether a letter is extracted on the text block, and when the text is extracted on the text block, the window corresponds to the size of the text It can be implemented by logic to determine.

In an alternative embodiment of a method for recognizing text on a document image, logic for determining the text block through extraction of linking elements on the document image includes: determining unit text blocks each extracting linking elements on the document image The logic may be implemented by logic for grouping the unit character block located within a predetermined distance among the unit character blocks into the one character block.

In an alternative embodiment of a method for recognizing text on a document image, logic 410 for determining a window based at least in part on a text block containing one or more text on the document image includes: the document image It may be implemented by logic for determining the attribute of the character located in the character block through the context analysis of and logic for determining the window based on the attribute of the character.

In an alternative embodiment of a method for recognizing a character on a document image, logic for determining an attribute of a character located in the character block through context analysis of the document image is located vertically or horizontally in the character block. The logic for performing semantic analysis on the recognized word recognized in another character block, the logic for determining the item of the character block based on the semantic analysis result for the recognized word, and the type of the character matching the item It can be implemented by logic for determining the item and attribute of the letter, and the logic for determining the window based on the attribute of the letter determines the window based on the type of character matching the item It can be implemented by logic.

In an alternative embodiment of a method for recognizing characters on a document image, logic 420 for recognizing characters by computing the character block and the window through a character recognition model including one or more pre-trained network functions is , It may be implemented by logic for generating a character image by cropping the character block based on the window and logic for recognizing a character by calculating the character image through the character recognition model.

In an alternative embodiment of a method for recognizing a character on a document image, logic for recognizing a character by calculating the character image through the character recognition model is recognized by calculating the character image through the character recognition model The logic for outputting the character and the recognition reliability of the character, the logic for determining whether the character of the character image is properly recognized based on the recognition reliability of the character and the predetermined reference recognition reliability, and the character of the character image When it is determined that it is properly recognized, it may be implemented by logic for outputting the text.

In an alternative embodiment of a method for recognizing a character on a document image, logic for determining whether a character of the character image is properly recognized based on the recognition reliability of the character and a predetermined reference recognition reliability, the character If the recognition reliability of is less than the predetermined reference recognition reliability, it may be implemented by logic for changing at least one of the size and shape of the window.

In an alternative embodiment of a method for recognizing a character on a document image, the logic 440 for sliding the window in the character block and outputting the recognized character as a character string is performed in the character block. At least one of logic for determining a sliding direction for moving a window, logic for recognizing a letter on the window and one or more letter images generated based on the sliding direction, and the position and order of the recognized letters in the text block It can be implemented by logic for generating the string based on one.

In an alternative embodiment of a method for recognizing text on a document image, logic for determining a sliding direction for moving the window within the text block determines the sliding direction based on the shape of the text block It can be implemented by logic.

In an alternative embodiment of a method for recognizing a character on a document image, logic for generating the character string based on at least one of the recognized position and order of the character in the character block includes: one or more of the character strings Based on the logic for extracting one or more recognized words by grouping the letters into semantic units and the result of comparing the recognized words with one or more reference words stored in memory, the window for one or more letters included in the string is changed. Or, it may be implemented by logic for changing the sliding direction.

Logic for implementing a method for recognizing text on a document image according to an embodiment of the present disclosure may be implemented by a module, circuit, or means for implementing a method for recognizing text on a document image.

5 is a block diagram of a computing device 100 according to an embodiment of the present disclosure.

5 shows a simplified and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure can be implemented.

Although the present disclosure has been described above in general with respect to computer-executable instructions that can be executed on one or more computers, those skilled in the art will appreciate that the present disclosure can be implemented in combination with other program modules and/or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, those of ordinary skill in the art may find that the methods of the present disclosure include single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like (each of which is). It will be appreciated that it may be implemented with other computer system configurations, including those that may operate in conjunction with one or more associated devices.

The described embodiments of the present disclosure can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer readable media. Any computer-accessible medium can be a computer-readable medium. Computer-readable media includes volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer readable media may include computer readable storage media and computer readable transmission media. Computer-readable storage media are volatile and non-volatile media, temporary and non-transitory media, removable and non-removable, implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data Includes media. Computer storage media includes RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, Or any other medium that can be accessed by a computer and used to store desired information.

Computer readable transmission media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and Includes all information delivery media. The term modulated data signal means a signal in which one or more of the characteristics of the signal are set or changed to encode information in the signal. By way of example, and not limitation, computer readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above described media are also intended to be included within the scope of computer readable transmission media.

An exemplary environment 1100 is shown that implements various aspects of the present disclosure, including a computer 1102, and the computer 1102 includes a processing device 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 connects system components, including, but not limited to, system memory 1106 to processing device 1104. The processing device 1104 can be any of a variety of commercial processors. Dual processor and other multiprocessor architectures may also be used as the processing unit 1104.

The system bus 1108 can be any of several types of bus structures that can be further interconnected to a memory bus, peripheral bus, and local bus using any of a variety of commercial bus architectures. System memory 1106 includes read-only memory (ROM) 1110 and random access memory (RAM) 1112. The basic input/output system (BIOS) is stored in a non-volatile memory 1110 such as ROM, EPROM, EEPROM, etc. This BIOS is a basic to help transfer information between components in the computer 1102 at the same time as during startup. Contains routines. The RAM 1112 may also include high-speed RAM, such as static RAM for caching data.

Computer 1102 is also an internal hard disk drive (HDD) 1114 (eg, EIDE, SATA)-this internal hard disk drive 1114 can also be configured for external use within a suitable chassis (not shown). Yes, magnetic floppy disk drive (FDD) 1116 (e.g., for reading from or writing to removable diskette 1118), and optical disk drive 1120 (e.g., CD-ROM Disc 1122, or for reading from or writing to other high-capacity optical media such as DVDs). The hard disk drive 1114, the magnetic disk drive 1116, and the optical disk drive 1120 are the system bus 1108 by the hard disk drive interface 1124, the magnetic disk drive interface 1126, and the optical drive interface 1128, respectively. ). The interface 1124 for implementing an external drive includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the above description of computer readable media refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those of ordinary skill in the art can use a zip drive, magnetic cassette, flash memory card, cartridge, etc. It will be appreciated that other types of media readable by a computer, etc., may also be used in the exemplary operating environment and any such media may include computer-executable instructions for performing the methods of the present disclosure.

A number of program modules may be stored in the drive and RAM 1112, including the operating system 1130, one or more application programs 1132, other program modules 1134, and program data 1136. All or part of the operating system, applications, modules, and/or data may also be cached in RAM 1112. It will be appreciated that the present disclosure can be implemented in various commercially available operating systems or combinations of operating systems.

The user may input commands and information to the computer 1102 through one or more wired/wireless input devices, for example, pointing devices such as a keyboard 1138 and a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, etc. These and other input devices are often connected to the processing unit 1104 through an input device interface 1142 connected to the system bus 1108, but the parallel port, IEEE 1394 serial port, game port, USB port, IR interface, And other interfaces.

The monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, the computer generally includes other peripheral output devices (not shown), such as speakers, printers, and the like.

The computer 1102 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communication. The remote computer(s) 1148 may be a workstation, computing device computer, router, personal computer, portable computer, microprocessor-based entertainment device, peer device, or other conventional network node, typically in computer 1102. It includes many or all of the components described with respect to, but for simplicity, only the memory storage device 1150 is shown. The illustrated logical connections include wired/wireless connections to a local area network (LAN) 1152 and/or a larger network, such as a wide area network (WAN) 1154. Such LAN and WAN networking environments are common in offices and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to computer networks around the world, such as the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 via a wired and/or wireless communication network interface or adapter 1156. The adapter 1156 may facilitate wired or wireless communication to the LAN 1152, which also includes a wireless access point installed therein to communicate with the wireless adapter 1156. When used in a WAN networking environment, the computer 1102 may include a modem 1158, connect to a communication computing device on the WAN 1154, or establish communication over the WAN 1154, such as through the Internet. Have other means. The modem 1158, which may be an internal or external and wired or wireless device, is connected to the system bus 1108 via a serial port interface 1142. In a networked environment, program modules described for the computer 1102 or portions thereof may be stored in the remote memory/storage device 1150. It will be appreciated that the illustrated network connection is exemplary and other means of establishing a communication link between computers can be used.

The computer 1102 is associated with any wireless device or entity that is deployed and operates in wireless communication, such as a printer, scanner, desktop and/or portable computer, a portable data assistant (PDA), communication satellite, or wireless detectable tag. It operates to communicate with any equipment or place and telephone. This includes at least Wi-Fi and Bluetooth wireless technology. Accordingly, the communication may be a predefined structure as in a conventional network or simply ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) enables a connection to the Internet or the like without a wired connection. Wi-Fi is a wireless technology such as a cell phone that allows a computer, for example, a computer to transmit and receive data indoors and outdoors, ie anywhere within the base station's coverage area. Wi-Fi networks use a wireless technology called IEEE 802.11 (a,b,g, etc.) to provide a secure, reliable and high-speed wireless connection. Wi-Fi can be used to connect computers to each other, to the Internet, and to a wired network (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz radio bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). have.

Those of ordinary skill in the art of the present disclosure will understand that information and signals can be represented using any of a variety of different technologies and techniques. For example, data, instructions, instructions, information, signals, bits, symbols and chips that can be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields Or particles, or any combination thereof.

Those of ordinary skill in the art of the present disclosure may use various examples of logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein in electronic hardware, (convenience For this, it will be understood that it may be implemented by various forms of program or design code (referred to herein as “software”) or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art of the present disclosure may implement the functions described in various ways for each particular application, but such implementation decisions should not be interpreted as being outside the scope of the present disclosure.

Various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term “manufactured article” includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory Devices (eg, EEPROMs, cards, sticks, key drives, etc.), but are not limited to these. In addition, various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is understood that the specific order or hierarchy of steps in the processes presented is an example of exemplary approaches. Based on design priorities, it is understood that within the scope of the present disclosure, a specific order or hierarchy of steps in the processes may be rearranged. The accompanying method claims provide elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

Descriptions of the presented embodiments are provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art of the present disclosure. The general principles defined herein can be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure should not be limited to the embodiments presented herein, but should be interpreted in the broadest scope consistent with the principles and novel features presented herein.

Claims (14)

A computer program stored on a computer readable storage medium, the computer program, when executed on one or more processors, performs the following operations for recognizing characters on a document image, the operations being:
Determining a window based at least in part on a character block containing one or more letters on the document image;
Recognizing a character by calculating the character block and the window through a character recognition model including one or more pre-trained network functions;
Re-adjusting the window based at least in part on the recognized character; And
Sliding the window in the character block to output characters recognized as character strings;
Containing,
A computer program stored on a computer readable storage medium.
According to claim 1,
The determining of a window based at least in part on a character block including one or more letters on the document image may include:
Determining the character block based on a pixel having a color value on the document image;
Determining a pixel line having a color value among pixel lines in one direction in the character block as a reference line; And
Determining the window based at least in part on the reference line;
Containing,
A computer program stored on a computer readable storage medium.
According to claim 1,
The determining of a window based at least in part on a character block including one or more letters on the document image may include:
Determining the text block through extraction of connection elements on the document image;
Determining whether a character is extracted on the character block; And
When the text is extracted on the text block, determining the window to correspond to the size of the text;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 3,
The operation of determining the text block through extraction of a connection element on the document image may include:
Determining a unit character block from which each connection element on the document image is extracted; And
Grouping the unit character blocks located within a predetermined distance among the unit character blocks into the one character block;
Containing,
A computer program stored on a computer readable storage medium.
According to claim 1,
The determining of a window based at least in part on a character block including one or more letters on the document image may include:
Determining an attribute of a character located in the character block through context analysis of the document image; And
Determining the window based on the attribute of the letter;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 5,
The operation of determining the attribute of the text located in the text block through context analysis of the document image may include:
Performing a semantic analysis on recognized words recognized in other character blocks positioned vertically or horizontally of the character block;
Determining an item of the character block based on a result of semantic analysis of the recognized word; And
Determining a type of character matching the item as an item and attribute of the character;
And
The operation of determining the window based on the attribute of the letter,
Determining the window based on the type of character matching the item;
Containing,
A computer program stored on a computer readable storage medium.
According to claim 1,
The operation of recognizing a character by calculating the character block and the window through a character recognition model including one or more pre-trained network functions includes:
Generating a character image by cropping the character block based on the window; And
Recognizing a character by calculating the character image through the character recognition model;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 7,
The operation of recognizing a character by calculating the character image through the character recognition model may include:
Outputting a character recognized by calculating the character image through the character recognition model and a recognition reliability of the character;
Determining whether the characters of the character image are properly recognized based on the character recognition reliability and a predetermined reference recognition reliability; And
Outputting the text when it is determined that the text of the text image is properly recognized;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 8,
The operation of determining whether the characters of the character image is properly recognized based on the recognition reliability of the characters and the predetermined reference recognition reliability,
Changing at least one of the size and shape of the window when the recognition reliability of the text is less than a predetermined reference recognition reliability;
Containing,
A computer program stored on a computer readable storage medium.
According to claim 1,
The operation of sliding the window in the character block and outputting the recognized character as a character string,
Determining a sliding direction for moving the window within the character block;
Recognizing a letter on one or more letter images generated based on the window and the sliding direction; And
Generating the character string based on at least one of the recognized position and order of the characters in the character block;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 10,
The operation of determining a sliding direction for moving the window in the character block is:
Determining the sliding direction based on the shape of the character block;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 10,
The operation of generating the character string based on at least one of the recognized position and order of the characters in the character block may include:
Extracting one or more recognized words by grouping one or more letters included in the string into semantic units; And
Changing the window for one or more letters included in the character string or changing the sliding direction based on a result of comparing the recognized word with one or more reference words stored in a memory;
Containing,
A computer program stored on a computer readable storage medium.
A method for recognizing text on a document image,
Determining a window based at least in part on a character block containing one or more letters on the document image;
Recognizing a character by calculating the character block and the window through a character recognition model including one or more pre-trained network functions;
Re-adjusting the window based at least in part on the recognized character; And
Sliding the window in the character block to output characters recognized as character strings;
Containing,
A method for recognizing text on document images.
A computing device for recognizing text on a document image,
A processor comprising one or more cores; And
Memory;
Including,
The processor,
Determine a window based at least in part on a character block containing one or more letters on the document image,
Characters are recognized by calculating the character block and the window through a character recognition model including one or more pre-trained network functions,
Readjust the window based at least in part on the recognized text, and
Sliding the window in the character block to output the recognized character as a character string,
A computing device for recognizing text on a document image.

Images