KR102351578B1 - Apparatus and method for recognizing character - Google Patents

Abstract

A method for recognizing characters in input data by a character recognition device, the method comprising: inputting the input data into a character detection model; obtaining position information of a word region in input data based on output data output from the character detection model; extracting partial data corresponding to the acquired location information from the input data; and inputting partial data into a character recognition model to recognize characters in the partial data.

Description

Character recognition device and method for character recognition using the same

The present disclosure relates to the field of data processing. More specifically, the present disclosure relates to a character recognition apparatus and method for recognizing characters in data such as an image.

According to the development of fintech (fintech) technology, a service for storing card information in a mobile phone or the like and enabling easy payment is provided. A technology for recognizing and storing information such as a card number and expiration date in a real card image such as a credit card or a check card is one of the core technologies for a simple payment service.

However, in recognizing characters in a card image, there are a large number of embossed printed characters in the card, and since the card background is various, there is a technical barrier in accurately recognizing the card number and the expiration date.

A character recognition apparatus and a character recognition method according to an embodiment have a technical task of accurately and quickly recognizing characters from data such as images.

In addition, a character recognition apparatus and a character recognition method according to an embodiment have a technical task of contributing to the development of the fintech industry by accurately recognizing characters in an image of a real card.

A character recognition method according to an embodiment includes inputting input data into a character detection model; obtaining position information of a word region in the input data based on output data output from the character detection model; extracting partial data corresponding to the obtained location information from the input data; and inputting the partial data into a character recognition model to recognize characters in the partial data.

A character recognition apparatus according to another embodiment includes: a processor; and a memory for storing at least one instruction, wherein the processor inputs input data into a character detection model according to the at least one instruction, and enters the input data based on output data output from the character detection model. A character in the partial data may be recognized by acquiring positional information of a word region, extracting partial data corresponding to the acquired positional information from the input data, and inputting the partial data into a character recognition model.

A character recognition apparatus and a character recognition method according to an embodiment may accurately and quickly recognize characters from data such as images.

In addition, the character recognition apparatus and the character recognition method according to an embodiment can accurately recognize characters in the image of a real card, thereby contributing to the development of the fintech industry.

However, the effects achievable by the character recognition device and the character recognition method according to the embodiment are not limited to those mentioned above, and other effects not mentioned above are described below in the technical field to which the present disclosure belongs. It will be clearly understood by those of ordinary skill in the art.

In order to more fully understand the drawings cited herein, a brief description of each drawing is provided.
1 is a diagram illustrating a character recognition apparatus according to an exemplary embodiment.
2 is a flowchart illustrating a character recognition method according to an exemplary embodiment.
3 is a view for explaining a process of recognizing a character through a character recognition apparatus according to an exemplary embodiment.
4 is an exemplary diagram illustrating output data output by a character detection model.
5 is a diagram for explaining a method of acquiring position information of a word region in input data based on output data output from a character detection model.
FIG. 6 is a view for explaining the binarization process and the merging process shown in FIG. 5 .
FIG. 7 is a diagram for explaining a process of determining a word box shown in FIG. 5 .
8 is a diagram for explaining the structure of a feature extraction model according to an embodiment.
9 is a diagram for explaining a structure of a character recognition model according to an exemplary embodiment.
10 is a flowchart illustrating a method of training a character detection model according to an embodiment.
11 is a diagram for explaining a method of generating a first GT score map.
12 is a diagram for explaining a method of generating a second GT score map.
13 is a diagram for explaining a method of determining a connection box between text boxes adjacent to each other.
14 is a block diagram illustrating a configuration of a character recognition apparatus according to an exemplary embodiment.
15 is a diagram illustrating a server device and a client device to which a character recognition device according to an embodiment can be applied.

Since the present disclosure can make various changes and can have various embodiments, specific embodiments are illustrated in the drawings, and this will be described through detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present disclosure.

In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter, the detailed description thereof will be omitted. In addition, numbers (eg, first, second, etc.) used in the description process of the embodiment are only identification symbols for distinguishing one component from other components.

Also, in this specification, when a component is referred to as “connected” or “connected” to another component, the component may be directly connected to or directly connected to the other component, but the opposite is particularly true. Unless there is a description to be used, it will be understood that it may be connected or connected through another element in the middle.

In addition, in the present specification, components expressed as '~ part (unit)', 'module', etc. are two or more components combined into one component, or two or more components for each more subdivided function. may be differentiated into In addition, each of the components to be described below may additionally perform some or all of the functions of other components in addition to the main functions they are responsible for, and some of the main functions of each component may have different functions. It goes without saying that it may be performed exclusively by the component.

In addition, in the present specification, 'letter' may mean a basic character unit constituting a word or sentence. For example, in the case of English, each alphabet may correspond to a letter, and in the case of a number, '0' Each number of '9' may correspond to a letter, and in the case of Korean, a character in which a consonant and a vowel are combined (eg, 'a'), a character in which a consonant, a vowel, and a consonant are combined (eg, 'Strong'), a single consonant (eg, 'a'), and a singly written vowel (eg, 'a') may correspond to the letter. Also, the character may include a symbol (eg, '/', '-', etc.).

Also, in the present specification, a 'word' may mean a character unit including at least one character. The letters constituting the 'word' may not be spaced apart from each other by more than a predetermined interval. A 'word' may consist of one letter. For example, the infinitive 'a' in English may correspond to a 'word' if it consists of one letter but is spaced apart from the surrounding letters by a predetermined distance or more.

Also, in the present specification, a 'character group' may mean at least one character recognized in any one piece of data to be described later.

Hereinafter, embodiments according to the technical spirit of the present disclosure will be described in detail in turn.

1 is a diagram illustrating a character recognition apparatus 100 according to an exemplary embodiment.

The character recognition apparatus 100 according to an embodiment acquires input data 10 and recognizes a character 50 in the input data 10 . The input data 10 may include an image obtained by photographing a physical card such as a debit card or a credit card, or as described below, outputted from the feature extraction model 800 based on an image photographed of a physical card, etc. It may include a feature map.

The character recognition apparatus 100 may recognize and store card information such as a card number and an expiration date in the input data 10 . The card information recognized and stored by the character recognition device 100 may be used to pay a price for purchasing a product or the like.

Hereinafter, an operation of the character recognition apparatus 100 will be described with reference to FIGS. 2 and 3 .

2 is a flowchart for explaining a method for recognizing a character according to an embodiment, and FIG. 3 is a diagram for explaining a process for recognizing a character through the character recognition apparatus 100 according to an embodiment.

In step S210 , the character recognition apparatus 100 inputs the input data 10 to the character detection model 410 . The character recognition apparatus 100 may store the character detection model 410 in advance. The character detection model 410 may be trained based on learning data.

In step S220 , the character recognition apparatus 100 acquires position information of the word region in the input data 10 based on the output data 30 output from the character detection model 410 .

The output data 30 output from the character detection model 410 indicates the position of a point where the character is expected to exist in the input data 10 . The character recognition apparatus 100 obtains position information of a word region including at least one character in the input data 10 based on the output data 30 .

In step S230 , the character recognition apparatus 100 extracts the partial data 40 corresponding to the location information of the word area from the input data 10 . In an embodiment, when a plurality of location information of a word region is obtained, a plurality of partial data 40 corresponding to each location information may be extracted from the input data 10 .

In step S240 , the character recognition apparatus 100 inputs the partial data 40 into the character recognition model 420 to recognize the character 50 included in the partial data 40 . When there are a plurality of partial data 40 , the character recognition apparatus 100 inputs each of the plurality of partial data 40 to the character recognition model 420 , and the character 50 included in each of the plurality of partial data 40 . can recognize them

In an embodiment, the character recognition apparatus 100 may input the output data 30 of the character detection model 410 together with the partial data 40 into the character recognition model 420 . Since the output data 30 of the character detection model 410 may include location information of individual characters in the input data 10 , the character recognition accuracy of the character recognition model 420 may be further improved.

The character recognition apparatus 100 may store the recognized character or transmit it to an external device through a network.

4 is an exemplary diagram illustrating an example of the output data 30 output by the character detection model 410 .

The output data 30 includes a first score map 31 indicating the probability that characters in the input data 10 exist on a data space (eg, image space) corresponding to the input data 10, and the input data ( 10) It may include a second score map 33 indicating the connectivity between the characters in the data space corresponding to the input data 10 .

A value (eg, a pixel value) stored at each position in the first score map 31 may indicate a probability that a character exists in the input data 10 corresponding to the position. In addition, a value (eg, a pixel value) stored at each position in the second score map 33 may indicate a probability that a plurality of characters are adjacent to each other in the input data 10 corresponding to the position.

The size of the first score map 31 and the second score map 33 may be the same as that of the input data 10 to facilitate calculation of the positional correspondence relationship.

As will be described later, the character detection model 410 includes a first score map 31 and a second score map 33 similar to the first GT (ground truth) score map and the second GT score map generated in response to the training data. ) can be trained to output.

The character recognition apparatus 100 may determine the location information of the word region in the input data 10 based on the first score map 31 and the second score map 33 . For this, refer to FIGS. 5 to 7 . to explain

5 is a diagram for explaining a method of acquiring position information of a word region in the input data 10 based on the output data 30 output from the character detection model 410, and FIG. 6 is the diagram shown in FIG. It is a diagram for explaining a binarization process and a merging process, and FIG. 7 is a diagram for explaining a word box determination process shown in FIG. 5 .

In steps S510 and S520, the character recognition apparatus 100 compares the data values in the first score map 31 with a threshold value to binarize the first score map 31, and the second score map 33 ) binarize the second score map 33 by comparing the data values in In one example, the character recognition apparatus 100 changes data values greater than or equal to a threshold value among data values in the first score map 31 and the second score map 33 to a first value, and sets data values less than the threshold value. It can be changed to the second value.

As shown in FIG. 6 , data having a value greater than or equal to a threshold in the first score map 31 and the second score map 33 is a binarized first score map 601 and a binarized second score map 603 . . The map 603 may be changed to have the second value.

The threshold value for binarization of the first score map 31 and the threshold value for binarization of the second score map 33 may be the same as or different from each other.

In step S530 , the character recognition apparatus 100 merges the binarized first score map 601 and the binarized second score map 603 . For example, the character recognition apparatus 100 may generate the merge map 605 by adding data values in the first binarized score map 601 and the binarized second score map 603 or performing an or operation. . For example, as shown in FIG. 6 , data having a first value in the first binarized score map 601 and the second binarized score map 603 may be included in the merge map 605 . In this way, the merge map 605 can be divided into regions 606 in which characters in the input data 10 are likely to exist and regions where it is not.

In step S540 , the character recognition apparatus 100 determines a word box 610 indicating an area including characters by using the merge map 605 .

For example, in the merge map 605, at least some of the regions 606 adjacent to each other having the same (or in the same range) value are determined as the word region, and a word box 610 including the determined word region. can be decided In an embodiment, the character recognition apparatus 100 may label each word area in order to distinguish the word areas in the merge map 605 .

In an embodiment, the character recognition apparatus 100 may perform additional verification to verify whether each of the recognized regions 606 includes an actual word using the merge map 605 . Specifically, for example, regions 606 adjacent to each other having the same (or within the same range) value in the merge map 605 are set as word candidate regions, and a first score corresponding to each data in the word candidate region is set. If one or more of the values of the map 601 are higher than the predetermined threshold, the corresponding word candidate area may be determined as the word area. That is, it is possible to verify whether each word candidate region corresponds to a word region by comparing the largest value among the values of the first score map 601 corresponding to each word candidate region with a threshold value.

In this way, it is possible to filter the cases determined as the word candidate area because there is a background similar to the character.

In an embodiment, the character recognition apparatus 100 may determine a word box 610 having a minimum size including an area of data verified as corresponding to the word area.

The character recognition apparatus 100 uses the determined position information of the word box 610 (eg, the position value of the corners of the word box 610 on the input data 10 or the merge map 605) to determine the position of the word region. information can be determined.

When the position information of the word region is determined, the character recognition apparatus 100 extracts partial data 40 corresponding to the position information from the input data 10 , and uses the extracted partial data 40 as a character recognition model 420 . ) to recognize characters in the partial data 40 .

As described above, the input data 10 input to the character detection model 410 may include a feature map output from the feature detection model 800 based on the original image. 8 is a diagram for explaining the structure of the feature detection model 800 .

The original image 20 may be input to the feature detection model 800 . Here, the original image 20 refers to an image input to the feature detection model 800 , and does not mean that it is not a copied image or a modified image of an image captured by the first card.

The original image 20 is a first convolutional layer 805 , a second convolutional layer 810 , a third convolutional layer 815 , a fourth convolutional layer 820 , and a fifth convolutional layer 825 . and the sixth convolutional layer 830 is convolutional. The output of the sixth convolution layer 830 and the output of the fifth convolution layer 825 are concatenated and input to the first up-convolution layer 835 , and the first up-convolution layer 835 . The values input as ? are input to the first up-sampling layer 840 through a convolution process 836 , a batch normalization 837 , a convolution process 838 , and a batch normalization 839 . The output of the first up-sampling layer 840 is concatenated with the output of the fourth convolutional layer 820 and processed in the second up-convolutional layer 845 and the second up-sampling layer 850 . The output of the second up-sampling layer 850 is concatenated with the output of the third convolutional layer 815 and processed in the third up-convolutional layer 855 and the third up-sampling layer 860, and the processing result is It is concatenated with the output of the second convolution layer 810 and is input to the fourth up-convolution layer 865 . In addition, a result output from the fourth up-convolution layer 865 may be used as the input data 10 .

In an embodiment, the horizontal size and vertical size of the input data 10 may be 1/2 of the horizontal size and vertical size of the original image 20 , but is not limited thereto.

The structure of the feature detection model 800 shown in FIG. 8 is only one example, and the number and processing order of the convolutional layer, the up-convolutional layer, and the up-sampling layer may be variously modified.

9 is a diagram for explaining the structure of a character recognition model 420 according to an embodiment.

The character recognition model 420 receives the partial data 40 extracted from the input data 10 , and recognizes the characters 50 in the partial data 40 . The character recognition model 420 may include a convolutional network 421 , a recurrent neural network (RNN) 423 , and a decoder 425 .

The convolutional network 421 includes at least one convolutional layer, and convolution processes the partial data 40 to extract a feature map. In one example, the convolutional network 421 may include known VGG, ResNet, etc., but in one embodiment the character recognition model 420 is extracted from a feature map (ie, input data) of the original image 20 . Since the partial data 40 may be input, the number of required convolution layers may be small.

The RNN 423 extracts a sequence of feature vectors from the feature map corresponding to the partial data 40 . The RNN 423 may determine a context relationship between consecutive feature vectors through bidirectional long-short-term memory (bi-LSTM).

The decoder 425 extracts characters from sequence information of feature vectors. The decoder 425 may perform an attention step and a generation step. In the attention step, the decoder 425 calculates a weight indicating whether to extract information from which sequence, and a weight in the generation step. can be applied to a sequence, and individual letters can be extracted through long-short-term memory (LSTM).

Meanwhile, according to an embodiment, the character recognition apparatus 100 may classify character groups recognized in each of the various partial data 40 according to a predetermined criterion. In one example, the character recognition apparatus 100 may determine the character group as the first type of information if a predetermined symbol (eg, /) is included in the character group recognized in the partial data 40 . have. Since a predetermined symbol for distinguishing a year and a month is generally included in the validity period of the card, the character recognition apparatus 100 detects a corresponding character when the character group recognized in the partial data 40 includes a predetermined symbol. The group can be determined by the validity period information.

If the number of character groups including a predetermined symbol is plural, the character recognition apparatus 100 selects a large character group with a number corresponding to the year (eg, a number located to the right of the symbol). It can be determined by the validity period information. If the card includes the validity period and issue date, the year included in the validity period will be greater than the year included in the issuance date. can be determined with

Also, according to an embodiment, the character recognition apparatus 100 may determine character groups that do not include a predetermined symbol among character groups recognized in each of the plurality of partial data 40 as the second type of information. The second type of information may include, for example, card number information.

Also, in an embodiment, the character recognition apparatus 100 aligns character groups recognized in each of the plurality of partial data 40 according to positions of the plurality of partial data 40 in the input data 10 . can do. For example, the character recognition apparatus 100 may align the character groups in a Z-scan method based on the upper left corner in the input data 10 .

The character recognition apparatus 100 may determine whether character recognition is necessary based on the number of characters included in a predetermined number of consecutively arranged character groups among the sorted character groups. For example, the character recognition apparatus 100 performs character recognition correctly when there is a predetermined number of consecutively arranged character groups each including a predetermined number of numbers among the aligned character groups, so that the character recognition is not required. can decide not to. In general, the card number includes 16 numbers, but in the sense that four numbers form one letter group, the letter recognition apparatus 100 includes four letter groups including four numbers among the aligned letter groups. If they are arranged consecutively, it may be determined that re-recognition of the characters is not necessary.

Also, according to an exemplary embodiment, when a predetermined symbol does not exist in the character groups recognized from the various partial data 40 , the character recognition apparatus 100 may determine that the character needs to be re-recognized.

When re-recognition of characters is required, the character recognition apparatus 100 may output information indicating that the image needs to be re-photographed through a speaker, a monitor, or the like, or inform an external device through a network. In an embodiment, when the character recognition apparatus 100 determines that the character needs to be re-recognized while recognizing the character from the preview image of the camera, the character recognition apparatus 100 may re-recognize the character from the preview image continuously captured through the camera.

Hereinafter, a method of training the character detection model 410 will be described with reference to FIGS. 10 to 13 .

10 is a flowchart illustrating a training method of the character detection model 410 according to an exemplary embodiment.

In step S1010 , the character recognition apparatus 100 provides a first GT score map 71 indicating the probability of the existence of a character in the training data 60 on the data space, and the data connection between the characters in the training data 60 . A second GT score map 73 shown in space is obtained. The horizontal size and vertical size of the training data 60 may be the same as the horizontal size and vertical size of the input data 10 . In addition, the horizontal size and vertical size of the training data 60 are the same as the horizontal size and vertical size of the first GT score map 71, and the horizontal size and vertical size of the second GT score map 73. can

In an embodiment, the training data 60 may include an image obtained by photographing an object such as a card or a feature map extracted based on the image, similarly to the above-described original image 20 .

The character recognition device 100 may directly generate at least one of the first GT score map 71 and the second GT score map 73 from the training data 60, or the first GT through a network or an external manager At least one of a score map 71 and a second GT score map 73 may be received.

* Values in the first GT score map 71 may represent the probability that a letter is located in the learning data 60 at the corresponding point. In addition, values in the second GT score map 73 may indicate a probability that a plurality of letters are adjacent to each other at a corresponding point.

In step S1020 , the character recognition apparatus 100 inputs the training data 60 into the character detection model 410 .

In step S1030, the first and second score maps output from the character detection model 410 corresponding to the training data 60, respectively, and the first GT score map 71 and the second GT score map 73 are obtained. The internal weight of the character detection model 410 may be updated according to the comparison result.

A loss value may be calculated according to a result of comparison between the first and second score maps, respectively, and the first GT score map 71 and the second GT score map 73 . The loss value may correspond to, for example, an L2 Loss value. In addition to the loss value, various methods such as L1 loss and smooth L1 loss can be used. The calculated loss value is input to the character detection model 410 , and the character detection model 410 may update an internal weight according to the loss value.

* FIG. 11 is a diagram for explaining a method of generating the first GT score map 71 , and FIG. 12 is a diagram for explaining a method of generating the second GT score map 73 . Also, FIG. 13 is a view for explaining a method of determining the connection box 63a between the letter boxes 62a and 62b adjacent to each other.

Referring to FIG. 11 , word boxes 61a , 61b , 61c , 61d and 61e are determined for word regions including at least one letter in the training data 60 . And, according to the number of letters included in the word boxes 61a, 61b, 61c, 61d, and 61e, the word boxes 61a, 61b, 61c, 61d, and 61e have at least one letter box 62a, 62b, 62c, 62d. ) is divided into For example, when four letters are included in any one word box, the corresponding word box may be divided into a total of four letter boxes. A first GT score map 71 may be generated by synthesizing a predetermined image 1100, for example, a 2D Gaussian image, in each of the text boxes 62a, 62b, 62c, and 62d.

12 and 13 , connection boxes 63a, 63b, and 63c positioned on the boundary line L between adjacent letter boxes among the plurality of letter boxes 62a, 62b, 62c, and 62d are determined. and a predetermined image 1100 , for example, a 2D Gaussian image, may be synthesized in the connection boxes 63a, 63b, and 63c to generate a second GT score map 73 .

The connection boxes 63a , 63b , and 63c may be determined by connecting a plurality of points set in internal spaces of adjacent letter boxes. Specifically, as shown in FIG. 13 , a connection box 63a connecting two points in the left text box 62a and two points in the right text box 62b among the adjacent text boxes 62a and 62b ) can be determined.

In one example, the upper and lower triangles are formed by connecting the lower left corner and the upper right corner among the corners of the left letter box 62a and the right letter box 62b adjacent to each other, and connecting the upper left corner and the lower right corner The connection box 63a may be determined by determining and connecting the midpoints of the corresponding triangles.

14 is a block diagram illustrating a configuration of a character recognition apparatus 100 according to an exemplary embodiment.

Referring to FIG. 14 , the character recognition apparatus 100 may include a memory 1410 , a communication module 1430 , and a processor 1450 . At least one instruction may be stored in the memory 1410 , and the processor 1450 may control the character detection and training of the character detection model 410 according to the at least one instruction.

14 illustrates only one memory 1410 and one processor 1450, the character recognition apparatus 100 may include a plurality of memories and/or a plurality of processors.

The memory 1410 may store a character detection model 410 and a character recognition model 420 . Also, the memory 1410 may further store the feature extraction model 800 .

The processor 1450 may input the input data 10 into the character detection model 410 and obtain position information of the word region in the input data 10 based on the output data output from the character detection model 410 . have. The processor 1450 may input partial data corresponding to the acquired location information into the character recognition model 420, and store the character information output from the character recognition model 420 in the memory 1410 or other storage device. have.

In an embodiment, the processor 1450 may train at least one of the character detection model 410 , the character recognition model 420 , and the feature extraction model 800 based on the training data 60 .

The communication module 1430 transmits/receives data to and from an external device through a network. For example, the communication module 1430 may transmit/receive an image to/from an external device or may transmit/receive character information recognized in the input data 10 to/from the external device.

15 is a diagram illustrating a server device 1510 and a client device 1520 to which the character recognition device 100 according to an embodiment can be applied.

The character recognition apparatus 100 may be implemented as a server device 1510 or as a client device 1520 .

When the character recognition device 100 is implemented as the server device 1510 , the server device 1510 may receive an image from the client device 1520 , recognize and store characters in the received image. In one example, the server device 1510 may transmit character information recognized in the image received from the client device 1520 to the client device 1520 . In addition, the server device 1510 receives training data from an external device including the client device 1520 or uses the training data stored therein to detect a character detection model 410 , a character recognition model 420 , and a feature extraction model. At least one of (800) may control training.

When the character recognition device 100 is implemented as the client device 1520 , the client device 1520 recognizes characters in an image captured by the camera of the client device 1520 or an image stored in the client device 1520 . can be saved

In an embodiment, the client device 1520 may receive data for executing at least one of the character detection model 410 , the character recognition model 420 , and the feature extraction model 800 from the server device 1510 . . The client device 1520 selects at least one of a character detection model 410 , a character recognition model 420 , and a feature extraction model 800 using an image captured through the camera module, an image stored in the internal memory, or an image received from an external device. You can recognize characters by entering them into

The client device 1520 receives training data from an external device or trains at least one of the character detection model 410, the character recognition model 420, and the feature extraction model 800 using the learning data stored therein. You can also control it. According to an embodiment, the server device 1510 that provides data for executing at least one of the character detection model 410 , the character recognition model 420 , and the feature extraction model 800 to the client device 1520 is training data Training of at least one of the character detection model 410 , the character recognition model 420 , and the feature extraction model 800 may be controlled based on the . In this case, the server device 1510 transmits only the weight information updated as a result of training to the client device 1520 , and the client device 1520 performs a character detection model 410 , a character recognition model 420 and At least one of the feature extraction models 800 may be updated.

15 shows a desktop PC as a client device 1520, but is not limited thereto, and the client device 1520 includes a notebook computer, a smartphone, a tablet PC, an artificial intelligence (AI) robot, an AI speaker, a wearable device, and the like. may include

Meanwhile, the above-described embodiments of the present disclosure can be written as a program that can be executed on a computer, and the written program can be stored in a medium.

The medium may continuously store a computer executable program, or may be a temporary storage for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributedly on a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store for distributing applications, sites supplying or distributing other various software, and servers.

In the above, the technical idea of the present disclosure has been described in detail with reference to preferred embodiments, but the technical idea of the present disclosure is not limited to the above embodiments, and those of ordinary skill in the art within the scope of the technical spirit of the present disclosure Various modifications and changes are possible by the person.

100: character recognition device
1410: memory
1430: communication module
1450: Processor
1510: server device
1520: client device

Claims (14)

A method for recognizing characters in input data by a character recognition device, the method comprising:
inputting the input data into the character detection model;
obtaining position information of a word region in the input data based on output data output from the character detection model;
extracting partial data corresponding to the obtained location information from the input data; and
Recognizing characters in the partial data by inputting the partial data into a character recognition model;
and the output data includes at least one of a first score map indicating a probability that a character exists in the input data and a second score map indicating a connection between characters in the input data.
According to claim 1,
The character recognition method is
and determining whether re-recognition of characters is necessary based on the number of characters included in the character groups recognized in the partial data.
According to claim 1,
The step of obtaining location information of the word region includes:
binarizing the first score map and the second score map according to a result of comparing values in the first score map and the second score map with a threshold value;
merging the binarized first score map and the binarized second score map;
determining an area having a predetermined value in the merge map; and
and determining position information of a word region including the determined region.
4. The method of claim 3,
The step of determining the location information of the word region includes:
determining a word box having a minimum size including the determined area; and
and determining the determined position information of the word box as position information of the word region.
According to claim 1,
The character recognition method is
obtaining a first GT score map indicating a probability that a letter exists in the training data and a second GT score map indicating a connection between the letters in the training data; and
Further comprising the step of inputting the learning data into the character detection model,
The internal weight of the character detection model is determined according to a result of comparison between the first and second score maps respectively output from the character detection model in response to the learning data and the first GT score map and the second GT score map. Character recognition method, characterized in that it is updated.
6. The method of claim 5,
Acquiring the first GT score map comprises:
determining a word box including a word in the training data;
dividing the word box into a plurality of letter boxes according to the determined number of letters included in the word box; and
and generating the first GT score map by synthesizing a predetermined image in each of the plurality of letter boxes.
7. The method of claim 6,
The step of generating the second GT score map comprises:
determining a connection box positioned on a boundary line between adjacent text boxes among the plurality of text boxes; and
and generating the second GT score map by synthesizing a predetermined image in the connection box.
According to claim 1,
The character recognition method is
and determining the character group as the first type of information when a predetermined symbol is included in the character group recognized from the partial data.
According to claim 1,
The character recognition method is
arranging character groups recognized in the partial data according to positions in the input data; and
and determining whether re-recognition of a character is necessary based on the number of characters included in the consecutively arranged character group among the sorted character groups.
According to claim 1,
The step of recognizing the letter is,
and recognizing characters in the partial data by further inputting output data output from the character detection model to the character recognition model.
According to claim 1,
The input data is
Character recognition method comprising a feature map output from the feature extraction model in correspondence to the original image.
A program stored in a medium to execute the character recognition method of claim 1 in combination with hardware.
processor; and
a memory for storing at least one instruction;
The processor according to the at least one instruction,
Input the input data to the character detection model,
obtaining position information of a word region in the input data based on the output data output from the character detection model;
extracting partial data corresponding to the obtained location information from the input data,
Input the partial data to a character recognition model to recognize the characters in the partial data,
and the output data includes at least one of a first score map indicating a probability that a character exists in the input data and a second score map indicating a connection between characters in the input data.
A method for recognizing characters in input data by a character recognition device, the method comprising:
inputting the input data into the character detection model;
obtaining position information of a word region in the input data based on output data output from the character detection model;
extracting partial data corresponding to the obtained location information from the input data;
inputting the partial data into a character recognition model to recognize characters in the partial data; and
and determining the character group as a predetermined type of information when a predetermined symbol is included in the character group recognized from the partial data.

Images