KR102206604B1 - Apparatus and method for recognizing character - Google Patents

Abstract

A method of recognizing a character in input data by a character recognition device, the method comprising: inputting the input data into a character detection model; Acquiring location information of a word area in the input data based on output data output from the character detection model; Extracting partial data corresponding to the obtained location information from the input data; And recognizing a character in the partial data by inputting the partial data into a character recognition model.

Description

Character recognition device and character recognition method by it {APPARATUS AND METHOD FOR RECOGNIZING CHARACTER}

The present disclosure relates to the field of data processing. More specifically, the present disclosure relates to a character recognition apparatus and method for recognizing characters from data such as images.

With the development of fintech technology, a service that allows convenient payment by storing card information in a mobile phone or the like has been provided. A technology for recognizing and storing information such as card number and expiration date in real card images such as credit cards and debit cards is one of the core technologies for simple payment service.

However, in recognizing letters in a card image, there are many embossed letters in the card and various card backgrounds, so there is a technical barrier to accurately recognizing the card number and expiration date.

A character recognition apparatus according to an embodiment and a character recognition method therefor is a technical task to accurately and quickly recognize characters from data such as images.

In addition, the character recognition apparatus according to an exemplary embodiment and a method of character recognition according to the character recognition apparatus accurately recognizes characters in an image of a physical card, thereby contributing to the development of the fintech industry as a technical task.

A character recognition method according to an embodiment includes the steps of inputting input data into a character detection model; Acquiring location information of a word area in the input data based on output data output from the character detection model; Extracting partial data corresponding to the acquired location information from the input data; And inputting the partial data into a character recognition model to recognize a character in the partial data.

A character recognition apparatus according to another embodiment includes: a processor; And a memory for storing at least one instruction, wherein the processor inputs input data into a character detection model according to the at least one instruction, and the input data is stored based on output data output from the character detection model. A character in the partial data may be recognized by obtaining location information of a word area, extracting partial data corresponding to the obtained location information from the input data, and inputting the partial data into a character recognition model.

The apparatus for recognizing characters and the method for recognizing characters according to the embodiment may accurately and quickly recognize characters from data such as images.

In addition, the apparatus for recognizing letters and the method for recognizing letters according to the embodiment may contribute to the development of the fintech industry by accurately recognizing letters within an image of a physical card.

However, the effects that can be achieved by the character recognition apparatus and the character recognition method according to the embodiment are not limited to those mentioned above, and other effects not mentioned are the technical fields to which the present disclosure belongs from the following description. It will be able to be clearly understood by those of ordinary skill.

In order to more fully understand the drawings cited in the present specification, a brief description of each drawing is provided.
1 is a diagram illustrating a character recognition apparatus according to an exemplary embodiment.
2 is a flow chart illustrating a method for recognizing letters according to an exemplary embodiment.
3 is a diagram for describing a process of recognizing a character through a character recognition apparatus according to an exemplary embodiment.
4 is an exemplary diagram showing output data output by a character detection model.
5 is a diagram for explaining a method of acquiring position information of a word region in input data based on output data output from a character detection model.
6 is a diagram illustrating a binarization process and a merging process shown in FIG. 5.
FIG. 7 is a diagram illustrating a process of determining a word box shown in FIG. 5.
8 is a diagram illustrating a structure of a feature extraction model according to an exemplary embodiment.
9 is a diagram illustrating a structure of a character recognition model according to an embodiment.
10 is a flowchart illustrating a method of training a character detection model according to an exemplary embodiment.
11 is a diagram for explaining a method of generating a first GT score map.
12 is a diagram for describing a method of generating a second GT score map.
13 is a diagram for describing a method of determining a connection box between text boxes adjacent to each other.
14 is a block diagram illustrating a configuration of a character recognition apparatus according to an exemplary embodiment.
15 is a diagram illustrating a server device and a client device to which a character recognition device according to an exemplary embodiment can be applied.

In the present disclosure, various changes may be made and various embodiments may be provided, and specific embodiments are illustrated in the drawings, and will be described through detailed description. However, this is not intended to limit the present disclosure to a specific embodiment, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present disclosure.

In describing the embodiments, when it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter, the detailed description thereof will be omitted. In addition, numbers (eg, first, second, etc.) used in the description process of the embodiment are merely identification symbols for distinguishing one component from another component.

In addition, in the present specification, when one component is referred to as "connected" or "connected" to another component, the one component may be directly connected to the other component or may be directly connected, It should be understood that as long as there is no substrate to be used, it may be connected or may be connected via another component in the middle.

In addition, in the components expressed as'~ unit (unit)','module', etc. in the present specification, two or more components are combined into one component, or two or more components are divided into more subdivided functions. It can also be differentiated into. In addition, each of the components to be described below may additionally perform some or all of the functions that other components are responsible for in addition to its own main function, and some of the main functions that each component is responsible for are different. It goes without saying that it may be performed exclusively by components.

In addition, in the present specification,'character' may mean a basic unit of characters constituting a word or sentence. For example, in English, each alphabet may correspond to a letter, and in the case of numbers, '0' Each of the numbers of to '9' may correspond to a letter, and in the case of Korean, a character combined with a consonant and a vowel (for example,'ga'), a character combined with a consonant, vowel, and consonant (for example, 'Kang'), consonants written alone (for example,'ㄱ'), and vowels written alone (for example,'ㅏ') may correspond to letters. In addition, the letters may include symbols (eg,'/','-', etc.).

In addition, in the present specification,'word' may mean a character unit including at least one letter. The letters constituting the'word' may not be spaced apart from each other by more than a predetermined distance. The'word' can also consist of a single letter. For example, the infinitive'a' in English may be composed of a single letter, but may correspond to a'word' if it is separated by a predetermined distance from the surrounding letters.

In addition, in the present specification, the'character group' may mean at least one character recognized in any one piece of data to be described later.

Hereinafter, embodiments according to the technical idea of the present disclosure will be described in detail in order.

1 is a diagram illustrating a character recognition apparatus 100 according to an exemplary embodiment.

The character recognition apparatus 100 according to an embodiment acquires the input data 10 and recognizes the character 50 in the input data 10. The input data 10 may include an image photographed of a real card such as a debit card or a credit card, or, as will be described later, output from the feature extraction model 800 based on an image photographed of the physical card. It may also include a feature map.

The character recognition device 100 may recognize and store card information such as a card number and an expiration date from the input data 10. Card information recognized and stored by the character recognition device 100 may be used to pay for purchases of goods.

Hereinafter, the operation of the character recognition apparatus 100 will be described with reference to FIGS. 2 and 3.

2 is a flowchart illustrating a method of recognizing a character according to an exemplary embodiment, and FIG. 3 is a diagram illustrating a process of recognizing a character through the character recognition apparatus 100 according to an exemplary embodiment.

In step S210, the character recognition apparatus 100 inputs the input data 10 into the character detection model 410. The character recognition apparatus 100 may store the character detection model 410 in advance. The character detection model 410 may be trained based on training data.

In step S220, the character recognition apparatus 100 acquires location information of a word area in the input data 10 based on the output data 30 output from the character detection model 410.

The output data 30 output from the character detection model 410 indicates a position of a point in the input data 10 where a character is expected to exist. The character recognition apparatus 100 acquires location information of a word area including at least one character in the input data 10 based on the output data 30.

In step S230, the character recognition apparatus 100 extracts partial data 40 corresponding to the location information of the word region from the input data 10. In an embodiment, when a plurality of pieces of location information of a word area are acquired, a plurality of pieces of partial data 40 corresponding to each location information may be extracted from the input data 10.

In step S240, the character recognition apparatus 100 recognizes the character 50 included in the partial data 40 by inputting the partial data 40 into the character recognition model 420. When there are a plurality of partial data 40, the character recognition apparatus 100 inputs each of the plurality of partial data 40 to the character recognition model 420, and the characters 50 included in each of the plurality of partial data 40 Can recognize them.

In an embodiment, the character recognition apparatus 100 may input the output data 30 of the character detection model 410 together with the partial data 40 as the character recognition model 420. Since the output data 30 of the character detection model 410 may include position information of individual characters in the input data 10, the accuracy of character recognition of the character recognition model 420 may be further improved.

The character recognition device 100 may store the recognized character or transmit it to an external device through a network.

4 is an exemplary diagram illustrating an example of output data 30 output by the character detection model 410.

The output data 30 includes a first score map 31 indicating a probability of a character in the input data 10 on a data space (eg, an image space) corresponding to the input data 10, and the input data ( 10) It may include a second score map 33 that indicates connectivity between internal characters on a data space corresponding to the input data 10.

A value (eg, a pixel value) stored at each location in the first score map 31 may indicate a probability that a character exists in the input data 10 corresponding to the location. In addition, a value (eg, a pixel value) stored at each location in the second score map 33 may indicate a probability that a plurality of letters are adjacent to each other in the input data 10 corresponding to the location.

In order to facilitate the calculation of the position correspondence relationship, the size of the first score map 31 and the second score map 33 may be the same as the input data 10.

As described later, the character detection model 410 includes a first ground truth (GT) score map and a first score map 31 and a second score map 33 similar to the second GT score map generated corresponding to the training data. ) Can be trained to output.

The character recognition apparatus 100 may determine the location information of the word area in the input data 10 based on the first score map 31 and the second score map 33, see FIGS. 5 to 7 This will be explained.

5 is a diagram for explaining a method of acquiring position information of a word area in the input data 10 based on the output data 30 output from the character detection model 410, and FIG. 6 is A diagram for explaining the binarization process and the merging process, and FIG. 7 is a diagram for explaining the word box determination process shown in FIG. 5.

In steps S510 and S520, the character recognition apparatus 100 compares the data values in the first score map 31 with a threshold value to binarize the first score map 31, and performs a second score map 33 ) The second score map 33 is binarized by comparing the data values within the threshold value. In an example, the character recognition apparatus 100 changes data values greater than or equal to a threshold value among data values in the first score map 31 and the second score map 33 to a first value, and changes data values less than the threshold value. It can be changed to a second value.

As shown in FIG. 6, data having a value equal to or greater than a threshold value in the first score map 31 and the second score map 33 are binarized first score map 601 and binarized second score map 603. ) Is changed to have a first value, and data having a value less than the threshold value in the first score map 31 and the second score map 33 are binarized first score map 601 and binarized second score It may be changed to have a second value in the map 603.

A threshold value for binarization of the first score map 31 and a threshold value for binarization of the second score map 33 may be the same or different from each other.

In step S530, the character recognition apparatus 100 merges the binarized first score map 601 and the binarized second score map 603. For example, the character recognition apparatus 100 may generate the merge map 605 by adding data values in the binarized first score map 601 and the binarized second score map 603 or performing an or operation. . For example, as shown in FIG. 6, data having a first value in the binarized first score map 601 and the binarized second score map 603 may be included in the merge map 605 together. In this way, the merge map 605 may be divided into areas 606 in which characters in the input data 10 are likely to exist and areas that do not.

In step S540, the character recognition apparatus 100 determines a word box 610 representing an area including a character using the merge map 605.

For example, in the merge map 605, at least some of the regions 606 that are adjacent to each other having the same (or within the same range) value are determined as a word region, and a word box 610 including the determined word region Can be determined. In an embodiment, the character recognition apparatus 100 may label each word region in order to distinguish the word regions in the merge map 605.

In an embodiment, the character recognition apparatus 100 may perform additional confirmation to verify whether each of the recognized regions 606 using the merge map 605 includes an actual word. Specifically, for example, in the merge map 605, a region 606 having the same (or within the same range) value and connected adjacent to each other is a word candidate region, and a first score corresponding to each data in the word candidate region If at least one of the values of the map 601 is higher than a predetermined threshold, the corresponding word candidate region may be determined as the word region. That is, it is possible to verify whether each word candidate region corresponds to a word region by comparing the largest value among the values of the first score map 601 corresponding to each word candidate region with a threshold value.

In this way, it is possible to filter out cases that have a background similar to a letter and are determined as a word candidate region.

In an embodiment, the character recognition apparatus 100 may determine a word box 610 having a minimum size including an area of data that has been verified to correspond to a word area.

The character recognition apparatus 100 uses the determined position information of the word box 610 (eg, the position values of the corners of the word box 610 on the input data 10 or the merge map 605) to the position of the word area. Can be determined by information.

When the location information of the word area is determined, the character recognition apparatus 100 extracts partial data 40 corresponding to the corresponding location information from the input data 10, and uses the extracted partial data 40 into a character recognition model 420 ), the character in the partial data 40 can be recognized.

As described above, the input data 10 input to the character detection model 410 may include a feature map output from the feature detection model 800 based on the original image. 8 is a diagram for explaining the structure of the feature detection model 800.

The original image 20 may be input as a feature detection model 800. Here, the original image 20 refers to an image input to the feature detection model 800, and does not mean that the image photographed by the first card or the like is not a copied image or a modified image.

The original image 20 includes a first convolution layer 805, a second convolution layer 810, a third convolution layer 815, a fourth convolution layer 820, and a fifth convolution layer 825. And convolution processing is performed in the sixth convolution layer 830. The output of the sixth convolution layer 830 and the output of the fifth convolution layer 825 are concatenated and input to the first up-convolution layer 835, and the first up-convolution layer 835 The values inputted to are input to the first up-sampling layer 840 through convolution processing 836, batch normalization 837, convolution processing 838, and batch normalization 839. The output of the first up-sampling layer 840 is concatenated with the output of the fourth convolutional layer 820 and processed in the second up-convolutional layer 845 and the second up-sampling layer 850. The output of the second up-sampling layer 850 is concatenated with the output of the third convolutional layer 815 and processed in the third up-convolutional layer 855 and the third up-sampling layer 860, and the processing result is It is connected to the output of the second convolution layer 810 and is input to the fourth up-convolution layer 865. In addition, the result output from the fourth up-convolution layer 865 may be used as the input data 10.

In one embodiment, the horizontal size and the vertical size of the input data 10 may be 1/2 of the horizontal size and the vertical size of the original image 20, but are not limited thereto.

The structure of the feature detection model 800 illustrated in FIG. 8 is only an example, and the number and processing order of the convolution layer, the up-convolution layer, and the up-sampling layer may be variously modified.

9 is a diagram illustrating a structure of a character recognition model 420 according to an exemplary embodiment.

The character recognition model 420 receives the partial data 40 extracted from the input data 10 and recognizes the characters 50 in the partial data 40. The character recognition model 420 may include a convolutional network 421, a recurrent neural network (RNN) 423, and a decoder 425.

The convolutional network 421 includes at least one convolutional layer, and extracts a feature map by convolving the partial data 40. In one example, the convolutional network 421 may include a known VGG, ResNet, etc., but in one embodiment, the character recognition model 420 is extracted from a feature map (ie, input data) of the original image 20 Since the partial data 40 can be input, the number of convolutional layers required may be small.

The RNN 423 extracts a sequence of feature vectors from the feature map corresponding to the partial data 40. The RNN 423 may grasp a context relationship between consecutive feature vectors through bi-LSTM (bidirectional long-short-term memory).

The decoder 425 extracts a character from sequence information of feature vectors. The decoder 425 may perform an attention step and a generation step. In the attention step, the decoder 425 calculates a weight indicating whether information is to be extracted from a sequence, and a weight in the generation step. Can be applied to the sequence, and individual letters can be extracted through long-short-term memory (LSTM).

Meanwhile, in an embodiment, the character recognition apparatus 100 may classify character groups recognized in each of the various partial data 40 according to a predetermined criterion. In one example, the character recognition apparatus 100 may determine the character group as the first type of information when a predetermined symbol (eg, /) is included in the character group recognized by the partial data 40. have. Since the validity period in the card generally includes a predetermined symbol for separating the year and the month, the character recognition apparatus 100 may use the character recognition device 100 if a predetermined symbol is included in the character group recognized by the partial data 40. The group can be determined by the expiration date information.

If the number of character groups including a predetermined symbol is plural, the character recognition apparatus 100 selects a character group having a large number corresponding to the year (eg, a number located on the right side of the symbol). It can be determined by the expiration date information. When the card includes an expiration date and an issuance date, since the year included in the validity period will be larger than the year included in the issuance date, the character recognition device 100 uses the character group with a large number corresponding to the year to be used as the valid period information. It can be decided by.

In addition, in an embodiment, the character recognition apparatus 100 may determine character groups that do not include a predetermined symbol among character groups recognized in each of the plurality of partial data 40 as the second type of information. The second type of information may include, for example, card number information.

In addition, in an embodiment, the character recognition apparatus 100 arranges the character groups recognized in each of the plurality of partial data 40 according to the position of the plurality of partial data 40 in the input data 10. can do. As an example, the character recognition apparatus 100 may arrange character groups in a Z scan method based on the upper left corner of the input data 10.

The character recognition apparatus 100 may determine whether or not re-recognition of a character is necessary based on the number of characters included in a predetermined number of character groups arranged in succession among the aligned character groups. As an example, when a predetermined number of character groups that are arranged in succession while each including a predetermined number of the arranged character groups exist, the character recognition is accurately performed, so that re-recognition of the characters is not required. It can be decided not. In general, the card number includes 16 numbers, but in that the 4 numbers form one character group, the character recognition device 100 includes 4 character groups including 4 numbers among the aligned character groups. If they are arranged in a row, it can be determined that re-recognition of the letters is not required.

In addition, in an embodiment, the character recognition apparatus 100 may determine that re-recognition of the character is required if a predetermined symbol does not exist in character groups recognized in the various partial data 40.

When re-recognition of a character is required, the character recognition apparatus 100 may output information indicating that the image needs to be re-captured through a speaker, a monitor, or the like, or may notify an external device through a network. In an embodiment, while the character recognition apparatus 100 recognizes a character from a preview image of a camera, when it is determined that the character needs to be re-recognized, the character may be re-recognized from the preview image continuously photographed through the camera.

Hereinafter, a method of training the character detection model 410 will be described with reference to FIGS. 10 to 13.

10 is a flowchart illustrating a training method of the character detection model 410 according to an embodiment.

In step S1010, the character recognition apparatus 100 data the first GT score map 71 representing the probability of the existence of the characters in the learning data 60 on the data space, and the connectivity between the characters in the learning data 60. A second GT score map 73 shown in space is acquired. The horizontal and vertical sizes of the training data 60 may be the same as the horizontal and vertical sizes of the input data 10. In addition, the horizontal and vertical dimensions of the training data 60 are the same as the horizontal and vertical dimensions of the first GT score map 71, and the horizontal and vertical dimensions of the second GT score map 73 are also the same. I can.

In an embodiment, like the original image 20 described above, the learning data 60 may include an image photographing an object such as a card or a feature map extracted based on the image.

The character recognition apparatus 100 may directly generate at least one of the first GT score map 71 and the second GT score map 73 from the learning data 60, or the first GT score map through a network or an external administrator. At least one of the score map 71 and the second GT score map 73 may be received.

Values in the first GT score map 71 may indicate a probability that a character is located in the learning data 60 at a corresponding point. Also, the values in the second GT score map 73 may indicate a probability that a plurality of letters are adjacent to each other at a corresponding point.

In step S1020, the character recognition apparatus 100 inputs the learning data 60 into the character detection model 410.

In step S1030, each of the first score map and the second score map output from the character detection model 410 in response to the training data 60, and the first GT score map 71 and the second GT score map 73 The internal weight of the character detection model 410 may be updated according to the comparison result.

A loss value may be calculated according to a result of comparing each of the first score map and the second score map with the first GT score map 71 and the second GT score map 73. The loss value may correspond to, for example, the L2 Loss value. In addition to the loss value, various methods such as L1 loss and smooth L1 loss can be used. The calculated loss value is input to the character detection model 410, and the character detection model 410 may update the internal weight according to the loss value.

FIG. 11 is a diagram for explaining a method of generating the first GT score map 71, and FIG. 12 is a diagram for describing a method of generating the second GT score map 73. In addition, FIG. 13 is a diagram for explaining a method of determining a connection box 63a between adjacent character boxes 62a and 62b.

Referring to FIG. 11, word boxes 61a, 61b, 61c, 61d, and 61e are determined for word regions including at least one letter in the learning data 60. In addition, the word boxes 61a, 61b, 61c, 61d, 61e are at least one letter box 62a, 62b, 62c, 62d according to the number of letters included in the word boxes 61a, 61b, 61c, 61d, and 61e. ). For example, when four letters are included in any one word box, the corresponding word box may be divided into a total of four letter boxes. A predetermined image 1100, for example, a 2D Gaussian image, may be synthesized in each of the character boxes 62a, 62b, 62c, and 62d to generate the first GT score map 71.

12 and 13, among a plurality of text boxes 62a, 62b, 62c, 62d, connection boxes 63a, 63b, 63c positioned on the boundary line L between adjacent text boxes are determined. Then, a predetermined image 1100, for example, a 2D Gaussian image may be synthesized in the connection boxes 63a, 63b, and 63c to generate a second GT score map 73.

The connection boxes 63a, 63b, and 63c may be determined by connecting a plurality of points set in the inner spaces of adjacent character boxes. Specifically, as shown in FIG. 13, a connection box 63a connecting two points in the left text box 62a and two points in the right text box 62b among the text boxes 62a and 62b adjacent to each other. ) Can be determined.

In one example, among the corners of the left and right text boxes 62a and 62b adjacent to each other, the lower left corner and the upper right corner are connected, and the upper and lower triangles are connected by connecting the upper left corner and the lower right corner. The connection box 63a may be determined by determining and connecting the midpoints of the corresponding triangles.

14 is a block diagram illustrating a configuration of a character recognition apparatus 100 according to an exemplary embodiment.

Referring to FIG. 14, the character recognition apparatus 100 may include a memory 1410, a communication module 1430, and a processor 1450. At least one instruction may be stored in the memory 1410, and the processor 1450 may control character detection and training of the character detection model 410 according to the at least one instruction.

14 illustrates only one memory 1410 and one processor 1450, the character recognition apparatus 100 may include a plurality of memories and/or a plurality of processors.

The memory 1410 may store a character detection model 410 and a character recognition model 420. In addition, the memory 1410 may further store the feature extraction model 800.

The processor 1450 may input the input data 10 into the character detection model 410 and obtain position information of the word area in the input data 10 based on the output data output from the character detection model 410. have. In addition, the processor 1450 may input partial data corresponding to the acquired location information into the character recognition model 420, and store the character information output from the character recognition model 420 in the memory 1410 or other storage device. have.

In an embodiment, the processor 1450 may train at least one of the character detection model 410, the character recognition model 420, and the feature extraction model 800 based on the training data 60.

The communication module 1430 transmits and receives data to and from an external device through a network. For example, the communication module 1430 may transmit/receive an image with an external device or transmit/receive text information recognized in the input data 10 with an external device.

15 is a diagram illustrating a server device 1510 and a client device 1520 to which the character recognition device 100 according to an exemplary embodiment can be applied.

The character recognition device 100 may be implemented as a server device 1510 or a client device 1520.

When the character recognition device 100 is implemented as the server device 1510, the server device 1510 may receive an image from the client device 1520 and may recognize and store a character in the received image. In an example, the server device 1510 may transmit text information recognized in the image received from the client device 1520 to the client device 1520. In addition, the server device 1510 receives training data from an external device including the client device 1520, or uses the training data stored therein to provide a character detection model 410, a character recognition model 420, and a feature extraction model. At least one of (800) may be controlled.

When the character recognition device 100 is implemented as a client device 1520, the client device 1520 recognizes characters in an image captured by a camera of the client device 1520 or an image stored in the client device 1520 Can be saved.

In one embodiment, the client device 1520 may receive data for execution of at least one of the character detection model 410, the character recognition model 420, and the feature extraction model 800 from the server device 1510. . The client device 1520 uses at least one of a character detection model 410, a character recognition model 420, and a feature extraction model 800 for an image captured through a camera module, an image stored in an internal memory, or an image received from an external device. You can recognize the letters by typing in.

The client device 1520 receives training data from an external device, or trains at least one of a character detection model 410, a character recognition model 420, and a feature extraction model 800 using training data stored therein. You can also control it. Depending on the implementation, the server device 1510 providing data for execution of at least one of the character detection model 410, the character recognition model 420, and the feature extraction model 800 to the client device 1520 is The training of at least one of the character detection model 410, the character recognition model 420, and the feature extraction model 800 may be controlled based on. In this case, the server device 1510 transmits only the weight information updated as a result of training to the client device 1520, and the client device 1520 transmits the text detection model 410, the text recognition model 420, and At least one of the feature extraction models 800 may be updated.

15 illustrates a desktop PC as a client device 1520, but is not limited thereto, and the client device 1520 includes a notebook, a smart phone, a tablet PC, an artificial intelligence (AI) robot, an AI speaker, a wearable device, and the like. Can include.

Meanwhile, the above-described embodiments of the present disclosure can be written as programs that can be executed on a computer, and the written programs can be stored in a medium.

The medium may be one that continuously stores a program executable by a computer, or temporarily stores a program for execution or download. In addition, the medium may be a variety of recording means or storage means in a form in which a single piece of hardware or several pieces of hardware are combined. The medium is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks, and And ROM, RAM, flash memory, and the like may be configured to store program instructions. In addition, examples of other media include an app store that distributes applications, a site that supplies or distributes various software, and a recording medium or storage medium managed by a server.

Above, the technical idea of the present disclosure has been described in detail with reference to a preferred embodiment, but the technical idea of the present disclosure is not limited to the above embodiments, and those having ordinary knowledge in the art within the scope of the technical idea of the present disclosure Various modifications and changes are possible by the user.

100: character recognition device
1410: memory
1430: communication module
1450: processor
1510: server device
1520: client device

Claims (15)

In a method for recognizing a character in input data by a character recognition device,
Inputting input data into a character detection model;
Acquiring location information of a word area in the input data based on output data output from the character detection model;
Extracting partial data corresponding to the acquired location information from the input data; And
Including the step of recognizing a character in the partial data by inputting the partial data into a character recognition model,
And the output data includes a first score map indicating a probability of the existence of a character in the input data and a second score map indicating connectivity between the characters in the input data.
delete The method of claim 1,
Acquiring the location information of the word area,
Binarizing the first score map and the second score map according to a result of comparing values in the first score map and the second score map with a threshold value;
Merging the binarized first score map and the binarized second score map;
Determining an area having a predetermined value in the merge map; And
And determining location information of a word area including the determined area.
The method of claim 3,
The step of determining the location information of the word area,
Determining a word box having a minimum size including the determined area; And
And determining position information of the determined word box as position information of the word area.
The method of claim 1,
The character recognition method,
Obtaining a first GT score map indicating a probability of the existence of a character in the learning data, and a second GT score map indicating a connection between the characters in the learning data; And
Further comprising the step of inputting the learning data into the character detection model,
According to the comparison result of each of the first score map and the second score map output from the character detection model and the first GT score map and the second GT score map corresponding to the training data, the internal weight of the character detection model is Character recognition method, characterized in that updated.
The method of claim 5,
The step of obtaining the first GT score map,
Determining a word box including words in the training data;
Dividing the word box into a plurality of letter boxes according to the number of letters included in the determined word box; And
And generating the first GT score map by synthesizing a predetermined image on each of the plurality of text boxes.
The method of claim 6,
The step of generating the second GT score map,
Determining a connection box positioned on a boundary line between adjacent text boxes among the plurality of text boxes; And
And generating the second GT score map by synthesizing a predetermined image in the connection box.
The method of claim 1,
The character recognition method,
And when a predetermined symbol is included in a character group recognized in the partial data, determining the character group as a first type of information.
The method of claim 1,
The number of partial data extracted from the input data is plural,
The character recognition method,
And aligning the character groups recognized from each of the plurality of partial data according to the positions of the plurality of partial data in the input data.
The method of claim 9,
The character recognition method,
And determining whether re-recognition of a letter is required based on the number of letters included in a predetermined number of letter groups arranged in succession among the sorted letter groups.
The method of claim 1,
Recognizing the letter,
And recognizing a character in the partial data by further inputting output data output from the character detection model to the character recognition model.
The method of claim 1,
The input data,
Character recognition method comprising a feature map output from the feature extraction model corresponding to the original image.
A program stored in a medium to execute the character recognition method of claim 1 in combination with hardware.
Processor; And
Including a memory for storing at least one instruction,
The processor according to the at least one instruction,
Input data into the character detection model,
Obtaining location information of a word area in the input data based on output data output from the character detection model,
Extracting partial data corresponding to the acquired location information from the input data,
By inputting the partial data into a character recognition model to recognize characters in the partial data,
The output data includes a first score map indicating a probability of the existence of a character in the input data and a second score map indicating connectivity between the characters in the input data.
In a method for recognizing a character in input data by a character recognition device,
Inputting input data into a character detection model;
Acquiring location information of a word area in the input data based on output data output from the character detection model;
Extracting partial data corresponding to the acquired location information from the input data; And
Including the step of recognizing a character in the partial data by inputting the partial data into a character recognition model,
Recognizing the letter,
And recognizing a character in the partial data by further inputting output data output from the character detection model to the character recognition model.

Images