KR102200608B1 - Apparatus and method for character detection - Google Patents

Abstract

Disclosed are a character detection device and a method thereof. According to one embodiment of the present invention, the character detection device comprises: an image input unit receiving a test image including one or more cursive characters; a detection unit detecting one or more cursive character regions from the input test image using a character detection model learned based on a plurality of learning images each including one or more cursive characters; and an erroneous detection region removing unit removing an erroneous detection region among the plurality of cursive character regions based on whether overlapping between the detected plurality of cursive character regions is detected, when a plurality of cursive character regions are detected in the test image.

Description

Character detection device and method {APPARATUS AND METHOD FOR CHARACTER DETECTION}

Embodiments of the present invention relate to character detection technology.

Conventional Optical Character Recognition (OCR) techniques include a post-processing method using a projection method, a recognition method based on segmentation, a method of combining words after detecting each character, and a recognition and detection method based on a tree structure. Etc. are suggested. However, this existing method has a problem in that it is vulnerable to various noises.

In order to solve this problem, a deep learning-based optical character recognition technique has been recently proposed, but most of these techniques detect and recognize sentences or words rather than individual characters, and the target is also mainly limited to English. There is a limit in detecting cursive fonts written in old documents.

Republic of Korea Patent Publication No. 10-1777601 (2017.09.06.)

Embodiments of the present invention are to provide a character detection apparatus and method.

The character detection apparatus according to an embodiment of the present invention uses an image input unit that receives a test image including one or more cursive characters, and a character detection model learned based on a plurality of training images each including one or more cursive characters. And a detection unit for detecting at least one cursive character area in the input test image, and when a plurality of cursive character areas are detected in the test image, the plurality of cursive characters based on whether or not overlap between the detected plurality of cursive character areas And an erroneous detection area removal unit for removing the erroneous detection area among the character areas.

The plurality of learning images may include scanned images of documents created using one or more cursive characters.

The plurality of training images may also include an image generated by combining at least one of individual character images for each of the plurality of cursive characters and one or more background images not including the cursive characters.

The character detection model may include a convolutional neural network (CNN) structure.

The CNN structure includes a plurality of first convolution layers to which a feature map extracted from the test image is input, and a plurality of second convolution layers to which a feature map passing through the first convolution layer is input. It may include a convolutional layer.

The character detection model may also generate each of the detected one or more cursive character regions in the form of a bounding box.

The character detection model may also generate a confidence score for each of the detected one or more cursive character regions.

The confidence score may be a probability that a cursive character is included in the detected one or more cursive character regions.

The erroneous detection area removal unit may detect the erroneous detection area based on a size of each of the plurality of cursive character areas and an overlapping area between the plurality of cursive character areas.

The erroneous detection area removing unit further sets one area of the plurality of cursive character areas as a reference area based on the confidence score, and the following Equation 1

[Equation 1]

는 상기 기준 영역의 크기,

는 상기 나머지 영역 중 하나의 크기,

는 기 설정된 값)을 이용하여 상기 기준 영역과 상기 복수의 초서체 문자 영역 중 상기 기준 영역을 제외한 나머지 영역 각각 사이의 IOU(Intersection Over Union) 값을 산출하며, 상기 나머지 영역 중 상기 IOU 값이 기 설정된 값 이상인 영역을 상기 오검출 영역으로 판단할 수 있다.(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is a preset value) to calculate an IOU (Intersection Over Union) value between the reference area and the remaining areas except for the reference area among the plurality of cursive character areas, and the IOU value among the remaining areas is preset. An area greater than or equal to the value may be determined as the erroneous detection area.

The character detection method according to an embodiment of the present invention includes receiving a test image including one or more cursive characters, each using a character detection model learned based on a plurality of training images including at least one cursive character. Detecting at least one cursive character area in the input test image, and when a plurality of cursive character areas are detected in the test image, the plurality of cursive character areas based on whether or not overlap between the detected plurality of cursive character areas And removing the erroneous detection area from among the areas.

The plurality of learning images may include scanned images of documents created using one or more cursive characters.

The plurality of training images may also include an image generated by combining at least one of individual character images for each of the plurality of cursive characters and one or more background images not including the cursive characters.

The character detection model may include a convolutional neural network (CNN) structure.

The CNN structure includes a plurality of first convolution layers to which a feature map extracted from the test image is input, and a plurality of second convolution layers to which a feature map passing through the first convolution layer is input. It may include a convolutional layer.

The character detection model may also generate each of the detected one or more cursive character regions in the form of a bounding box.

The character detection model may also generate a confidence score for each of the detected one or more cursive character regions.

The confidence score may be a probability that a cursive character is included in the detected one or more cursive character regions.

In the removing of the erroneous detection area, the erroneous detection area may be detected based on a size of each of the plurality of cursive character areas and a size of an overlapping area between the plurality of cursive character areas.

The step of removing the erroneous detection area may further include setting one of the plurality of cursive character areas as a reference area based on the confidence score, Equation 1 below

[Equation 1]

는 상기 기준 영역의 크기,

는 상기 나머지 영역 중 하나의 크기,

는 기 설정된 값)을 이용하여 상기 기준 영역과 상기 복수의 초서체 문자 영역 중 상기 기준 영역을 제외한 나머지 영역 각각 사이의 IOU(Intersection Over Union) 값을 산출하는 단계 및 상기 나머지 영역 중 상기 IOU 값이 기 설정된 값 이상인 영역을 아래의 수학식 1을 이용하여 오검출 영역으로 판단하는 단계를 더 포함할 수 있다.(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is a preset value), calculating an IOU (Intersection Over Union) value between the reference region and the remaining regions other than the reference region among the plurality of cursive character regions, and the IOU value among the remaining regions. It may further include determining an area greater than or equal to the set value as an erroneous detection area using Equation 1 below.

The character detection method may further include generating a performance index using Equation 2 below after removing the erroneous detection area.

[Equation 2]

(At this time, # of FA = number of true positive detection boxes + number of false positive detection boxes-number of detected ground truth boxes, # of GT = number of correct answer boxes)

According to embodiments of the present invention, by considering the areas of bounding boxes generated as a result of detecting a cursive area, it is possible to detect a cursive area including a cursive character without overlapping.

In addition, according to the embodiments of the present invention, a plurality of cropped characters and a plurality of background images not including characters are arbitrarily transformed and then combined to use the generated image for learning a character detection model, It is possible to improve the detection performance of a cursive area containing characters.

In addition, according to embodiments of the present invention, instead of False Positives Per Image (FPPI), which determines the number of false positives per image, as an indicator of false detection performance, False Positives Per Character (FPPC), which determines the number of false positives per character, is used. By using it, it is possible to efficiently evaluate erroneous detection performance.

1 is a diagram for explaining a conventional character detection network structure
2 is a diagram for explaining the structure of a character detection network according to an embodiment of the present invention
3 is a block diagram of a character detection apparatus according to an embodiment of the present invention
4 is a diagram for explaining a data set according to an embodiment of the present invention
5 is a diagram for explaining a structure of a convolutional neural network (CNN) in a character detection model according to an embodiment of the present invention
6 is a flowchart of a character detection method according to an embodiment of the present invention
7 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to aid in a comprehensive understanding of the methods, devices, and/or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, when it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout this specification. The terms used in the detailed description are only for describing embodiments of the present invention, and should not be limiting. Unless explicitly used otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "comprising" or "feature" are intended to refer to certain features, numbers, steps, actions, elements, some or combination thereof, and one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, actions, elements, any part or combination thereof.

Meanwhile, an embodiment of the present invention may include a program for performing the methods described in the present specification on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include a program command, a local data file, a local data structure, or the like alone or in combination. The media may be specially designed and configured for the present invention, or may be commonly used in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and specially configured to store and execute program instructions such as ROM, RAM, and flash memory. Hardware devices are included. Examples of the program may include not only machine language codes created by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

1 is a diagram 100 for explaining the structure of a conventional cursive area detection network. As shown, the conventional cursive region detection network structure includes multi-scale convolution layers, and the detection result obtained by passing through the multi-scale convolution layer is a confidence score. ) And areas.

At this time, in the conventional cursive region detection network structure, the detection result is aligned based on the confidence score, and the region corresponding to the detection result having the highest confidence score value is set as the reference region, and Equation 1 below between the reference region and the remaining regions When the IOU value exceeds a preset value, the classical non-maximum suppression (NMS) algorithm is used to remove the remaining detection results except for the detection result with the highest confidence score. .

[Equation 1]

는 기준 영역의 크기,

는 나머지 영역 중 하나의 크기)(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas)

Due to this structure, the convolution layer used for detecting some large objects among the convolution layers of multiple scales becomes unnecessary when detecting the cursive area of the cursive character written on the document, and the cursive area detected in the process of sorting and removing the detection results. Since the area of the bounding box for indicating the Bounding box is not considered, detection of an overlapped cursive area may occur.

FIG. 2 is a diagram 200 for explaining the structure of a cursive area detection network according to an embodiment of the present invention. As shown, the structure of a cursive area detection network according to an embodiment of the present invention includes a character detection model trained using a plurality of training images each including one or more cursive characters, and the character detection model is a single scale (single-scale) convolutional layer. The detection result passing through the convolutional layer is displayed in the form of a score based on a preset item and a box representing the detected cursive character, and the corresponding score and area are grouped and aligned. After that, if the calculation result through a preset formula based on the score-area pair with the highest score exceeds the preset value, a modified ratio that removes the remaining detection result pairs except for the detection result pair with the highest corresponding item value. Apply the maximum suppression algorithm.

Due to the single-scale convolutional layer structure, the cursive area of cursive characters, which is relatively small compared to general objects (for example, cars, people, etc.), is compared to the existing structure for Optical Character Recognition (OCR). It can be detected relatively simply, and redundant detection of the cursive area can be prevented by considering the area of the bounding box for indicating the detected cursive character area.

3 is a block diagram of a character detection apparatus 300 according to an embodiment of the present invention.

Referring to FIG. 3, a character detection apparatus 300 according to an embodiment of the present invention includes an image input unit 310, a detection unit 320, and an erroneous detection area removal unit 330.

The image input unit 310 receives a test image including one or more cursive characters.

The detection unit 320 detects at least one cursive character region from the input test image using a character detection model learned based on a plurality of training images each including at least one cursive character.

According to an embodiment of the present invention, the plurality of training images may include scanned images of documents created using one or more cursive characters.

4 is a diagram illustrating a data set 400 according to an embodiment of the present invention. The dataset 400 refers to a set of images including a plurality of training images used for learning a character detection model in the present invention.

Referring to FIG. 4, the dataset 400 includes a cropped character 410 for each of a plurality of cursive characters, at least one image 420 including cursive characters, and at least one background not including cursive characters. It may be composed of an image 430.

Specifically, the dataset 400 further includes an image generated by combining at least one of individual character images 410 for each of a plurality of cursive characters and one or more background images 430 not including cursive characters. Can include.

In more detail, the dataset 400 is arbitrarily adjusted among one or more background images 430 that do not include cursive characters, one image rotated at an arbitrary angle, and individual character images 410 for each of a plurality of cursive characters. An image generated by combining one or more images having a font size or sharpness may be further included.

In this case, the sharpness may be adjusted to a predetermined value or more so that one or more individual character images 410 included in the generated image do not disappear.

According to an embodiment of the present invention, the character detection model may include a convolutional neural network (CNN) structure.

5 is a diagram illustrating a CNN structure 500 in a character detection model according to an embodiment of the present invention.

5, the CNN structure 500 is a feature map passing through a plurality of first convolutional layers 510 and first convolutional layers 510 into which a feature map extracted from a test image is input. It may include a plurality of second convolutional layers 520 are input.

Specifically, for example, the first convolution layer 510 may be composed of the sum of seven lower convolution layers, but the number of lower convolution layers is not limited thereto.

Also, for example, the second convolutional layer 520 may be configured as a sum of eight lower convolutional layers, but the number of lower convolutional layers is not limited thereto.

Also, the feature map that has passed through the first convolutional layer 510 may have a smaller scale than before passing through the first convolutional layer 510 by a convolution operation.

According to an embodiment of the present invention, the CNN structure 500 may include a part of the Resnet-34 network as part of the structure. For example, a feature map that has passed through the Resnet-34 network passes through 7 3x3 convolution layers, then passes through one convolution layer with a stride of 2 and is halved in size, then 8 3x3 convolutions. Can pass through layers.

According to an embodiment of the present invention, the character detection model may calculate a localization loss by calculating a mean square error (MSE) through Equations 2 to 4 below.

[Equation 2]

[Equation 3]

는 위치 검출 오차, i는 초서체 문자가 검출된 바운딩 박스의 인덱스 값, j는 정답(ground truth) 바운딩 박스의 인덱스(index) 값, N은 정답 바운딩 박스에 대응되는 디폴트 바운딩 박스의 수, d는 디폴트 바운딩 박스, m은 바운딩 박스의 좌상단의 가로축 좌표값(cx), 바운딩 박스의 좌상단의 세로축 좌표값(cy), 바운딩 박스의 폭(width, w) 및 바운딩 박스의 높이(height, h)로 구성된 집합의 원소, k는 특정 카테고리(category), l은 예측된 바운딩 박스, g는 정답 바운딩 박스를 나타낸다.At this time,

Is the position detection error, i is the index value of the bounding box in which the cursive character is detected, j is the index value of the ground truth bounding box, N is the number of default bounding boxes corresponding to the correct answer bounding box, d is Default bounding box, m is the horizontal coordinate value of the upper left corner of the bounding box (cx), the vertical axis coordinate value of the upper left corner of the bounding box (cy), the width of the bounding box (width, w), and the height of the bounding box (height, h). An element of the configured set, k denotes a specific category, l denotes a predicted bounding box, and g denotes a correct answer bounding box.

는 아래의 수학식 4를 이용하여 산출될 수 있다.Also,

Can be calculated using Equation 4 below.

[Equation 4]

However, the character detection model can calculate the position detection error by calculating a smooth L1 error or an L2 error instead of the mean square error, and the method of calculating the position detection error is necessarily limited to this. no.

In addition, the character detection model may calculate a confidence loss through Equation 5 below, and combine the confidence error with the above-described position detection error through Equation 6 below to derive an error function for learning. .

[Equation 5]

[Equation 6]

는 신뢰 오차,

은 학습을 위한 오차 함수,

는 초서체 문자가 검출된 바운딩 박스의 인덱스 값,

는 초서체 문자가 검출되지 않은 바운딩 박스의 인덱스 값, j는 정답(ground truth) 바운딩 박스의 순서, N은 정답 바운딩 박스에 대응되는 디폴트 바운딩 박스의 수, d는 디폴트 바운딩 박스, p는 특정 카테고리, l은 예측된 바운딩 박스, g는 정답 바운딩 박스, c는 카테고리 p에 대한 신뢰 점수를 나타낸다.At this time,

Is the confidence error,

Is the error function for learning,

Is the index value of the bounding box where the cursive character was detected,

Is the index value of the bounding box where no cursive character is detected, j is the order of the ground truth bounding box, N is the number of default bounding boxes corresponding to the correct answer bounding box, d is the default bounding box, p is a specific category, l is the predicted bounding box, g is the correct answer bounding box, and c is the confidence score for category p.

는 아래의 수학식 7을 이용하여 산출될 수 있다.Also,

Can be calculated using Equation 7 below.

[Equation 7]

In addition, according to an embodiment of the present invention, the character detection model is in the form of a confidence score and a bounding box using a feature map that has passed through the first convolution layer 510 and the second convolution layer 520. You can generate results.

Specifically, the character detection model may generate each of the detected one or more cursive character regions in the form of a bounding box. In addition, the character detection model may generate a confidence score for each of the detected one or more cursive character regions.

In this case, the bounding box may refer to a rectangular section indicating the area in which the cursive character is predicted to be present in the document image, and the result value of the bounding box type is the horizontal axis coordinate value of the upper left corner of the bounding box, It can be generated by calculating four values of the vertical axis coordinate value, the width of the bounding box, and the height of the bounding box. However, the result value in the form of a bounding box is not necessarily limited to this, and if it is a method that can specify a rectangular section indicating the area in which the cursive character is expected to be present in the document image, the result value in the form of a bounding box is generated. Can be used to

In addition, the confidence score may be a probability that a cursive character is included in the detected one or more cursive character regions. For example, if the confidence score corresponding to the detected cursive character area is 0.9, it may indicate that the cursive character is included in the bounding box representing the cursive character area with a 90% probability.

When a plurality of cursive character regions are detected in the test image, the erroneous detection region removal unit 330 removes the erroneous detection region from among the plurality of cursive character regions based on whether or not overlapping between the detected plurality of cursive character regions is detected.

For example, when there is another character area included in any one character area among a plurality of detected cursive character areas, the erroneous detection area removal unit 330 may determine the included character area as a false detection area and remove it. have.

According to an embodiment of the present invention, the erroneous detection area removal unit 330 may detect the erroneous detection area based on the size of each of the plurality of cursive character areas and the size of an overlapping area between the plurality of cursive character areas.

According to an embodiment of the present invention, the erroneous detection area removal unit 330 may use a modified non-maximum value suppression algorithm that considers the area of the bounding box generated as a result value of the detection unit 320.

In addition, according to an embodiment of the present invention, the erroneous detection area removal unit 330 sets one of the plurality of cursive character areas as a reference area based on the confidence score, and Equation 8 below

[Equation 8]

는 기준 영역의 크기,

는 나머지 영역 중 하나의 크기,

는 기 설정된 값)을 이용하여 기준 영역과 복수의 초서체 문자 영역 중 기준 영역을 제외한 나머지 영역 각각 사이의 IOU(Intersection Over Union) 값을 산출하며, 나머지 영역 중 IOU 값이 기 설정된 값 이상인 영역을 오검출 영역으로 판단할 수 있다.(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is a preset value) to calculate the IOU (Intersection Over Union) value between the reference area and the remaining areas except for the reference area among the plurality of cursive character areas, and the area in which the IOU value is greater than or equal to the preset value is calculated. It can be determined as a detection area.

In more detail, the erroneous detection area removal unit 330 aligns a plurality of bounding boxes generated as a result of the detection unit 320 and a plurality of confidence scores corresponding thereto, based on the bounding box having the highest confidence score. As a result, an IOU (Intersection Over Union) value with other bounding boxes may be calculated through Equation 8 above. In addition, the erroneous detection area removal unit 330 determines and removes other bounding boxes having IOU values greater than or equal to a preset value as duplicate detection for the same cursive character, and this process can be repeated for all remaining bounding boxes. have.

6 is a flowchart illustrating a method of detecting a character according to an embodiment of the present invention.

The method illustrated in FIG. 6 may be performed, for example, by the character detection apparatus 300 illustrated in FIG. 3. In the illustrated flowchart, the method is described by dividing the method into a plurality of steps, but at least some of the steps are performed in a different order, combined with other steps, performed together, omitted, divided into detailed steps, or not shown. One or more steps may be added and performed.

Referring to FIG. 6, first, the character detection device 300 receives a test image including one or more cursive characters (610).

Thereafter, the character detection apparatus 300 detects at least one cursive character region from the input test image using a character detection model learned based on a plurality of training images each including at least one cursive character (operation 620).

According to an embodiment of the present invention, the plurality of training images may include scanned images of documents created using one or more cursive characters.

In addition, according to an embodiment of the present invention, the plurality of training images further includes an image generated by combining at least one of individual character images for each of the plurality of cursive characters and one or more background images not including cursive characters. Can include.

According to an embodiment of the present invention, as shown in FIG. 5, the character detection model may include a CNN structure 500.

In this case, the character detection model may calculate the position detection error by calculating the mean square error through Equations 2 to 4 above.

However, the character detection model may calculate a position detection error by calculating a smooth L1 error or an L2 error instead of a mean square error, and a method of calculating the position detection error is not necessarily limited thereto.

In addition, the character detection model may calculate a confidence error through Equation 5 above, and combine the confidence error with the position detection error described above through Equation 6 below to derive an error function for learning.

According to an embodiment of the present invention, the CNN structure includes a plurality of first convolution layers into which a feature map extracted from a test image is input, and a feature map passing through the first convolution layer. It may include a plurality of second convolutional layers.

According to an embodiment of the present invention, the character detection model generates a confidence score and a result value in the form of a bounding box using a feature map that has passed through the first convolution layer 510 and the second convolution layer 520. I can.

Specifically, the character detection model may generate each of the detected one or more cursive character regions in the form of a bounding box. In addition, the character detection model may generate a confidence score for each of the detected one or more cursive character regions.

In this case, the confidence score may be a probability that a cursive character is included in the detected one or more cursive character regions.

Thereafter, when a plurality of cursive character areas are detected in the test image, the character detection apparatus 300 removes the erroneous detection area from among the plurality of cursive character areas based on whether or not overlapping between the detected cursive character areas (630). ).

According to an embodiment of the present invention, the character detection apparatus 300 may use a modified non-maximum value suppression algorithm that considers the area of the bounding box generated as a result of the step 630 of detecting the cursive area.

In addition, according to an embodiment of the present invention, the character detection apparatus 300 may detect an erroneous detection area based on the size of each of the plurality of cursive character areas and the size of an overlapping area between the plurality of cursive character areas.

Specifically, based on the confidence score, the character detection apparatus 300 sets one of the plurality of cursive character areas as a reference area, and Equation 9 below

[Equation 9]

는 기준 영역의 크기,

는 나머지 영역 중 하나의 크기,

는 기 설정된 값)를 이용하여 기준 영역과 복수의 초서체 문자 영역 중 기준 영역을 제외한 나머지 영역 각각 사이의 IOU(Intersection Over Union) 값을 산출하며, 나머지 영역 중 IOU 값이 기 설정된 값 이상인 영역을 오검출 영역으로 판단할 수 있다.(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is a preset value) to calculate the IOU (Intersection Over Union) value between the reference area and the rest of the areas except the reference area among the plurality of cursive character areas, and the area where the IOU value is greater than or equal to the preset value is calculated. It can be determined as a detection area.

In more detail, the character detection apparatus 300 arranges a plurality of bounding boxes generated as a result of the detection unit 320 and a plurality of confidence scores corresponding thereto together, so that the character detection apparatus 300 is arranged based on the bounding box having the highest confidence score. By calculating the IOU (Intersection Over Union) value with other bounding boxes through Equation 9 of, other bounding boxes having an IOU value equal to or greater than a preset value are determined to be duplicate detection of the same cursive character and removed. This may mean a method of repeating the process for all remaining bounding boxes.

In the character detection method according to an embodiment of the present invention, after removing the erroneous detection area, generating a performance index may be further included.

Specifically, without using False Positives Per Image (FPPI), which corresponds to the false detection performance evaluation index that determines the number of false detections per image, the number of false positives per character is determined through Equation 10 below. Character detection performance can be evaluated using False Positives Per Character (FPPC), which is an indicator of false detection performance.

[Equation 10]

(At this time, # of FA = number of true positives + number of false positives-number of detected ground truth boxes, # of GT = number of correct answer boxes)

7 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12. In one embodiment, the computing device 12 may be a cursive area detection device.

The computing device 12 includes at least one processor 14, a computer-readable storage medium 16 and a communication bus 18. The processor 14 may cause the computing device 12 to operate according to the exemplary embodiments mentioned above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16. The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions are configured to cause the computing device 12 to perform operations according to an exemplary embodiment when executed by the processor 14 Can be.

The computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer-readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, the computer-readable storage medium 16 includes memory (volatile memory such as random access memory, nonvolatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other types of storage media that can be accessed by computing device 12 and store desired information, or a suitable combination thereof.

The communication bus 18 interconnects the various other components of the computing device 12, including the processor 14 and computer-readable storage medium 16.

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24. The input/output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input/output device 24 may be connected to other components of the computing device 12 through the input/output interface 22. The exemplary input/output device 24 includes a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touch pad or a touch screen), a voice or sound input device, and various types of sensor devices and/or a photographing device. Input devices and/or output devices such as display devices, printers, speakers, and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12, and may be connected to the computing device 102 as a separate device distinct from the computing device 12. May be.

Although the exemplary embodiments of the present invention have been described in detail above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications may be made to the above-described embodiments without departing from the scope of the present invention. . Therefore, the scope of the present invention should not be defined by being limited to the described embodiments, and should not be determined by the claims to be described later, but also by the claims and equivalents.

10: computing environment
12: computing device
14: processor
16: computer readable storage medium
18: communication bus
20: program
22: input/output interface
24: input/output device
26: network communication interface
300: character detection device
310: image input unit
320: detection unit
330: erroneous detection area removal unit

Claims (22)

An image input unit for receiving a test image including one or more cursive characters;
A detector configured to detect at least one cursive character region from the input test image using a character detection model learned based on a plurality of training images each including at least one cursive character; And
When a plurality of cursive character areas are detected in the test image, including a false detection area removal unit for removing a false detection area from among the plurality of cursive character areas based on whether or not overlapping between the detected plurality of cursive character areas,
The character detection model generates, for each of the detected one or more cursive character areas, a confidence score, which is a probability that a cursive character is included in each of the detected one or more cursive character areas,
The erroneous detection area removing unit sets one of the plurality of cursive character areas as a reference area based on the confidence score, and among the reference area and the plurality of cursive character areas using Equation 1 below. A character detection device for calculating an Intersection Over Union (IOU) value between each of the remaining areas excluding the reference area, and determining an area in which the IOU value is equal to or greater than a preset value among the remaining areas as the erroneous detection area.
[Equation 1]

(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is the preset value)
The method according to claim 1,
The plurality of learning images, including an image of a scanned document created using one or more cursive characters, character detection device.
The method according to claim 1,
The plurality of learning images includes an image generated by combining at least one of individual character images for each of the plurality of cursive characters and one or more background images that do not include cursive characters.
The method according to claim 1,
The character detection model includes a convolutional neural network (CNN) structure.
The method of claim 4,
The CNN structure includes a plurality of first convolution layers to which a feature map extracted from the test image is input, and a plurality of second convolution layers to which a feature map passing through the first convolution layer is input. Character detection device comprising a convolutional layer.
The method according to claim 1,
The character detection model generates each of the detected one or more cursive character regions in the form of a bounding box.
delete delete delete delete Receiving a test image including one or more cursive characters;
Detecting at least one cursive character region from the input test image using a character detection model learned based on a plurality of training images each including at least one cursive character; And
In the case where a plurality of cursive character areas are detected in the test image, removing an erroneous detection area from among the plurality of cursive character areas based on whether or not overlapping between the detected plurality of cursive character areas is detected,
The character detection model generates, for each of the detected one or more cursive character areas, a confidence score, which is a probability that a cursive character is included in each of the detected one or more cursive character areas,
The removing of the erroneous detection area may include setting one of the plurality of cursive character areas as a reference area based on the confidence score;
Calculating an Intersection Over Union (IOU) value between the reference region and the remaining regions of the plurality of cursive character regions except for the reference region using Equation 1 below; And
The character detection method further comprising determining an area in which the IOU value is equal to or greater than a preset value among the remaining areas as an erroneous detection area using Equation 1 below.
[Equation 1]

(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is the preset value)
The method of claim 11,
The plurality of learning images, including an image of a scanned document created using one or more cursive characters, character detection method.
The method of claim 11,
The plurality of training images includes an image generated by combining at least one of individual character images for each of the plurality of cursive characters and one or more background images not including cursive characters.
The method of claim 11,
The character detection model includes a convolutional neural network (CNN) structure.
The method of claim 14,
The CNN structure includes a plurality of first convolution layers to which a feature map extracted from the test image is input, and a plurality of second convolution layers to which a feature map passing through the first convolution layer is input. Character detection method comprising a convolutional layer.
The method of claim 11,
The character detection model generates each of the detected one or more cursive character regions in the form of a bounding box.
delete delete delete delete The method of claim 11,
After the step of removing the erroneous detection region, the method further comprising generating a performance index using Equation 2 below.
[Equation 2]

(At this time, # of FA = number of true positive detection boxes + number of false positive detection boxes-number of detected ground truth boxes, # of GT = number of correct answer boxes)
As a computer program stored in a non-transitory computer readable storage medium,
The computer program includes one or more instructions, and when the instructions are executed by a computing device having one or more processors, the computing device causes:
Receive a test image containing one or more cursive characters,
One or more cursive character regions are detected from the input test image using a character detection model learned based on a plurality of training images each including at least one cursive character,
When a plurality of cursive character areas are detected in the test image, an erroneous detection area among the plurality of cursive character areas is removed based on whether or not the detected plurality of cursive character areas overlap,
The character detection model generates, for each of the detected one or more cursive character areas, a confidence score, which is a probability that a cursive character is included in each of the detected one or more cursive character areas,
The computing device may set one of the plurality of cursive character areas as a reference area based on the confidence score, and the reference area among the plurality of cursive character areas using Equation 1 below. A computer program for calculating an Intersection Over Union (IOU) value between each of the remaining areas except for the area, and determining an area in which the IOU value is equal to or greater than a preset value among the remaining areas as the erroneous detection area.
[Equation 1]

(At this time,

Is the size of the reference area,

Is the size of one of the remaining areas,

Is the preset value)

Images