KR20190122178A - Method and apparatus for preprocessing image for recognition of character - Google Patents

Abstract

Disclosed are an image preprocessing apparatus for character recognition and a method thereof. The image preprocessing apparatus comprises: an I/O interface receiving an input image including a character; a memory storing the input image; and a processor performing preprocessing on the input image. When the processor performs preprocessing on the input image, the processor edits the input image based on crop area information received from the I/O interface to generate an edited image including the character, automatically performs preprocessing on the edited image to generate a first preprocessed image, and performs preprocessing on the first preprocessing image to generate a second preprocessing image based on the user input received from the I/O interface, when the first preprocessing image does not satisfy a first condition.

Description

Image preprocessing apparatus and method for character recognition {METHOD AND APPARATUS FOR PREPROCESSING IMAGE FOR RECOGNITION OF CHARACTER}

The description below relates to a technique for recognizing characters in a game image.

In order to increase the realism of the game, the characters used in the game may be displayed as part of an image in the game screen. Characters displayed as part of the image may be recognized as characters through preprocessing of the image. However, the accuracy of character recognition may be reduced due to the superposition of special effects or graphic components.

An image preprocessing apparatus according to an embodiment of the present invention includes an I / O interface for receiving an input image including a character, a memory for storing the input image, and a processor for preprocessing the input image. When the processor performs preprocessing on the input image, the processor edits the input image based on crop region information received from the I / O interface to generate an edited image including the character, and the edited image. Automatically preprocess the image to generate a first preprocessed image, and when the first preprocessed image does not meet a first condition related to accuracy related to accuracy, based on a user input received from the I / O interface. The second preprocessed image is generated by performing preprocessing on the first preprocessed image.

The processor may adjust the size of the edited video.

The processor may generate the second preprocessed image based on color information or area information of the character included in the user input.

The processor may generate the second preprocessed image by using a box surrounding a character included in the edited image.

The processor detects one or more edges in the first preprocessed image, generates an outline associated with the character using the one or more edges, generates a box surrounding the outline, and generates an inner region of the box. The second binarization result image may be generated by performing binarization.

The processor may detect an edge gradient from the first preprocessed image, and detect the one or more edges using the size and direction of the edge gradient.

The processor may select an edge that satisfies a second condition related to connectivity between the edges among the one or more edges, and generate the outline using the selected edge.

The processor may generate one or more boxes surrounding the outline, and select one of the one or more boxes that satisfies a third condition related to the characteristic of the character.

The processor generates an image for each channel by separating the inner region of the box for each channel, generates a plurality of binary image for each channel by performing binarization on each of the plurality of channel images, and generates a plurality of images. The second binarization result image may be generated by selecting a binarized channel image satisfying a fourth condition related to accuracy from among the binarized channel images of.

The processor may perform binarization on each of the plurality of channels based on the user input.

The processor may remove noise included in the second binarization result image.

The processor may remove noise included in the second binarization result image based on area information of the character included in the user input.

The processor may generate a first binarization result image by performing binarization on the first preprocessed image.

The processor may generate the plurality of channel-specific images by separating the first preprocessed image for each channel, and generate the first binarization result image by performing binarization on each of the plurality of channel-specific images.

The processor may generate the second preprocessed image based on the characteristic information of the character included in the first preprocessed image.

The processor determines whether the second preprocessed image satisfies the preset condition, and the memory corresponds to the character and the character recognized from the second preprocessed image when the preset condition is satisfied. Can store unique keys.

The character recognition apparatus according to an embodiment may include an I / O interface for receiving an input image including a character, a memory for storing the input image, preprocessing the input image, and outputting the input image from the output image. And a processor for recognizing a character and an output device for outputting the recognized character, wherein the processor is based on crop area information received from the I / O interface when preprocessing the input image. Edit the input image to generate an edited image including the character, and automatically perform preprocessing on the edited image to generate a first preprocessed image, wherein the first preprocessed image is subject to a first condition related to accuracy. If not satisfactory, pre-process the first pre-processed image based on user input received from the I / O interface. The processing may be performed to generate a second preprocessed image.

The output device may include a speaker or a display.

In an image preprocessing method according to an embodiment, in an image preprocessing method performed by a computer including a processor, generating an edited image including the character by editing an input image based on crop region information. Generating a first preprocessed image by automatically performing preprocessing on the edited image, and when the first preprocessed image does not satisfy a first condition related to accuracy, the user input received from the I / O interface. The method may include generating a second preprocessed image by performing preprocessing on the first preprocessed image as a basis.

In a character recognition method according to an embodiment, in a character recognition method performed by a computer including a processor, generating an edited image including the character by editing an input image based on crop area information. Generating a first preprocessed image by automatically performing preprocessing on the edited image; and when the first preprocessed image does not satisfy a first condition related to accuracy, receiving a user input received from the I / O interface. The method may include generating a second preprocessed image by performing preprocessing on the first preprocessed image, and recognizing the character included in the second preprocessed image.

1 is a diagram illustrating a situation in which a character is recognized in a game screen by an image preprocessing apparatus according to an exemplary embodiment.
2 is a diagram illustrating an overall configuration of an image preprocessing apparatus according to an exemplary embodiment.
3 is a flowchart illustrating overall operations of an image preprocessing method according to an exemplary embodiment.
4 is a flowchart illustrating a process of generating a second preprocessed image, according to an exemplary embodiment.
5 is a flowchart illustrating a process of generating a second preprocessed image according to another exemplary embodiment.
6 is a flowchart illustrating an overall operation of an image preprocessing method according to another exemplary embodiment.
7 is a diagram illustrating a process of generating a second preprocessed image by the image preprocessing apparatus according to an exemplary embodiment.
8 shows an example of a general binarization result.
9 illustrates an example of performing binarization on each channel-specific image by the image preprocessing apparatus according to an exemplary embodiment.
FIG. 10 illustrates an example in which binarization is performed on each channel-specific image based on color information input from a user by the image preprocessing apparatus. Referring to FIG.
11 is a diagram illustrating a result of removing noise of an image in which binarization is performed by an image preprocessing apparatus according to an exemplary embodiment.
12 is a flowchart illustrating an operation of a character recognition method, according to an exemplary embodiment.
13 is a diagram illustrating an overall configuration of a character recognition apparatus according to an embodiment.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. However, various changes may be made to the embodiments so that the scope of the patent application is not limited or limited by these embodiments. It is to be understood that all changes, equivalents, and substitutes for the embodiments are included in the scope of rights.

The terminology used herein is for the purpose of description and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

In addition, in the description with reference to the accompanying drawings, the same components regardless of reference numerals will be given the same reference numerals and duplicate description thereof will be omitted. In the following description of the embodiment, when it is determined that the detailed description of the related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

1 is a diagram illustrating a situation in which a character is recognized in a game screen by an image preprocessing apparatus according to an exemplary embodiment.

According to an embodiment, the image preprocessing apparatus 110 may preprocess the game screen to recognize a character included in the game screen. Characters in the preprocessed image can be recognized more accurately.

The game screen may include various special effects or graphic components. For this reason, the characters included in the game screen may not be correctly recognized. In order for characters to be recognized correctly, components other than characters must be removed. To this end, the image preprocessing apparatus 110 may preprocess the game screen.

The image preprocessing apparatus 110 may use an edge box model and an HSV color model. The HSV color model is a way of expressing color and arranging colors according to the method. In the HSV color model, colors are specified using the coordinates of Hue, Saturation, and Value. However, this is merely an example and various color models may be used. For example, a lightness (HSL) or an intensity (HSI) color model may be used.

The image preprocessor 110 may perform preprocessing for character recognition based on the HSV value of the pixel selected by the user or the region information selected by the user. The image preprocessing apparatus 110 may binarize an edited image including a character based on the HSV value of the pixel selected by the user or the region information selected by the user. The image preprocessing apparatus 110 may remove noise from the binarized image. The image preprocessing apparatus 110 may recognize a character more accurately from the binarized image from which the noise is removed.

Techniques for recognizing characters in the game screen can be applied to test the game. Test automation techniques are needed to mechanically test the user's response to the characters appearing in the developed game. Character recognition technology can be used for test automation of games. For example, the level of the character, the amount of goods, the price of the item, etc. are displayed as numbers in the game, and the recognition rate of the numbers needs to be evaluated during the game test process.

The image preprocessing apparatus 110 may perform preprocessing on a partial region including a character to be recognized in the game screen. For example, a partial region containing text may be cropped by the user. The cropped partial region may be referred to as an edited video. The shape of the edited video is not limited to a rectangle, and may have various shapes including a circle.

The image preprocessing apparatus 110 may automatically preprocess the edited image first. If the character is correctly recognized from the automatically preprocessed image, the image preprocessing apparatus 110 may end the preprocessing of the edited image. However, if a character recognized from an automatically preprocessed image is incorrect, the preprocessing of the edited image may be repeatedly performed based on a user input. Here, the preprocessing can be terminated on the condition of accuracy. For example, if the accuracy of the recognized character is greater than or equal to the threshold, the preprocessing may be set to end. The threshold of accuracy can be preset and adjusted.

In the following description, a description will be given focusing on an operation in which the preprocessing is performed based on a user input. However, an operation in which the preprocessing is automatically performed may be equally applied to a part not related to user input.

The image preprocessing apparatus 110 may adjust the size of the edited image. Scaling may be referred to as resizing. When the size of the characters in the edited image is small, the image preprocessing apparatus 110 may increase the size of the edited image, or fix the size of the edited image and increase the size of the partial region in the edited image including the character. The problem that the accuracy of character recognition is large is that the size of the character is small, so the size adjustment may be mainly an enlargement operation.

The image preprocessing apparatus 110 may receive color information of characters in the edited image from the user and perform binarization of the edited image based on the input. In addition, the image preprocessing apparatus 110 may receive region information in the edited image from the user, and may remove a background region from pixels that are the same as or similar to the color of the character based on the input of the region information. As such, the image preprocessing apparatus 110 may increase the accuracy of character recognition by performing preprocessing based on a user input.

The image preprocessing apparatus 110 may determine a pixel having a color not corresponding to the color information as a background area and perform binarization. The image preprocessing apparatus 110 may determine a portion that does not correspond to area information of a character acquired from a user as a background area and perform binarization. For example, the image preprocessing apparatus 110 may determine a portion that deviates from the threshold for the ratio of horizontal and vertical included in the area information of the character as the background area and perform binarization.

The image preprocessor 110 may binarize the edited image by using character characteristics. The image preprocessing apparatus 110 may limit the character area in the edited image by using the character characteristic. For example, in the case of an alphabet or a number, when a rectangle surrounding the connection point is drawn in the binary image, more than two rectangles may not be drawn in the rectangle of the character. The image preprocessing apparatus 110 may preprocess the edited image more accurately by using character characteristics of alphabets or numbers.

The image preprocessor 110 may recognize a character from the preprocessed image. The image preprocessing apparatus 110 may generate a color list and a text area list from the preprocessed image. The image preprocessing apparatus 110 may store each color range and text area in a database in correspondence with a unique key.

When character recognition is requested based on the unique key stored in the database, the edited image may be preprocessed using the color list and the text area list corresponding to the unique key, and the character recognition may be performed on the preprocessed result image.

Referring to FIG. 1, the image preprocessing apparatus 110 may include a display 111. The display 111 may provide a game screen to the user. The game screen may include a character 113. The user may set the editing area 115 including the character 113 by using the cursor 117. The image preprocessing apparatus 110 may perform preprocessing on the editing area 115.

2 is a diagram illustrating an overall configuration of an image preprocessing apparatus according to an exemplary embodiment.

Referring to FIG. 2, the image preprocessor 110 includes an I / O interface 210, a memory 220, and a processor 230. The image preprocessing apparatus 110 may further include a display 240.

The I / O interface 210 receives an input image including text. I / O interface 210 may include a keyboard, mouse (or other pointing device), touch pad, and touch screen. The touch screen may be combined with the display 240.

The memory 220 stores the input image. Memory 220 may include high speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices, and may include one or more magnetic disk storage devices, optical disk storage devices, flash devices, and the like. It may also include nonvolatile memory such as memory devices or other nonvolatile semiconductor storage devices. Memory 370 may optionally include one or more storage devices remotely located from processor 230.

The processor 230 performs preprocessing on the input image. When performing preprocessing on the input image, the processor 230 edits the input image based on the crop region information received from the I / O interface 210 to generate an edited image including characters. Since the edited video is smaller than the input video, the processor 230 may preprocess the edited video with less resources than preprocessing the entire input video.

The processor 230 automatically performs preprocessing on the edited image to generate the first preprocessed image. The processor 230 may automatically perform preprocessing without user input. If the first preprocessed image satisfies the first condition related to accuracy, the preprocessing process may be terminated. Here, the first condition may include a condition in which the accuracy of character recognition is greater than or equal to a threshold. As such, the processor 230 may automatically perform the preprocessing first, thereby providing a function of performing the preprocessing with an accuracy capable of quickly recognizing a character without user intervention.

When the first preprocessed image does not satisfy the first condition, the processor 230 generates a second preprocessed image by performing preprocessing on the first preprocessed image based on a user input received from the I / O interface 210. do. The processor 230 may perform a preprocessing operation to more accurately recognize a character through the assistance of a user input.

The processor 230 generates a second preprocessed image by performing preprocessing on the first preprocessed image based on the user input received from the I / O interface 210. First, the processor 230 may adjust the size of the edited video. When the size of the text in the edited video is small, the processor 230 may increase the size of the edited video or the size of the partial region in the edited video including the text while fixing the size of the edited video. The size of the edited video to be resized may be preset.

The processor 230 may generate a second preprocessed image based on color information or area information of a character included in a user input. The processor 230 may receive color information of characters in the edited video from the user and perform binarization of the edited video based on the color information. In addition, the processor 230 may receive area information in the edited image from the user, and may remove the background area from pixels that are the same as or similar to the color of the text based on the information. The processor 230 may perform binarization of the edited image based on the color information of the character, and then binarize the edited image based on the region information of the character, or may perform binarization of the edited image in the reverse order. have. In addition, the processor 230 may select a result having higher accuracy by comparing the result of binarization of the edited image based on the color information of the character and the result of binarization of the edited image based on the region information of the character. . In the following description, the color information is used as an example, and then the area information is used.

The processor 230 may generate a second binarization result image by performing binarization on the first preprocessed image. The processor 230 may generate a plurality of channel-specific images by separating the first preprocessed image for each channel, and generate a second binarization result image by performing binarization on each of the plurality of channel-specific images.

The processor 230 may perform binarization on each of the plurality of channels based on the user input. The image preprocessing apparatus 110 may receive color information of characters in the edited image from the user and perform binarization of the edited image based on the input. For example, the processor 230 may classify pixels that are the same as or similar to the pixel values of the color information of the text into the text area and perform binarization. Here, the processor 230 may classify pixels having a pixel value having a difference of less than or equal to a threshold from the color information of a character as pixels having similar pixel values. For example, the processor 230 may calculate the similarity based on the difference between the pixel values, and calculate the probability that the pixel belongs to the character region according to the similarity. The processor 230 may binarize the edited image based on the calculated probability.

The processor 230 may binarize the first preprocessed image for each channel and select the most accurate image. The processor 230 may generate a plurality of channels-specific images by separating the first preprocessed image by each channel. The processor 230 may generate a plurality of binarized channel-specific images by performing binarization on each of the plurality of channel-specific images. The processor 230 may generate a first binarization result image by selecting a binarized channel-specific image satisfying a fourth condition from among the plurality of binarized channel-specific images. The fourth condition may be related to the accuracy of the character.

The processor 230 may generate a second preprocessed image by using a box surrounding a character included in the edited image. For example, the processor 230 may use an edge box model. The processor 230 detects one or more edges in the first preprocessed image, generates an outline associated with a character using one or more edges, generates a box surrounding the outline, and binarizes an inner region of the box. The second binarization result image may be generated.

The processor 230 may detect an edge gradient from the first preprocessed image, and detect one or more edges using the size and direction of the edge gradient. The processor 230 may extract an edge included in the first preprocessed image. The processor 230 may search for a component to which each edge is connected and display the rectangle. This may be referred to as connected component labeling.

The processor 230 may modify the shape of the quadrangle such that the included element is represented by one letter. The processor 230 may use erosion / expansion operations. This may be referred to as Morphological Transformation. The processor 230 may remove the atypical rectangle based on the ratio or width of the rectangle. Through this, the processor 230 may purify the component.

The processor 230 may select an edge that satisfies the second condition among one or more edges, and generate an outline using the selected edge. Here, the second condition may be a condition related to connectivity between edges. The processor 230 may generate one or more boxes that surround the outline. This may be referred to as a bounding box. The processor 230 may select a box satisfying the third condition from one or more boxes. Here, the third condition may be determined based on the characteristic according to the type of the character. For example, the ratio of the width and height of the text may be included in the third condition.

According to another embodiment, the processor 230 may generate the second preprocessed image based on the characteristic information of the character included in the first preprocessed image. The processor 230 may change the number of internal components according to the characteristics of the character. When the processor 230 performs the shape transformation, the processor 230 may change the value of the kernel used for the operation based on the characteristic of the character.

The processor 230 may remove noise included in the second binarization result image. The processor 230 may remove noise included in the second binarization result image based on the area information of the character included in the user input. The processor 230 may receive area information in the edited image from the user, and may remove the background area from pixels that are the same as or similar to the color of the text based on the information.

As such, the processor 230 may preprocess the edited image by various combinations of channel-specific binarization, binarization by a bounding box, and preprocessing using character characteristics.

The processor 230 may build a database based on the character recognition result. The processor 230 may determine whether the second preprocessed image satisfies a preset condition. When the preset condition is satisfied, the memory 220 may store a character recognized from the second preprocessed image and a unique key corresponding to the character.

3 is a flowchart illustrating overall operations of an image preprocessing method according to an exemplary embodiment.

According to an embodiment, in operation 310, the image preprocessing apparatus 110 may generate an edited image including a character by editing the input image based on the crop region information. The image preprocessing apparatus 110 may reduce preprocessing time by performing preprocessing on the edited image reduced to the crop region. The crop area is not limited to the rectangle and may include various types of closed areas.

According to an embodiment, in operation 320, the image preprocessing apparatus 110 may automatically perform preprocessing on the edited image to generate the first preprocessed image. The image preprocessing apparatus 110 may automatically resize the edited image. The image preprocessing apparatus 110 may generate the first binarization result image by performing binarization on a channel basis for the automatically resized edited image. The image preprocessing apparatus 110 may automatically generate a second binarization result image by using a bounding box with respect to the first binarization result image. The image preprocessing apparatus 110 may automatically remove noise from the second binarization result image. The result of removing the noise may be referred to as a first preprocessed image.

According to an embodiment, in step 330, if the first preprocessed image does not satisfy the first condition, the image preprocessing apparatus 110 may apply to the first preprocessed image based on a user input received from the I / O interface. The preprocessing may be performed to generate a second preprocessed image. The image preprocessor 110 may generate a second preprocessed image based on a user input when the accuracy of the first preprocessed image is equal to or less than a preset threshold.

The image preprocessing apparatus 110 may automatically resize the first preprocessed image. The image preprocessing apparatus 110 may generate the first binarization result image by performing binarization on a resized image for each channel. The image preprocessing apparatus 110 may perform binarization based on color information of a character. The image preprocessing apparatus 110 may generate a second binarization result image by using a bounding box with respect to the first binarization result image. The image preprocessing apparatus 110 may remove noise from the second binarization result image. The image preprocessing apparatus 110 may remove noise from the second binarization result image based on the area information of the character. The result of removing the noise may be referred to as a second preprocessed image.

4 is a flowchart illustrating a process of generating a second preprocessed image, according to an exemplary embodiment.

According to an embodiment, in step 330, if the first preprocessed image does not satisfy the first condition, the image preprocessing apparatus 110 may apply to the first preprocessed image based on a user input received from the I / O interface. The preprocessing may be performed to generate a second preprocessed image.

According to an embodiment, in operation 410, the image preprocessing apparatus 110 may detect one or more edges in the first preprocessed image. The processor 230 may detect an edge gradient from the first preprocessed image, and detect one or more edges using the size and direction of the edge gradient. The processor 230 may extract an edge included in the first preprocessed image.

According to an embodiment, in operation 420, the image preprocessing apparatus 110 may generate an outline related to a character by using one or more edges. The image preprocessor 110 may search for a component to which each edge is connected and display the rectangle. The image preprocessor 110 may perform connected component labeling. The image preprocessing apparatus 110 may deform the shape of the quadrangle such that the elements included are represented by one letter. The image preprocessing apparatus 110 may remove the atypical rectangle based on the ratio or the width of the rectangle.

According to an embodiment, in operation 430, the image preprocessing apparatus 110 may generate a box surrounding an outline. The image preprocessing apparatus 110 may select a box satisfying the third condition from one or more boxes. According to an embodiment, in operation 440, the image preprocessing apparatus 110 may generate a second binarization result image by performing binarization on the inner region of the box.

5 is a flowchart illustrating a process of generating a second preprocessed image according to another exemplary embodiment.

According to an embodiment, in step 330, if the first preprocessed image does not satisfy the first condition, the image preprocessing apparatus 110 may apply to the first preprocessed image based on a user input received from the I / O interface. The preprocessing may be performed to generate a second preprocessed image.

According to an embodiment, in operation 510, the image preprocessing apparatus 110 may generate an image for each channel by separating an inner region of the box for each channel. According to an embodiment, in operation 520, the image preprocessing apparatus 110 may generate a plurality of binarized channel-specific images by performing binarization on each of the plurality of channel-specific images. For example, when the HSV color model is used, the image preprocessing apparatus 110 may generate an image for each channel of Hue, Saturation, and Brightness.

According to an embodiment, in operation 510, the image preprocessor 110 may generate a second binarization result image by selecting a binarized channel image satisfying a fourth condition from among a plurality of binarized channel images. have. The fourth condition may be a condition in which the accuracy of the binarized result is greater than or equal to a threshold.

6 is a flowchart illustrating an overall operation of an image preprocessing method according to another exemplary embodiment.

Referring to FIG. 6, the image preprocessing apparatus 110 may perform preprocessing using a user interface (UI). The image preprocessing apparatus 110 may perform preprocessing using character characteristics.

According to an embodiment, in operation 610, the image preprocessing apparatus 110 may crop the character area in the game. The image preprocessing apparatus 110 generates an edited image including characters by editing the input image based on the crop region information. Since the edited video is smaller than the input video, the processor 230 may preprocess the edited video with less resources than preprocessing the entire input video.

According to an embodiment, in operation 620, the image preprocessing apparatus 110 may automatically perform character recognition after the first preprocessing. The image preprocessing apparatus 110 may automatically perform preprocessing first to provide a function of performing preprocessing with an accuracy capable of quickly recognizing a character without user intervention.

According to an embodiment, in operation 621, the image preprocessing apparatus 110 may output a preprocessing result and a character recognition result to the UI. The accuracy of the result of the first preprocessing may be automatically calculated, but may also be determined by the user. The image preprocessing apparatus 110 may provide an opportunity for the user to determine the accuracy by outputting the preprocessing result and the character recognition result through the UI.

According to an embodiment, in operation 623, the image preprocessing apparatus 110 may identify a result including a preprocessing result and a character recognition result. The image preprocessing apparatus 110 may determine whether the accuracy condition is satisfied based on the character recognition result or the preprocessing result.

According to an embodiment, when the result satisfies a preset condition, in operation 650, the image preprocessing apparatus 110 may store the character recognition result in a database. The image preprocessing apparatus 110 may store a character recognized from the second preprocessed image and a unique key corresponding to the character.

According to an embodiment, when the result does not satisfy the preset condition, in operation 630, the image preprocessing apparatus 110 may receive a user input. In operation 631, the image preprocessor 110 may receive color information input. In operation 633, the image preprocessing apparatus 110 may receive area information input.

According to an embodiment, in operation 635, the image preprocessing apparatus 110 may perform a second preprocessing on the first preprocessed image based on a user input. The image preprocessing apparatus 110 may determine a pixel having a color not corresponding to the color information as a background area and perform binarization. The image preprocessing apparatus 110 may determine a pixel having a color corresponding to the color information as a character area and perform binarization. The image preprocessing apparatus 110 may receive region information in the edited image from the user, and may remove the background region from pixels that are the same as or similar to the color of the text based on the input of the region information.

According to an embodiment, in operation 640, the image preprocessing apparatus 110 may perform a third preprocessing using the character characteristic. The image preprocessor 110 may change the number of internal components according to the characteristics of the character. When the image preprocessing apparatus 110 performs shape transformation, the image preprocessing apparatus 110 may change the value of the kernel used for the calculation based on the characteristic of the character.

According to an embodiment, in operation 650, the image preprocessing apparatus 110 may store a character recognition result in a database. The image preprocessing apparatus 110 may store the recognized character and a unique key corresponding to the character.

7 is a diagram illustrating a process of generating a second preprocessed image by the image preprocessing apparatus according to an exemplary embodiment.

In operation 710, the image preprocessing apparatus 110 may adjust the size of the edited image. The image preprocessing apparatus 110 may first adjust the size of the numeric image having a predetermined size or less to perform the preprocessing. For example, the image preprocessing apparatus 110 may enlarge the size of the edited image having a height of 30 pixels or less to a size of 1.5 times.

In operation 720, the image preprocessing apparatus 110 may perform binarization on a channel-by-channel basis with respect to the edited size of the edited image. In operation 721, the image preprocessing apparatus 110 may separate the adjusted edited image for each channel. For example, when the HSV color model is used, the image preprocessing apparatus 110 may separate the edited image resized to three channels. In operation 723, the image preprocessing apparatus 110 may perform binarization for each channel of Hue, Saturation, and Brightness. The image preprocessing apparatus 110 may acquire a first binarization result image.

The image preprocessing apparatus 110 may select one binarization result image from among the channel-specific binarization result images as the first binarization result image. For example, the image preprocessing apparatus 110 may select a binarization result image corresponding to a channel having the least unnecessary bounding box candidate. As another example, the image preprocessing apparatus 110 may select a binarization result image corresponding to a channel having the most bounding boxes. Since the number in the game screen usually has eight digits (99,999,999) or less, the image preprocessing apparatus 110 corresponds to a channel corresponding to the channel with the largest number of bounding boxes without the number of bounding boxes. You can select an image.

In operation 730, the image preprocessing apparatus 110 may generate a second preprocessed image by using a box surrounding a character included in the edited image. For example, the image preprocessing apparatus 110 may use an edge box model.

In operation 735, the image preprocessing apparatus 110 may detect an edge from the first binarization result image. The image preprocessing apparatus 110 may detect an edge by using a canny edge detector. The image preprocessing apparatus 110 may detect an edge by acquiring the size and direction of an edge gradient of the first binarization result image by using a canny edge detector. The image preprocessing apparatus 110 may find an outline based on the detected edge.

In operation 733, the image preprocessing apparatus 110 may remove redundant or unnecessary outlines. For example, if you create a rectangle with the smallest area that encloses the outline, and the coordinates of all four vertices overlap or the rectangle is skewed more than 10 degrees to the left or right, the ratio of the width and height of the rectangle that is considered non-numeric is greater than 0.4. Smaller or larger than 4 cases may be determined as duplicated or unnecessary edges.

In operation 731, the image preprocessing apparatus 110 may generate a bounding box candidate using an edge box algorithm. The image preprocessing apparatus 110 may directly generate a bounding box candidate using the detected edge. Through this, the image preprocessing apparatus 110 may find a character in the edited image.

For example, the bounding box candidate may be generated only when the height of the bounding box candidate is greater than 40% of the height of the edited image and the Y coordinate of the center of the bounding box candidate is within ± 30% of the Y coordinate of the center of the edited image. have. Duplicate or unnecessary ones among the plurality of bounding box candidates may be excluded. In the case of two or more bounding box candidates having one or more edges overlapping, a smaller bounding box candidate may be excluded. If there are two or less bounding box candidates within one bounding box candidate, all of the inner bounding box candidates may be excluded, and vice versa, the outer bounding box candidate may be excluded. Further, if one of the two bounding box candidates is smaller than 70% of the height of the other bounding box candidate, it may be excluded.

In operation 740, the image preprocessing apparatus 110 may perform binarization on an area within the bounding box. The image preprocessing apparatus 110 may perform noise removal on the resultant image of the binarization. The image preprocessing apparatus 110 may receive region information in the edited image from the user, and may remove a background region from pixels that are the same as or similar to the color of the text based on the input of region information.

8 shows an example of a general binarization result. 9 illustrates an example of performing binarization on each channel-specific image by the image preprocessing apparatus according to an exemplary embodiment.

In the case where a complex background is mixed in addition to the character “5000” such as the image 810, when the general binarization method is applied, the image 820 may be derived. The image 820 may be an image including a lot of noise.

The image preprocessor 110 may generate an image 911, an image 921, and an image 931 by separating the image 810 for each HSV channel. The image preprocessor 110 may binarize the image 911, the image 921, and the image 931. As a result of binarization, a binarized image 913, an image 923, and an image 933 may be generated for each HSV channel. For example, the image preprocessing apparatus 110 may use an Otsu binarization method that automatically finds a threshold value through the brightness distribution of the image.

The image preprocessor 110 may select one image 923 from the binarized images 913, 923, and 933 for each channel. For example, the image preprocessor 110 may select an image 923 having the least noise among the images 913, 923, and 933.

FIG. 10 illustrates an example in which binarization is performed on each channel-specific image based on color information input from a user by the image preprocessing apparatus. Referring to FIG.

Referring to FIG. 10, the image preprocessing apparatus 110 may receive a pixel value of a text area and perform binarization using the pixel value. The user selects two pixels of the area “100”, and the image preprocessing apparatus 110 may perform binarization by classifying pixels corresponding to values between pixel values of the corresponding pixels into the character area. The image preprocessor 110 may perform binarization by receiving color information from the user until the recognition accuracy is greater than or equal to a threshold. Accordingly, the binarized result image may be improved in the image 1011, the image 1013, the image 1021, and the image 1023.

11 is a diagram illustrating a result of removing noise of an image in which binarization is performed by an image preprocessing apparatus according to an exemplary embodiment.

The image preprocessing apparatus 110 may perform noise removal on the resultant image of the binarization. The image preprocessing apparatus 110 may receive region information in the edited image from the user, and may remove a background region from pixels that are the same as or similar to the color of the text based on the input of region information.

Referring to FIG. 11, the image preprocessing apparatus 110 may calculate the number of pixels having the same or similar pixel value as the pixel value of the region estimated by the number “63”. The image preprocessing apparatus 110 may change another region having the number of pixels smaller than the number of pixels included in the number estimated region to be the pixel value of the background region. The image preprocessing apparatus 110 may receive the position of the background area from the user and change the pixel values of the pixels in the adjacent position having the pixel value similar to the corresponding area to the pixel value of the background area.

12 is a flowchart illustrating an operation of a character recognition method, according to an exemplary embodiment.

According to an embodiment, in operation 1210, the image preprocessing apparatus 110 may generate an edited image including a character by editing the input image based on the crop region information.

According to an embodiment, in operation 1220, the image preprocessing apparatus 110 may automatically perform preprocessing on the edited image to generate a first preprocessed image.

According to an embodiment, in operation 1230, when the first preprocessed image does not satisfy the first condition, the image preprocessing apparatus 110 may apply to the first preprocessed image based on a user input received from the I / O interface. The preprocessing may be performed to generate a second preprocessed image.

According to an embodiment, in operation 1240, the image preprocessing apparatus 110 may recognize a character included in the second preprocessed image.

13 is a diagram illustrating an overall configuration of a character recognition apparatus according to an embodiment.

According to an embodiment of the present disclosure, the character recognition apparatus 110 performs an I / O interface 1310 for receiving an input image including a character, a memory 1320 for storing the input image, and preprocessing the input image. The processor 1330 may recognize a character from the output image and the output device 1340 may output the recognized character. For example, the output device 1340 may include a speaker or a display.

When preprocessing the input image, the processor 1330 may edit the input image based on crop region information received from the I / O interface to generate an edited image including text.

The processor 1330 may automatically perform preprocessing on the edited image to generate the first preprocessed image. The processor 1330 may determine whether the first preprocessed image satisfies the first condition.

When the first preprocessed image does not satisfy the first condition, the processor 1340 may generate a second preprocessed image by performing preprocessing on the first preprocessed image based on a user input received from the I / O interface. .

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

Although the embodiments have been described with reference to the accompanying drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the following claims.

[One]

Claims (14)

An I / O interface for receiving an input image including text;
A memory for storing the input image; And
A processor configured to perform preprocessing on the input image,
When the processor performs preprocessing on the input image,
Edit the input image based on crop region information received from the I / O interface to generate an edited image including the character;
Automatically preprocessing the edited image to generate a first preprocessed image,
When the first preprocessed image does not satisfy the first condition related to accuracy, a second preprocessed image is generated by performing preprocessing on the first preprocessed image based on a user input received from the I / O interface.
When the first preprocessed image satisfies the first condition, the character is recognized from the first preprocessed image,
The processor,
Generating the second preprocessed image by using a box surrounding a character included in the edited image,
The processor,
Detect one or more edges in the first preprocessed image,
Create an outline associated with the text using the one or more edges,
Create a box surrounding the outline,
Generate a second binarization result image by performing binarization on an inner region of the box,
The processor,
Separating the inner region of the box for each channel to generate a plurality of images of the channel of the color channel, chroma channel and brightness channel,
Generate a plurality of binarized channel-specific images by performing binarization on each of the plurality of channel-specific images,
Among the plurality of binarized channel-specific images, a second channel-specific image is selected by satisfying a fourth condition related to accuracy, the number of generated boxes is less than a threshold value, and the number of generated boxes is the largest. To generate the binarization result image,
Image preprocessor. The method of claim 1,
The processor,
To adjust the size of the edited video,
Image preprocessor. The method of claim 1,
The processor,
Generating the second preprocessed image based on color information or area information of the character included in the user input;
Image preprocessor. The method of claim 1,
The processor,
Detect an edge gradient from the first preprocessed image,
Detecting the one or more edges using the size and direction of the edge gradient,
Image preprocessor. The method of claim 1,
The processor,
Selecting an edge satisfying a second condition related to connectivity between the edges among the one or more edges,
Generating the outline using the selected edge;
Image preprocessor. The method of claim 1,
The processor,
Create one or more boxes surrounding the outline,
Selecting a box from among the one or more boxes that satisfies a third condition related to the characteristic of the character,
Image preprocessor. The method of claim 1,
The processor,
Performing binarization on each of the plurality of channels based on the user input;
Image preprocessor. The method of claim 1,
The processor,
To remove noise included in the second binarization result image,
Image preprocessor. The method of claim 1,
The processor,
Removing noise included in the second binarization result image based on area information of the character included in the user input;
Image preprocessor. The method of claim 1,
The processor,
Generating a first binarization result image by performing binarization on the first preprocessed image;
Image preprocessor. The method of claim 10,
The processor,
Generating the first binarization result image by separating the first preprocessed image for each channel to generate a plurality of images for each channel, and performing binarization on each of the plurality of channels.
Image preprocessor. The method of claim 1,
The processor,
Generating the second preprocessed image based on the characteristic information of the character included in the first preprocessed image,
Image preprocessor. The method of claim 1,
The processor,
It is determined whether the second preprocessed image satisfies a preset condition.
The memory,
When the preset condition is satisfied, storing the character recognized from the second preprocessed image and a unique key corresponding to the character,
Image preprocessor. In the image preprocessing method performed by a computer comprising a processor,
Editing the input image based on crop region information to generate an edited image including characters;
Automatically preprocessing the edited image to generate a first preprocessed image;
Generating a second preprocessed image by performing preprocessing on the first preprocessed image based on a user input received from an I / O interface when the first preprocessed image does not satisfy a first condition related to accuracy; And
Recognizing the character from the first preprocessed image when the first preprocessed image satisfies the first condition
Including,
The generating of the second preprocessed image may include:
Generating the second preprocessed image by using a box surrounding a character included in the edited image,
The generating of the second preprocessed image by using the box may include:
Detecting one or more edges in the first preprocessed image;
Generating an outline associated with the text using the one or more edges;
Generating a box surrounding the outline; And
Generating a second binarization result image by performing binarization on an inner region of the box;
The generating of the second binarization result image may include:
Generating an image for each channel of the color channel, the chroma channel, and the brightness channel by separating the inner region of the box for each channel;
Generating a plurality of binarized channel-specific images by performing binarization on each of the plurality of channel-specific images; And
Among the plurality of binarized channel-specific images, a second channel-related image is selected by satisfying a fourth condition related to accuracy, the number of generated boxes is less than a threshold value, and the number of generated boxes is the largest. Generating a binarization result image;
Image preprocessing method.

Images