TWI766930B - Classification-based character skew correction apparatus and method thereof - Google Patents

Abstract

A classification-based character skew correction method is provided. The method includes the steps of: obtaining an input character image; recognizing a plurality of target objects from the input character image; classifying the target objects into at least one candidate character categories; determining a primary character category from the at least one candidate character categories; calculating a skew angle of the target objects in the primary character category; and performing image skew correction on the input character image according to the calculated skew angle of the primary character category.

Description

Classification-based character inclination correction device and method thereof

The present invention relates to image processing, in particular to a classification-based character inclination correction device and method thereof.

In visual image detection, the characters (text/character) in the input image to be recognized may be inclined due to the relationship between printing and positioning. In addition, in practical applications, there will be many characters skewed in different situations, such as: different fonts (dot-matrix, printed), different font sizes, cluttered backgrounds, etc., which will affect character recognition. the accuracy rate. Therefore, before character recognition processing is performed on the characters in the image, it is necessary to detect the character direction and correct the character direction.

In the traditional tilt correction algorithm, the overall tilt change of the characters in the image to be recognized is usually directly considered, such as Hough Transform, Radom Transform, minimum standard deviation method ( Minimum Standard Deviation), and the shortest distance square method, etc. In other words, the above-mentioned traditional tilt correction algorithm is to find the angle of the best solution of the whole character in the input image to determine the tilt angle, and cannot take into account the individual The relationship between characters, and therefore the inclination angle obtained by these methods, is unstable and easily affected by other factors (eg, different fonts, mixed uppercase and lowercase characters, noise interference), which leads to character correction errors.

Therefore, a classification-based character inclination correction device and method thereof are required to solve the above-mentioned problems.

The present invention provides a character inclination correction method based on classification, comprising: obtaining an input character image; identifying a plurality of target objects from the input character image; classifying the plurality of target objects into at least one candidate character category; Determine a main character type from the at least one candidate character type; calculate an inclination angle corresponding to each target object in the main character type; and image the input image according to the calculated inclination angle of the main character type Tilt correction to generate a corrected image.

The present invention further provides a classification-based character skew correction device, comprising: a memory unit for storing a character skew correction program; and a processing unit for reading and executing the character skew correction from the memory unit The program performs the following steps: obtaining an input character image; identifying a plurality of target objects from the input character image; classifying the plurality of target objects into at least one candidate character class; determining a main character from the at least one candidate character class character type; calculating an inclination angle corresponding to each target object in the main character type; and performing image inclination correction on the input image according to the calculated inclination angle of the main character type to generate a corrected image.

100‧‧‧Character tilt correction device

110‧‧‧Processing unit

120‧‧‧Memory cells

121‧‧‧Volatile memory

122‧‧‧Non-volatile memory

123‧‧‧Character Skew Correction Program

210‧‧‧Input character image

θ ₁ ‧‧‧Rotation angle

Y _TP1 , Y _BP1 , Y _TP2 , Y _BP2 ‧‧‧coordinates

S310-S360‧‧‧Steps

FIG. 1 is a functional block diagram of a character skew correction device according to an embodiment of the present invention.

FIG. 2A is a schematic diagram of calculating the object height of the target object according to an embodiment of the present invention.

FIG. 2B is a schematic diagram of calculating the object height of the target object in another embodiment of the present invention.

FIG. 3 is a flowchart showing a method for correcting character inclination according to a classification-based formula in an embodiment of the present invention.

FIGS. 4A to 4F are schematic diagrams showing classification conditions for classifying target objects according to an embodiment of the present invention.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, a preferred embodiment is exemplified below, and is described in detail as follows in conjunction with the accompanying drawings.

FIG. 1 is a functional block diagram of a character skew correction device according to an embodiment of the present invention. As shown in FIG. 1 , the character skew correction device 100 includes a processing unit 110 and a memory unit 120 . The memory unit 120 includes a volatile memory 121 and a non-volatile memory 122 . The volatile memory 121 may be a random access memory, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), but the invention is not limited thereto. The non-volatile memory 122 can be, for example, a hard disk drive, a solid state hard drive, A flash memory, or a ROM, but the invention is not limited thereto.

The non-volatile memory 122 stores a character skew correction program 123, and the processing unit 110 reads the character skew correction program 123 from the non-volatile memory 122 to the volatile memory 121 and executes, wherein the character skew correction program Line 123 includes code for a character skew correction method.

The character inclination correction method of the present invention can consider the direction change of some characters in the input character image, instead of directly considering the whole direction change of all characters. Because the overall direction change of all characters is easily affected by noise, differences in characters themselves also affect the calculation results. For example, there is a string "A---" in the input character image, in which A is obviously different from other characters. When calculating the inclination angle of the character string, because the relationship of the character A causes the direction of the whole character of the whole character string to be shifted, the calculated inclination angle is wrong. Relatively speaking, if only the direction change of the character "-" can be considered, the calculated inclination angle will be correct.

FIG. 2A is a schematic diagram of calculating the object height of the target object according to an embodiment of the present invention. FIG. 2B is a schematic diagram of calculating the object height of the target object in another embodiment of the present invention. In this embodiment, the target object is a character such as "D" (uppercase letter), "x" (lowercase letter), ";" (punctuation mark).

As shown in FIG. 2A , the target object can be, for example, a capital letter D. Assuming that the rotation angle of the target object is 0, the processing unit 110 can calculate the vertex coordinate TP and the base point coordinate BP of the target object, wherein the vertex refers to the highest coordinate point of the target object in the vertical direction (Y axis), and the base point refers to the target The coordinates of the lowest point of the object in the vertical direction (Y axis). It should be noted that when the input character image starts with When the center point is rotated, the vertex and base point of the target object are not always in a fixed position. Under different rotation angles, the vertex refers to the highest coordinate point of the target object in the vertical direction (Y axis) under the rotation angle, and the base point Both refer to the coordinates of the lowest point of the target object in the vertical direction (Y axis) under the rotation angle. The embodiment in FIG. 2A takes one of the target objects in the input character image as an example. If the number of target objects is equal to 1, no additional classification judgment is required, and the inclination angle of the target object can be directly determined.

If the number of target objects is greater than 1, the processing unit 110 calculates each rotation angle θ of each target object within the rotation angle range (eg, the rotation angle is between plus and minus 45 degrees) (for example, a predetermined interval, such as The vertex coordinates TP (X _TP , Y _TP ) and the base point coordinates BP (X _BP , Y _BP ) are sampled in degrees or 5 degrees). The vertical distance between the vertex and the base point is the object height h. Next, if the number of target objects is greater than 1 (that is, there are multiple target objects), the processing unit 110 compares the object height difference at each rotation angle θ between any two target objects in the input character image in a combined manner The value diff (e.g. considering only the Y-axis coordinates) can be represented, for example, by the following equation: diff _ij =| object _i (Y _TP )- object _j (Y _TP )|+| object _i (Y _BP )- object _j (Y _BP ) | (1)

where i and j are positive integers ranging from 1 to N, and i is not equal to j. In one embodiment, if the number of target objects is greater than 1 (that is, there are multiple target objects), the processing unit 110 calculates a diff pair of object height differences between any two target objects at each rotation angle. The two target objects are classified. For example, the processing unit 110 determines whether the object height difference diff is less than a predetermined value. If the object height difference is less than a predetermined value, the processing unit 110 determines that the two target objects belong to the same candidate character category. If the object height When the difference is not less than the predetermined value, the two selected target objects are classified into different candidate character categories.

，也就是10次。兩兩目標物件的比較關係為：(1)aD(2)ac(3)aD(4)ac(5)Dc(6)DD(7)Dc(8)cD(9)cc(10)Dc。意即以不同組合的方式對所有物件進行兩兩目標物件之高度差值diff的比較。舉例來說，對於左邊的大寫字母D來說，其頂點及端點在垂直方向(Y軸)的座標例如分別為Y_TP1及Y_BP1；對於右邊的大寫字母D來說，其頂點及端點在垂直方向(Y軸)的座標例如分別為Y_TP2及Y_BP2。就在此旋轉角度θ₁相應的高度差值diff=|Y_TP1-Y_TP2|+|Y_BP1-Y_BP2|。在此例中，目標物件D、D會被分為同一類、目標物件a、c、c亦會被分為同一類。 For example, if there are 5 target objects: "aDcDc" in the input character image 210, when the input character image is rotated (for example, rotated clockwise by an angle of _θ1 ), the 5 target objects will also be rotated correspondingly. As shown in Figure 2B. The total number of comparison operations of the height difference between the above two target objects is

, which is 10 times. The comparison relationship between pairwise target objects is: (1)aD(2)ac(3)aD(4)ac(5)Dc(6)DD(7)Dc(8)cD(9)cc(10)Dc. It means to compare the height difference diff between the two target objects for all objects in different combinations. For example, for the capital letter D on the left, the coordinates of the vertex and the end point in the vertical direction (Y axis) are, for example, Y _TP1 and Y _BP1 respectively; for the capital letter D on the right, the vertex and the end point are respectively Y TP1 and Y BP1 The coordinates in the vertical direction (Y axis) are, for example, Y _TP2 and Y _BP2 , respectively. The height difference value diff=|Y _TP1 -Y _TP2 |+|Y _BP1 -Y _BP2 | at this rotation angle θ ₁ . In this example, target objects D and D will be classified into the same class, and target objects a, c, and c will also be classified into the same class.

In detail, if characters with the same font, usually have similar object heights, such as English capital letters, these letters will have very close object height differences in several specific angles when making the above judgment, so they will be in the same category. However, the heights of English lowercase letters are not consistent, they must belong to the same letter, belong to the same general lowercase letters (such as w, n, c, e, a, etc.), or belong to the same special lowercase letters (such as g, j, p, q, y, etc., or f, h, b, d, k, etc.) are classified in the same category. In addition, the heights of punctuation marks (+-*/) are also inconsistent. Only the same punctuation marks can be classified into the same category, and different punctuation marks are not necessarily classified into the same category. middle.

For example, if there is a character string in the input character image as "WWWxxyy--", where W is an English uppercase letter, x and y are English lowercase letters, and - is a punctuation mark. According to the above judgment mechanism, four candidate character categories are obtained, such as (1) WWW, (2) xx, (3) yy, (4)--.

In one embodiment, the processing unit 110 uses the candidate character class with the largest number of target objects as the main character class to calculate the inclination angle thereof. The height difference of the object is the rotation angle corresponding to the minimum value, which is the inclination angle φ. The inclination angle φ can be expressed by the following formula, for example:

After determining the inclination angle of the main character type, the processing unit 110 performs image inclination correction on the input character image according to the determined inclination angle. For the process of Optical Character Recognition (OCR), it is necessary to perform image skew correction on the input character image before performing character segmentation and character recognition, and outputting the final character recognition result.

The character inclination correction method in the present invention can make the characters in the corrected image of the image inclination correction more in line with the actual character inclination, thereby increasing the accuracy of character recognition.

In some embodiments, the processing unit 110 may also use other algorithms to classify each target object in the input character image, for example: (1) the aspect ratio of the character itself, as shown in FIG. 4A; (2) the character The center point of the circumscribed rectangle, as shown in Figure 4B; (3) the area of the character circumscribed rectangle, as shown in Figure 4C; (4) The center of the circle circumscribed by the character, as shown in Figure 4D; (5) The area of the circle circumscribed by the character, as shown in Figure 4E; (6) The radius of the circle circumscribed by the character, as shown in Figure 4F as shown in the figure. That is, the processing unit 110 can use at least one method to classify each target object in the input character image to generate at least one candidate character classification.

In other embodiments, if the pattern type of the character to be recognized can be determined first, a traditional classifier can be used for classification. That is to say, each character to be recognized can be input into the classifier one by one for training, so as to generate the classification result of each character after training. The above-mentioned classifier can use, for example, Support Vector Machine (SVM), K-nearest neighbor classification (K-nearest neighbor classification), Convolutional Neural Networks (Convolutional Neural Networks), or deep learning (Deep learning), etc. method, but the present invention is not limited to this.

According to the above-mentioned embodiment, when the processing unit 110 determines the main character type from the at least one candidate character type, in addition to the candidate character type with the smallest sum of difference values in the foregoing embodiment, the processing unit 110 can also use various methods to determine the main character type from the at least one candidate character type. The main character class is determined in the class, for example, in the candidate character class: (1) has the largest sum of character area; (2) has the largest sum of edge strength; (3) has the largest number of characters; (4) has the smallest character (5) The sum of the standard deviations of the characters with the largest; (6) The sum of the variances of the characters with the largest; (7) The average brightness of the characters is minimum (black words)/maximum (white words), but the present invention does not limited to this.

FIG. 3 is a flowchart showing a method for correcting character inclination according to a classification-based formula in an embodiment of the present invention. In step S310, an input is obtained character image. The input character image may come from an external image capturing device, or may be obtained by other computer equipment through a wired or wireless transmission interface.

In step S320, a plurality of target objects are identified from the input character image. The above-mentioned target objects include characters or noise. For example, before identifying the target object, the processing unit 110 may employ at least one algorithm to remove noise from the input character image, such as setting thresholds, filters, and/or morphological operations to Remove noise. For example, the algorithm for setting the threshold may use the following conditions: gray value, edge magnitude, area size, height, width, or a combination thereof, but the invention is not limited thereto. Algorithms using filters, for example, can use the following filters to remove noise, such as: mean filter, median filter, min/max filter, peak A peak and valley filter, a Gaussian filter, a low-pass filter, a high-pass filter, etc., but the present invention is not limited thereto. The algorithms for removing noise using morphological operations include, for example, erosion, opening, etc., but the present invention is not limited thereto. After the noise removal process is performed, the processing unit 110 identifies a plurality of target objects from the input character image from which the noise has been removed.

In step S330, the plural target objects are classified into at least one candidate character category. For example, the processing unit 110 may classify the plurality of target objects into at least one candidate character category according to a first determination mechanism, wherein the first determination mechanism may not only use the object height difference to classify the target objects, but also Different algorithms can be used to classify each target object in the input character image, such as: (1) the aspect ratio of the character itself; (2) the rectangle surrounding the character (3) the area of the rectangle circumscribed by the character; (4) the center of the circle circumscribed by the character; (5) the area of the circle circumscribed by the character; (6) the radius of the circle circumscribed by the character. In other embodiments, the processing unit 110 may also use Support Vector Machine (SVM), K-nearest neighbor classification (K-nearest neighbor classification), Convolutional Neural Networks, or deep Various possible input characters are pre-trained and classified by methods such as deep learning.

In step S340, a main character class is determined from the at least one candidate character class. For example, the processing unit 110 may determine the main character category from the at least one candidate character category according to a second determination mechanism, wherein the second determination mechanism may use the candidate character category with the smallest difference sum to determine the main character category. In addition to the character category, various methods can also be used to determine the main character category, such as: for example, among the candidate character categories: (1) has the largest sum of character areas; (2) has the largest sum of edge strengths; (3) has the most (4) with the smallest character difference; (5) with the largest sum of standard deviations of the characters; (6) with the largest sum of the variances of the characters; (7) with the smallest average brightness of the characters (black characters)/ maximum (white words), but the present invention is not limited to this.

In step S350, an inclination angle corresponding to each target object in the main character category is calculated. To put it simply, the inclination angle of each target object in the input character image is determined by the inclination angle of each target object in the main character category.

In step S360, image tilt correction is performed on the input image according to the calculated tilt angle of the main character type to generate a corrected image. Because in the present invention, the target object is classified and selected from the candidate character category The method of determining the main character type can accurately calculate the inclination angle, so the image inclination correction according to the calculated inclination angle can obtain better effects and improve the accuracy of subsequent character recognition.

To sum up, the present invention provides a classification-based character inclination correction device and method thereof, which can classify the characters in the input character image, determine the main character type, calculate the corresponding inclination angle, and use the The calculated tilt angle is used for image tilt correction, so as to obtain a better corrected image. In addition, when a better corrected image is obtained, the accuracy of character segmentation and character recognition in the subsequent process of optical character recognition can also be increased.

The methods of the present invention, or specific forms or portions thereof, may be embodied in the form of code on physical media such as floppy disks, optical disks, hard disks, or any other machine-readable (eg, computer-readable) ) storage medium in which, when the code is loaded and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention. The method, system and device of the present invention can also be transmitted in the form of code through some transmission medium, such as wire or cable, optical fiber, or any transmission type, wherein when the code is received by a machine, such as a computer, loaded, loaded And when executed, the machine becomes a device or system for participating in the present invention. When implemented on a general-purpose processor, the code combines with the processor to provide a unique device that operates analogously to application-specific logic circuits.

Although the present invention is disclosed above with preferred embodiments, it is not intended to limit the scope of the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the appended patent application.

S310-S360‧‧‧Steps

Claims (10)

A character inclination correction method based on classification, comprising: obtaining an input character image; identifying a plurality of target objects from the input character image; classifying the plurality of target objects into at least one candidate character category; determining a main character type from the character type; calculating an inclination angle corresponding to each target object in the main character type; and performing image inclination correction on the input image according to the calculated inclination angle of the main character type to generate A calibration image, wherein the step of classifying the plurality of target objects into at least one candidate character category includes: calculating a vertex coordinate and a base point coordinate in each target object; calculating any two targets in the plurality of target objects an object height difference value at each rotation angle of the object within a rotation angle range; and when the object height difference value is less than a predetermined value, the two calculated target objects are classified into the same candidate character category; when the object When the height difference is not less than the predetermined value, the two calculated target objects are classified into different candidate character categories. The classification-based character inclination correction method described in item 1 of the scope of the application further comprises: determining a candidate character class with a maximum number of characters in the at least one candidate character class as the main character class. The classification-based character inclination correction method described in item 1 of the patent application scope further includes: according to the aspect ratio of each target object itself, the center point of the circumscribed rectangle, the area of the circumscribed rectangle, the center of the circumscribed circle, the circumscribed One of the area of the circle, the radius of the circumscribed circle, or a combination thereof to classify the plurality of target objects into the at least one candidate character category. The classification-based character inclination correction method as described in item 1 of the scope of the application, further comprising: classifying the plurality of target objects as the At least one candidate character class. The classification-based character inclination correction method described in item 1 of the scope of application for patent further comprises: determining the character with the largest sum of area, the largest sum of edge strength, the largest number of characters, and the smallest character from each candidate character class. The difference, the sum of the standard deviations of the largest characters, the sum of the variances of the largest characters, or the candidate character class with the smallest average brightness is used as the main character class. A character inclination correction device of a classification-based type, comprising: a memory unit for storing a character inclination correction program; and a processing unit for reading and executing the character inclination correction program from the memory unit to execute the following Steps: obtaining an input character image; identifying a plurality of target objects from the input character image; classifying the plurality of target objects into at least one candidate character class; determining a main character class from the at least one candidate character class; calculating an inclination angle corresponding to each target object in the main character class; Perform image tilt correction on the input image based on the inclination angle of the main character category to generate a corrected image, wherein in the step of classifying the plurality of target objects into at least one candidate character category, the processing unit further calculates each target object A vertex coordinate and a base point coordinate in the object, calculate an object height difference value of each rotation angle of any two target objects in the plurality of target objects within a rotation angle range, wherein when the object height difference value is less than When a predetermined value is used, the processing unit classifies the two target objects into the same candidate character category; when the height difference between the objects is not less than the predetermined value, the processing unit classifies the two target objects into different candidate characters category. The classification-based character inclination correction device as described in claim 6, wherein the processing unit determines a candidate character type having a maximum number of characters in the at least one candidate character type as the main character type. The classification-based character inclination correction device as described in item 6 of the scope of application, wherein the processing unit is based on the length and width of each target object itself ratio, the center point of the circumscribed rectangle, the area of the circumscribed rectangle, the center of the circumscribed circle, the area of the circumscribed circle, the radius of the circumscribed circle, or a combination thereof to classify the plurality of target objects as the at least one candidate character class. The classification-based character inclination correction device as described in claim 6, wherein the processing unit uses support vector machine, nearest neighbor classification, convolutional neural network, or deep learning to classify the plurality of target objects is the at least one candidate character category. The classification-based character inclination correction device according to item 6 of the scope of the application, wherein the processing unit is determined from each candidate character class to have the largest sum of character areas, the largest sum of edge strengths, the largest number of characters, the smallest The character difference of , the sum of the standard deviation of the largest character, the sum of the variation of the largest character, or the candidate character class with the smallest average brightness is used as the main character class.

Images