TWI747450B - Character recognition method, electric device and computer program product - Google Patents

Abstract

A character recognition method includes: performing an object detecting algorithm to detect a location of a character in an image; inputting the character to an image mapping neural network to output a dot-matrix image which includes multiple binary values; and recognizing the dot-matrix image as one of character classes.

Description

Character recognition method, electronic device and computer program product

This disclosure relates to a character recognition method, especially a method of first converting an image into a dot matrix image and then performing recognition.

In various industries, item identification is one of the most critical tasks in manufacturing, storage, and logistics operations. The item's identification code can be formed on the item by spray painting or printing, and then identified through the character recognition algorithm Origin code. In some conventional technologies, machine learning algorithms are used for character recognition. These methods will face some problems, such as requiring a large number of training samples, pre-processing of images, incomplete image or interference, etc. How to solve these problems is a topic of concern to those skilled in the art.

An embodiment of the present invention provides a character recognition method suitable for electronic devices. The character recognition method includes: executing an object detection algorithm to detect the position of a character in an image; and inputting the character to the image mapping nerve The network outputs a dot matrix image, the dot matrix image includes a plurality of binarized values; and the identification dot matrix image is one of a plurality of character types.

In some embodiments, the aforementioned image mapping neural network is a multilayer perception (MLP) or convolutional neural network.

In some embodiments, the step of recognizing that the dot matrix image is one of the character types includes: obtaining a plurality of preset dot matrix images, the preset dot matrix images respectively corresponding to the character type; and calculating the dots The difference or similarity between the matrix image and each preset dot matrix image is used to identify the character as the corresponding character category.

In some embodiments, the step of obtaining a preset dot matrix image includes: obtaining a plurality of standard dot matrix images, and changing the distribution of binarization values in the standard dot matrix image to generate the preset dot matrix image, wherein The total average difference of the preset dot matrix image is greater than the total average difference of the standard dot matrix image.

In some embodiments, the step of recognizing the dot matrix image as one of the character types includes: inputting the dot matrix image into a type of neural network to recognize the dot matrix image as one of the character types.

From another perspective, an embodiment of the present invention provides an electronic device including a memory and a processor. The memory stores multiple instructions, and the processor executes the instructions to complete multiple steps: execute object detection algorithm to detect the position of characters in an image; input characters to an image mapping neural network to output points A matrix image, the dot matrix image includes a plurality of binarized values; and the recognition dot matrix image is one of a plurality of character types.

From another perspective, the embodiment of the present invention provides a computer program product that is loaded and executed by a computer system to complete multiple steps: execute an object detection algorithm to detect the position of characters in an image; The element is input to an image-mapping neural network to output a dot matrix image, the dot matrix image includes a plurality of binary values; and the dot matrix image is recognized as one of a plurality of character types.

In the above-mentioned character recognition method, the image-mapping neural network can be independently trained and applied without too many training samples.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

Regarding the “first”, “second”, etc. used in this text, it does not particularly mean the order or sequence, but only distinguishes elements or operations described in the same technical terms.

Fig. 1 is a schematic diagram illustrating an electronic device according to an embodiment. Please refer to FIG. 1, the electronic device 100 may be a smart phone, a tablet computer, a personal computer, a notebook computer, a server, an industrial computer, or various electronic devices with computing capabilities, etc. The present invention is not limited thereto. The electronic device 100 includes a processor 110 and a memory 120. The processor 110 is communicatively connected to the memory 120. The processor 110 may be a central processing unit, a microprocessor, a microcontroller, an image processing chip, or a special application integrated circuit. Etc., the memory 120 can be random access memory, read-only memory, flash memory, floppy disk, hard disk, optical disk, flash drive, tape, or a database accessible through the Internet, in which storage There are multiple instructions, and the processor 110 executes these instructions to complete a character recognition method. The character recognition method will be described in detail below.

Fig. 2 is a flowchart illustrating a character recognition method according to an embodiment. Please refer to FIG. 2, where the characters in the image 210 are to be recognized. In this embodiment, the image 210 is about a steel coil, and there is a printed identification code on the steel coil, but in other embodiments, the image to be recognized may also be Regarding other arbitrary items, the characters to be recognized can be formed on the item by means of engraving, printing, spray painting, etc. The present invention does not limit the content of the image 210.

In step 220, an object detection algorithm is executed to detect the position of at least one character in the image 210. In some embodiments, the object detection algorithm can be a convolutional neural network, a support vector machine, etc. The invention is not limited thereto. In some embodiments, the above-mentioned convolutional neural network is YOLO (you only look once), which can be any version. This convolutional neural network has been trained in advance and can detect the position of characters. After the positions of the characters are detected, these characters can be cut out. In this embodiment, the detected character is “52619103”, and the character “6” can be cut out to become the image 230. In FIG. 2, a corresponding bounding box is drawn for each character to clearly indicate the position of the character, but in the subsequent processing, the image 230 does not need to include a bounding box.

In step 240, each detected character is input to an image-mapping neural network to output a dot matrix image. The dot matrix image includes the binary value of the matrix arrangement, such as "1 "Represents black, and "0" represents white. In this example, a total of 8 characters such as "52619103" are detected, and each character is input to the image mapping neural network, so a total of 8 dot matrix images are generated. For example, the dot matrix image 250 It contains the binary value representing the character "6". In some embodiments, the width of the dot matrix image is 5 and the height is 7, which means that each character is composed of a 5x7 binary value. However, the present invention does not limit the width and height of the dot matrix image. In other embodiments, the image-mapping neural network can also output larger or smaller dot matrix images. The aforementioned image mapping neural network is, for example, a convolutional neural network or a multilayer perception (MLP), and the present invention is not limited thereto. Take a multilayer perceptron as an example. The input layer of this multilayer perceptron receives each pixel of the image 230, and the output layer contains 35 neurons. These neurons output 35 binarized values. These 35 binarized values The value can form a dot matrix image of one character. In some embodiments, the multi-layer perceptron only includes two layers: the input layer and the output layer. Compared with the conventional technology that uses YOLO to recognize characters, the multi-layer perceptron can reduce the need for the number of training samples. In other words, even a very small number of training samples can still achieve high accuracy.

In some embodiments, the image 230 (excluding the bounding box) may be directly input to the image mapping neural network to output the dot matrix image 250. In some embodiments, a feature map corresponding to a character can be obtained from YOLO for input to the image-mapping neural network. In some embodiments, the image or feature map input to the image-mapping neural network may undergo operations such as up-sampling, down-sampling, and aspect ratio change, and the present invention is not limited thereto.

In step 260, the recognition point matrix image is one of a plurality of character categories, and these character categories include, for example, 10 categories such as "0" to "9". In some embodiments, the dot matrix image 250 may be input to a type of neural network to identify the character type, and the invention does not limit the type and architecture of such a neural network. In some embodiments, a plurality of preset dot matrix images may be obtained in advance, and these preset dot matrix images correspond to all character types, and then calculate the difference between the dot matrix image 250 and each preset dot matrix image The difference or similarity between the two to identify the character category. The preset dot matrix image will be explained below.

Please refer to Figures 2 and 3. Figure 3 shows a 5x7 dot matrix image 310 (also known as a standard dot matrix image) commonly used in general electronic instruments or computers, representing the characters "0" to "9" respectively . If the difference between the standard dot matrix images is analyzed in detail, the calculation formula for the difference is shown in the following mathematical formula (1). [Math 1]

為第i個字元與第j個字元之間的差異，N是字元類別的數量，

為第i個字元中位於座標(x,y)的二值化數值，

為第j個字元中位於座標(x,y)的二值化數值，例如“1”代表黑色，“0”代表白色，

為點矩陣圖像的寬度，

為點矩陣圖像的高度。計算結果請參照圖3的表格320，舉例來說，字元“0”與字元“1”之間的差異為4.2；字元“0”與字元“2”之間的差異為3.5，以此類推。此外，若把關於字元“0”所有的差異平均起來可以得到第12列的平均差，字元“0”的平均差為3.36；字元“1”的平均差為3.92，以此類推。從表格320中可以看出字元“0”、“5”、“6”、“8”、“9”的平均差相對比較小，也就是說比較難辨識出這些字元。表格420的最後一列是總平均差，是將所有字元的平均差再取平均所得出，在此例子中標準點矩陣圖像310的總平均差為3.52。 in

Is the difference between the i-th character and the j-th character, N is the number of character categories,

Is the binarized value at the coordinate (x, y) in the i-th character,

Is the binarized value at the coordinate (x, y) in the j-th character, for example, "1" represents black, and "0" represents white,

Is the width of the dot matrix image,

Is the height of the dot matrix image. Please refer to the table 320 in Figure 3 for the calculation result. For example, the difference between the character "0" and the character "1" is 4.2; the difference between the character "0" and the character "2" is 3.5, And so on. In addition, if all the differences about the character "0" are averaged, the average difference of the 12th column can be obtained. The average difference of the character "0" is 3.36; the average difference of the character "1" is 3.92, and so on. It can be seen from the table 320 that the average difference of the characters "0", "5", "6", "8", and "9" is relatively small, that is, it is difficult to identify these characters. The last column of the table 420 is the total average difference, which is obtained by averaging the average difference of all characters. In this example, the total average difference of the standard dot matrix image 310 is 3.52.

In this embodiment, in order to make the image mapping to the dot matrix image output by the neural network have a larger difference to improve the recognition of characters, the distribution of the binarized values in the standard dot matrix image 310 is changed to produce A new dot matrix image (called a preset dot matrix image). Please refer to FIG. 4, which shows a newly generated dot matrix image 410. The difference between these characters is shown in Table 420, and the drawing method is the same as Table 320. For example, the difference between the character "0" and the character "1" is 4; the difference between the character "0" and the character "2" is 4.2, and so on. Similarly, averaging all the differences about the same character can get the average difference of the 12th column, the average difference of the character "0" is 4.21; the average difference of the character "1" is 4.26, and so on. Finally, the total average difference corresponding to the dot matrix image 410 is 4.26. Comparing the table 320 with the table 420, it can be found that the total average difference of the preset dot matrix image 410 is greater than the total average difference of the standard dot matrix image 310, that is, the preset dot matrix image 410 has better recognition. After experimental testing, this can accelerate the convergence speed of the image mapping neural network and reduce the misjudgment rate of the characters "0", "5", "6", "8", and "9", thereby improving the recognition accuracy of the system Rate.

It is worth noting that when the preset point matrix image 410 is used, the preset point matrix image 410 is regarded as the ground truth when training the image mapping neural network, that is to say, in Figure 2 The dot matrix image output in step 240 will be similar to the preset dot matrix image 410, but for clarity of expression, FIG. 2 does not show the preset dot matrix image 410, but a standard dot matrix image.

[數學式3]

In an embodiment of step 260, the difference or similarity between the dot matrix image 250 and each preset dot matrix image is calculated. The calculation formula for the difference is shown in the following mathematical formula (2), the calculation of the similarity The formula is shown in mathematical formula (3). [Math 2]

[Math 3]

為圖像映對神經網路輸出的點矩陣圖像(例如點矩陣圖像250)中位於座標(x,y)的二值化數值，

為第i個預設點矩陣圖像中位於座標(x,y)的二值化數值，

為點矩陣圖像250與第i個預設點矩陣圖像之間的差異，

點矩陣圖像250與第i個預設點矩陣圖像之間的相似度。若採用差異，則可以採用最小的差異

，也就是說最小差異

所對應的第i個字元類別是點矩陣圖像250所屬的字元類別；若採用相似度，則可以採用最大的相似度

，也就是說最大相似度

所對應的第i個字元類別是點矩陣圖像250所屬的字元類別。對於圖像映對神經網路所輸出的每個點矩陣圖像都可經過上述的運算找到所屬的字元類別。 in

Is the binarized value at the coordinate (x, y) in the dot matrix image (for example, the dot matrix image 250) output by the image mapping neural network,

Is the binarized value at the coordinate (x, y) in the i-th preset point matrix image,

Is the difference between the dot matrix image 250 and the i-th preset dot matrix image,

The similarity between the dot matrix image 250 and the i-th preset dot matrix image. If the difference is used, the smallest difference can be used

, Which means that the smallest difference

The corresponding i-th character category is the character category to which the dot matrix image 250 belongs; if the similarity is used, the maximum similarity can be used

, Which means that the maximum similarity

The corresponding i-th character category is the character category to which the dot matrix image 250 belongs. For each dot matrix image output by the image mapping neural network, the character category to which it belongs can be found through the above-mentioned operations.

One advantage of the above-mentioned embodiment is that the image mapping neural network can be independently trained and applied. A major difficulty when using machine learning algorithms is obtaining training samples. The conventional technology uses algorithms such as YOLO to recognize characters. Due to the large number of parameters in these algorithms, a large number of training samples are also required. In this embodiment, the required image mapping neural network can be independently trained, and this network does not require too many training samples. Has a very good recognition rate. When the font of the character image to be recognized changes, the above-mentioned architecture can also be used to obtain good recognition results without retraining the image mapping neural network. Furthermore, the step of converting characters into a dot matrix image does not require pre-processing, incomplete filling, noise rate division and other processing to have good results. The experimental results show that even if there is interference in the image (such as steel strip crossing In the case of character), the overall recognition rate of the system is still very high.

The steps in FIG. 2 above can be implemented as multiple program codes or circuits, and the present invention is not limited thereto. From another perspective, the present invention also proposes a computer program product, which can be written in any programming language and/or platform. When the computer program product is loaded into the computer system and executed, it can execute Figure 2 Character recognition method.

In some embodiments, the aforementioned character recognition method can also be used to recognize English letters or any suitable characters.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to those defined by the attached patent scope.

100: electronic device 110: processor 120: memory 210,230: image 220, 240, 260: steps 250, 310, 410: dot matrix image 320,420: table

[Fig. 1] is a schematic diagram showing an electronic device according to an embodiment. [Fig. 2] is a flowchart of a character recognition method according to an embodiment. [Fig. 3] is a schematic diagram showing a standard dot matrix image and its difference table according to an embodiment. [Fig. 4] is a schematic diagram of a preset dot matrix image and its difference table according to an embodiment.

210,230: image

220, 240, 260: steps

250: dot matrix image

Claims (6)

A character recognition method suitable for an electronic device. The character recognition method includes: obtaining a plurality of standard dot matrix images, and changing the distribution of binary values in the standard dot matrix images to generate a plurality of preset dots Matrix image, calculating the average difference between each of the preset dot matrix images and other preset dot matrix images, calculating a total average difference of the average differences of the preset dot matrix images, where The preset dot matrix images respectively correspond to a plurality of character types, and the total average difference of the preset dot matrix images is greater than a total average difference of the standard dot matrix images; an object detection algorithm is executed To detect the position of a character in an image; input the character to an image-mapping neural network to output a dot matrix image, wherein the dot matrix image includes a plurality of binary values, the image The mapping neural network only includes an input layer and an output layer. The output layer includes a plurality of neurons. The number of these neurons is the same as the number of the binarized values, and the neurons output the two Value the value, the image mapping neural network is trained according to the preset dot matrix images; and identify the dot matrix image as one of a plurality of character types. The character recognition method according to claim 1, wherein the image mapping neural network is a multilayer perception (MLP). The character recognition method as described in claim 1, wherein the recognition The step of the dot matrix image being one of the character types includes: calculating the difference or similarity between the dot matrix image and each of the preset dot matrix images to identify the character as One of these character categories. The character recognition method of claim 1, wherein the step of recognizing the dot matrix image as one of the character types includes: inputting the dot matrix image to a type of neural network to recognize the dot matrix The image is one of these character categories. An electronic device includes: a memory for storing a plurality of instructions; and a processor for executing the instructions to complete a plurality of steps: obtaining a plurality of standard dot matrix images, and changing the standard dot matrix images Binarize the distribution of values to generate a plurality of preset point matrix images, calculate the average difference between each of the preset point matrix images and other preset point matrix images, and calculate the preset point matrix images A total average difference of the average differences of images, wherein the preset dot matrix images respectively correspond to a plurality of character types, and the total average difference of the preset dot matrix images is greater than the standard dot matrix images Perform an object detection algorithm to detect the position of a character in an image; input the character to an image-mapping neural network to output a dot matrix image, where the dot matrix The image includes multiple binary values. The image mapping neural network only includes an input layer and an output layer. The output layer includes Contains multiple neurons, the number of these neurons is the same as the number of the binarized values, the neurons output the binarized values respectively, and the image mapping neural network is based on the presets The dot matrix image is trained; and the dot matrix image is recognized as one of a plurality of character types. A computer program product loaded and executed by a computer system to complete multiple steps: obtaining multiple standard dot matrix images, and changing the distribution of binary values in the standard dot matrix images to generate multiple preset points Matrix image, calculating the average difference between each of the preset dot matrix images and other preset dot matrix images, calculating a total average difference of the average differences of the preset dot matrix images, where The preset dot matrix images respectively correspond to a plurality of character types, and the total average difference of the preset dot matrix images is greater than a total average difference of the standard dot matrix images; an object detection algorithm is executed To detect the position of a character in an image; input the character to an image-mapping neural network to output a dot matrix image, wherein the dot matrix image includes a plurality of binary values, the image The mapping neural network only includes an input layer and an output layer. The output layer includes a plurality of neurons. The number of these neurons is the same as the number of the binarized values, and the neurons output the two Value the value, the image mapping neural network is trained according to the preset dot matrix images; and identify the dot matrix image as one of a plurality of character types.

Images