CN114882511A

CN114882511A - Handwritten Chinese character recognition method, system, equipment and storage medium based on AMNN and Chinese character structure dictionary

Info

Publication number: CN114882511A
Application number: CN202210591264.3A
Authority: CN
Inventors: 邓伟廷; 邓智升
Original assignee: Shengma Intelligent Technology Shenzhen Co ltd
Current assignee: Shengma Intelligent Technology Shenzhen Co ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2022-08-09

Abstract

The invention provides a handwritten Chinese character recognition method, system, equipment and storage medium based on AMNN and Chinese character structure dictionary, the method includes preprocessing the image data of the handwritten Chinese character image; classifying the preprocessed handwritten Chinese character structures; extracting the classified handwritten Chinese character structure; recognizing the structure of the handwritten Chinese character; and recognizing the handwritten Chinese characters. The invention classifies, extracts and identifies the handwritten Chinese character structure, and identifies the handwritten Chinese character finally, and the handwritten Chinese character is identified after being split, so the invention has the advantages of high processing speed, high identification accuracy, small system volume, wide identification range and the like. Compared with the method which is continuously complicated in the depth and the complexity of the neural network, the method has more practicability and universality.

Description

Handwritten Chinese character recognition method, system, equipment and storage medium based on AMNN and Chinese character structure dictionary

Technical Field

The invention relates to the technical field of handwritten Chinese character recognition, in particular to a handwritten Chinese character recognition method, a handwritten Chinese character recognition system, handwritten Chinese character recognition equipment and a storage medium based on AMNN and a Chinese character structure dictionary.

Background

With the development of the times, the recognition of Chinese characters is concerned in application scenes such as document retrieval, mail classification, automatic appraisal and the like. Due to the fact that requirements of people on functions of electronic products are increased, nowadays, all smart phones, tablet computers and handwriting pens achieve handwriting input functions, and therefore the requirements for handwritten Chinese character recognition are greatly improved, however, handwritten Chinese character recognition is still a hotspot and a difficulty of current mode recognition due to the fact that the number of Chinese characters is large, the styles of handwritten Chinese characters are different, and the like. Mainly present four distinct challenges: namely speed of recognition, accuracy, recognition range and volume of the recognition system.

In recent years, due to the development of scientific technology, some paper documents need to be digitally processed, and the traditional handwriting recognition is slow in speed, low in accuracy, large in size and limited in recognition range. The efficiency of using manual input is low, but the method using image recognition can improve the recognition efficiency, but has the problems of recognition speed, accuracy and the like.

Disclosure of Invention

The invention provides a handwritten Chinese character recognition method, a system, equipment and a storage medium based on AMNN and a Chinese character structure dictionary, which improves the speed of handwritten Chinese character recognition, improves the accuracy rate of handwritten Chinese character recognition, reduces the volume of a recognition system and expands the recognition range of handwritten Chinese characters.

The invention has a technical scheme that: a handwritten Chinese character recognition method based on AMNN and Chinese character structure dictionary is provided, which comprises the following steps:

preprocessing the image data of the handwritten Chinese character image;

classifying the preprocessed handwritten Chinese character structures;

extracting the classified handwritten Chinese character structure;

recognizing the handwritten Chinese character structure;

and recognizing the handwritten Chinese characters.

As an improvement of the present invention, in the step of preprocessing the image data of the handwritten chinese character image, the following is further included:

carrying out gray processing on the handwritten Chinese character image;

carrying out binarization processing on the gray level image;

and obtaining a binary image of the single handwritten Chinese character.

As an improvement to the present invention, in the step of obtaining the binarized image of a single chinese character, the following is also included:

and detecting the position of a single Chinese character on the binary image by using a projection method, and cutting the binary image word by word according to a detection result to obtain the binary image of the single handwritten Chinese character.

As an improvement of the present invention, in the step of classifying the preprocessed handwritten Chinese character structure, the following contents are also included:

extracting image texture characteristics of the binarized image of each handwritten Chinese character by using a gray level co-occurrence matrix, and the contrast, energy and entropy of the gray level co-occurrence matrix;

and classifying the handwritten Chinese character structures in the binarized image of each handwritten Chinese character by using a support vector machine.

As an improvement of the present invention, in the step of classifying the handwritten Chinese character structure in the binarized image of each handwritten Chinese character by using a support vector machine, the following contents are further included:

inputting the image texture characteristics of the binarization image of each handwritten Chinese character and the contrast, energy and entropy of the gray level co-occurrence matrix into a support vector machine;

and classifying the handwritten Chinese character structures in the binarized image of each handwritten Chinese character according to all Chinese character types.

As an improvement of the present invention, in the step of extracting the classified handwritten Chinese character structure, the following contents are also included:

extracting black character lines in the binary image of each handwritten Chinese character by using a sliding window method;

and identifying and splitting the radicals of the Chinese characters in the binary image of the handwritten Chinese characters to obtain all the components of the Chinese characters in the binary image of the handwritten Chinese characters.

As an improvement to the present invention, in the step of recognizing the structure of the handwritten Chinese character, the following contents are further included:

inputting each component of the Chinese character into an attention mechanism neural network;

and identifying to obtain the actual Chinese character part corresponding to each component of the Chinese character.

As an improvement of the present invention, in the step of recognizing handwritten Chinese characters, the following contents are also included:

and searching in a Chinese character feature dictionary according to the classification result and the recognition result of the handwritten Chinese character structure to obtain a Chinese character result.

The other technical scheme of the invention is as follows: a handwritten Chinese character recognition system based on AMNN and a Chinese character structure dictionary is provided, which comprises:

the preprocessing module is used for preprocessing image data of the handwritten Chinese character image;

the classification module is used for classifying the preprocessed handwritten Chinese character structures;

the extraction module is used for extracting the classified handwritten Chinese character structures;

the structure recognition module is used for recognizing the handwritten Chinese character structure;

and the Chinese character recognition module is used for recognizing the handwritten Chinese characters.

As an improvement to the present invention, the preprocessing module comprises:

the gray processing module is used for carrying out gray processing on the handwritten Chinese character image;

the binarization processing module is used for carrying out binarization processing on the gray level image;

and the single character processing module is used for obtaining a binary image of a single handwritten Chinese character.

As an improvement of the invention, the single word processing module comprises:

a position detection model, which is used for detecting the position of a single Chinese character on the binary image by using a projection method;

and the cutting module cuts the Chinese characters word by word according to the detection result to obtain the binary image of the single handwritten Chinese character.

As an improvement to the present invention, the classification module comprises:

the image texture feature module is used for extracting the image texture features of the binarization image of each handwritten Chinese character by using a gray level co-occurrence matrix, and the contrast, the energy and the entropy of the gray level co-occurrence matrix;

and the vector machine classification module is used for classifying the handwritten Chinese character structures in the binarization image of each handwritten Chinese character by using a support vector machine.

As an improvement to the present invention, the vector machine classification module comprises:

the vector machine input module is used for inputting the image texture characteristics of the binarization image of each handwritten Chinese character and the contrast, the energy and the entropy of the gray level co-occurrence matrix into a support vector machine;

and the vector machine structure classification module is used for classifying the handwritten Chinese character structures in the binarization image of each handwritten Chinese character according to all Chinese character types.

As an improvement to the present invention, the extraction module comprises:

the line extraction module is used for extracting black character lines in the binarization image of each handwritten Chinese character by using a sliding window method;

and the recognition and splitting module is used for recognizing and splitting the radicals of the Chinese characters in the binary image of the handwritten Chinese characters to obtain all the components of the Chinese characters in the binary image of the handwritten Chinese characters.

As an improvement to the present invention, the structure recognition module includes:

each component input module inputs each component of the Chinese character into the attention mechanism neural network;

and the component part identification module is used for identifying and obtaining the actual Chinese character part corresponding to each component part of the Chinese character.

As an improvement of the invention, the Chinese character recognition module searches in a Chinese character feature dictionary according to the classification result and the recognition result of the handwritten Chinese character structure and obtains a Chinese character result.

The third technical scheme of the invention is as follows: there is provided a computer device comprising a memory and a processor, the memory storing a computer program, the processor, when executing the computer program, implementing the steps of any of the above-mentioned methods for handwritten Chinese character recognition based on the AMNN and the dictionary of Chinese character structures.

The fourth technical scheme of the invention is as follows: there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for handwritten Chinese character recognition based on the AMNN and the chinese character structure dictionary as described in any one of the above.

The invention classifies, extracts and identifies the handwritten Chinese character structure, and identifies the handwritten Chinese character finally, and the handwritten Chinese character is identified after being split, so the invention has the advantages of high processing speed, high identification accuracy, small system volume, wide identification range and the like. Compared with the method which is continuously complicated in the depth and the complexity of the neural network, the method has more practicability and universality.

Drawings

FIG. 1 is a flow chart of a handwritten Chinese character recognition method in the present invention.

FIG. 2 is a handwritten Chinese character image in accordance with the present invention.

Fig. 3 is a grayscale diagram of fig. 2.

Fig. 4 is the binarized image of fig. 3.

FIG. 5 is a binarized image of a single handwritten Chinese character in the present invention.

FIG. 6 is a diagram of the structural types of all Chinese characters in the present invention.

Fig. 7 is a schematic diagram of the extraction result of the black line of the characters in fig. 6.

Fig. 8 is a diagram illustrating the result of splitting each part of the text in fig. 6.

FIG. 9 is a schematic flow chart of an attention mechanism neural network according to the present invention.

FIG. 10 is a diagram illustrating the search in a Chinese character feature dictionary in accordance with the present invention.

FIG. 11 is a block diagram of a handwritten Chinese character recognition system in accordance with the present invention.

FIG. 12 is a block diagram showing the structure of a computer apparatus according to the present invention.

Wherein:

1. a preprocessing module; 2. a classification module; 3. an extraction module; 4. a structure identification module; 5. a Chinese character recognition module; 71. a processor; 72. an input interface; 73. a network port; 74. a display unit; 75. a memory.

Detailed Description

In the description of the present invention, it is to be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience in describing the present invention and simplifying the description, but do not indicate or imply that the device or assembly referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; the two components can be directly connected or indirectly connected through an intermediate medium, and the two components can be communicated with each other. The specific meaning of the above terms in the present invention can be understood in specific cases to those skilled in the art.

The invention provides a handwritten Chinese character recognition method based on AMNN and a Chinese character structure dictionary, please refer to FIG. 1, which comprises the following steps:

100. preprocessing the image data of the handwritten Chinese character image;

200. classifying the preprocessed handwritten Chinese character structures;

300. extracting the classified handwritten Chinese character structure;

400. recognizing the structure of the handwritten Chinese character;

500. and recognizing the handwritten Chinese characters.

In the method, a camera is used for shooting handwritten texts to obtain clear handwritten Chinese character images, as shown in figure 2.

In the above step 100 of the method, the image data preprocessing is performed on the handwritten Chinese character image, and the method further comprises the following steps.

101. And carrying out graying processing on the handwritten Chinese character image.

The handwritten Chinese character image is subjected to graying processing to obtain a grayscale image of the handwritten Chinese character image, as shown in fig. 3. The method aims to simplify the pixel information of the picture and improve the calculation speed of detecting characters and processing the picture by the neural network in the subsequent steps.

Graying treatment: the color image is composed of three color channels of RGB (R: red, G: green and B: blue), each pixel point on the color image is represented by three color variables of RGB, the three channels of RGB of each pixel point of the image are calculated to obtain a new pixel value according to a formula 1, when the RGB values of each pixel point on the image are equal, the color image is changed into a single-channel gray image, and the processing of the image can be accelerated by converting the three-channel color image into the single-channel gray image.

Gray ═ R0.299 + G0.587 + B0.114 formula 1

Wherein Gray represents the Gray value of the image.

102. And carrying out binarization processing on the gray-scale image.

And (3) carrying out binarization processing on the gray level image to obtain a binarized image of the handwritten Chinese characters only with black and white colors, as shown in figure 4. The method aims to further eliminate redundant characteristic information on the handwritten Chinese character image, retain main characteristic information of the handwritten Chinese character image and further improve the calculation speed when the neural network processes the handwritten Chinese character image in the subsequent steps. In addition, the binarization processing aims to highlight the edges of characters in the handwritten Chinese character image to the maximum extent, increase the accuracy of subsequent character position detection, character cutting and character area extraction, and further increase the accuracy of subsequent character recognition by a neural network.

And (3) binarization processing: because target characters, backgrounds and noises exist in the handwritten Chinese character images, the noises and the backgrounds of the handwritten Chinese character images need to be removed when the target characters are extracted and identified from the handwritten Chinese character images. Typically the text is black or darker. Therefore, binarization processing is used for the grayscale image. In the invention, when the pixel value of the handwritten Chinese character image is more than 100, the pixel value is assigned to be 0; and assigning 255 for the pixel points with the pixel values of less than 100 of the handwritten Chinese character images. After the binary processing is carried out on the handwritten Chinese character image, the characters in the handwritten Chinese character image are white, and the background is black.

103. And obtaining a binary image of the single handwritten Chinese character.

And (3) carrying out position detection on the single Chinese character on the binary image by using a projection method, and cutting the binary image word by word according to a detection result to obtain the binary image of the single handwritten Chinese character, as shown in figure 5.

Projection method: the main idea of the projection method is to record the number of pixels of the corresponding value of each row or each column, and then judge whether it is a boundary or a target object according to the number. The number of pixels is just like a threshold value, and finally the number of each row of points can be drawn to facilitate visual observation.

The principle of the projection method segmentation is that the binarized image is respectively subjected to vertical projection and horizontal projection, because the literal characters are black and the background is white, peaks are formed at the literal characters and troughs are formed at the background as a result of the projection, and segmentation of the literal characters can be completed by finding boundary points of the peaks and the troughs and taking the boundary points as character segmentation positions. Horizontally segmenting the image through horizontal projection to obtain an image of each line; and vertically dividing each divided line of image through vertical projection, finally determining the coordinate position of each character, and dividing each character.

In the above step 200 of the method, the step of classifying the preprocessed handwritten Chinese character structure further includes the following steps.

201. And extracting image texture characteristics of the binary image of each handwritten Chinese character by using the gray level co-occurrence matrix, and the contrast, energy and entropy of the gray level co-occurrence matrix.

Respectively calculating the image texture characteristics (LBP- - -Local Binary Pattern) of the Binary image of each handwritten Chinese character, the contrast of the gray level co-occurrence matrix, the energy of the gray level co-occurrence matrix and the entropy of the gray level co-occurrence matrix.

Image texture features: the image texture features are operators used for describing local texture features of the image, and have the advantages of rotation invariance, gray scale invariance and the like. The LBP operator is defined as that in the window of 3 × 3, the central pixel of the window is used as a threshold value, the gray values of the adjacent 8 pixels are compared with the central pixel, if the values of the surrounding pixels are greater than the value of the central pixel, the position of the pixel point is marked as 1, otherwise, the position is 0. Thus, 8 pixel points in the 3 × 3 neighborhood can generate 8-bit binary numbers through comparison, and the LBP value of the pixel point in the center of the window is obtained. We perform the extraction of LBP features on the image using the grayscale image obtained at Step 2.

202. And classifying the handwritten Chinese character structures in the binary image of each handwritten Chinese character by using a support vector machine.

2021. And inputting the image texture characteristics of the binary image of each handwritten Chinese character and the contrast, energy and entropy of the gray level co-occurrence matrix into a support vector machine.

Contrast of gray level co-occurrence matrix: the values of the metric matrix are how distributed and how much of the local variation in the image reflects the sharpness of the image and the depth of the texture. The larger the contrast value is, the deeper the furrows of the texture are, the larger the contrast is, and the clearer the effect is; otherwise, if the contrast value is small, the grooves are shallow and the effect is blurred. And calculating the contrast of the gray level co-occurrence matrix of the image through the gray level image obtained by the formula 1. And calculating the contrast of the gray level co-occurrence matrix through a formula 3, wherein i and j represent the pixel values of any two pixel points in the image, and p (i, j) represents the value of the gray level co-occurrence matrix of the pixel values of the two points.

Energy of gray level co-occurrence matrix: the energy transformation reflects the uniformity of the image gray level distribution and the texture thickness. If the element values of the gray level co-occurrence matrix are similar, the energy is smaller, and the texture is detailed; if some of these values are large, while others are small, the energy values are large. A large energy value indicates a more uniform and regularly varying texture pattern. And calculating the energy of the gray level co-occurrence matrix of the image through the gray level image obtained by the formula 1. And calculating the energy of the gray level co-occurrence matrix through a formula 4, wherein i and j represent the pixel values of any two pixel points in the image, and p (i, j) represents the value of the gray level co-occurrence matrix of the pixel values of two points.

Entropy of gray level co-occurrence matrix: the energy transformation reflects the uniformity of the image gray level distribution and the texture thickness. If the element values of the gray level co-occurrence matrix are similar, the energy is smaller, and the texture is detailed; if some of these values are large, while others are small, the energy values are large. A large energy value indicates a more uniform and regularly varying texture pattern. And calculating the energy of the gray level co-occurrence matrix of the image through the gray level image obtained by the formula 1. The entropy of the gray level co-occurrence matrix is calculated by formula 5, where i and j represent pixel values of any two pixel points in the image, and p (i, j) represents a value of the gray level co-occurrence matrix of the pixel values of two points.

2022. And classifying the handwritten Chinese character structures in the binary image of each handwritten Chinese character according to all Chinese character types.

And classifying character structures in the binary image of the handwritten Chinese characters by using a Support Vector Machine (SVM), and storing a classification result as one feature of the handwritten Chinese characters. The invention proposes to divide all Chinese characters into twelve classes, each class is as shown in figure 6, and the twelve classes are respectively: upper and lower structures, such as Lu; left and right structures, such as the eye; left, right, middle, and left structures, such as plaques; upper, middle and lower structures, such as yellow; fully enclosed structures, such as countries; the top three enclosure structures, as questions; the lower three surrounding structures, such as pictures; a left three enclosure structure, such as a box; upper left surrounding structures, such as temple; upper right bounding structures, such as sentences; lower left surrounding structure, if constructed; damascene structures such as strings.

Support Vector Machine (SVM): the SVM is called a support vector machine, is mainly used for solving the classification problem, and belongs to one of supervised learning algorithms. In the binary problem, the SVM goal is to find a hyperplane so that the greater the distance between the two types of data from the hyperplane, the better. That is, the decision boundary of the SVM is to solve the maximum edge distance hyperplane for the samples. For a given data set T and hyperplane w · x + b ═ 0, the optimal hyperplane is required and hyperplane parameters w and b can be obtained by solving equation 6. Where c is the penalty factor and ε is the error.

For the nonlinear classification problem, since in the dual problem of the linear SVM, only the inner product between samples needs to be obtained, the nonlinear transformation does not need to be specified, but the inner product in the middle is replaced by a kernel function. Assuming that there is a mapping of phi (x) from the input space to the feature space, for (x) in any input space _i ,x _j ) All have a kernel function (equation 7). The kernel function is used for replacing the mapped sample inner product, so that the problem of linear inseparable sample classification can be effectively solved, and a formula 8 is a classification function of the SVM. Wherein (x) _i ,y _i ) The number of points of the sample is represented,

for lagrange multiplier

K(x _i ,x _j )＝φ(x _i ) ^T φ(x _j ) Equation 7

In the above step 300 of the method, the step of extracting the classified handwritten Chinese character structure further comprises the following steps.

301. And extracting black character lines in the binary image of each handwritten Chinese character by using a sliding window method.

302. And identifying and splitting the radicals of the Chinese characters in the binary image of the handwritten Chinese characters to obtain all the components of the Chinese characters in the binary image of the handwritten Chinese characters.

And (3) extracting black character lines in the obtained binary image of the single Chinese character by using a Sliding Window method (Sliding Window), and simultaneously identifying and splitting radicals of the Chinese character in the binary image of the handwritten Chinese character to obtain each component of the Chinese character in the binary image of the handwritten Chinese character. The method aims to reserve the character area with the actual meaning, so that the burden of subsequent neural network calculation is further reduced, and the calculation speed is greatly improved. Meanwhile, extracting the black text lines is equivalent to removing noise points in the background, so that the accuracy of subsequent network identification is further improved. The extraction result of the black lines of the characters is shown in fig. 7, and the splitting result of each part of the characters is shown in fig. 8.

Sliding Window method (Sliding Window): and sliding the image by using sliding windows with different window sizes (from left to right and from top to bottom), executing a classifier on the current window during each sliding, and judging whether the window contains text content. After detecting each sliding window with different sliding window sizes, the object marks detected by different windows can be obtained, and the window sizes can have parts with higher repetition. And finally, screening out a window with the highest probability by adopting a non-maximum value suppression algorithm.

When a window is screened by using non-maximum suppression, several candidate frames with high repetition rate sometimes appear when candidate frame prediction is carried out, and the non-maximum suppression is to remove the candidate frames with high repetition rate. The non-maximum suppression method deletes the candidate frame with small score by calculating the IOU value of two windows and selecting the window with larger score if the IOU value of two windows exceeds the specified threshold.

The IOU (interaction Over Unit) is called a cross-Over ratio, and the IOU is used to measure the correlation between two windows. The IOU is the intersection of the areas of the two windows compared to the union of the areas of the two windows. The larger the value of IOU, the higher the coincidence of the two windows.

In the above step 400 of the method, the handwritten Chinese character structure is recognized, which further includes the following steps.

401. And inputting all the components of the Chinese characters into an attention mechanism neural network.

402. And identifying to obtain the actual Chinese character part corresponding to each component part of the Chinese character, and storing each identified component part of the Chinese character as another characteristic of the handwritten Chinese character.

Attention Mechanism Neural Network (AMNN): an attention-driven neural network (AMNN) is composed of a convolutional layer, a pooling layer, a normalization layer and a fully-connected layer. The convolution network is divided into two paths to extract the characteristics of the image, the first path generates an Attention Map (Attention Map) through a 1 x 1 convolution layer, the second path extracts the characteristics of the image by performing the operations of the convolution layer, a pooling layer and normalization on the text and digital image, finally the Attention Map is mapped on the original image, the extracted image characteristics are subjected to convolution operation through the convolution layer and then are identified through a full connection layer, and a final result is obtained. It should be noted that the specific flow of AMNN is shown in fig. 9.

The first path, the lowest convolution layer with 1 × 1 generates the attention map, the convolution kernel with 1 × 1 makes the network not lose the content in the learning process, and the size of the obtained attention feature map is unchanged with the original map, and if the attention feature map is combined through the top line, the mapping of the attention feature map onto the original map is equivalent.

The second path is the common image feature extraction, and the features of the image are obtained after normalization through a convolution layer of 3 × 3 and a pooling layer of 3 × 3. The 3 × 3 convolutional layer not only can learn the characteristics of the image more finely, but also can reduce the training parameters of the image. 3 the pooling layer greatly reduces the size of the feature map, and improves the training speed of the network. The addition of the normalization layer avoids the situation of gradient explosion in network training, so that the convergence rate of the network is higher.

And finally, mapping the attention diagram on the original image, splicing the image features extracted by the second path, sending the image features into a 3 x 3 convolutional layer, performing convolution operation to obtain the features of the image (the 3 x 3 convolutional layer can not only learn the features of the image more finely, but also reduce the training parameters of the image), classifying through the full-connection layer to obtain a final classification result, and outputting the full-connection layer as 1, namely the classification result.

In the above step 500 of the method, the method for recognizing handwritten Chinese characters further comprises the following steps.

According to the classification result and the recognition result of the handwritten Chinese character structure, searching is carried out in a Chinese character feature dictionary (as shown in figure 10) and a Chinese character result is obtained. It should be noted that the classification result indicates which category of the twelve categories the handwritten Chinese character structure is, for example, whether the handwritten Chinese character structure is a left-right structure, a top-bottom structure, or the like; the recognition result is the actual chinese character part corresponding to each component of the chinese character, for example, when the handwritten chinese character structure is a left-right structure, the left-side structure and the right-side structure correspond to what content of the actual chinese character part respectively, please refer to fig. 10, which is only an example, and the invention is not limited thereto.

The invention provides a method for identifying handwritten Chinese characters, which has the following advantages:

the invention has the characteristic of higher processing speed. The method provided by the invention fully preprocesses the acquired high-definition images, gradually reduces the volume of the original large-volume handwritten Chinese character images shot by the CCD camera on the basis of keeping the characteristics, and shortens the time of processing the images by a computer in each subsequent step.

The invention has the characteristic of high identification accuracy. The invention provides a retrieval and recognition method combining a Support Vector Machine (SVM), a Sliding Window method (Sliding Window), an AMNN network and a Chinese character structure dictionary. Compared with the prior method for integrally identifying the Chinese characters, the method increases the accuracy of Chinese character identification.

The invention has the characteristic of small volume. The traditional method for increasing the recognition accuracy of handwritten Chinese characters is to innovate on a neural network, and continuously increase the number of layers of the neural network or complicate the layers of the network. This adds virtually to the volume of the neural network, resulting in an inordinately long time-consuming training of the network. The recognition method provided by the invention puts innovation points on Chinese character splitting recognition, abandons the traditional concept of complicated neural network, and emphatically considers the requirement of small volume when the neural network AMNN is used for designing.

The invention has the characteristic of wide identification range. The traditional recognition mode of the neural network can only recognize Chinese characters existing in a training set, and if the network recognition meets the Chinese characters not existing in the training set, the recognition cannot be carried out or the recognition is wrong. The invention provides a method for classifying handwritten Chinese characters into 12 structures by using a Support Vector Machine (SVM) in cooperation with a Sliding Window method (Sliding Window), and then carrying out split recognition, wherein the AMNN only needs to train and recognize the existing radicals, and the method greatly widens the recognizable range of the invention.

The invention provides a handwritten Chinese character recognition system based on AMNN and a Chinese character structure dictionary, please refer to FIG. 11, the handwritten Chinese character recognition system comprises a preprocessing module 1, a classification module 2, an extraction module 3, a structure recognition module 4 and a Chinese character recognition module 5.

In this embodiment, the preprocessing module 1 is configured to perform image data preprocessing on a handwritten Chinese character image; the preprocessing module 1 is used for preprocessing image data of the handwritten Chinese character image; the classification module 2 is used for classifying the preprocessed handwritten Chinese character structures; the extraction module 3 is used for extracting the classified handwritten Chinese character structures; the structure recognition module 4 is used for recognizing the structure of the handwritten Chinese character; and the Chinese character recognition module 5 is used for recognizing the handwritten Chinese characters.

In this embodiment, the preprocessing module 1 includes a gray scale processing module (not shown), a binarization processing module (not shown), and a single character processing module (not shown). The gray processing module is used for carrying out gray processing on the handwritten Chinese character image; the binarization processing module is used for carrying out binarization processing on the gray level image; and the single character processing module is used for obtaining a binary image of a single handwritten Chinese character.

In this embodiment, the word processing module includes a position detection pattern (not shown) and a clipping module (not shown). Detecting the position of a single Chinese character in the binary image by using a projection method; and the cutting module cuts the Chinese characters word by word according to the detection result to obtain a binary image of the single handwritten Chinese character.

In this embodiment, the classification module 2 includes an image texture feature module (not shown) and a vector machine classification module (not shown). The image texture feature module is used for extracting the image texture features of the binary image of each handwritten Chinese character by using the gray level co-occurrence matrix, and the contrast, the energy and the entropy of the gray level co-occurrence matrix; and the vector machine classification module is used for classifying the handwritten Chinese character structures in the binary image of each handwritten Chinese character by using a support vector machine.

In this embodiment, the vector machine classification module includes a vector machine input module (not shown) and a vector machine structure classification module (not shown). The vector machine input module is used for inputting the image texture characteristics of the binarization image of each handwritten Chinese character and the contrast, the energy and the entropy of the gray level co-occurrence matrix into the support vector machine; and the vector machine structure classification module is used for classifying the handwritten Chinese character structures in the binary image of each handwritten Chinese character according to all Chinese character types.

In this embodiment, the extracting module 3 includes a line extracting module (not shown) and an identifying and splitting module (not shown). The line extraction module is used for extracting black character lines in the binary image of each handwritten Chinese character by using a sliding window method; and the recognition and splitting module is used for recognizing and splitting the radicals of the Chinese characters in the binary image of the handwritten Chinese characters to obtain all the components of the Chinese characters in the binary image of the handwritten Chinese characters.

In this embodiment, the structure recognition module 4 includes a component input module (not shown) and a component recognition module (not shown). Each component input module inputs each component of the Chinese character into the attention mechanism neural network; and the component part identification module is used for identifying and obtaining the actual Chinese character part corresponding to each component part of the Chinese character.

In this embodiment, the chinese character recognition module 5 retrieves from the chinese character feature dictionary and obtains a chinese character result according to the classification result and recognition result of the handwritten chinese character structure.

It should be noted that the contents in the foregoing method embodiments are all applicable to the corresponding system embodiments, and thus the functions specifically implemented by the system embodiments are the same as those in the foregoing method embodiments, and the beneficial effects achieved by the system embodiments are also the same as those in the foregoing method embodiments. Further, details of the system embodiment are not repeated, and please refer to the above method for details.

Thirdly, the invention provides a best implementation scheme by combining the contents of the method and the system.

In this embodiment, the handwritten Chinese character recognition algorithm is installed in a computer, and the computer needs to be configured with a Win10 system, an NVIDA 3080Ti GPU, a machine vision library Opencv, a Pytorch deep learning framework based on Python language, and an 8G RAM. The CCD camera (charge coupled camera) is connected with the computer, so that the handwritten Chinese character image shot by the CCD camera can be smoothly transmitted to the computer.

11. And shooting a handwritten Chinese character image. The handwritten Chinese character data such as student homework and the like are placed in a bright environment, a CCD camera (charge coupled camera) is used for shooting handwritten Chinese character images page by page right above the handwritten Chinese characters, and the shot images are transmitted to a computer.

12. And carrying out gray processing on the handwritten Chinese character image to obtain a gray image of the handwritten Chinese character.

13. And carrying out binarization processing on the gray level image of the handwritten Chinese character image to obtain the handwritten Chinese character binarization image with only black and white colors.

14. And (3) carrying out position detection on the single character on the binary image of the handwritten Chinese character by using a projection method, and cutting the binary image word by word according to a detection result to obtain the binary image of the handwritten form of the single Chinese character.

15. Respectively calculating the image texture characteristics (LBP- - -Local Binary Pattern) of the Binary image of each handwritten Chinese character, the contrast of the gray level co-occurrence matrix, the energy of the gray level co-occurrence matrix and the entropy of the gray level co-occurrence matrix.

16. And inputting the four items of data (image texture characteristics, contrast of the gray level co-occurrence matrix, energy of the gray level co-occurrence matrix and entropy of the gray level co-occurrence matrix) obtained by calculation into a Support Vector Machine (SVM).

17. And classifying character structures in the binary image of the handwritten Chinese character by using a Support Vector Machine (SVM), and storing a classification result as one feature of the binary image of the handwritten Chinese character.

18. And (3) extracting black character lines in the binary image of the single character by using a Sliding Window method (Sliding Window), and simultaneously identifying and splitting radicals of the Chinese characters in the binary image of the handwritten Chinese characters to obtain each component of the Chinese characters in the binary image of the handwritten Chinese characters.

19. Inputting each component of the Chinese characters in the binary image of the handwritten Chinese characters into an Attention Mechanism Neural Network (AMNN) provided by the invention to obtain the actual Chinese characters corresponding to each component of the Chinese characters in the image.

20. Inputting each component of the Chinese characters in the binary image of the handwritten Chinese characters into an Attention Mechanism Neural Network (AMNN) provided by the invention to obtain the actual Chinese characters corresponding to each component of the Chinese characters in the image.

21. The obtained Chinese character structure and each part of the actual Chinese character are used as the characteristics of the handwritten Chinese character, retrieval is carried out in a Chinese character characteristic dictionary, and the Chinese character result obtained by the retrieval is the recognition result of the handwritten Chinese character.

In the identification process, a blank document (word document, txt format document) is correspondingly established for each handwritten Chinese character image. And storing the Chinese characters recognized from the same handwritten Chinese character image into a blank document to finish the recognition of the handwritten Chinese characters of the students.

Fourthly, the present invention provides a computer device, please refer to fig. 12, and a structure diagram of a computer device provided in the embodiment of the present application includes a memory 75 and a processor 71, wherein the memory 75 stores a computer program, and the processor 71 implements any of the steps of the method for identifying handwritten chinese characters based on the AMNN and the dictionary of chinese character structure as disclosed above when executing the computer program.

Specifically, the memory 75 includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operating system and the computer-readable instructions in the non-volatile storage medium to run. Processor 71, which in some embodiments may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data processing chip, provides computing and control capabilities for the computing device.

The computer device further comprises: and an input interface 72 connected to the processor 71, for acquiring computer programs, parameters and instructions imported from the outside, and storing the computer programs, parameters and instructions into the memory 75 under the control of the processor 71. The input interface 72 may be coupled to an input device for receiving parameters or instructions manually entered by a user. The input device may be a touch layer covered on a display screen, or a button, a track ball or a touch pad arranged on a terminal shell, or a keyboard, a touch pad or a mouse, etc.

A display unit 74, connected to the processor 71, for displaying data processed by the processor 71 and for displaying a visualized user interface. The display unit 74 may be an LED display, a liquid crystal display, a touch-controlled liquid crystal display, an OLED (organic light-emitting diode) touch device, and the like.

And a network port 73 connected to the processor 71 for communication connection with external terminal devices. The communication technology adopted by the communication connection can be a wired communication technology or a wireless communication technology, such as a mobile high definition link (MHL) technology, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a wireless fidelity (WiFi), a bluetooth communication technology, a low power consumption bluetooth communication technology, an ieee802.11 s-based communication technology, and the like.

While FIG. 12 shows only a computer device having components 71-75, those skilled in the art will appreciate that the configuration shown in FIG. 12 does not constitute a limitation of computer devices, and may include fewer or more components than shown, or some components may be combined, or a different arrangement of components.

The invention provides a computer readable storage medium, which may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk. The storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of any of the above-disclosed methods of handwritten Chinese character recognition based on AMNN and Chinese character structure dictionaries.

The contents in the foregoing method embodiments are all applicable to the corresponding storage medium embodiments, so that the functions specifically implemented by the present storage medium embodiment are the same as those in the foregoing method embodiments, and the beneficial effects achieved by the present storage medium embodiment are also the same as those in the foregoing method embodiments.

As will be understood by those skilled in the art, the steps can be reversed in order or processed in parallel as required in actual operation. It should be recognized that the embodiments of the present application can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.

Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.

Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the application may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it is readable by a programmable computer, which when read by the storage medium or device can be used to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The present application also includes the computer itself when programmed according to the methods and techniques described herein.

A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the present application, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.

It should be noted that the detailed explanation of the above embodiments is only for the purpose of explaining the present invention so as to better explain the present invention, but the descriptions should not be construed as limiting the present invention for any reason, and particularly, the features described in the different embodiments may be arbitrarily combined with each other to constitute other embodiments, and the features should be understood as being applicable to any one embodiment and not limited to only the described embodiments except for the explicit contrary description.

Claims

1. A handwritten Chinese character recognition method based on AMNN and Chinese character structure dictionary is characterized by comprising the following steps:

preprocessing the image data of the handwritten Chinese character image;

classifying the preprocessed handwritten Chinese character structures;

extracting the classified handwritten Chinese character structure;

recognizing the handwritten Chinese character structure;

and recognizing the handwritten Chinese characters.

2. The method of claim 1, wherein the step of preprocessing the image data of the handwritten Chinese character image further comprises:

carrying out gray processing on the handwritten Chinese character image;

carrying out binarization processing on the gray level image;

and obtaining a binary image of the single handwritten Chinese character.

3. The method according to claim 2, wherein said step of classifying said preprocessed handwritten Chinese character structure further comprises the steps of:

4. The method according to claim 3, wherein said step of extracting said classified handwritten Chinese character structure further comprises the steps of:

and identifying and splitting the radicals of the Chinese characters in the binary image of the handwritten Chinese characters to obtain all components of the Chinese characters in the binary image of the handwritten Chinese characters.

5. The method of claim 4, wherein the step of identifying the handwritten Chinese character structure further comprises:

6. The method according to claim 1 or 2, wherein the step of recognizing handwritten Chinese characters further comprises:

7. A handwritten Chinese character recognition system based on AMNN and a Chinese character structure dictionary is characterized by comprising:

8. The system of claim 7, wherein the pre-processing module comprises:

9. The system of claim 8, wherein the classification module comprises:

10. The system of claim 9, wherein the vector machine classification module comprises:

11. The system of claim 10, wherein the extraction module comprises:

12. The system of claim 11, wherein the structure identification module comprises:

13. The system of claim 7 or 8, wherein the Chinese character recognition module retrieves and obtains the Chinese character result from the Chinese character feature dictionary according to the classification result and the recognition result of the handwritten Chinese character structure.

14. A computer device comprising a memory and a processor, said memory storing a computer program, wherein said processor, when executing said computer program, performs the steps of the method for handwritten chinese character recognition based on AMNN and a dictionary of chinese character structures according to any of claims 1 to 6.

15. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for handwritten chinese character recognition based on AMNN and chinese character structure dictionary according to any of claims 1 to 6.