JP2022533065A - Character recognition methods and devices, electronic devices and storage media - Google Patents

Abstract

The present invention relates to a character recognition method and apparatus, an electronic device, and a storage medium, wherein the character recognition method comprises obtaining a target image to be recognized, determining a position vector and a first position vector of the target image. obtaining character features of the target image based on image features, wherein the position vector is determined based on position features of characters in a preset information sequence; and recognizing characters in the target image to obtain character recognition results for the target image. Embodiments of the present invention can improve the accuracy of character recognition. [Selection drawing] Fig. 1

Description

[Cross-reference to related applications]
This application is submitted based on a Chinese patent application with an application number of 20100301340.3, which was filed with the Chinese Patent Office on April 16, 2020, claiming the priority of the Chinese patent application. The entire contents of the Chinese patent application are incorporated herein by reference.
[Technical field]
The present invention relates to the field of electronic technology, and more particularly to character recognition methods and devices, electronic devices and storage media.

With the development of electronic technology, more and more tasks can be performed with electronic devices or with the support of electronic devices, which brings convenience to people. For example, the efficiency of manual processing can be improved by automatically recognizing characters using a computer.

Currently, character recognition can recognize regular characters, such as parsing documents. Character recognition can also recognize irregular characters, and can recognize irregular characters in natural scenes such as traffic signs and shop signs. However, it is difficult to accurately recognize irregular characters due to factors such as changes in viewing angle and changes in illumination.

The present invention proposes a technical solution for character recognition.

According to one aspect of the present invention, a character recognition method is provided, in which the character recognition method obtains a target image to be recognized and uses a determined position vector and a first image feature of the target image. Based on the acquisition of the character feature of the target image, the position vector is determined based on the position feature of the character in the preset information sequence, and the target is based on the character feature. It includes recognizing characters in an image and acquiring a character recognition result of the target image.

In one possible embodiment, acquiring the character features of the target image based on the determined position vector and the first image feature of the target image encodes the first image feature of the target image. Then, the coding result of the first image feature is acquired, the second image feature of the target image is determined according to the coding result of the first image feature, and the determined position vector, the first. It includes acquiring character features of the target image based on one image feature and the second image feature. Here, since the second image feature has a stronger position feature, the corresponding acquired character feature of the target image also has a stronger position feature, so that the character recognition result acquired based on the character feature is It is more accurate and further reduces the semantic effect on character recognition results.

In one possible embodiment, encoding the first image feature of the target image and obtaining the coding result of the first image feature can result in a plurality of one-dimensional feature vectors of the first image feature. On the other hand, at least one level of the first coding process is sequentially executed, and the coding result of the first image feature is acquired. Positional features included in the first image feature can be emphasized by sequentially executing one-level or multiple-level first coding processing on a plurality of first-dimensional feature vectors of the first image feature. , The obtained encoded result of the first image feature can have more obvious positional features between the characters.

In one possible embodiment, at least one level of first coding processing is sequentially executed on a plurality of first-dimensional feature vectors of the first image feature, and the coding result of the first image feature is obtained. To obtain the first coding process of the first level in the first coding process of at least one level using N (N is a positive integer) first coding nodes. It is to sequentially encode the input information of the coding node and acquire the output result of N first coding nodes, and when 1 <i ≦ N, i (i is positive). The input information of the first coded node (which is an integer) includes the output result of the i-1st first coded node, and according to the output result of the N first coded nodes, the first coded node is described. 1 Acquiring the coding result of the image feature includes. In this way, since the input information of the first first coded node can be transferred to the last first coded node, the input information of the first coded node can be stored for a long period of time and acquired. The output result can be made more accurate.

In one possible embodiment, the input information of the first coding node further includes the output result of the first dimension feature vector of the first image feature or the first level first coding process. In this way, in the first-level first coding process, the output result of the first-dimensional feature vector of the first image feature or the first-level first coding process of the previous level is finally output via the first coding node. Since it can be transferred to one coded node, the output result of the first level coded processing can be made more accurate.

In one possible embodiment, acquiring the character features of the target image based on the determined position vector, the first image feature and the second image feature is the position vector and the second image. It includes determining the attention weight according to the feature and acquiring the character feature of the target image by performing feature weighting on the first image feature using the attention weight. In this way, the attention weight can be used to emphasize the feature to be noted in the first image feature, so that the character feature acquired by performing the feature weighting on the first image feature using the attention weight is , The more important feature portion of the first image feature can be reflected more accurately.

In one possible embodiment, the character recognition method obtains a preset information sequence that includes at least one first preset information and a second code of at least one level with respect to the at least one first preset information. Further including the acquisition of the position vector by sequentially performing the conversion process. In the process of performing the second coding process on at least one first preset information using the neural network, at least one first preset information is sequentially encoded, so that at least one position vector is generated. Related to the order of the first preset information, the position vector can represent the position features between the characters.

In one possible embodiment, acquiring the position vector by sequentially performing at least one level of second coding processing on the at least one first preset information is a process of sequentially performing the at least one level second coding process. Regarding the 1-level second coding process in the conversion process, M (M is a positive integer) second coding nodes are used to sequentially encode the input information of the second coding node. This is to acquire the output results of M second coded nodes, and when 1 <j ≦ M, the input of the j (j is a positive integer) second coded node is input. The information includes including the output result of the j-1st second coded node and acquiring the position vector according to the output result of the M second coded nodes. In this way, since the input information of the first second coding node can be transferred to the last second coding node, the input information of the second coding node can be stored for a long period of time and acquired. The position vector can be made more accurate.

In one possible embodiment, the input information of the second coding node further includes the first preset information or the output result of the second coding process at the previous level. In this way, in the first-level second coding process, the output result of the first preset information or the previous level second coding process is transferred to the last first coding node via the second coding node. Therefore, the output result of the first level coding process can be made more accurate.

In one possible embodiment, recognizing a character in the target image based on the character feature and acquiring the character recognition result of the target image is to extract the semantic feature of the target image. Acquiring the character recognition result of the target image based on the semantic feature of the target image and the character feature. In this way, in the process of acquiring the character recognition result of the target image, the accuracy of the character recognition result can be improved by combining the semantic feature and the character feature.

In one possible embodiment, extracting the semantic features of the target image sequentially determines the semantic features of the target image in at least one time step based on the acquired second preset information. Acquiring the character recognition result of the target image, including, based on the semantic features and the character features of the target image, is at least based on the semantic features and the character features of the target image in at least one time step. It includes acquiring the character recognition result of the target image in one time step. Here, when there are a plurality of characters in the target image, the character recognition results can be sequentially acquired according to the character positions (character features) and semantics (semantic features), so that the accuracy of the character recognition results can be improved. ..

In one possible embodiment, sequentially determining the semantic features of the target image in at least one time step based on the acquired second preset information is at least one with respect to the second preset information. The third coding process of the level is performed to acquire the semantic feature of the first time step of the at least one time step, and the k-1 (k is an integer greater than 1) th time. The character recognition result of the target image in the step is subjected to at least one level of third coding processing, and the semantic feature of the target image in the kth time step is acquired. In this way, the input information of the third coded node in the previous order can be transferred to the third coded node in the later order, whereby the input information of the third coded node is stored for a long period of time. It is possible to make the acquired semantic features more accurate.

According to one aspect of the present invention, a character recognition device is provided, and the character recognition device is a character recognition device.
An acquisition unit configured to acquire the target image to be recognized,
A determination unit configured to acquire character features of the target image based on the determined position vector and the first image feature of the target image, wherein the position vector is the position of the character in the preset information sequence. The decision part, which is decided based on the characteristics,
A recognition unit configured to recognize characters in the target image based on the character features and acquire a character recognition result of the target image is provided.

In one possible embodiment, the determination unit further encodes the first image feature of the target image to obtain the encoded result of the first image feature and according to the encoded result of the first image feature. , The second image feature of the target image is determined, and the character feature of the target image is acquired based on the determined position vector, the first image feature, and the second image feature.

In one possible embodiment, the determination unit further sequentially performs at least one level of first coding processing on the plurality of first-dimensional feature vectors of the first image feature to sequentially perform the first image. It is configured to acquire the coded result of the feature.

In one possible embodiment, the determination unit further describes N (where N is a positive integer) first for the 1st level 1st coding process in the at least 1st level 1st coding process. When the input information of the first coding node is sequentially coded using the coding node and the output results of N first coding nodes are acquired, where 1 <i ≦ N. , I (i is a positive integer) th input information includes the output result of the i-1st first coded node, and of the N first coded nodes. It is configured to acquire the coding result of the first image feature according to the output result.

In one possible embodiment, the input information of the first coding node further includes the output result of the first dimension feature vector of the first image feature or the first level first coding process.

In one possible embodiment, the determination unit further determines the attention weight according to the position vector and the second image feature, and uses the attention weight to perform feature weighting on the first image feature. This is configured to acquire the character features of the target image.

In one possible embodiment, the character recognition device further acquires a preset information sequence that includes at least one first preset information and at least one level of second coding for the at least one first preset information. A coding unit configured to sequentially perform processing and acquire the position vector is provided.

In one possible embodiment, the coding unit further describes an M (where M is a positive integer) th-order for the one-level second coding process in the at least one-level second coding process. The input information of the second coding node is sequentially coded using the two coding nodes, and the output results of the M second coding nodes are acquired, where 1 <j ≦ M. In the case, the input information of the j (j is a positive integer) second coding node includes the output result of the j-1st second coding node, and the M second coding nodes are included. It is configured to acquire the position vector according to the output result of.

In one possible embodiment, the input information of the second coding node further includes the first preset information or the output result of the second coding process at the previous level.

In one possible embodiment, the recognition unit further extracts the semantic features of the target image and obtains the character recognition result of the target image based on the semantic features of the target image and the character features. It is composed.

In one possible embodiment, the recognizer further sequentially determines the semantic features of the target image in at least one time step based on the acquired second preset information, said in at least one time step. Based on the semantic features of the target image and the character features, it is configured to acquire the character recognition result of the target image in at least one time step.

In one possible embodiment, the recognition unit further performs at least one level of third coding processing on the second preset information, and the semantic of the first time step of the at least one time step. The feature is acquired, and at least one level of third coding processing is performed on the character recognition result of the target image in the k-1 (k is an integer greater than 1) th time step, and the kth time is obtained. It is configured to acquire the semantic features of the target image in the step.

According to one aspect of the present invention, an electronic device is provided, wherein the electronic device is a device.
With the processor
With memory configured to store processor executable instructions,
Here, the processor is configured to execute the character recognition method by calling and executing an instruction stored in the memory.

According to one aspect of the present invention, a computer-readable storage medium in which computer program instructions are stored is provided, and when the computer program instructions are executed by a processor, the above-mentioned character recognition method is realized.

According to one aspect of the present invention, a computer program including a computer-readable code is provided, and when the computer-readable code is executed in an electronic device, the processor of the electronic device is made to execute the above-mentioned character recognition method. ..

In the embodiment of the present invention, the target image to be recognized is acquired, then the character features of the target image are acquired based on the determined position vector and the first image feature of the target image, and then the character features. The characters in the target image can be recognized based on the above, and the character recognition result of the target image can be obtained. Here, since the position vector is determined based on the character position feature in the preset information sequence and can represent the position feature between the characters, in the character recognition process, the character spacing to the character recognition result is obtained. The influence of the position feature can be enhanced and the accuracy of character recognition can be improved. For example, in the case of irregular characters or non-semantic characters, a better recognition effect can be obtained.

It should be understood that the general description described above and the detailed description described below are merely examples and description and are not intended to limit the present disclosure.

In the following, a detailed description of an exemplary embodiment with reference to the drawings will reveal other features and embodiments of the present disclosure.

The drawings herein are incorporated herein to constitute a portion thereof, and these drawings are intended to provide examples in accordance with the present invention and to illustrate the technical solutions of the present invention together with the specification. used.
It is a flowchart of the character recognition method which concerns on embodiment of this invention. It is a block diagram which shows an example which determines the 2nd image feature of the target image which concerns on Example of this invention. It is a block diagram which shows an example which acquires the character recognition result using the neural network which concerns on embodiment of this invention. It is a block diagram of an example of the character recognition apparatus which concerns on embodiment of this invention. It is a block diagram of an example of the character recognition apparatus which concerns on embodiment of this invention. It is a block diagram of an example of the electronic device which concerns on embodiment of this invention.

Hereinafter, various exemplary examples, features and embodiments of the present invention will be described in detail with reference to the drawings. The same reference number in the drawing indicates an element of the same or similar function. Various aspects of the examples are shown in the drawings, but it is not necessary to draw the drawings to scale unless otherwise specified.

The specialized description of "exemplary" in the specification means "used as an example, example or description". Here, any example described as "exemplary" need not be construed as more appropriate or superior to the other examples.

The term "and / or" in the present specification merely refers to the relational relationship of the related object, and indicates that three kinds of relations can exist. For example, A and / or B means that A exists independently. , A and B exist at the same time, and represent three cases such as the case where B exists independently. Further, the term "at least one" as used herein refers to any combination of one or more of a plurality, including, for example, at least one of A, B, C. This indicates that it contains any one or more elements selected from the set composed of A, B and C.

In addition, a number of details are provided in the following embodiments to better illustrate the present disclosure. It will be appreciated by those skilled in the art that the present disclosure can be carried out without some details being provided. In some embodiments, methods, means, elements and circuits well known to those of skill in the art will not be described in detail in order to emphasize the gist of the present disclosure.

According to the character recognition solution according to the embodiment of the present invention, after acquiring the target image to be recognized, the character feature of the target image is determined based on the determined position vector and the first image feature of the target image. Then, the characters in the target image can be recognized based on the character features, and the character recognition result of the target image can be acquired. Here, since the position vector is determined based on the position feature of the character in the preset information sequence and can represent the position feature of the character, the position feature between the characters is emphasized in the character recognition process. This makes it possible to make the acquired character recognition result more accurate.

In related techniques, character sequences are usually recognized using semantic features between characters, but the semantic relevance between characters in some character sequences is low. For example, since the semantic relevance between characters of a character sequence such as a license plate number or a room number is low, the effect of recognizing a character sequence by a semantic feature is poor. The character recognition solution according to the embodiment of the present invention enhances the influence of character position features on character recognition, reduces the dependence of the character recognition process on semantic features, and recognizes characters with low semantic relevance. It is possible to obtain a better recognition effect on the recognition of irregular characters.

The technical solution according to the embodiment of the present invention can be applied to the extension of application scenarios such as character recognition in an image and conversion from an image to text, and the embodiment of the present invention particularly relates to these. Not limited. For example, it is possible to bring convenience to the user by performing character recognition on irregular characters of a traffic sign and determining a traffic instruction represented by the traffic sign.

FIG. 1 shows a flowchart of a character recognition method according to an embodiment of the present invention. The character recognition method can be performed by a terminal device, server or other type of electronic device, where the terminal device is a user device (UE: User Computing), mobile device, user terminal, terminal, cellular. It may be a telephone, a cordless telephone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or the like. In some possible embodiments, the character recognition method can be implemented by the processor calling and implementing a computer-readable instruction stored in memory. Hereinafter, the character recognition method according to the embodiment of the present invention will be described by taking as an example that the electronic device is the execution subject.

In step S11, the target image to be recognized is acquired.

In the embodiment of the present invention, the electronic device can have an image collecting function and can collect a target image to be recognized. Alternatively, the electronic device may acquire the target image to be recognized from another device. For example, the electronic device acquires the target image to be recognized from a device such as a photographing device or a monitoring device. Can be done. The target image to be recognized may be an image waiting for character recognition. The target image may include characters, and the characters may be one character or a character string. The characters in the target image may be regular characters (for example, text written in a standard font). Regular letters can have features such as neat placement, uniform size, no deformation, and no shading. In some embodiments, the characters in the target image may be irregular characters (eg, artistic text on the cover of a store sign or advertisement). Irregular characters can have features such as uncluttered placement, uneven size, deformation, and shielding.

In step S12, the character feature of the target image is acquired based on the determined position vector and the first image feature of the target image, where the position vector is based on the position feature of the character in the preset information sequence. It was decided.

In an embodiment of the present invention, a position vector for representing a character position feature can be determined based on the character position feature of the preset information sequence, for example, after obtaining a preset information sequence of a specific length. , The position feature of the character in the preset information sequence can be extracted. The position vector is associated with the location position of the character. For example, when the position of the character to be recognized in the character sequence is the third character position, the position vector is the character to be recognized in the character sequence. The position of, that is, the position of the third character can be represented. The characters in the preset information sequence may be the same to reduce the correlation between the position vector and the character semantics. In some embodiments, each character in the preset information sequence can also be set to non-semantic information, further reducing the correlation between the position vector and the character semantics. Since the position vector has a low correlation with the character semantics, the position vector may be the same or different for different target images.

The first image feature of the target image may be obtained by performing image extraction on the target image, for example, using a neural network to perform at least one convolution operation on the target image. It can be executed to acquire the first image feature of the target image. The character features of the target image can be determined according to the determined position vector and the first image feature of the target image. For example, the determined position vector and the first image feature of the target image are fused to form the target image. Character features can be obtained. Here, since the character features are acquired based on the position vector and the first image feature, the influence of the character semantics on the character features is low.

In step S13, the characters in the target image are recognized based on the character features, and the character recognition result of the target image is acquired.

In the embodiment of the present invention, the character feature can be processed by using the neural network, for example, the activation operation is performed on the character feature, or the character feature is output to the fully connected layer of the neural network. The character recognition result of the target image can be obtained by executing the full join operation. The character recognition result can be the recognition result of the characters in the target image. If the target image contains one character, the character recognition result can be one character. If the target image contains a character sequence, the character recognition result can be one character sequence, and the order of each character in the character recognition result is the same as the order of the corresponding characters in the target image.

Since the character recognition result obtained by the character feature is not easily affected by the semantics of the characters, a better recognition effect can be obtained even in the case of a character sequence having a low semantic relevance between the characters. For example, a license. Character recognition can be performed on character sequences that are not semantically relevant within the plate number.

In step S12 above, the character features of the target image can be acquired based on the determined position vector and the first image feature of the target image, thereby reducing the semantic influence on the character features. Hereinafter, an embodiment for acquiring the character features of the target image will be provided.

In one possible embodiment, the first image feature of the target image is encoded to obtain the encoded result of the first image feature, and then the second image feature of the target image is according to the encoded result of the first image feature. Is determined, and the character features of the target image can be acquired based on the determined position vectors, the first image feature, and the second image feature.

In this embodiment, a neural network can be used to encode the first image feature of the target image, eg, the first image feature can be encoded row by row or column by column, thereby. The positional feature included in the first image feature can be emphasized. After that, the second image feature of the target image can be acquired according to the coding result obtained by encoding the first image feature. For example, the first image feature and the encoding result are fused to obtain the first image feature. The second image feature of the target image is acquired, and the second image feature has a stronger positional feature than the first image feature. After that, the character features of the target image can be acquired based on the determined position vector, the first image feature, and the second image feature. For example, the determined position vector, the first image feature, and the second image feature can be acquired. The character features of the target image are acquired by fusing, and since the second image feature has a stronger position feature, the character feature of the acquired target image also has a stronger position feature, thereby based on the character feature. The acquired character recognition result is more accurate, further reducing the semantic influence on the character recognition result.

In the above embodiment, the positional feature included in the first image feature can be emphasized by encoding the first image feature of the target image. In the following, the process of acquiring the coding result of the first image feature will be described with reference to one example.

In one example, at least one level of first coding processing can be sequentially executed on a plurality of first-dimensional feature vectors of the first image feature, and the coding result of the first image feature can be obtained.

In this example, the first image feature may include a plurality of one-dimensional feature vectors. The first image feature may include features in multiple dimensions, for example, the first image feature may include features in multiple dimensions such as length, width, and depth. The number of feature dimensions in different dimensions may be different. The one-dimensional feature vector may be a feature of the first image feature on one dimension, for example, the first-dimensional feature vector may be a feature on the length dimension or the width dimension. The first coding process may be coding to the first image feature, correspondingly the neural network may include at least one first coding layer and corresponds to the first coding layer. The coding process may be the first coding process. Here, it is possible to perform one-level or multiple-level first coding processing on a plurality of first-dimensional feature vectors using a neural network, and obtain the processing results of the plurality of first-dimensional feature vectors. One one-dimensional feature vector can correspond to one processing result, and then a plurality of processing results of the plurality of first-dimensional features can be merged to form a coding result of the first image feature. can. Positional features included in the first image feature can be emphasized by sequentially executing one-level or multiple-level first coding processing on a plurality of first-dimensional feature vectors of the first image feature. , The obtained encoded result of the first image feature can have a more pronounced positional feature between the characters.

In this example, for the 1st level 1st coding process in at least the 1st level 1st coding process, the 1st coding process is performed using N (N is a positive integer) first coding nodes. The input information of the nodes is sequentially coded, and the output results of N first coded nodes can be obtained. When 1 <i ≦ N, i (i is a positive integer). The input information of the first coding node of the) th th code includes the output result of the first coding node of the i-1st. The coding result of the first image feature is acquired according to the output result of N first coding nodes.

In this example, the coding result of the first image feature can be obtained by executing at least one level of the first coding process on the first image feature using the neural network. The neural network can include at least one level of the first coding layer, the first coding layer can perform the first coding process, and each level of the first coding process has a plurality of coding processes. Realized by nodes. When the first coding process is a plurality of levels of processing, the operations related to the first coding process of each level may be the same. For the 1st level 1st coding process in at least the 1st level 1st coding process, the input information of the 1st level coding process can be encoded by using N 1st coding nodes. One first coded node can correspond to one input information, and the input information of different first coded nodes may be different. Correspondingly, one output result can be acquired by one first coding node. The input information of the first coding node of the first level first coding process may be the first-dimensional feature vector of the first image feature. The output result of the first coding node in the first level first coding process can be used as the input information of the first coding node having the same order in the second level first coding process. , The same applies to the first level of the first coding process. The output result of the first coding node in the first level first coding processing may be the processing result of the above-mentioned one-dimensional feature vector. The one-level first coding process can include N first coding nodes, where 1 <i ≦ N, that is, the first coding node is in the current level of the first coding process. In the case of a first coding node other than the first first coding node, the input information of the first coding node further adds the output result of the previous first coding node in the first coding process of the relevant level. It can include, thereby transferring the input information of the first first coded node to the last first coded node, so that the input information of the first coded node can be stored for a long period of time. The acquired output result can be made more accurate.

を取得することができる。 FIG. 2 shows a block diagram showing an example of determining a second image feature of a target image according to an embodiment of the present invention. In this example, a neural network (eg, a long short-term memory (LSTM) network, etc.) can be used to encode the first image feature F of the target image. The neural network may include two first coding layers, each first coding layer may contain a plurality of first coding nodes (corresponding to the coding nodes of FIG. 2). Here, the first image feature F of the target image is input to the first coding layer of the neural network, and the plurality of first coding nodes of the first image feature F are used by using the plurality of first coding nodes of the first coding layer. The output result of each first coded node can be obtained by encoding each of the dimensional feature vectors (width-dimensional feature vectors). Here, the input information of the first first coding node is the first one-dimensional feature vector, and the input information of the second first coding node is the output of the first first coding node. It is the result and the second one-dimensional feature vector, and by analogy with this, the output result of the last first coding node can be obtained. The output results of the plurality of first coding points are input to the second first coding layer, and the processing of the second first coding layer is the same as the processing of the first first coding layer. It will not be explained repeatedly here. Finally, the coding result F2 of the ^first image feature can be obtained. After that, feature fusion (may be special addition or merger) is executed on the first image feature F and the coding result F ² of the first image feature, and the second image feature of the target image is executed.

Can be obtained.

を取得することができる。

式（１）

式（２）

式（３） Here, taking the example of encoding the first image feature F of the target image using the two-layer LSTM as an example, the second image feature F is used by the first image feature F according to the following equation.

Can be obtained.

Equation (1)

Equation (2)

Equation (3)

は、位置（ｉ，ｊ）における第１画像特徴Ｆの特徴ベクトル（第１次元特徴ベクトル）であり得、

は、位置（ｉ，ｊ）における１番目の第１符号化層の出力結果Ｆ¹の特徴ベクトルを表すことができ、

は、位置（ｉ，ｊ－１）における出力結果Ｆ¹の特徴ベクトルを表すことができ、

は、位置（ｉ，ｊ）における符号化結果Ｆ^２の特徴ベクトルを表すことができ、

は、位置（ｉ，ｊ－１）における符号化結果Ｆ^２の特徴ベクトルを表すことができ、

は、取得された第２画像特徴を表すことができ、

は、ベクトルの加算演算を表すことができる。ここで、ｉ及びｊは、自然数である。 here,

Can be a feature vector (first dimension feature vector) of the first image feature F at position (i, j).

Can represent the feature vector of the output result F ¹ of the first first coding layer at position (i, j).

Can represent the feature vector of the output result F ¹ at position (i, j-1).

Can represent the feature vector of the coding result F2 at position ⁽ i, j).

Can represent the feature vector of the coding result F2 at position (i, j ^- 1).

Can represent the acquired second image feature,

Can represent a vector addition operation. Here, i and j are natural numbers.

In the above embodiment, the character features of the target image can be acquired based on the determined position vector, the first image feature, and the second image feature. In the following, the process of acquiring the character features of the target image will be described with reference to one example.

In one example, the attention weight is determined according to the determined position vector and the second image feature, and then the feature weight is applied to the first image feature using the attention weight to acquire the character feature of the target image. Can be done.

In one example, since both the position vector and the second image feature include prominent position features, the attention weight can be determined according to the position vector and the second image feature, eg, between the position vector and the second image feature. The correlation can be determined, and the attention weight can be determined according to the correlation. The correlation between the position vector and the second image feature can be obtained by the inner product of the position vector and the second image feature. Using the determined attention weight, feature weighting can be performed on the first image feature. For example, by multiplying the attention weight and the first image feature to obtain the total, the character feature of the target image is obtained. Can be obtained. Since the attention weight can be used to emphasize the feature to be noted in the first image feature, the character feature acquired by performing the feature weighting on the first image feature using the attention weight is the first. More important feature parts of image features can be reflected more accurately.

式（４） In this example, the attention weight can be determined by the following equation (4).

Equation (4)

は、アテンション重みを表し、

は、活性化関数を表し、

は、位置ベクトル

の転置を表し、

は、特徴位置（ｉ，ｊ）における第２画像特徴

の特徴ベクトルを表す。上記の式（４）を使用して、位置ベクトル及び第２画像に従って、特徴アテンション重みを決定することができる。 here,

Represents the attention weight

Represents the activation function

Is the position vector

Represents the transpose of

Is the second image feature at the feature position (i, j)

Represents the feature vector of. The feature attention weight can be determined according to the position vector and the second image using the above equation (4).

式（５） In this example, the character feature can be determined by the following equation (5).

Equation (5)

は、文字特徴を表し、

は、アテンション重みを表し、

は、特徴位置（ｉ，ｊ）における第１画像特徴Fの特徴ベクトルを表す。上記の式（５）を使用して、アテンション重み及び第１画像特徴に基づいて文字特徴を取得することができる。 here,

Represents a character feature,

Represents the attention weight

Represents the feature vector of the first image feature F at the feature position (i, j). Using the above equation (5), character features can be obtained based on the attention weight and the first image feature.

In the above embodiment, the attention weight can be determined according to the determined position vector and the second image feature. The position vector can represent the position feature of the character, that is, the relative position between the characters. The process of determining the position vector via one embodiment will be described below.

In one possible embodiment, after acquiring a preset information sequence containing at least one first preset information, at least one level of second coding processing is sequentially performed on the at least one first preset information to obtain a position. You can get the vector.

In this embodiment, the preset information sequence may include one or more first preset information. The first preset information may be information set according to an actual case, or may have no specific meaning. For example, the first preset information may be a count instruction. Using a neural network, a position vector can be acquired by sequentially performing a one-level or a plurality of levels of second coding processing on at least one first preset information. Since at least one first preset information is the same and has no specific meaning, the semantic relevance between at least one first preset information is low, so that one level for at least one first preset information. Alternatively, the position vector obtained by performing the second coding process of a plurality of levels has low semantic relevance. Further, in the process of performing the second coding process on at least one first preset information using the neural network, at least one first preset information is sequentially encoded, so that the generated position vector is at least It is related to the order of one first preset information (ie, it can be understood that it is related to the position between at least one first preset information), whereby the position vector represents the position feature between characters. be able to.

In an example of the present embodiment, for the 1st level 2nd coding process in the 1st level 2nd coding process, the input information of the 2nd coding node is sequentially input by using M 2nd coding nodes. By encoding, the output results of M second coding nodes can be obtained. When 1 <j≤M (M and j are positive integers), the input information of the jth second coding node includes the output result of the j-1st second coding node. The position vector is acquired according to the output result of the M second coding nodes.

In this example, using a neural network, a position vector can be acquired by sequentially performing a one-level or a plurality of levels of second coding processing on at least one first preset information. When the second coding process is a plurality of levels of processing, the operations related to the second coding process of each level may be the same. For the 1st level 2nd coding process in at least 1st level 2nd coding process, the input information of the 1st level 2nd coding process can be encoded by using M 2nd coding nodes. One second coding node can correspond to one input information, and the input information of different second coding nodes may be different. Correspondingly, one output result can be acquired by one second coding node. The input information of one second coding node of the first level second coding process may be one first preset information. The output result of the second coding node in the first level second coding process can be used as the input information of the second coding node having the same order in the second level second coding process. , The same applies to the second coding process at the final level. The output result of the last second coding node in the last level second coding process can be used as a position vector or the last second coding in the last level second coding process. The position vector can be acquired by performing further processing such as convolution processing and pooling processing on the output result of the node. The one-level second coding process can include M second coding nodes, where 1 <j ≦ M, that is, the second coding node is in the current level second coding process. In the case of a second coding node other than the first second coding node, the input information of the second coding node further adds the output result of the previous second coding node in the second coding process of the relevant level. It can include, thereby transferring the input information of the first second coded node to the last second coded node, so that the input information of the second coded node can be stored for a long period of time. The acquired position vector can be made more accurate.

を決定することができる。

式（６）

式（７） Here, taking as an example that the first preset information is a constant "<next>" and the second coding process is a two-layer LSTM, the position vector is based on the following equations (6) and (7).

Can be determined.

Equation (6)

Equation (7)

は、第１レベルの第２符号化処理におけるｔ番目の第２符号化ノードの出力結果を表すことができ、

は、第１レベルの第２符号化処理におけるｔ－１番目の第２符号化ノードの出力結果を表すことができ、

は、第２レベルの第２符号化処理におけるｔ番目の第２符号化ノードの出力結果（すなわち、位置ベクトル）を表すことができ、

は、第２レベルの第２符号化処理におけるｔ－１番目の第２符号化ノードの出力結果を表すことができる。ここで、ｔは自然数である。 here,

Can represent the output result of the t-th second coding node in the first level second coding process.

Can represent the output result of the t-1st second coding node in the first level second coding process.

Can represent the output result (ie, position vector) of the t-th second coding node in the second level second coding process.

Can represent the output result of the t-1st second coding node in the second level second coding process. Here, t is a natural number.

It should be noted that the process of acquiring the position vector with at least one first preset information can be implemented by the neural network shown in FIG. 2, where the position vector is the output of the plurality of second coded nodes. It may not be formed by the result, but may be the output result of the last second coding node in the second level second coding process.

In step S13 described above, the characters in the target image can be recognized based on the character features, and the character recognition result of the target image can be acquired. In order to improve the accuracy of the character recognition result, the semantic characteristics of the characters in the target image can also be considered in the process of recognizing the characters in the target image. Hereinafter, the process of acquiring the character recognition result of the target image will be described through one embodiment.

In one possible embodiment, after extracting the semantic features of the target image, the character recognition result of the target image can be obtained based on the semantic features and the character features of the target image.

In this embodiment, the semantic features of the target image can be extracted, for example, the semantic features of the target image can be extracted using the semantic extraction model in some scenarios, followed by the semantic features and character features of the target image. Can be fused to obtain the fusion result. For example, the semantic feature and the character feature can be connected, or the semantic feature and the character feature can be connected and then the feature weighting can be further performed to obtain the fusion result. Here, the feature weighting may be preset or calculated according to semantic features and character features. After that, the character recognition result of the target image can be acquired according to the fusion result. For example, the fusion result is subjected to at least one convolution operation, a fully combined operation, and the like to acquire the character recognition result of the target image. Can be done. In this way, in the process of acquiring the character recognition result of the target image, the accuracy of the character recognition result can be improved by combining the semantic feature and the character feature.

で表すことができ、文字特徴は、

で表すことができ、以下の式（８）及び式（９）により、セマンティック特徴と文字特徴との融合結果を取得することができる。

式（８）

式（９） For example, the semantic feature is

Can be represented by, character features,

It can be expressed by, and the fusion result of the semantic feature and the character feature can be obtained by the following equations (8) and (9).

Equation (8)

Equation (9)

は、融合結果を表すことができ、ｗ_ｔは、セマンティック特徴

及び文字特徴

に対して特徴加重を行うための重みを表すことができ、

は、第１マッピング行列を表すことができ、ここで、第１マッピング行列を用いてセマンティック特徴

と文字特徴

を二次元ベクトル空間にマッピングすることができ、

は、第１バイアス項を表すことができる。 here,

Can represent the fusion result, _wt is a semantic feature

And character features

Can represent the weight for performing feature weighting on

Can represent a first mapping matrix, where semantic features using the first mapping matrix

And character features

Can be mapped to a two-dimensional vector space,

Can represent the first bias term.

を取得した後、以下の式（１０）により、目標画像の文字認識結果を取得することができる。

式（１０） Fusion result

After obtaining the above, the character recognition result of the target image can be obtained by the following equation (10).

Equation (10)

は、文字認識結果を表すことができ、Ｗは、第２マッピング行列を表すことができ、ここで、第２マッピング行列を用いて融合結果

に対して線形変換を行うことができ、ｂは、第２バイアス項であり得る。 here,

Can represent the character recognition result, W can represent the second mapping matrix, where the fusion result using the second mapping matrix.

Can be linearly transformed against, where b can be the second bias term.

In one example of the present embodiment, the semantic features of the target image in at least one time step are sequentially determined based on the acquired second preset information, and then the semantic features and characters of the target image in at least one time step are determined. Based on the features, the character recognition result of the target image in at least one time step can be acquired.

In this example, the acquired second preset information can be selected according to the actual case, and the second preset information may not have a specific meaning. For example, the second preset information may be a start command. The step width of the time step can be set according to the actual need. One semantic feature can be determined for each time step, and the semantic features acquired by different time steps may be different. Here, the second preset information is encoded using a neural network to sequentially acquire the semantic features of at least one time step, and then the semantic features of the target image at at least one time step and at least one time step. According to the character characteristics of the target image in, the character recognition result of the target image in at least one time step can be acquired. The semantic feature of one time step and the character feature of the same time step can correspond to the character recognition result of one time step, that is, when multiple characters are present in the target image, the position of the character (character feature). ) And semantics (semantic features), the character recognition results can be acquired in sequence, so that the accuracy of the character recognition results can be improved.

In this example, the second preset information is subjected to at least one level of third coding processing to acquire the semantic characteristics of the first time step of at least one time step, and then the k-1th. The character recognition result of the target image in the time step of 1 can be subjected to at least one level of third coding processing, and the semantic features of the target image in the kth time step can be acquired. Here, k is an integer greater than 1.

In this example, the second preset information can be used as input information for at least one level of the third coding process of the neural network. The third coding process at each level may include a plurality of third coding nodes, and each third coding node can correspond to the input information of one time step. The input information of different third coded nodes may be different. Correspondingly, one output result can be acquired by one third coding node. The input information of the first third coding node in the first level third coding process may be the second preset information. The output result of the third coding node in the first level third coding process can be used as the input information of the third coding node having the same order in the second level third coding process. , The same applies to the second coding process at the final level. In this way, it is possible to perform at least one level of third coding processing on the second preset information and acquire the output result of the first third coding node of the last level third coding processing. The output result may be a semantic feature of the first time step of at least one time step. Further, the character recognition result of the first time step can be acquired according to the semantic feature of the first time step and the character feature of the same time step. The input information of the second third coding node of the third process of the first level may be the character recognition result of the first time step. After that, at least one level of third coding processing can be performed on the character recognition result of the first time step, and the semantic features of the second time step can be acquired. Further, according to the semantic feature of the second time step and the character feature of the same time step, the character recognition result of the second time step can be acquired. The same applies to the final level of the third coding process. In the third level of third coding processing, the output result of the last third coding node may be a semantic feature of the last time step. That is, at least one level of third coding processing can be performed on the character recognition result of the target image in the k-1st time step, and the semantic features of the target image in the kth time step can be acquired. A third if k is an integer greater than 1, i.e., if the third coded node is a third coded node other than the first third coded node in the current level of third coded processing. The input information of the coding node may further include the output result of the previous third coding node in the third coding process of the level, thereby the input information of the third coding node in the previous order later. It can be transferred to the third coded node in order, whereby the input information of the third coded node can be stored for a long period of time, and the acquired semantic features can be made more accurate.

It should be noted that the process of determining the semantic features with the second preset information can be realized by the neural network shown in FIG. 2, where the semantic features of the kth time step are the third level of the second level. It may be the output result of the kth third coding node of the coding process.

In the embodiment of the present invention, the character recognition result of the target image can be acquired by using the neural network. In the following, the process of acquiring the character recognition result of the target image using the neural network will be described with reference to one example.

を取得することができ、位置情報強調モジュールのネットワークアーキテクチャは、図２に示すとおりであり得る。その後、第２画像特徴

をデコーダのアテンションモジュールに入力し、アテンションモジュールを用いて第２画像特徴

と位置ベクトル

とに対して行列乗算及び活性化操作を行い、アテンション重みを取得し、次に、アテンション重みを用いて第１画像特徴Ｆに対して特徴加重を行い（すなわち、アテンション重みと第１画像特徴とに対して行列乗算を行う）、目標画像の文字特徴を受信することができる。デコーダは更に、動的融合モジュールを備え、動的融合モジュールを用いて、文字特徴とセマンティック特徴とを融合することができ、その後、融合結果を全結合層に入力して、文字認識結果を取得することができる。 FIG. 3 is a block diagram showing an example of acquiring a character recognition result using a neural network according to an embodiment of the present invention. In this example, the neural network may include encoders and decoders. First, the first image feature F of the target image can be acquired by inputting the target image into the encoder of the neural network and extracting the image feature of the target image using the encoder. Here, image feature extraction can be performed on the target image using the network architecture of a 31-layer residual neural network (ResNet: Residual Neural Network). The encoder may include a position information enhancement module, which uses the position information enhancement module to emphasize the position information of the first image feature to enhance the position information of the second image feature of the target image.

The network architecture of the location information enhancement module can be as shown in FIG. After that, the second image feature

Is input to the attention module of the decoder, and the second image feature is used using the attention module.

And position vector

Matrix multiplication and activation operations are performed on the image to obtain the attention weight, and then the feature weight is applied to the first image feature F using the attention weight (that is, the attention weight and the first image feature). Matrix multiplication is performed on the target image), and the character features of the target image can be received. The decoder is further equipped with a dynamic fusion module, which can be used to fuse character features and semantic features, after which the fusion result is input to the fully coupled layer to obtain the character recognition result. can do.

を取得することができる。ここで、位置符号化モジュールは、２つの符号化層を含み得る。デコーダは更に、セマンティックモジュールを備え、１番目のタイムステップの入力情報として１つの特殊トークン「＜ｓｔａｒｔ＞」（第２プリセット情報）をセマンティックモジュールに入力して、セマンティックモジュールによって出力された１番目のタイムステップのセマンティック特徴を取得することができる。その後、１番目のタイムステップの文字認識結果

を、セマンティックモジュールの２番目のタイムステップの入力情報として使用でき、セマンティックモジュールによって出力された２番目のタイムステップのセマンティック特徴を取得し、以下同様に、ｔ番目のタイムステップにおけるセマンティックモジュールによって出力されたセマンティック特徴

を取得することができる。セマンティックモジュールは、２層の符号化層を含み得る。位置符号化モジュール及びセマンティックモジュールのネットワークアーキテクチャは、図２のネットワークアーキテクチャと同様であってもよく、ここでは繰り返して説明しない。 Here, the decoder further includes a position coding module, and a plurality of constants "<next>" (first preset information) can be sequentially input to the position coding module, that is, one for each time step. The constant "<next>" can be input. The position coding module may include two coding layers (corresponding to the first coding process), encode the input "<next>", and position vector of the t-th time step.

Can be obtained. Here, the position coding module may include two coding layers. The decoder also includes a semantic module, which inputs one special token "<start>" (second preset information) into the semantic module as input information for the first time step, and is output by the semantic module. You can get the semantic features of the time step. After that, the character recognition result of the first time step

Can be used as input information for the second time step of the semantic module, and the semantic characteristics of the second time step output by the semantic module are acquired, and similarly, output by the semantic module at the t-th time step. Semantic features

Can be obtained. The semantic module may include two coding layers. The network architecture of the position coding module and the semantic module may be similar to the network architecture of FIG. 2, and will not be described repeatedly here.

Illustratively, the encoder includes a position information enhancement module, and the decoder includes a position coding module, an attention module, a semantic module, and a dynamic fusion module. Here, the position information enhancement module includes a two-layer LSTM (see FIG. 2), and the two-layer LSTM is used to encode the first image feature of the target image in order from left to right. The coding result of one image feature is acquired, the coding result of the first image feature is added to the first image feature, and the coding result of the second feature of the target image is obtained, whereby the second image feature is obtained. Is determined, and the second image feature is output as the output of the position information enhancement module. The position coding module is essentially a character length counter because the position coding module includes two layers of LSTMs and each input of the position coding module is one particular input. A two-level second coding process is performed on at least one preset information using the position coding module, a position vector is acquired, the position vector and the second image feature are input to the attention module, and the attention module performs the second coding process. 2 Matrix multiplication and activation operations are performed on the image features and the position vector to acquire the attention weight. Next, the weighted average value of the first image feature is taken according to the attention weight, and the character feature of the target image is acquired. The second preset information is input to the semantic module, the semantic feature of the target image is acquired, the weight prediction is performed for the semantic feature and the character feature using the dynamic fusion module, and the weighted average of the semantic feature and the character feature is performed. The value can be output as a fusion result output, the fusion result can be input to the prediction module, character classification can be performed by the prediction module, and the character recognition result can be obtained.

According to the character encoding solution according to the embodiment of the present invention, the position information between characters is emphasized, the dependency of the character recognition result on the semantics is reduced, and thereby the accuracy of character recognition is improved. The character coding solution according to the present invention can be applied to more complex character recognition scenarios (eg, irregular character recognition, non-semantic character recognition, etc.) and scenarios such as image recognition (eg, image review and image analysis). Etc.) can also be applied.

The examples of the above-mentioned methods described in the present invention can be combined with each other to form a combined example without violating the principle and logic, and the number of papers is limited. Please understand that I will not explain it repeatedly.

In addition, the invention also provides character recognition devices, electronic devices, computer readable storage media, programs, all of which can be used to implement any of the character recognition methods according to the invention and the corresponding technical solutions. For the measures and explanations, the corresponding description of the embodiment of the method can be referred to and will not be repeated here.

As will be obvious to those skilled in the art, in the method of the above specific embodiments, the order of each step described does not mean a strict execution order that limits the implementation process, but a specific of each step. The order of execution must be determined by its function and possible internal logic.

FIG. 4 shows a block diagram of the character recognition device according to the embodiment of the present disclosure, and as shown in FIG. 4, the character recognition device is
An acquisition unit 41 configured to acquire a target image to be recognized, and
A determination unit 42 configured to acquire the character features of the target image based on the determined position vector and the first image feature of the target image, wherein the position vector is a character in the preset information sequence. The determination unit 42, which is determined based on the positional feature,
A recognition unit 43 configured to recognize characters in the target image based on the character features and acquire a character recognition result of the target image is provided.

In one possible embodiment, the determination unit 42 further encodes the first image feature of the target image, obtains the coding result of the first image feature, and encodes the first image feature. According to the above, the second image feature of the target image is determined, and the character feature of the target image is acquired based on the determined position vector, the first image feature, and the second image feature.

In one possible embodiment, the determination unit 42 further sequentially executes at least one level of first coding processing on the plurality of first-dimensional feature vectors of the first image feature to perform the first coding process. It is configured to acquire the encoded result of the image feature.

In one possible embodiment, the determination unit 42 further describes N (where N is a positive integer) number of 1st level 1st coding processes in the at least 1st level 1st coding process. The input information of the first coding node is sequentially coded using the one coding node, and the output results of N first coding nodes are acquired, where 1 <i ≦ N. In some cases, the input information of the i-th first coding node (i is a positive integer) includes the output result of the i-1th first coding node, and the N first codings are included. It is configured to acquire the coding result of the first image feature according to the output result of the node.

In one possible embodiment, the input information of the first coding node further includes the output result of the first dimension feature vector of the first image feature or the first level first coding process.

In one possible embodiment, the determination unit 42 further determines the attention weight according to the position vector and the second image feature, and uses the attention weight to perform feature weighting on the first image feature. By doing so, it is configured to acquire the character features of the target image.

In one possible embodiment, the character recognition device further
A preset information sequence including at least one first preset information is acquired, and at least one level of second coding processing is sequentially performed on the at least one first preset information to acquire the position vector. It is provided with an encoding unit to be used.

In one possible embodiment, the coding unit further describes an M (where M is a positive integer) th-order for the one-level second coding process in the at least one-level second coding process. The input information of the second coding node is sequentially coded using the two coding nodes, and the output results of the M second coding nodes are acquired, where 1 <j ≦ M. In some cases, the input information of the j (j is a positive integer) second coding node includes the output result of the j-1st second coding node, and the M second coding nodes are described. It is configured to acquire the position vector according to the output result of the node.

In one possible embodiment, the input information of the second coding node further includes the first preset information or the output result of the second coding process at the previous level.

In one possible embodiment, the recognition unit 43 further extracts the semantic features of the target image and obtains the character recognition result of the target image based on the semantic features of the target image and the character features. It is composed of.

In one possible embodiment, the recognition unit 43 further sequentially determines the semantic features of the target image in at least one time step, based on the acquired second preset information, in at least one time step. Based on the semantic features of the target image and the character features, it is configured to acquire the character recognition result of the target image in at least one time step.

In one possible embodiment, the recognition unit 43 further performs at least one level of third coding processing on the second preset information, and the first time step of the at least one time step. The semantic feature is acquired, and at least one level of third coding processing is performed on the character recognition result of the target image in the k-1 (k is an integer greater than 1) th time step, and the kth th coding process is performed. It is configured to acquire the semantic features of the target image in the time step.

In an embodiment of the invention and other embodiments, the "part" may be a partial circuit, a partial processor, a partial program or software, etc., and of course, a unit, modular or non-modular. You can understand that it may be.

In some embodiments, the functions or modules of the apparatus provided in the embodiments of the present invention can be configured to perform the methods described in the embodiments of the methods described above, and the specifics thereof. The implementation can be referred to in the description of the embodiments of the above method and will not be repeated here for brevity.

FIG. 5 is a block diagram of the character recognition device 800 shown by an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a mobile information terminal, or the like.

Referring to FIG. 5, the apparatus 800 includes processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816. It can include one or more of these components.

The processing component 802 generally controls general operations of the device 800, such as operations relating to displays, telephone calls, data communications, camera operations and recording operations. The processing component 802 may include one or more processors 820 for executing instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and the other components. For example, the processing component 802 may include a multimedia module for facilitating the interaction between the multimedia component 808 and the processing component 802.

Memory 804 is configured to store various types of data to support operations on device 800. Examples of these data include instructions, contact data, phonebook data, messages, photos, videos, etc. of any application or method running on device 800. The first memory 804 includes static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), and programmable read-only memory (EPROM). It can be achieved by any type of volatile or non-volatile storage device or a combination thereof, such as PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

The power component 806 provides power to various components of the device 800. The power component 806 can include a power management system, one or more power sources, and other components related to the generation, management, and distribution of power for the device 800.

The multimedia component 808 includes a screen provided as an output interface between the device 800 and the user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen comprises a touch panel, the screen can be implemented as a touch screen for receiving input signals from the user. The touch panel comprises one or more touch sensors for detecting touches, swipes and gestures on the touch panel. The touch sensor can not only detect the boundaries of the touch or swipe operation, but also the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 comprises one front camera and / or rear camera. When the device 800 is in an operating mode such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or may have a focal length and an optical zoom function.

The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 comprises one microphone (MIC), and the microphone is configured to receive an external audio signal when the device 800 is in an operating mode such as a call mode, a recording mode, and a voice recognition mode. .. The received audio signal may be further stored in memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further comprises a speaker for outputting an audio signal.

The I / O interface 812 provides an interface between the processing component 802 and the peripheral interface module, which peripheral interface module may be a keyboard, click wheel, buttons, or the like. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

The sensor component 814 comprises one or more sensors for providing the device 800 with an assessment of the state of each aspect. For example, the sensor component 814 can detect the on / off state of the device 800 and the relative positioning of components such as the display and keypad of the device 800, and the sensor component 814 is a component of the device 800 or the device 800. Changes in the position of the device 800, the presence or absence of contact of the device 800 with the user, the orientation or acceleration / deceleration of the device 800, and changes in the temperature of the device 800 can also be detected. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without physical contact. The sensor component 814 can also further include an optical sensor such as a CMOS or CCD image sensor for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. The device 800 can access a wireless network based on communication standards such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further comprises a Near Field Communication (NFC) module to facilitate short range communication. For example, NFC modules can be embodied on the basis of radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the device 800 is configured to perform the above method, one or more application-specific integrated circuits (ASICs), a digital signal processor (DSP), a digital signal processor (DSPD). ), Programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microprocessors, microprocessors or other electronic components.

In an exemplary embodiment, a computer-readable storage medium, such as a memory 804 containing computer program instructions, is further provided, the computer program instructions described above being executed by processor 820 of device 800 to perform the above method. can do.

An embodiment of the present invention further proposes an electronic device, wherein the electronic device comprises a processor and a memory configured to store processor-executable instructions, wherein the processor is provided by the memory 804. It is configured to execute the above method by calling and executing the stored instruction.

Electronic devices are provided as terminals, servers or other forms of device.

FIG. 6 is a block diagram of an electronic device 1900 according to one exemplary embodiment. For example, the electronic device 1900 can be provided as a server. Referring to FIG. 6, electronic device 1900 is represented as a processing component 1922 that includes one or more processors and a memory resource that is configured to store instructions (such as applications) that can be executed by the processing component 1922. It includes a memory 1932. An application stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Further, the processing component 1922 is configured to execute the above method by executing an instruction.

The electronic device 1900 also includes an input / output with a power component 1926 configured to perform power management of the electronic device 1900 and a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network. (I / O) interface 1958 and can be provided. The electronic device 1900 can be operated on the basis of an operating system stored in memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

In an exemplary embodiment, the above method is completed by further providing a computer-readable storage medium, such as memory 1932 containing computer program instructions, and executing the computer program instructions described above by processing component 1922 of electronic device 1900. can do.

The present invention may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium, which includes computer-readable program instructions for the processor to realize various aspects of the embodiments of the present invention.

The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by the instruction executing device. The computer-readable storage medium can be, for example, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination described above, but is not limited thereto. More specific examples (non-exhaustive lists) of computer-readable storage media are portable computer disksets, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory). , Static Random Access Memory (SRAM), Portable Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disk (DVD), Memory Stick, Floppy Disk, Punch Card or Groove Convex Structure in which Instructions Are Stored, And include mechanical coding equipment such as any suitable combination described above. Computer-readable storage media used herein are radio waves, other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other propagating media (such as optical pulses through fiber optic cables), or wires. It should not be interpreted as a temporary signal, such as an electronic signal transmitted via.

The computer-readable program instructions described herein are downloaded from a computer-readable storage medium to each computing / processing device or via a network such as the Internet, local area networks, wide area networks and / or wireless networks to external computers. Or it can be downloaded to an external storage device. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and / or edge servers and the like. A network adapter card or network interface in each computing / processing device receives a computer-readable program instruction from the network and issues the computer-readable program instruction for storage in a computer-readable storage medium of another computing / processing device. Forward.

The computer programming instructions for performing the operations of the present invention are assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state setting data, or programming of one or more. It may be source code or target code programmed in any combination of languages, said programming language being an object-oriented programming language such as Smalltalk, C ++, and common programming languages such as "C" or similar programming languages. Includes procedural programming languages. Computer-readable program instructions may be executed entirely on the user's computer, some may be executed on the user's computer, or some may be executed as a separate software package, some of which may be executed by the user. It may be run on one computer and the rest may be run on a remote computer or entirely on a remote computer or server. For remote computers, the remote computer can connect to the user's computer or to an external computer via any type of network, including local area networks (LANs) or wide area networks (WANs). Yes (for example, you can use an internet service provider to access an external computer over the internet). In some embodiments, computer-readable instruction state information is used to customize an electronic circuit, such as a programmable logic circuit, field programmable gate array (FPGA) or programmable logic array (PLA). , Computer-readable program instructions can be executed, thereby realizing each aspect of the present invention.

Here, each aspect of the present invention has been described with reference to the flowcharts and / or blocks of the method, apparatus (system) and computer program product according to the embodiment of the present invention, but each block of the flowchart and / or block diagram has been described. It should be understood that each block combination of, and the flowchart and / or block diagram can be achieved by computer-readable program instructions.

These computer-readable program instructions can be provided to the processor of a general purpose computer, dedicated computer or other programmable data processor, whereby these instructions can be provided to the processor of the computer or other programmable data processor. Creates means to achieve the function / operation specified by one or more blocks in the flowchart and / or block diagram when executed by. These computer-readable program instructions may be stored on a computer-readable storage medium, allowing a computer, programmable data processing device, and / or other device to operate in a particular manner in response to these instructions. Thus, a computer-readable medium in which instructions are stored can include a product that includes instructions for each aspect of a function / operation specified by one or more blocks in a flowchart and / or block diagram.

Also, loading computer-readable program instructions into a computer, other programmable data processor, or other device causes the computer, programmable number of processor, or other device to perform a series of operational steps. Thereby generating a process implemented by a computer, thereby one or more in a flow chart and / or a block diagram by instructions executed by a computer, other programmable number processing device, or other equipment. The function / operation specified by the block can be realized.

The flowcharts and block diagrams in the accompanying drawings show the feasible implementation architectures, functions and operations of the systems, methods and computer program products according to a plurality of embodiments of the present invention. In this regard, each block in a flowchart or block diagram can represent a module, program segment or part of an instruction, said module, program segment or part of the instruction to achieve a specified logical function. Includes one or more executable instructions of. In some alternative implementations, the functions displayed in blocks can also be performed in a different order than they are displayed in the drawing. For example, two consecutive blocks can actually be executed in parallel and, depending on the related functions, in reverse order. Each block in the block diagram and / or flowchart, and a combination of blocks in the block diagram and / or flowchart, can be achieved by a dedicated system based on the hardware performing the specified function or operation, or with dedicated hardware. Note that this can be achieved by a combination of computer instructions.

Although each embodiment of the present invention has been described above, the above description is exemplary, not exhaustive, and is not limited to each disclosed embodiment. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of each of the embodiments described. The choice of terminology used herein best describes the principles of each embodiment, the actual application or technical improvement of the art in the market, or each embodiment disclosed herein by one of ordinary skill in the art. Intended to be understandable.

In the embodiment of the present invention, the target image to be recognized is acquired, then the character features of the target image are acquired based on the determined position vector and the first image feature of the target image, and then the character features. The characters in the target image can be recognized based on the above, and the character recognition result of the target image can be obtained. Here, the position vector is determined based on the position feature of the character in the preset information sequence, and can represent the position feature between the characters. Therefore, in the character recognition process, the character spacing to the character recognition result is obtained. The influence of positional features can be increased, the dependence of the character recognition process on semantic features can be reduced, and the accuracy of character recognition can be improved.

Claims (16)

It is a character recognition method
Acquiring the target image to be recognized and
Acquiring the character features of the target image based on the determined position vector and the first image feature of the target image, the position vector is determined based on the position features of the characters in the preset information sequence. That it is a vector
The character recognition method including recognizing a character in the target image based on the character feature and acquiring a character recognition result of the target image. Acquiring the character features of the target image based on the determined position vector and the first image feature of the target image is
To obtain the coding result of the first image feature by encoding the first image feature of the target image, and
Determining the second image feature of the target image according to the coding result of the first image feature,
Acquiring the character features of the target image based on the determined position vector, the first image feature and the second image feature, and the like.
The character recognition method according to claim 1. Encoding the first image feature of the target image to obtain the coding result of the first image feature is possible.
The first coding process of at least one level is sequentially executed on a plurality of first-dimensional feature vectors of the first image feature, and the coding result of the first image feature is acquired.
The character recognition method according to claim 2. Obtaining the coding result of the first image feature by sequentially executing at least one level of the first coding process on the plurality of first-dimensional feature vectors of the first image feature is possible.
For the 1st level 1st coding process in the at least 1st level 1st coding process, the input of the 1st coding node is performed by using N (N is a positive integer) first coding nodes. Information is sequentially coded and the output results of N first coded nodes are acquired. When 1 <i ≦ N, the i (i is a positive integer) th. The input information of the first coded node of the above includes the output result of the i-1st first coded node.
Acquiring the coding result of the first image feature according to the output result of the N first coding nodes, and the like.
The character recognition method according to claim 3. The input information of the first coding node further includes the output result of the first dimension feature vector of the first image feature or the first coding process of the previous level.
The character recognition method according to claim 4. Acquiring the character features of the target image based on the determined position vector, the first image feature, and the second image feature can be done.
Determining the attention weight according to the position vector and the second image feature,
Acquiring the character features of the target image by performing feature weighting on the first image feature using the attention weights, including.
The character recognition method according to any one of claims 2 to 5. The character recognition method is
Acquiring a preset information sequence that includes at least one first preset information,
Further comprising sequentially performing at least one level of second coding processing on the at least one first preset information and acquiring the position vector.
The character recognition method according to any one of claims 1 to 6. Acquiring the position vector by sequentially performing at least one level of second coding processing on the at least one first preset information is possible.
For the 1st level 2nd coding process in the at least 1st level 2nd coding process, the input of the 2nd coding node is performed by using M (M is a positive integer) number of 2nd coding nodes. Information is sequentially coded and the output results of M second coded nodes are acquired. When 1 <j ≦ M, the j (j is a positive integer) th. The input information of the second coded node of the above includes the output result of the j-1st second coded node.
Acquiring the position vector according to the output result of the M second coding nodes, and the like.
The character recognition method according to claim 7. The input information of the second coding node further includes the output result of the first preset information or the second coding process of the previous level.
The character recognition method according to claim 8. Recognizing the characters in the target image based on the character features and acquiring the character recognition result of the target image is possible.
Extracting the semantic features of the target image and
Acquiring the character recognition result of the target image based on the semantic feature of the target image and the character feature, and the like.
The character recognition method according to any one of claims 1 to 9. Extracting the semantic features of the target image
Including sequentially determining the semantic features of the target image in at least one time step based on the acquired second preset information.
Acquiring the character recognition result of the target image based on the semantic feature of the target image and the character feature is
Acquiring the character recognition result of the target image in at least one time step based on the semantic feature and the character feature of the target image in at least one time step.
The character recognition method according to claim 10. It is possible to sequentially determine the semantic features of the target image in at least one time step based on the acquired second preset information.
Performing at least one level of third coding processing on the second preset information to acquire the semantic characteristics of the first time step of the at least one time step.
The character recognition result of the target image in the k-1 (k is an integer greater than 1) th time step is subjected to at least one level of third coding processing, and the target image in the kth time step. To get the semantic features of, including,
The character recognition method according to claim 11. It is a character recognition device
An acquisition unit configured to acquire the target image to be recognized,
A determination unit configured to acquire character features of the target image based on the determined position vector and the first image feature of the target image, wherein the position vector is the position of the character in the preset information sequence. The decision part, which is decided based on the characteristics,
The character recognition device including a recognition unit configured to recognize characters in the target image based on the character features and acquire a character recognition result of the target image. It ’s an electronic device,
With the processor
With memory configured to store processor executable instructions,
The electronic device, wherein the processor is configured to execute the method according to any one of claims 1 to 12 by calling and executing an instruction stored in the memory. A computer-readable storage medium that stores computer program instructions.
The computer-readable storage medium that implements the method of any one of claims 1-12 when the computer program instructions are executed by a processor. A computer program that contains computer-readable code
The computer program that causes the processor of the electronic device to perform the method according to any one of claims 1 to 12 when the computer-readable code is executed in the electronic device.

Images