JP2021197165A - Information processing apparatus, information processing method and computer readable storage medium - Google Patents

Abstract

To provide an information processing apparatus, an information processing method and a computer readable storage medium.SOLUTION: An information processing apparatus comprises: a probability vector acquisition unit that acquires an M-dimensional probability vector of each of N segments which are obtained by dividing a target to be classified; a candidate class selection unit that selects a class corresponding to upper maximum K elements of elements other than a H-th element in the M-dimensional probability vector of each segment as a candidate class of the segments; a route vector generation unit that generates a route vector on the basis of the candidate class of each segment, and that calculates a score of the route vector on the basis of probability corresponding to an element included in each route vector and an association degree between adjacent elements; and a classification result acquisition unit that acquires a route vector having the highest score as a classification result of the target to be classified, and the association degree between the adjacent elements is calculated on the basis of semantic information between the adjacent elements, and variable weight associated with a distance between segments corresponding to the adjacent elements.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to the field of information processing, specifically to information processing devices, information processing methods and computer readable storage media.

The classification technique is widely applied to, for example, image recognition, character recognition, voice recognition, and the like.

The following is a brief overview of the present disclosure in order to provide a basic understanding of aspects of the present disclosure. It should be noted that this brief overview is not an exhaustive overview of the present disclosure, does not intentionally specify the points or important parts of the present disclosure, and does not intentionally limit the scope of the present disclosure. As a preamble to a more detailed explanation, which will be described later, the purpose is to explain a mere concept in a simple form.

It is an object of the present disclosure to provide an improved information processing device, an information processing method, and a computer-readable storage medium.

In one aspect of the present disclosure, it is a random vector acquisition unit that acquires the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified, and M is the number of classes, and each is The first element to the M element in the M-dimensional probability vector represent the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1, the probability vector acquisition unit and the above. For each of the N segments, in the candidate class selection unit that selects the class corresponding to the highest K maximum elements among the elements other than the H element in the M-dimensional random variable of the segment as the candidate class of the segment. Therefore, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H class corresponding to the H element is a class that does not include semantic information. Candidate class selection. A path vector is generated based on the unit and each candidate class of the N segments, and for each of the generated path vectors, the probability corresponding to each element included in the path vector and the relationship between adjacent elements. It includes a route vector generation unit that calculates the score of the route vector based on the degree, and a classification result acquisition unit that acquires the route vector having the highest score among the route vectors as the classification result of the target to be classified. Provided is an information processing apparatus in which the degree of association between adjacent elements is calculated based on the semantic information between the adjacent elements in the path vector and the variable weight regarding the distance between the segments corresponding to the adjacent elements. ..

In another aspect of the present disclosure, a random vector acquisition step of acquiring the M-dimensional random variables of each of the N segments obtained by dividing the object to be classified, where M is the number of classes. The first element to the M element in each M-dimensional probability vector represent the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1, the probability vector acquisition step and For each of the N segments, a candidate class selection step of selecting the class corresponding to the highest K maximum elements among the elements other than the H element in the M-dimensional random variable of the segment as the candidate class of the segment. Therefore, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element is a class that does not include semantic information. A path vector is generated based on the selection step and each candidate class of the N segments, and for each of the generated path vectors, the probability corresponding to each element included in the path vector and between adjacent elements. A route vector generation step for calculating the score of the route vector based on the degree of relevance, and a classification result acquisition step for acquiring the route vector having the highest score among the route vectors as the classification result of the target to be classified. Provided is an information processing method in which the degree of association between adjacent elements including and included is calculated based on semantic information between adjacent elements in the path vector and variable weights regarding the distance between segments corresponding to the adjacent elements. do.

In another aspect of the present disclosure, a computer-readable computer in which the computer program code and computer program product for realizing the method of the present disclosure described above, and the computer program code for realizing the method of the present disclosure described above are recorded. Further provides a storage medium.

The following describes other embodiments of the present disclosure, particularly preferred embodiments of the present disclosure, but the present disclosure is not limited to these examples.

In order to understand the principles and advantages of the present disclosure, each embodiment of the present disclosure will be described with reference to the drawings. In all drawings, the same or similar components are indicated by the same or similar reference numerals. The drawings described herein are for illustration purposes only, and are not all possible examples and are not intended to limit the scope of the present disclosure.
It is a block diagram which shows the example of the functional configuration of the information processing apparatus which concerns on embodiment of this disclosure. 2A and 2B are diagrams showing an example of an object to be classified and an example of a classification result, respectively. 2A and 2B are diagrams showing an example of an object to be classified and an example of a classification result, respectively. It is a figure which shows an example of three scenarios which are considered when updating a path vector in the case of character recognition. It is a flowchart which shows an example of the flow of the process which the path vector generation part of the information processing apparatus which concerns on embodiment of this disclosure executes in the process of the i-th (i ≧ 2) round. It is a flowchart which shows an example of the flow of the information processing method 400 which concerns on embodiment of this disclosure. FIG. 3 is a block diagram illustrating an exemplary configuration of a personal computer applicable to the embodiments of the present disclosure.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. For convenience of explanation, the specification does not show all the features of the actual embodiment. In the actual implementation, a specific embodiment may be changed in order to realize the specific goal of the developer, for example, the embodiment may be changed according to the restriction conditions related to the system and business. good. In addition, the development work is very complicated and time-consuming, but for those skilled in the art of this publication, this development work is merely an example work.

It should be noted that, in order to clarify the present disclosure, the drawings show only the configuration requirements or processing steps of the apparatus closely related to the present disclosure and omit details unrelated to the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

First, an example of the functional configuration of the information processing apparatus according to the embodiment of the present disclosure will be described with reference to FIG. FIG. 1 is a block diagram showing an example of a functional configuration of the information processing apparatus according to the embodiment of the present disclosure. As shown in FIG. 1, the information processing apparatus 100 according to the embodiment of the present disclosure may include a probability vector acquisition unit 102, a candidate class selection unit 104, a path vector generation unit 106, and a classification result acquisition unit 108.

The probability vector acquisition unit 102 acquires the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified. M is the number of classes, the first element to the M element in each M-dimensional probability vector represents the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1. Is. For example, the probability vector acquisition unit 102 may acquire the M-dimensional probability vector of each segment by a neural network such as a pre-trained convolutional recurrent neural network (CRNN) (for example, Shi B, Bai X, Yao C.I. an End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition [J] IEEE transactions on pattern analysis & machine intelligence, 2017, 39 (11):. referring to the 2298-2304). The method of acquiring the M-dimensional probability vector of each segment is not limited to this, and those skilled in the art may adopt another method as necessary to acquire the M-dimensional probability vector of each segment. The explanation is omitted in.

As an example, the object to be classified may be an image containing text. In this case, the first class to the M class may represent different characters (for example, Chinese characters, English alphabets, symbols, etc.), respectively, and the Hth corresponding to the H element in the first class to the M class may be represented. The class may represent a whitespace character (spacing character). The objects to be classified are not limited to this. For example, the object to be classified may be voice. In this case, the classes other than the H class among the first class to the M class may represent different characters (for example, Chinese characters, English words, etc.), and the H class may be blank (for pause in voice). Corresponding) may be represented.

For example, N may be related to, but not limited to, the size of the object to be classified. For example, when the object to be classified is an image containing text, the larger the size of the image, the larger N. Further, for example, when the object to be classified is voice, the longer the time corresponding to the voice, the larger N becomes.

For each of the N segments, the candidate class selection unit 104 uses a class corresponding to the largest K elements other than the H element in the M-dimensional probability vector of the segment as the candidate class for the segment. You may choose. Here, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element may be a class that does not include semantic information. For example, the candidate class selection unit 104 sorts the elements other than the H element in the M-dimensional probability vector of each segment in descending order, and selects the class corresponding to the upper K elements as the candidate class of the corresponding segment. You may. Here, those skilled in the art may set the value of K as needed.

The route vector generation unit 106 generates a route vector based on each candidate class of N segments, and for each of the generated route vectors, the probability corresponding to each element included in the route vector and the adjacent elements. The score of the path vector may be calculated based on the degree of relevance between them. Here, the degree of relevance between adjacent elements may be calculated based on semantic information between adjacent elements in a path vector and variable weights with respect to distances between segments corresponding to adjacent elements.

The classification result acquisition unit 108 may acquire the route vector having the highest score among the route vectors as the classification result of the target to be classified.

The classification technique is widely applied to, for example, image recognition, character recognition, voice recognition, and the like. In character recognition, the core technology is to perform character recognition by extracting features from images containing text. Usually, the extracted features do not contain semantic information, so long-term and short-term memory (LSTM: Long Short-Term Memory) networks (eg, Hochreiter S, Schmidhuber J. Long short-term memory [J]. 1997, 9 (8): see 1735-1780), n-gram model (eg, Brown PF, Desouza PV, Mercer RL, et al. Class-based n-gram models of natural lang. A technique has been proposed for introducing semantic information into the decoding process, such as Computational Linguistics, 1992, 18 (4): 467-479). In the prior art, the semantic information between two specific adjacent non-blank characters, words or phrases is constant regardless of the actual distance between them. In addition, "adjacent" here means that there is no character between these two specific non-blank characters, words or phrases, or there is no other character other than the blank.

As described above, the information processing apparatus 100 according to the embodiment of the present disclosure is adjacent based on the semantic information between adjacent elements in the path vector and the variable weight regarding the distance between the segments corresponding to the adjacent elements. Calculate the degree of association between elements. If the object to be classified is an image containing text, the distance between the segments corresponding to the adjacent elements corresponds to the actual distance between the adjacent elements, and when using the above relevance calculation method, Since it is equivalent to adding a variable weight regarding the actual distance between adjacent elements to the semantic information between adjacent elements, the text can be recognized in consideration of the actual distance between adjacent elements.

In one embodiment of the present disclosure, semantic information between adjacent elements is represented by values calculated via a pre-trained n-gram model. It should be noted that other methods may be used to acquire or represent semantic information between adjacent elements. For example, a recurrent neural network (RNN) such as LSTM may be used to acquire semantic information between adjacent elements.

In one embodiment of the present disclosure, when the distance between segments corresponding to adjacent elements is less than or equal to a predetermined threshold, the variable weight is set to 1 and the distance between segments corresponding to adjacent elements is greater than or equal to the predetermined threshold. If is also large, the variable weight is set to a value less than 1, and the variable weight decreases with increasing distance. For example, the distance between segments corresponding to adjacent elements may be represented by the difference in the number of corresponding segments. For example, suppose that two adjacent elements are the letters "B" and "C", and the segment numbers corresponding to the letters "B" and "C" are TS _"B" = 2 and TS _"C" = 4, respectively. If so, the distance between the segments corresponding to the letters "B" and "C" may be expressed as _{TS "C"} -TS _{"B" = 2.}

For example, if the distance between segments corresponding to adjacent elements is greater than or equal to a predetermined threshold, the variable weight may be set to the reciprocal of the difference between the distance and the predetermined threshold, but is not limited thereto. For example, the variable weight may be calculated according to the following equation (1).

In the equation (1), D represents the distance between the segments corresponding to the adjacent elements, and D _t represents a predetermined threshold value. Moreover, those skilled in the art may determine the predetermined threshold value D _t If the actual need. For example, the predetermined threshold D _t may be determined based on the number of segments occupied by one blank character.

As mentioned above, in the prior art, the semantic information between two specific adjacent non-blank characters, words or phrases is constant regardless of the actual distance between them. On the other hand, there are often delimiters (for example, blanks) in the text to be recognized, and the semantic relevance of the sentences before and after the delimiter is weak. For example, as shown in FIG. 2A, the contextual semantic information of the phrase “1-1” and the subsequent phrase “(magara construction” is relatively weak. However, the text shown in FIG. 2A is recognized using the prior art. In the case, a specific semantic information is introduced between the phrase "1-1" and the phrase "(magara construction", and the specific semantic information is between the phrase "1-1" and the phrase "(magara construction"). It has nothing to do with whether or not there is a space in, and the space (ie, the number of blank characters).

The information processing apparatus according to the embodiment of the present disclosure determines the degree of relevance between adjacent elements based on the semantic information between adjacent elements in the path vector and the variable weight of the distance between the segments corresponding to the adjacent elements. You may calculate. If the object to be classified is an image containing text, the distance between the segments corresponding to the adjacent elements corresponds to the actual distance between the adjacent elements, and by the delimiter between the adjacent elements (eg, blank). , The actual distance between adjacent elements can be increased. In other words, the information processing apparatus according to the embodiment of the present disclosure may add a variable weight regarding a delimiter (for example, a blank) between adjacent elements to the semantic information between adjacent elements. For example, in the text shown in FIG. 2A, when the distance between the segments corresponding to the adjacent characters "1" and "(" is larger than a predetermined threshold value, the meaning between the adjacent characters "1" and "("). A variable weight calculated according to the above equation (1) may be added to the information.

If the distance between the segments corresponding to the adjacent elements is greater than a predetermined threshold, the variable weight is set to a value less than 1, and the variable weight decreases as the distance increases. This ensures that when recognizing text, if there is a delimiter (eg, blank) between adjacent elements, a degree of relevance that decreases with increasing delimiters (eg, blank) is applied between the adjacent elements. Therefore, the performance of character recognition (for example, the accuracy of character recognition) can be improved.

In one embodiment of the present disclosure, the information processing apparatus 100 may further include a preprocessing unit 110. The pre-processing unit 110 performs pre-processing on the target to be classified. Here, the pretreatment may include at least one of noise reduction, normalization, binarization and tilt correction. Since noise removal, normalization, binarization, and tilt correction are well known in the art, the details thereof will be omitted.

In one embodiment of the present disclosure, the route vector generation unit 106 may generate a route vector by the following N rounds of processing and acquire the score of the route vector.

In the processing of the first round, the route vector generation unit 106 generates K + 1 route vectors based on the candidate class of the first segment of the N segments and the H class corresponding to the H element. Scores for the path vectors may be generated based on the probabilities corresponding to the elements in each path vector.

For example, as shown in FIG. 4, in the processing of the i-th (i ≧ 2) round, the route vector generation unit 106 selects the upper L route vectors having the maximum score as candidate route vectors (step S4102). ), Here, L is a natural number larger than 1, and excludes other candidate route vectors other than the candidate route vector having the highest score among the same two or more candidate route vectors (step S4104), and at least. The score of the remaining candidate route vectors after exclusion is updated based on the M-dimensional probability vector of the i-segment (step S4106), and for each of the remaining candidate route vectors, each of the candidate classes of the i-segment is the remaining. Each of the candidate route vectors is added to generate a new route vector, and the score before updating the remaining candidate route vectors, the probability corresponding to the newly added candidate class, and the newly added candidate class and the remaining candidates. The score of the newly generated route vector is calculated based on the degree of association with the route vector (step S4108). Here, those skilled in the art may set the value of L as needed.

In the process of the i-th (i ≧ 2) round, the candidate vectors selected by the path vector generation unit 106 in step S4102 include the same two or more candidate path vectors, and the same two as described above. If there are more than one candidate route vector with the highest score among the above candidate route vectors (that is, two or more candidate route vectors have the highest score), any one of the candidates in step S4104. You may leave the path vector.

The following describes the above-mentioned iteration process executed by the path vector generation unit 106, taking as an example that the object to be classified is an image containing text. For convenience of explanation, the following has only uppercase English alphabets and blanks in the text, ie only M = 27 classes such as blank, A, B, C, D, E, F, G, ..., Z. Suppose there is.

In the processing of the first round, the path vector generation unit 106 is a candidate class for the first segment of the N segments (in this example, K = 2 and the candidate classes for the first segment are A and B). (Assuming) and the probability of generating K + 1 = 3 path vectors [A], [B] and [blank] based on the H class (that is, the first class blank) and corresponding to the elements in each path vector. Generates a score for the path vector based on. For example, the probabilities corresponding to the elements A, B, and blank in the M-dimensional probability vector of the first segment may be used as the scores of the path vectors [A], [B], and blank, respectively. Further, in an actual application, the probabilities corresponding to the elements of each route vector may be further processed. For example, the logarithm of each probability may be obtained and the obtained result may be used as the score of the corresponding route vector. For example, the M-dimensional probability vector of the first segment acquired by the probability vector acquisition unit 102 is P ₁ = [p _blank1 , p _A1 , p _B1 , p _C1 , p _D1 , p _E1 , p _F1 , p _G1 , ..., p _Z1 ], where p _blank1 , p _A1 , p _B1 , p _C1 , p _D1 , p _E1 , p _F1 , p _G1 , ..., p _Z1 have the first segment of classes blank, A, Represents the probabilities of belonging to B, C, D, E, F, G, ..., Z, respectively. In this case, the scores of the route vectors [A], [B] and [blank] are log (p _blank1 ), log (p _A1 ) and log (p _B1 ), respectively. In addition, those skilled in the art will process the probabilities corresponding to each element using other methods as needed, and will handle the processing results so that the score of the path vector can increase as the probabilities of the corresponding elements increase. It may be used as the score of the path vector to be used.

In the processing of the second round, the path vector generation unit 106 has the highest score from the current path vector (that is, the path vector generated in the sled of the first round) (L = in this example). The route vector of 2) may be selected as the candidate route vector (assuming that the candidate route vectors selected in the processing of the second round are [A] and [blank]) (step S4102). In this example, since the candidate route vectors selected in the processing of the second round are different from each other, the exclusion processing in step S4104 (that is, other than the candidate route vector having the highest score among the same two or more candidate route vectors). Exclude other candidate path vectors). The route vector generation unit 106 updates the scores of the candidate route vectors [A] and [blank] based on at least the M-dimensional probability vector of the second segment (step S4106), and the candidate class of the second segment (of the second segment). (Assuming that the candidate classes are C and D) are added to the candidate path vectors [A] and [blank], respectively, and the path vectors [A, C], [A, D], [blank, C] and Newly generated [blank, D], pre-updated scores of candidate path vectors [A] and [blank], probabilities corresponding to newly added candidate classes C and D, and newly added candidate classes C and D. The score of the newly generated route vector may be calculated based on the degree of association between the candidate route vector [A] and [blank] (step S4108).

For example, the route vector generation unit 106 may calculate the score Score (“AC”) of the newly generated route vector [A, C] according to the following equation (2).

In the equation (2), the score (“A”) represents the score before the update of the candidate path vector [A], and p _C2 is the M-dimensional probability vector of the second segment acquired by the probability vector acquisition unit 102. The probability corresponding to the class C is set, and the LM (“C” / “A”) * weight represents the degree of association between the newly added candidate class C and the candidate route vector [A]. Here, LM (“C” / “A”) represents semantic information between “A” and “C”, and weight represents a variable weight, which is calculated according to the above equation (1). May be good.

Further, the route vector generation unit 106 may calculate the scores of the newly generated route vectors [A, D], [blank, C] and [blank, D] by using the same method. The semantic information between the blank and the letters A, B, C, D, E, F, G, ..., Z may be represented by 0.

In the processing of the third round, the route vector generation unit 106 uses the current route vector, that is, the candidate route vector selected in the processing of the second round (when the exclusion processing is executed, the remaining candidate route vector after exclusion). ) (Ie [A] and [blank]) and the newly generated path vectors in the second round (ie [A, C], [A, D], [blank, C] and [blank, D]). The upper L (L = 2) route vectors having the highest score may be selected as candidate route vectors, and the same processing as the processing of the second round may be executed. The description thereof will be omitted here. If there are two or more candidate route vectors that are the same, it is necessary to execute the exclusion process.

Further, in the subsequent round processing, the path vector generation unit 106 may execute the same processing as the processing of the second round and the third round described above, but the description thereof will be omitted here. If there are two or more candidate route vectors that are the same, it is necessary to execute the exclusion process.

The above-mentioned N-round processing is performed by a beam search method (for example, Hannun A Y, MaaS A L, Jurafsky D, et al. First-pass vocabulary vocabulary reference computer science selection Computer Science. The accuracy of classification can be improved by combining the semantic information between adjacent elements and the degree of association based on variable weights. Further, in the processing of the i-th (i ≧ 2) round, the L route vectors having the highest score are selected as the candidate route vectors, and the subsequent processing is performed only on the candidate route vectors to perform the classification processing. The amount of calculation can be reduced and the calculation time can be saved.

For convenience of explanation, a path vector containing only a blank is represented by [blank]. However, in reality, since the route vector usually does not include blank characters, the so-called route vector containing only blanks is actually a blank route vector, that is, a route vector containing no characters. Similarly, when the first element acquired based on the route vector [blank] is a route vector having a blank, the corresponding route vector does not actually contain a blank character. For example, the path vector [blank, C] is actually [C].

For example, in character recognition, it is necessary to merge adjacent duplicate characters and remove the blank character blank according to the decoding rule. Therefore, when updating the score of the route vector, the following three scenarios are taken into consideration. May be good. For example, in the example shown in FIG. 3, for the "new world", these three scenarios may be summarized as follows. (1) Stationion 1 (scenario 1): A character that overlaps with the last character, that is, a "world" appears. (2) Station 2 (scenario 2): blank, "New World" + blank appears. (3) Stationion 3 (Scenario 3): The path vector "new world" exists, and the "world" appears. Since all of these three scenarios can be decoded into the "New World", it is necessary to add up the scores of these route vectors and update them to the score of the route vector "New World". In the example shown in FIG. 3, in Stationion 3, only one blank character between "new world" and "world" is shown, but in reality, between "new world" and "world". There may be one or more whitespace characters in.

As an example, in the processing of the i-th (i ≧ 2) round, the route vector generation unit 106 may perform the following processing when updating the scores of the remaining candidate route vectors.

If the remaining candidate path vectors contain one element and the element is not of the H class (in the character recognition example, the H class is blank), the path vector generator 106 of the i-segment The score of the remaining candidate path vector may be updated based on the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector and the H element in the M-dimensional probability vector of the i-segment.

Further, when one element is included in the remaining candidate route vector and the element is not the H class, the remaining candidate route vector in the i-th round has the remaining candidate route vector containing only the H class. When, the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment, the H element in the M-dimensional probability vector of the i-segment, and the remaining candidate paths including only the H class. The score of the remaining candidate path vector is updated based on the score of the vector.

Further, when the remaining candidate route vectors include only the H class, the score of the remaining candidate route vectors is updated based on the H element in the M-dimensional probability vector of the i-segment.

For example, in the above example where the object to be classified is an image containing text, in the processing of the second round, there is a remaining candidate route vector [blank] containing only the H class (that is, blank), so that the route vector _{The generation unit 106 has a probability p A2} corresponding to the first element A of the remaining candidate path vector [A] in the M-dimensional probability vector of the second segment (the probability p _A2 is a second acquired by the probability vector acquisition unit 102. The remaining candidate path vector containing only the class A in the M-dimensional random vector of the segment), the H element (ie, the first element p _{blank2) in the M-dimensional random vector of the second segment, and the H class only.} The score of the candidate route vector [A] may be updated based on the score of (that is, [blank]). For example, the score of the candidate route vector [A] may be updated according to the following equations (3) to (6).

In formula (5), LM (“blank” / “A”) represents semantic information between “blank” and “A”, weight represents a variable weight, and is calculated according to the above formula (1). You may. As mentioned above, LM (“blank” / “A”) may be set to 0. Further, in the equation (6), Score'(“ A ”) represents the score after the update of the candidate route vector [A].

Further, the probability vector acquisition unit 102 may update the score of another candidate route vector including one element by the same method as the above method of updating the score of the candidate route vector [A]. If the remaining candidate path vectors contain one element and the element is not in class H, then there are no remaining candidate path vectors containing only class H (ie, there is no scenario 3), then the remaining candidates. It is not necessary to consider scenario 3 when updating the path vector score. For example, regarding the candidate route vector [A], if there is no remaining candidate route vector [blank], the above equation (6) may be converted into the following equation (7).

Further, when updating the score of the candidate route vector [blank], since scenario 1, scenario 2 and scenario 3 are actually one scenario, only one scenario needs to be considered. For example, in the processing of the second round, the score of the candidate route vector [blank] may be updated according to the following equation (8).

In the formula (8), Score'(“ blank”) represents the score after the update of the candidate route vector [blank].

When the remaining candidate route vectors include m (m ≧ 2) elements, the route vector generation unit 106 has the probability corresponding to the mth element of the remaining candidate route vectors in the M-dimensional probability vector of the i-segment. The scores of the remaining candidate path vectors may be updated based on the H element in the M-dimensional probability vector of the i-segment.

Further, when the remaining candidate route vectors include m (m ≧ 2) elements, the remaining candidate route vectors in the i-th round have the remaining candidate route vectors containing m-1 elements, and the remaining candidate route vectors have m-1 elements. When the first element to the m-1 element of the remaining candidate path vector including the above m-1 elements and the first element to the m-1 element of the remaining candidate path vector are the same, respectively. The probability corresponding to the mth element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment, the H element in the M-dimensional probability vector of the i-segment, and the remaining candidates including the above m-1 elements. The score of the remaining candidate route vector is updated based on the score of the route vector and the degree of association between the mth element of the remaining candidate route vector and the remaining candidate route vector including the above m-1 elements. do.

For example, in the specific example where the object to be classified is an image containing text, in the processing of the fourth round, the remaining candidate path vectors are [A, B] and [A, B, C]. Suppose. For the remaining candidate path vectors [A, B, C] (for the candidate path vectors [A, B, C], m = 3), m-1 = the remaining candidate path vectors [A, B] containing two elements. ], And the first and second elements of the remaining candidate path vectors [A, B] and the first and second elements of the candidate path vector [A, B, C] are the same, respectively. _{The path vector generation unit 106 has a probability p C4} (probability p _C4 is a probability vector) corresponding to the m (m = 3) element C of the candidate path vector [A, B, C] in the M-dimensional probability vector of the fourth segment. The element corresponding to class C in the M-dimensional probability vector of the fourth segment acquired by the acquisition unit 102), the H element (that is, the first element p _blank4 ) in the M-dimensional probability vector of the fourth segment, and the candidate route vector. The candidate route vector [A, B] is based on the score of [A, B] and the degree of association between the m (m = 3) element C of the candidate route vector [A, B, C] and the candidate route vector [A, B]. A, B, C] scores may be updated. For example, the score of the candidate route vector [A, B, C] may be updated according to the following equations (9) to (12).

In the equation (11), the LM (“C” / “AB”) * weight is the degree of association between the third element C of the candidate route vector [A, B, C] and the candidate route vector [A, B]. Here, LM (“C” / “AB”) represents semantic information between “AB” and “C”, and weight represents a variable weight, which is calculated according to the above equation (1). You may. In the equation (12), Score'(“ABC”) represents the updated score of the candidate route vector [A, B, C].

Further, the probability vector acquisition unit 102 can use the same method as the above method for updating the score of the candidate route vector [A, B, C] to obtain m (m ≧ 2) elements of other candidate route vectors. You may update the score. When the remaining candidate route vector contains m (m ≧ 2) elements, m-1 elements are included, and the first element to the m-1 element is the first element of the remaining candidate route vector. When there is no remaining candidate route vector that is the same as the m-1th element (that is, there is no scenario 3), it is not necessary to consider scenario 3 when updating the score of the remaining candidate route vector. For example, for the candidate route vector [A, B, C], if there is no remaining candidate route vector [A, B], the above equation (12) may be converted into the following equation (13).

By updating the scores of the remaining candidate path vectors as described above, it is possible to merge adjacent duplicate characters and remove the blank character blank in the character recognition decoding process, further improving the accuracy of character recognition. Can be made to.

FIG. 2B shows the results obtained by recognizing the text in FIG. 2A using the information processing apparatus 100 according to the embodiment of the present disclosure. As can be seen from FIG. 2B, the information processing apparatus 100 according to the embodiment of the present disclosure can accurately recognize the text of FIG. 2A.

Although the information processing apparatus according to the embodiment of the present disclosure has been described above with reference to FIGS. 1 to 3, the present disclosure is an embodiment of an information processing method corresponding to the above-mentioned embodiment of the information processing apparatus. Further provide.

FIG. 5 is a flowchart showing an example of the flow of the information processing method 400 according to the embodiment of the present disclosure. As shown in FIG. 5, the information processing method 400 according to the embodiment of the present disclosure may include a probability vector acquisition step S406, a candidate class selection step S408, a route vector generation step S410, and a classification result acquisition step S412. The information processing method starts from the start step S402 and ends in the end step S414.

In the probability vector acquisition step S406, the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified may be acquired. M is the number of classes, the first element to the M element in each M-dimensional probability vector represents the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1. be. For example, in the probability vector acquisition step S406, the M-dimensional probability vector of each segment may be acquired by a neural network such as a pre-trained convolutional recurrent neural network (CRNN). For example, the probability vector acquisition step S406 may be performed by the probability vector acquisition unit 102 of the information processing apparatus 100, and the specific description thereof will be omitted here.

As an example, the object to be classified may be an image containing text. In this case, the first class to the M class may represent different characters (for example, Chinese characters, English alphabets, symbols, etc.), respectively, and the Hth corresponding to the H element in the first class to the M class may be represented. The class may represent a whitespace character (spacing character). The objects to be classified are not limited to this. For example, the object to be classified may be voice. In this case, the classes other than the H class among the first class to the M class may represent different characters (for example, Chinese characters, English words, etc.), and the H class may be blank (for pause in voice). Corresponding) may be represented.

For example, N may be related to, but not limited to, the size of the object to be classified. For example, when the object to be classified is an image containing text, the larger the size of the image, the larger N. Further, for example, when the object to be classified is voice, the longer the time corresponding to the voice, the larger N becomes.

In the candidate class selection step S408, for each of the N segments, the class corresponding to the largest K elements other than the H element in the M-dimensional probability vector of the segment is set as the candidate class for the segment. You may choose. Here, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element may be a class that does not include semantic information. For example, in the candidate class selection step S408, the elements other than the H element in the M-dimensional probability vector of each segment are sorted in descending order, and the class corresponding to the upper K elements is selected as the candidate class of the corresponding segment. You may. Here, those skilled in the art may set the value of K as needed. For example, the candidate class selection step S408 may be performed by the candidate class selection unit 104 of the information processing apparatus 100, and the specific description thereof will be omitted here.

In the route vector generation step S410, a route vector is generated based on each candidate class of N segments, and for each of the generated route vectors, the probability corresponding to each element included in the route vector and the adjacent elements. The score of the path vector may be calculated based on the degree of relevance between them. Here, the degree of relevance between adjacent elements may be calculated based on semantic information between adjacent elements in a path vector and variable weights with respect to distances between segments corresponding to adjacent elements. For example, the route vector generation step S410 may be performed by the route vector generation unit 106 of the information processing apparatus 100, and the specific description thereof will be omitted here.

In the classification result acquisition step S412, the route vector having the highest score among the route vectors may be acquired as the classification result of the target to be classified. For example, the classification result acquisition step S412 may be performed by the classification result acquisition unit 108 of the information processing apparatus 100, and the specific description thereof will be omitted here.

The classification technique is widely applied to, for example, image recognition, character recognition, voice recognition, and the like. In character recognition, the core technology is to perform character recognition by extracting features from images containing text. Since the extracted features usually do not contain semantic information, techniques have been proposed to introduce semantic information such as long short-term memory (LSTM) network, n-gram model, etc. into the decoding process. .. In the prior art, the semantic information between two specific adjacent non-blank characters, words or phrases is constant regardless of the actual distance between them. In addition, "adjacent" here means that there is no character between these two specific non-blank characters, words or phrases, or there is no other character other than the blank.

As described above, the information processing method 400 according to the embodiment of the present disclosure is adjacent based on semantic information between adjacent elements in a path vector and variable weights with respect to distances between segments corresponding to the adjacent elements. Calculate the degree of association between elements. If the object to be classified is an image containing text, the distance between the segments corresponding to the adjacent elements corresponds to the actual distance between the adjacent elements, and when using the above relevance calculation method, Since it is equivalent to adding a variable weight regarding the actual distance between adjacent elements to the semantic information between adjacent elements, the text can be recognized in consideration of the actual distance between adjacent elements.

In one embodiment of the present disclosure, semantic information between adjacent elements is represented by values calculated via a pre-trained n-gram model.

In one embodiment of the present disclosure, when the distance between segments corresponding to adjacent elements is less than or equal to a predetermined threshold, the variable weight is set to 1 and the distance between segments corresponding to adjacent elements is greater than or equal to the predetermined threshold. If is also large, the variable weight is set to a value less than 1, and the variable weight decreases with increasing distance.

For example, if the distance between segments corresponding to adjacent elements is greater than or equal to a predetermined threshold, the variable weight may be set to the reciprocal of the difference between the distance and the predetermined threshold, but is not limited thereto. For example, the variable weight may be calculated according to the above equation (1).

Further, those skilled in the art may determine a predetermined threshold value as needed in practice.

As mentioned above, in the prior art, the semantic information between two specific adjacent non-blank characters, words or phrases is constant regardless of the actual distance between them. On the other hand, there are often delimiters (for example, blanks) in the text to be recognized, and the semantic relevance of the sentences before and after the delimiter is weak.

The information processing method 400 according to an embodiment of the present disclosure has a degree of relevance between adjacent elements based on semantic information between adjacent elements in a path vector and variable weights with respect to distances between segments corresponding to the adjacent elements. May be calculated. If the object to be classified is an image containing text, the distance between the segments corresponding to the adjacent elements corresponds to the actual distance between the adjacent elements, and by the delimiter between the adjacent elements (eg, blank). , The actual distance between adjacent elements can be increased. In other words, the information processing apparatus according to the embodiment of the present disclosure may add a variable weight regarding a delimiter (for example, a blank) between adjacent elements to the semantic information between adjacent elements. For example, in the text shown in FIG. 2A, when the distance between the segments corresponding to the adjacent characters "1" and "(" is larger than a predetermined threshold value, the meaning between the adjacent characters "1" and "("). A variable weight calculated according to the above equation (1) may be added to the information.

If the distance between the segments corresponding to the adjacent elements is greater than a predetermined threshold, the variable weight is set to a value less than 1, and the variable weight decreases as the distance increases. This ensures that when recognizing text, if there is a delimiter (eg, blank) between adjacent elements, a degree of relevance that decreases with increasing delimiters (eg, blank) is applied between the adjacent elements. Therefore, the performance of character recognition (for example, the accuracy of character recognition) can be improved.

In one embodiment of the present disclosure, the information processing method 400 may further include preprocessing step S404. In the preprocessing step S404, preprocessing is performed on the target to be classified. Here, the pretreatment may include at least one of noise reduction, normalization, binarization and tilt correction. Since noise removal, normalization, binarization, and tilt correction are well known in the art, the details thereof will be omitted.

In one embodiment of the present disclosure, in the route vector generation step S410, the route vector may be generated by the following N rounds of processing, and the score of the route vector may be acquired.

In the processing of the first round, K + 1 path vectors are generated based on the candidate class of the first segment of the N segments and the H class corresponding to the H element, and the elements in each path vector are supported. The score of the path vector may be generated based on the probability of doing so.

For example, as shown in FIG. 4, in the processing of the i-th (i ≧ 2) round, the upper L route vectors having the maximum score are selected as candidate route vectors (step S4102), where L is Exclude candidate path vectors other than the candidate path vector that is a natural number larger than 1 and has the highest score among the same two or more candidate path vectors (step S4104), and at least the M-dimensional probability of the i-segment. The score of the remaining candidate route vectors after exclusion is updated based on the vector (step S4106), and for each of the remaining candidate route vectors, each of the candidate classes of the i-segment is added to the remaining candidate route vectors. To generate a new route vector, the score before updating the remaining candidate route vector, the probability corresponding to the newly added candidate class, and the degree of association between the newly added candidate class and the remaining candidate route vector. Based on this, the score of the newly generated path vector is calculated (step S4108).

In the process of the i-th (i ≧ 2) round, the candidate vector selected in step S4102 includes the same two or more candidate route vectors, and among the above two or more candidate route vectors. If there are more than one candidate route vector with the highest score (that is, two or more candidate route vectors have the highest score), even if any one of the candidate route vectors is left in step S4104. good.

The above-mentioned N-round processing can improve the accuracy of classification by combining the beam search method, the semantic information between adjacent elements, and the relevance based on the variable weight. Further, in the processing of the i-th (i ≧ 2) round, the L route vectors having the highest score are selected as the candidate route vectors, and the subsequent processing is performed only on the candidate route vectors to perform the classification processing. The amount of calculation can be reduced and the calculation time can be saved.

For example, in character recognition, it is necessary to merge adjacent duplicate characters and remove the blank character blank according to the decoding rule. Therefore, when updating the score of the route vector, the following three scenarios are taken into consideration. May be good.

As an example, in the processing of the i-th (i ≧ 2) round, the following processing may be performed when updating the scores of the remaining candidate route vectors.

If the remaining candidate path vectors contain one element and the element is not of the H class (in the character recognition example, the H class is blank), the path vector generator 106 of the i-segment The score of the remaining candidate path vector may be updated based on the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector and the H element in the M-dimensional probability vector of the i-segment.

Further, when one element is included in the remaining candidate route vector and the element is not the H class, the remaining candidate route vector in the i-th round has the remaining candidate route vector containing only the H class. When, the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment, the H element in the M-dimensional probability vector of the i-segment, and the remaining candidate paths including only the H class. The score of the remaining candidate path vector is updated based on the score of the vector.

Further, when the remaining candidate route vectors include only the H class, the score of the remaining candidate route vectors is updated based on the H element in the M-dimensional probability vector of the i-segment.

When the remaining candidate path vector contains m (m ≧ 2) elements, the probability corresponding to the mth element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment and the M-dimensional probability of the i-segment. The scores of the remaining candidate path vectors may be updated based on the Hth element in the vector.

Further, when the remaining candidate route vectors include m (m ≧ 2) elements, the remaining candidate route vectors in the i-th round have the remaining candidate route vectors containing m-1 elements, and the remaining candidate route vectors have m-1 elements. When the first element to the m-1 element of the remaining candidate path vector including the above m-1 elements and the first element to the m-1 element of the remaining candidate path vector are the same, respectively. The probability corresponding to the mth element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment, the H element in the M-dimensional probability vector of the i-segment, and the remaining candidates including the above m-1 elements. The score of the remaining candidate route vector is updated based on the score of the route vector and the degree of association between the mth element of the remaining candidate route vector and the remaining candidate route vector including the above m-1 elements. do.

By updating the scores of the remaining candidate path vectors as described above, it is possible to merge adjacent duplicate characters and remove the blank character blank in the character recognition decoding process, further improving the accuracy of character recognition. Can be made to.

Although the functional configuration and operation of the information processing apparatus and the information processing method according to the embodiment of the present disclosure have been described above, the functional configuration and operation are merely exemplary, and the present disclosure is limited. It's not something to do. Those skilled in the art may modify the above embodiments according to the principles of the present disclosure, eg, add, remove or combine functional modules in each embodiment, these modifications are within the scope of the present disclosure. be.

Further, since the embodiment of the apparatus here corresponds to the embodiment of the above method, the corresponding description of the above method embodiment may be referred to for the contents not described in detail in the embodiment of the apparatus. The explanation is omitted.

The disclosure also provides storage media and program products. The instruction that can be executed by the device in the storage medium and the program product according to the embodiment of the present disclosure may execute the above method, and for the contents not described in detail here, refer to the corresponding description of the embodiment of the above method. However, the description thereof will be omitted here.

Accordingly, the present disclosure further includes storage media on which program products containing instructions that the device can execute are recorded. The storage medium includes, but is not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

The above processing and device may be realized by software and / or firmware. When implemented by software and / or firmware, a program constituting software for performing the above method on a computer having a dedicated hardware configuration from a storage medium or network, for example, the general purpose personal computer 500 shown in FIG. The computer may perform various functions when various programs are installed.

In FIG. 6, the central processing unit (CPU) 501 executes various processes by a program stored in the read-only memory (ROM) 502 or a program loaded from the storage unit 508 into the random access memory (RAM) 503. do. The RAM 503 stores data necessary for the CPU 501 to execute various processes, if necessary.

The CPU 501, ROM 502, and RAM 503 are connected to each other via the bus 504. The input / output interface 505 is also connected to the bus 504.

Input unit 506 (including keyboard, mouse, etc.), output unit 507 (including displays such as brown tube (CRT), liquid crystal display (LCD), and speakers), storage unit 508 (including hard disk, etc.), communication. The unit 509 (including, for example, a network interface card, for example, a LAN card, a modem, etc.) is connected to the input / output interface 505. The communication unit 509 executes communication processing via a network, for example, the Internet.

If desired, the driver 510 may be connected to the input / output interface 505. The removable medium 511 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like. It is installed in 508.

When performing the above processing by software, a program constituting the software is installed via a network such as the Internet or a storage medium such as a removable medium 511.

Note that these storage media are not limited to the removable medium 511 shown in FIG. 6, which stores the program and provides the program to the user separately from the device. The removable medium 511 is, for example, a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including an optical disk-read-only memory (CD-ROM), and a digital multipurpose disk (DVD)), and an optical magnetic disk (mini). Includes optical disc (MD) (registered trademark)) and semiconductor memory. Alternatively, the storage medium may be a ROM 502, a hard disk included in the storage unit 508, or the like, which stores programs and is provided to the user together with a device containing them.

Although the preferred embodiments of the present disclosure have been described above with reference to the drawings, the above examples and examples are exemplary and not restrictive. Those skilled in the art may make various modifications, improvements, and equalities to the present disclosure within the scope and purpose of the claims. These modifications, improvements or changes to equal ones are within the scope of this disclosure.

For example, the functions included in one unit of the above embodiment may be realized by separate devices. Further, the plurality of functions realized by the plurality of units of the above embodiment may be realized by different devices. Further, one of the above functions may be realized by a plurality of units. It should be noted that these configurations are within the scope of the present disclosure.

Further, the method of the present disclosure is not limited to the one executed in the temporal order described in the specification, and may be executed sequentially, in parallel, or independently in another temporal order. For this reason, the order of execution of the methods described herein does not limit the technical scope of the present disclosure.

Further, the following additional notes will be disclosed with respect to the embodiments including each of the above-mentioned embodiments, but the present invention is not limited to these additional notes.
(Appendix 1)
It is a probability vector acquisition unit that acquires the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified, M is the number of classes, and the first element in each M-dimensional probability vector. The M element represents the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1, a probability vector acquisition unit and the like.
For each of the N segments, a candidate class selection unit that selects the class corresponding to the highest K maximum elements among the elements other than the H element in the M-dimensional probability vector of the segment as the candidate class of the segment. Therefore, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element is a class that does not include semantic information. Selection part and
A route vector is generated based on each candidate class of the N segments, and for each of the generated route vectors, the probability corresponding to each element included in the route vector and the degree of relevance between adjacent elements are used. And a path vector generator that calculates the score of the path vector,
A classification result acquisition unit for acquiring the path vector having the highest score among the path vectors as the classification result of the object to be classified is included.
The relevance between adjacent elements is calculated based on the semantic information between the adjacent elements in the path vector and the variable weight of the distance between the segments corresponding to the adjacent elements.
(Appendix 2)
The information processing apparatus according to Appendix 1, wherein the semantic information between adjacent elements is represented by a value calculated via a pre-trained n-gram model.
(Appendix 3)
If the distance between segments corresponding to adjacent elements is less than or equal to a predetermined threshold, the variable weight is set to 1.
If the distance between the segments corresponding to the adjacent elements is greater than the predetermined threshold, the variable weight is set to a value less than 1, and the variable weight decreases as the distance increases. The information processing apparatus according to 1.
(Appendix 4)
The path vector generation unit generates the path vector by the following N rounds of processing, obtains the score of the path vector, and obtains the score.
In the processing of the first round, the route vector generation unit generates K + 1 route vectors based on the candidate class of the first segment among the N segments and the H class, and in each route vector. Generate a score for the path vector based on the probabilities corresponding to the elements
In the processing of the i-th (i ≧ 2) round, the path vector generation unit is
The top L route vectors with the highest score are selected as candidate route vectors, where L is a natural number greater than 1.
Exclude other candidate path vectors other than the one with the highest score among the same two or more candidate path vectors.
Update the scores of the remaining candidate path vectors after exclusion, at least based on the M-dimensional probability vector of the i-segment.
For each of the remaining candidate route vectors, each of the candidate classes in the i-segment is added to the remaining candidate route vectors to generate a new route vector, and the score before the update of the remaining candidate route vectors is newly added. Any of appendices 1 to 3, which calculates the score of the newly generated route vector based on the probability corresponding to the created candidate class and the degree of association between the newly added candidate class and the remaining candidate route vectors. The information processing device described in.
(Appendix 5)
In the processing of the i-th (i ≧ 2) round, the path vector generation unit updates the scores of the remaining candidate path vectors.
When the remaining candidate path vectors contain one element and the element is not of the H class.
The score of the remaining candidate path vector is updated based on the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment and the H element in the M-dimensional probability vector of the i-segment. Or,
When the remaining candidate path vectors in the i-th round include the remaining candidate path vectors containing only the H class, the probability corresponding to the first element of the remaining candidate path vectors in the M-dimensional probability vector of the i-segment. , The score of the remaining candidate route vector is updated based on the score of the remaining candidate route vector including only the H element in the M-dimensional probability vector of the i-th segment and the H class.
When the remaining candidate route vectors include only the H class, the score of the remaining candidate route vectors is updated based on the H element in the M-dimensional probability vector of the i-segment.
When the remaining candidate path vectors include m (m ≧ 2) elements,
The score of the remaining candidate route vector is calculated based on the probability corresponding to the mth element of the remaining candidate route vector in the M-dimensional probability vector of the i-segment and the H element in the M-dimensional probability vector of the i-segment. Update or
The remaining candidate path vectors in the i-th round have the remaining candidate path vectors containing m-1 elements, and the first element to the m-th of the remaining candidate path vectors containing the m-1 elements. When one element and the first element to the m-1 element of the remaining candidate route vector are the same, it corresponds to the mth element of the remaining candidate route vector in the M-dimensional probability vector of the i-segment. The probability, the H element in the M-dimensional probability vector of the i-segment, the score of the remaining candidate route vector including the m-1 element, and the mth element of the remaining candidate route vector and the m-1 element. 4. The information processing apparatus according to Appendix 4, which updates the score of the remaining candidate route vectors based on the degree of association with the remaining candidate route vectors including the elements of.
(Appendix 6)
The information processing apparatus according to any one of Supplementary note 1 to 3, wherein the probability vector acquisition unit acquires M-dimensional probability vectors of each of the N segments by a pre-trained convolutional recurrent neural network.
(Appendix 7)
Further includes a pretreatment unit that performs pretreatment on the object to be classified.
The information processing apparatus according to any one of Supplementary note 1 to 3, wherein the preprocessing includes at least one of noise removal, normalization, binarization, and tilt correction.
(Appendix 8)
The object to be classified is an image containing text.
The first class to the M class represent different characters, respectively.
The information processing apparatus according to any one of Supplementary note 1 to 3, wherein the H-class represents a blank character.
(Appendix 9)
N is the information processing apparatus according to any one of Supplementary note 1 to 3, which is related to the size of the object to be classified.
(Appendix 10)
It is a probability vector acquisition step to acquire the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified, and M is the number of classes and is the first element in each M-dimensional probability vector. The M element represents the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1, a probability vector acquisition step and
For each of the N segments, a candidate class selection step of selecting the class corresponding to the highest K maximum elements among the elements other than the H element in the M-dimensional probability vector of the segment as the candidate class of the segment. Therefore, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element is a class that does not include semantic information. Selection steps and
A route vector is generated based on each candidate class of the N segments, and for each of the generated route vectors, the probability corresponding to each element included in the route vector and the degree of relevance between adjacent elements are used. And the path vector generation step to calculate the score of the path vector,
Includes a classification result acquisition step of acquiring the route vector having the highest score among the route vectors as the classification result of the object to be classified.
Relevance between adjacent elements is an information processing method calculated based on semantic information between adjacent elements in the path vector and variable weights with respect to distances between segments corresponding to the adjacent elements.
(Appendix 11)
The information processing method according to Appendix 10, wherein the semantic information between adjacent elements is represented by a value calculated via a pre-trained n-gram model.
(Appendix 12)
If the distance between segments corresponding to adjacent elements is less than or equal to a predetermined threshold, the variable weight is set to 1.
If the distance between the segments corresponding to the adjacent elements is greater than the predetermined threshold, the variable weight is set to a value less than 1, and the variable weight decreases as the distance increases. The information processing method according to 10.
(Appendix 13)
In the path vector generation step, the path vector is generated by the following N rounds of processing, and the score of the path vector is acquired.
In the processing of the first round, K + 1 path vectors are generated based on the candidate class of the first segment of the N segments and the H class, and based on the probability corresponding to the element in each path vector. To generate a score for the path vector
In the processing of the i-th (i ≧ 2) round,
The top L route vectors with the highest score are selected as candidate route vectors, where L is a natural number greater than 1.
Exclude other candidate path vectors other than the one with the highest score among the same two or more candidate path vectors.
Update the scores of the remaining candidate path vectors after exclusion, at least based on the M-dimensional probability vector of the i-segment.
For each of the remaining candidate route vectors, each of the candidate classes in the i-segment is added to the remaining candidate route vectors to generate a new route vector, and the score before the update of the remaining candidate route vectors is newly added. Any of Appendix 10-12, which calculates the score of the newly generated route vector based on the probability corresponding to the created candidate class and the degree of association between the newly added candidate class and the remaining candidate route vectors. Information processing method described in.
(Appendix 14)
In the processing of the i-th (i ≧ 2) round, when updating the score of each of the remaining candidate route vectors,
When the remaining candidate path vectors contain one element and the element is not of the H class.
The score of the remaining candidate path vector is updated based on the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment and the H element in the M-dimensional probability vector of the i-segment. Or,
When the remaining candidate path vectors in the i-th round include the remaining candidate path vectors containing only the H class, the probability corresponding to the first element of the remaining candidate path vectors in the M-dimensional probability vector of the i-segment. , The score of the remaining candidate route vector is updated based on the score of the remaining candidate route vector including only the H element in the M-dimensional probability vector of the i-th segment and the H class.
When the remaining candidate route vectors include only the H class, the score of the remaining candidate route vectors is updated based on the H element in the M-dimensional probability vector of the i-segment.
When the remaining candidate path vectors include m (m ≧ 2) elements,
The score of the remaining candidate route vector is calculated based on the probability corresponding to the mth element of the remaining candidate route vector in the M-dimensional probability vector of the i-segment and the H element in the M-dimensional probability vector of the i-segment. Update or
The remaining candidate path vectors in the i-th round have the remaining candidate path vectors containing m-1 elements, and the first element to the m-th of the remaining candidate path vectors containing the m-1 elements. When one element and the first element to the m-1 element of the remaining candidate route vector are the same, it corresponds to the mth element of the remaining candidate route vector in the M-dimensional probability vector of the i-segment. The probability, the H element in the M-dimensional probability vector of the i-segment, the score of the remaining candidate route vector including the m-1 element, and the mth element of the remaining candidate route vector and the m-1 element. 13. The information processing method according to Appendix 13, wherein the score of the remaining candidate route vectors is updated based on the degree of association with the remaining candidate route vectors including the elements of.
(Appendix 15)
The information processing method according to any one of Supplementary note 10 to 12, wherein in the probability vector acquisition step, the M-dimensional probability vector of each of the N segments is acquired by a pre-trained convolutional recurrent neural network.
(Appendix 16)
Further including a pretreatment step of performing pretreatment on the object to be classified.
The information processing method according to any one of Supplementary Provisions 10 to 12, wherein the preprocessing includes at least one of noise removal, normalization, binarization, and tilt correction.
(Appendix 17)
The object to be classified is an image containing text.
The first class to the M class represent different characters, respectively.
The information processing method according to any one of Supplementary Provisions 10 to 12, wherein the H-class represents a blank character.
(Appendix 18)
N is the information processing method according to any one of Supplementary note 10 to 12, which is related to the size of the object to be classified.
(Appendix 19)
A computer-readable storage medium in which a program instruction is stored, wherein the information processing method according to any one of the appendices 10 to 18 is executed when the program instruction is executed by the computer.

Claims (10)

It is a probability vector acquisition unit that acquires the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified, M is the number of classes, and the first element in each M-dimensional probability vector. The M element represents the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1, a probability vector acquisition unit and the like.
For each of the N segments, a candidate class selection unit that selects the class corresponding to the highest K maximum elements among the elements other than the H element in the M-dimensional probability vector of the segment as the candidate class of the segment. Therefore, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element is a class that does not include semantic information. Selection part and
A route vector is generated based on each candidate class of the N segments, and for each of the generated route vectors, the probability corresponding to each element included in the route vector and the degree of relevance between adjacent elements are used. And a path vector generator that calculates the score of the path vector,
A classification result acquisition unit for acquiring the path vector having the highest score among the path vectors as the classification result of the object to be classified is included.
The relevance between adjacent elements is calculated based on the semantic information between the adjacent elements in the path vector and the variable weight of the distance between the segments corresponding to the adjacent elements. The information processing apparatus according to claim 1, wherein the semantic information between adjacent elements is represented by a value calculated via a pre-trained n-gram model. If the distance between segments corresponding to adjacent elements is less than or equal to a predetermined threshold, the variable weight is set to 1.
If the distance between the segments corresponding to the adjacent elements is greater than the predetermined threshold, the variable weight is set to a value less than 1, and the variable weight decreases as the distance increases. Item 1. The information processing apparatus according to item 1. The path vector generation unit generates the path vector by the following N rounds of processing, obtains the score of the path vector, and obtains the score.
In the processing of the first round, the route vector generation unit generates K + 1 route vectors based on the candidate class of the first segment among the N segments and the H class, and in each route vector. Generate a score for the path vector based on the probabilities corresponding to the elements
In the processing of the i-th (i ≧ 2) round, the path vector generation unit is
The top L route vectors with the highest score are selected as candidate route vectors, where L is a natural number greater than 1.
Exclude other candidate path vectors other than the one with the highest score among the same two or more candidate path vectors.
Update the scores of the remaining candidate path vectors after exclusion, at least based on the M-dimensional probability vector of the i-segment.
For each of the remaining candidate route vectors, each of the candidate classes of the i-segment is added to the remaining candidate route vectors to generate a new route vector, and the score before the update of the remaining candidate route vectors is newly added. Any of claims 1 to 3, which calculates the score of the newly generated route vector based on the probability corresponding to the created candidate class and the degree of association between the newly added candidate class and the remaining candidate route vectors. Information processing device described in Crab. In the processing of the i-th (i ≧ 2) round, the path vector generation unit updates the scores of the remaining candidate path vectors.
When the remaining candidate path vectors contain one element and the element is not of the H class.
The score of the remaining candidate path vector is updated based on the probability corresponding to the first element of the remaining candidate path vector in the M-dimensional probability vector of the i-segment and the H element in the M-dimensional probability vector of the i-segment. Or,
When the remaining candidate path vectors in the i-th round include the remaining candidate path vectors containing only the H class, the probability corresponding to the first element of the remaining candidate path vectors in the M-dimensional probability vector of the i-segment. , The score of the remaining candidate route vector is updated based on the score of the remaining candidate route vector including only the H element in the M-dimensional probability vector of the i-th segment and the H class.
When the remaining candidate route vectors include only the H class, the score of the remaining candidate route vectors is updated based on the H element in the M-dimensional probability vector of the i-segment.
When the remaining candidate path vectors include m (m ≧ 2) elements,
The score of the remaining candidate route vector is calculated based on the probability corresponding to the mth element of the remaining candidate route vector in the M-dimensional probability vector of the i-segment and the H element in the M-dimensional probability vector of the i-segment. Update or
The remaining candidate path vectors in the i-th round have the remaining candidate path vectors containing m-1 elements, and the first element to the m-th of the remaining candidate path vectors containing the m-1 elements. When one element and the first element to the m-1 element of the remaining candidate route vector are the same, it corresponds to the mth element of the remaining candidate route vector in the M-dimensional probability vector of the i-segment. The probability, the H element in the M-dimensional probability vector of the i-segment, the score of the remaining candidate route vector including the m-1 element, and the mth element of the remaining candidate route vector and the m-1 element. The information processing apparatus according to claim 4, wherein the score of the remaining candidate route vectors is updated based on the degree of association with the remaining candidate route vectors including the elements of. The information processing apparatus according to any one of claims 1 to 3, wherein the probability vector acquisition unit acquires M-dimensional probability vectors of each of the N segments by a pre-trained convolutional recurrent neural network. Further includes a pretreatment unit that performs pretreatment on the object to be classified.
The information processing apparatus according to any one of claims 1 to 3, wherein the preprocessing includes at least one of noise reduction, normalization, binarization, and tilt correction. The object to be classified is an image containing text.
The first class to the M class represent different characters, respectively.
The information processing apparatus according to any one of claims 1 to 3, wherein the H-class represents a blank character. It is a probability vector acquisition step to acquire the M-dimensional probability vector of each of the N segments obtained by dividing the object to be classified, and M is the number of classes and is the first element in each M-dimensional probability vector. The M element represents the probability that the corresponding segment belongs to the first class to the M class, respectively, and M and N are natural numbers larger than 1, a probability vector acquisition step and
For each of the N segments, a candidate class selection step of selecting the class corresponding to the highest K maximum elements among the elements other than the H element in the M-dimensional probability vector of the segment as the candidate class of the segment. Therefore, H and K are natural numbers, 1 ≦ H ≦ M, and 1 ≦ K ≦ M-1, and the H-class corresponding to the H element is a class that does not include semantic information. Selection steps and
A route vector is generated based on each candidate class of the N segments, and for each of the generated route vectors, the probability corresponding to each element included in the route vector and the degree of relevance between adjacent elements are used. And the path vector generation step to calculate the score of the path vector,
Includes a classification result acquisition step of acquiring the route vector having the highest score among the route vectors as the classification result of the object to be classified.
Relevance between adjacent elements is an information processing method calculated based on semantic information between adjacent elements in the path vector and variable weights with respect to distances between segments corresponding to the adjacent elements. A computer-readable storage medium in which a program instruction is stored, wherein the information processing method according to claim 9 is executed when the program instruction is executed by the computer.

Images