JP2024020728A - Learning model learning apparatus, character string image position prediction apparatus, character position estimation apparatus, method thereof, and program - Google Patents

Abstract

To reduce the time required for identifying a position of a character in a character string image in a recognition target, the character being obtained as a result of character recognition, in a text-sequence scheme.SOLUTION: A learning model learning system learns positions of a character string described in a character string image and a partial character string including one or more characters in the character string image, in character string image recognition which recognizes characters from a character string image representing a character string. The learning model learning system includes a learning unit which generates a learning model by performing learning with a character string image including one character string including two or more partial character strings, the character string written on the character string image, and start and end positions of partial character strings in the character string image.SELECTED DRAWING: Figure 1

Description

The present invention relates to a learning model learning device, a character string image position prediction device, a character position estimation device, a method, and a program for handwritten character string image recognition.

Conventionally, in character recognition technology, when recognizing a character string image containing one line of character string as an image, the character area (character frame) of each character is recognized and the characters in that character area are identified. was recognized character by character.

Patent Document 1 discloses a technique for generating learning data including a handwritten character string image and a correct label, and constructing a machine learning recognizer using the above method. However, when building a machine learning recognizer using this method, it is necessary to create a character frame for each character and assign a character label (character code) to each character when creating training data. There was a problem that the load of creating learning data was high.

On the other hand, in recent years, instead of creating learning data by adding a character label to each character, as in Non-Patent Document 1, learning data (hereinafter referred to as text sequence data) in which text labels are added to entire lines is created. A technique for learning using text sequences (hereinafter referred to as the text sequence method) has been widely adopted. This method has the advantage of eliminating the need to create frames for each character and assign character labels.

International Publication No. 2020/218512

“An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition”, Baoguang Shi, Xiang Bai and Cong Yao School of Electronic Information and Communications Huazhong University of Science and Technology, Wuhan, China, 21 Jul 2015, arXiv

However, the aforementioned text sequence method has a problem in that it takes time to recognize the position of a character obtained as a recognition result in a character string image to be recognized.

The present invention was made in view of this situation, and even with the text sequence method, it is difficult to know where the character obtained as a result of character recognition is located in the character string image to be recognized. An object of the present invention is to provide a learning model learning device, a character string image position prediction device, a character position estimation device, a method thereof, and a program, which can reduce the time required for the learning model.

In order to solve the above-mentioned problems, one aspect of the present invention provides a character string written in a character string image and a character string in the character string image in character string image recognition that recognizes characters from a character string image representing a character string. A learning model learning device that learns the position of a partial string containing one or more characters, comprising a character string image in which a single line of character strings containing two or more of the partial strings is written, and the character string. A learning model learning device comprising a learning unit that generates a learning model by learning using a character string written on an image and information on the start and end positions of each sub-character string in the character string image. It is.

In addition, one aspect of the present invention provides character string image recognition that recognizes characters from a character string image representing a character string, including a character string written in the character string image and one or more characters in the character string image. A character string image position prediction device that predicts the position of a sub-character string that is By inputting a character string image to be recognized into a learning model that is generated by learning using the character string and the starting and ending position information of each substring in the character string image, the character string A character string comprising: a prediction unit that obtains a character string included in an image and start and end position information in the character string image to be recognized; and an output unit that outputs a prediction result of the prediction unit. This is an image position prediction device.

Further, one aspect of the present invention provides a character position for estimating the position in the character string image of a partial character string of one or more characters obtained by performing character string image recognition to recognize characters from a character string image representing a character string. The estimation device includes a character string image in which one line of character strings including two or more of the partial character strings is written, a character string written on the character string image, and each partial character in the character string image. The character string included in the character string image obtained by inputting the character string image to be recognized into a learning model generated by learning using the start and end position information of the string and the character The information on the starting and ending positions of the character string image to be recognized is acquired, and the acquired character is generated based on the same conditions as those for generating the partial character string from the character string written on the character string image. a dividing unit that divides a string into partial character strings; and an estimation unit that estimates a range of the divided partial character strings in the character string image to be recognized, which includes the partial character strings divided by the dividing unit. , and an output unit that outputs the range estimated by the estimation unit and the partial character strings obtained by the division unit included within the range, respectively.

Moreover, one aspect of the present invention is characterized in that, in the learning model learning device described above, the learning model learned by the learning unit is learned by deep learning using text sequence data.

In addition, one aspect of the present invention provides character string image recognition that recognizes characters from a character string image representing a character string, including a character string written in the character string image and one or more characters in the character string image. A learning model learning method for learning the positions of partial character strings that are written on the character string image, the character string image having one line of character strings containing two or more of the partial character strings written on the character string image. This learning model learning method is characterized in that a learning model is generated by learning using information on the start and end positions of each partial character string in a character string and the character string image.

In addition, one aspect of the present invention provides character string image recognition that recognizes characters from a character string image representing a character string, including a character string written in the character string image and one or more characters in the character string image. A character string image position prediction method for predicting the position of a partial character string that is By inputting a character string image to be recognized into a learning model that is generated by learning using the character string and the starting and ending position information of each substring in the character string image, the character string This is a character string image prediction method characterized in that a character string included in an image and start and end position information in the character string image to be character recognized are acquired as prediction results.

Further, one aspect of the present invention provides a character position for estimating the position in the character string image of a partial character string of one or more characters obtained by performing character string image recognition to recognize characters from a character string image representing a character string. The estimation method includes a character string image in which a single line of character strings containing two or more of the partial character strings is written, a character string written on the character string image, and each partial character in the character string image. The character string included in the character string image obtained by inputting the character string image to be recognized into a learning model generated by learning using the start and end position information of the string and the character The information on the starting and ending positions of the character string image to be recognized is acquired, and the acquired character is generated based on the same conditions as those for generating the partial character string from the character string written on the character string image. A string is divided into partial character strings, the range of the divided partial character strings in the character string image to be recognized that includes the divided partial character strings is estimated, and the range of the divided partial character strings and the corresponding This character position estimation method is characterized in that each of the divided partial character strings included within the range is output.

In addition, one aspect of the present invention provides character string image recognition that recognizes characters from a character string image representing a character string, including a character string written in the character string image and one or more characters in the character string image. A computer that learns the positions of partial character strings that are written is provided with a character string image in which one line of character strings containing two or more of the partial character strings is written, a character string written on the character string image, and the character mentioned above. This is a program that functions as a learning unit that generates a learning model by learning using information on the start and end positions of each partial character string in a string image.

In addition, one aspect of the present invention provides character string image recognition that recognizes characters from a character string image representing a character string, including a character string written in the character string image and one or more characters in the character string image. A computer that predicts the position of a partial character string that is to be written is provided with a character string image in which a single line of character strings containing two or more of the partial character strings is written, a character string written on the character string image, and the character string described above. By inputting a character string image to be recognized into a learning model that is generated by learning using the starting and ending position information of each partial character string in a column image, the character string contained in the character string image and , a prediction unit that obtains start and end position information in a character string image to be recognized, and an output unit that outputs a prediction result of the prediction unit.

Further, one aspect of the present invention provides a computer that estimates the position of a partial character string of one or more characters obtained by performing character string image recognition that recognizes characters from a character string image representing a character string. , a character string image in which a single line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the start and end of each sub-character string in the above-mentioned character string image. A character string included in the character string image obtained by inputting a character string image to be recognized into a learning model generated by learning using position information, and the character string to be recognized. The starting and ending position information in the image is acquired, and a partial string is generated from the acquired character string based on the same conditions as those used to generate the partial string from the character string written on the character string image. a dividing section that divides the partial string into two parts, an estimation section that estimates the range of the divided partial string in the character string image to be character recognized that includes the partial string divided by the dividing section; This is a program for functioning as an output unit that outputs a range obtained by dividing the range and a partial character string included in the range obtained by the dividing unit.

As explained above, according to the present invention, even in the text sequence method, the time required to determine the position of a character obtained as a character recognition result in a character string image to be recognized is reduced. can do.

1 is a schematic block diagram showing the configuration of a learning model learning system S according to an embodiment of the present invention. 3 is a diagram illustrating an example of a learning data set stored in a character string image database 101. FIG. FIG. 3 is a diagram illustrating starting and ending position information of a partial character string on a character string image. It is a flow chart showing an example of the operation of the deep learning model learning system of this embodiment. 3 is a diagram showing an example of a display screen displayed on a display 4. FIG.

Hereinafter, a learning model learning system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a learning model learning system S according to an embodiment of the present invention.
The learning model learning system S includes a deep learning model learning device 1, a character string image prediction device 2, a character position estimation device 3, and a display 4.
The deep learning model learning device 1 includes a character string image database 101, a learning section 102, and a learning model 103. The character string image prediction device 2 includes a predicted image database 201, a prediction section 202, a prediction model 203, and an output section 204. The character position estimating device 3 includes a dividing section 301, an estimating section 302, and an output section 303.

The character string image database 101 stores a plurality of learning data sets including one line of character string images, corresponding text sequence data, and starting and ending position information of partial character strings into which the sequence data is divided.
FIG. 2 is a diagram showing an example of a learning data set stored in the character string image database 101.
In this example, the learning data set includes character string images 110, text sequence data 111, and position information 112.
The character string image 110 is an image in which a character string of at least two characters is represented as an image. The character string image 110 is, for example, an image that includes a character string written in handwritten characters. In addition to handwritten characters, the character string image 110 may include printed characters or characters written and printed using a word processor. The character string image 110 is an image representing the written content of one line extracted from the document to be read by optically reading the document with a scanner and performing character recognition processing (OCR (Optical Character Recognition) processing). It may be. The lines may be written vertically or horizontally. This document may be an official document, a contract, or an application form for various services.
These handwritten characters may be written in a unique font depending on the writer, or may be written in old characters, or may be written in old kana. Furthermore, handwritten characters may be written using various writing instruments such as brushes, pencils, pens, and the like. Therefore, the person in charge of checking the document may check whether the correspondence between the character string obtained as a result of character recognition processing and the character string written in the document to be read is correct. In this case, if the document is written in old characters and old kana, when a person in charge who does not have sufficient knowledge of old characters and old kana tries to check the correspondence, the characters obtained by character recognition processing , it may take time to figure out which part of the character string image it corresponds to.

The text sequence data 111 is a character string obtained by performing character recognition processing on the character string image 110, and is, for example, text data. This text sequence data 111 is given to the character string image 110 as a correct label for the character string image 110.
Here, the text sequence data 111 is divided into partial character strings according to certain division conditions. For example, if a character string represented by text sequence data includes an address and a name, it is divided into prefectures, town characters, street numbers, names, etc. If the character string is a sentence, morphological analysis is performed to divided into This division condition is determined in advance.
For example, the text sequence data 111 obtained from the character string image 110 is "○○ Hanako, Suido ○-chome, Bunkyo-ku, Tokyo, ○○ Printing Co., Ltd."; A partial character string 111a that is "Waterworks District", a partial character string 111b that is "○-chome ○ address ○", a partial character string 111c that is "○○ Printing Co., Ltd.", and a partial character string 111d that is "○○ Hanako". It is divided into

Next, an example of start end/end end information will be explained below.
FIG. 3 is a diagram showing an example of start end/end end position information.
The start/end position information is information representing the start position and end position of the area where the character string represented by the partial character string exists as an image, within the range from the beginning to the end of one line represented by the character string image.
The character string image 410 is an example of a character string image registered in the character string image database 101. The character string image 420 shows an example of the case where starting and ending position information is expressed for the character string image 410. In the character string image 420, four areas are set, such as a first area 421, a second area 422, a third area 423, and a fourth area 424. Each region is adjacent to each other, and the settings are such that there are no gaps between the regions. In this case, the coordinates of the boundary of each region indicate the start and end position information of the partial character string, with the left end of the character string image 410 (the beginning of the character string) as a reference.
For example, the coordinates of the starting end of the character string image 420 are P1, and the coordinates of the ending end are position P5.
The coordinates of the starting end of the first region 421 are position P1, and the coordinates of the ending end are position P2.
The coordinates of the starting end of the second region 422 are position P2, and the coordinates of the ending end are position P3.
The coordinates of the starting end of the third region 423 are position P3, and the coordinates of the ending end are position P4.
The coordinates of the starting end of the fourth region 424 are position P4, and the coordinates of the ending end are position P5.
Here, since the character string image 410 is a horizontally written image, the left end of the character string image 410 is used as a reference, but in the case of a vertically written image, the upper end may be used as a reference.

Further, a character string image 430 is shown as an example of start/end position information other than the above example. The contents described in the character string image 430 are the same as those in the character string image 410, but the way of setting the start and end position information is different.
In the character string image 430, four areas are set, such as a fifth area 431, a sixth area 432, a seventh area 433, and an eighth area 434, but they are set in the entire image of the character string image 430 without any gaps. Instead, blank areas between partial character strings are not included in the partial character string area, and areas are set according to the portions corresponding to each partial character string.
For example, the coordinates of the starting end of the character string image 430 are P10, and the coordinates of the ending end are position P50.
The coordinates of the starting end of the fifth region 431 are position P11, and the coordinates of the ending end are position P12. There is a gap between the starting end of the character string image 430 and the starting end of the fifth area 431.
The coordinates of the starting end of the sixth region 432 are position P21, and the coordinates of the ending end are position P22. There is a gap between the end of the fifth region 431 and the start of the sixth region 432.
The coordinates of the starting end of the seventh region 433 are position P31, and the coordinates of the ending end are position P32. There is a gap between the end of the sixth region 432 and the beginning of the seventh region 433.
The coordinates of the starting end of the eighth region 434 are position P41, and the coordinates of the ending end are position P42. There is a gap between the end of the seventh region 433 and the start of the eighth region 434.
Further, there is a gap between the end of the eighth area 434 and the end of the character string image 430.

Here, in FIG. 2, a position Pa, a position Pb, a position PC, and a position Pd are determined in the character string image 110. Such a position may be set by a person in charge of creating the learning data set by specifying the position via an input device such as a mouse, keyboard, or touch panel.
In the text sequence data 111, the coordinates of the starting end of the partial character string 111a are the position Pa, and the coordinates of the ending end are the position Pb. Regarding the partial character string 111b, the coordinates of the starting end are position Pb, and the coordinates of the ending end are position Pc. Regarding the partial character string 111c, the coordinates of the starting end are position Pc, and the coordinates of the ending end are position Pd. Regarding the partial character string 111d, the coordinates of the starting end are position Pd, and the coordinates of the ending end are position Pe.
The combination of each partial character string and start/end position information may be set by being specified by the above-mentioned person in charge of creation via an input device.
In addition, in the character string image 110, an area 110a between positions Pa and Pb, an area 110b between positions Pb and Pc, an area 110c between positions Pc and Pd, and an area between positions Pd and Pe. The four areas 110d in between are arranged in order from the beginning of the character string image 110, and the partial character strings 111a, 111b, 111c, and 111d are also arranged in order from the beginning of the text sequence data. Therefore, the correspondence between partial character strings and areas may be determined according to the order in which they are arranged.

Returning to FIG. 1, the learning unit 102 includes a learning model 103. The learning unit 102 uses the character string image data and text sequence data registered in the character string image database 101, and the start and end position information of each partial character string to generate character string images, partial character strings, and partial character strings. The learning model 103 is generated by learning the relationship with the region in the character string image where the character string image exists. The learning method used by the learning unit 102 is, for example, deep learning. The learning unit 102 temporarily stores the generated learning model 103 and outputs it to the character string image prediction device 2.

The learning model 103 is a model that has learned the relationship between a character string image, the character string written in the character string image, and where the starting and ending positions of a partial character string are in the character string image.

The predicted image database 201 has registered character string image data.
The character string image data stored in the predicted image database 201 is a character string image extracted from an image generated by optically reading a document to be subjected to character recognition.

The prediction unit 202 includes a prediction model 203.
The prediction unit 202 acquires the learning model 103 generated by the learning unit 102 from the deep learning model learning device 1 and stores it.
The prediction unit 202 inputs character string images registered in the predicted image database 201 to the prediction model 203, and outputs the prediction result output from the prediction model 203 to the output unit 204. This prediction result includes a character string (text sequence data) corresponding to the character string image and start and end position information determined for the character string image. If the text sequence data obtained here includes at least two or more partial character strings, multiple sets of start/end position information are obtained.

The prediction model 203 is a learning model 103 trained by the deep learning model learning device 1, receives a character string image as input, and outputs a character string and start/end position information.

The output unit 204 receives the output (character string and start/end position information) of the prediction model 203 into which an image is input from the prediction image database 201 and outputs it to the character position estimation device 3 and the display 4 . The display 4 displays the character string and starting and ending position information. Here, the output unit 204 may output not only the output of the prediction model 203 but also the character string image input to the prediction model 203 to the display 4. In this case, the display 4 can display a character string image, a character string, and start/end position information.

The dividing unit 301 receives the prediction result (character string and start/end position information) obtained from the prediction model 203 from the output unit 204 . The dividing unit 301 generates partial character strings obtained from the output unit 204 based on the same dividing conditions as those used when obtaining partial character strings from text sequence data corresponding to character strings registered in the character string image database 101. Divide a string (text sequence data) into substrings.

The estimation unit 302 inputs the start/end position information obtained as a prediction result from the output unit 204 and each partial character string divided by the dividing unit 301, and calculates a value corresponding to each partial character string in the character string image. Estimate the start and end positions.

The output unit 303 receives each partial character string estimated by the estimating unit 302 and the start and end positions corresponding to each partial character string, and outputs and displays them on the display 4.

The above-described deep learning model learning device 1, character string image prediction device 2, and character position estimation device 3 are each computers. The learning unit 102, the prediction unit 202, the output unit 204, the division unit 301, the estimation unit 302, and the output unit 303 may be configured with a processing device such as a CPU (central processing unit) or a dedicated electronic circuit, for example.

The character string image database 101 and the predicted image database 201 are stored in a storage medium such as an HDD (Hard Disk Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), or a RAM (Random Access). read/write Memory), ROM (Read Only Memory), or any combination of these storage media.
For example, nonvolatile memory can be used for the character string image database 101 and the predicted image database 201.

Next, FIG. 4 is a flowchart showing an example of the operation of the deep learning model learning system of this embodiment.
<Step S1>
The learning unit 102 generates a learning model 103 by learning by inputting character string images, text sequence data, and start/end position information stored in the character string image database 101 as a learning data set.

<Step S2>
The prediction unit 202 receives the learning model 103 generated in step S1 from the deep learning model learning device 1, and uses it as a prediction model 203. Using this prediction model 203, the prediction unit 202 inputs the character string image selected from the predicted image database 201 into the prediction model 203, thereby generating a prediction result (text sequence data and a starting point determined for the character string image).・Terminal position information) is passed to the output unit 204.
The output unit 204 outputs the prediction result to the character position estimation device 3.

<Step S3>
The dividing unit 301 divides the text sequence data received from the output unit 204 in step S2 into partial character strings by dividing it based on division conditions.

<Step S4>
The estimation unit 302 uses the start/end position information received from the output unit 204 in step S2 and the partial strings divided by the dividing unit 301 to determine which start/end position information each partial string corresponds to. Estimate.

<Step S5>
The output unit 303 outputs the entire character string (text sequence data) predicted in step S2, each partial character string estimated in step S4, and position information corresponding to each partial character string to the display 4.

FIG. 5 is a diagram showing an example of a display screen displayed on the display 4. As shown in FIG. This display screen 500 is an example of a display screen output from the output unit 303 of the character position estimation device 3.
A character string image 510 to be character recognized is displayed on the upper side of the screen. At the bottom of the screen, text sequence data 520 obtained by subjecting the character string image 510 to character recognition processing is displayed.
The person in charge of checking the text sequence data aligns the position of the operator 511 with the character to be checked in the text sequence data 520 by inputting an operation using an input device. The character position estimating device 3 identifies the area to which the character located at the position of the operator 511 belongs based on the start and end position information of the partial character string to which the character belongs. Then, the character position estimation device 3 displays the specified region in the character string image 510 in a display mode different from that of other regions. Here, for example, the character string image 510 is made transparent so that the character string image 510 can be visually recognized, and an arbitrary color (for example, any one of red, yellow, green, etc.) is applied to the specified area. Display overlapping. As a result, the person in charge can grasp the range of the character string image to which the character belongs by pointing to the character to be confirmed in the text sequence data using the operator 511. This allows the person in charge to check only that range rather than the character string image, so the checking time can be prevented from taking too long.

According to the embodiment described above, instead of creating a frame or a character label for each character in a character string image, the character string image is divided into two or more substrings of one or more characters, and then A machine learning recognizer is trained using the start and end position information for substrings and text sequence data, and the machine learning recognizer is used to determine the position of character strings and substrings on character string images ( By estimating the starting and ending positions), it saves time compared to creating a frame or label for each character, and during recognition, the rough position (area) of each character in a partial string can be estimated in the character string image. be able to. Therefore, for example, a person in charge of checking text sequence data can quickly understand in which block in a character string image a partial character string or a single character belonging to the text sequence data exists, which saves time on checking. This can reduce the prolongation of the problem.

Note that the deep learning model learning device 1 described above is an example of a learning model learning device.
The learning model learning device performs character string image recognition that recognizes characters from images representing character strings, and recognizes character strings written in character string images and partial character strings that include one or more characters in the character string image. Learn location.
The learning model learning device uses a character string image in which a single line of character strings containing two or more partial character strings is written, a character string written on the character string image, and each partial character string in the character string image. It has a learning section that generates a learning model by learning using the starting and ending position information. An example of this learning section is the learning section 102.

Furthermore, the above-described character string image prediction device 2 is an example of a character string image position prediction device.
The character string image position prediction device performs character string image recognition that recognizes characters from a character string image representing a character string, and includes a character string written in the character string image and one or more characters in the character string image. Predict the position of a substring.
The character string image position prediction device detects a character string image in which a single line of character strings containing two or more partial character strings is written, a character string written on the character string image, and each part in the character string image. By inputting a character string image to be recognized into a learning model generated by learning using the starting and ending position information of a character string, the character string included in the character string image and the character recognition target can be input. A prediction unit that obtains start and end position information in a character string image, and an output unit that outputs a prediction result of the prediction unit. An example of this prediction unit is the prediction unit 202, and an example of the output unit is the output unit 204.

Further, the character position estimation device 3 described above is an example of a character position estimation device.
The character position estimation device estimates the position in the character string image of a partial character string of one or more characters obtained by performing character string image recognition to recognize characters from a character string image representing a character string.
The character position estimation device detects a character string image in which one line of character strings containing two or more partial character strings is written, a character string written on the character string image, and each partial character string in the character string image. The character string included in the character string image obtained by inputting a character string image to be recognized into a learning model generated by learning using the start and end position information of The obtained character string is obtained by acquiring start and end position information in the target character string image and generating the partial character string from the character string written on the character string image based on the same conditions as those used to generate the partial character string. a dividing unit that divides into partial character strings; an estimating unit that estimates the range of the divided partial character strings in the character string image to be recognized, which includes the partial character strings divided by the dividing unit; The apparatus further includes an output section that outputs the range estimated by the estimation section and the partial character strings obtained by the division section included within the range. An example of this dividing section is the dividing section 301, an example of the estimating section is the estimating section 302, and an example of the output section is the output section 303.

The deep learning model learning device 1, character string image prediction device 2, and character position estimation device 3 in the embodiments described above may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Note that the "computer system" herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, a "computer-readable recording medium" refers to a storage medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a device that retains a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. Further, the above-mentioned program may be one for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configuration is not limited to these embodiments, and includes designs within the scope of the gist of the present invention.

DESCRIPTION OF SYMBOLS 1... Deep learning model learning device, 2... Character string image prediction device, 3... Character position estimation device, 4... Display, 101... Character string image database, 102... Learning unit, 103... Learning model, 110... Character string image, 110a...area, 110b...area, 110c...area, 110d...area, 111...text sequence data, 111a...partial character string, 111b...partial character string 111c...partial character string, 111d...partial character string, 112...position information, 201... Predicted image database, 202... Prediction unit, 203... Prediction model, 204... Output unit, 301... Division unit, 302... Estimation unit, 303... Output unit, 410... Character string image, 420... Character string image, 421... 1st area, 422...2nd area, 423...3rd area, 424...4th area, 430...character string image, 431...5th area, 432...6th area, 433...7th area, 434...8th area Area, 500...Display screen, 510...Character string image, 511...Manipulator, 520...Text sequence data, S...Learning model learning system

Claims (10)

In character string image recognition, which recognizes characters from a character string image representing a character string, the character string written in the character string image and the position of a partial character string containing one or more characters in the character string image are learned. A learning model learning device,
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. A learning model learning device characterized by comprising a learning section that generates a learning model by learning using information. A character string image position prediction device that predicts the position of a character string written in a character string image and a partial character string containing one or more characters in the character string image in character string image recognition,
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. By inputting a character string image for character recognition into a learning model generated by learning using information, the character string included in the character string image and the starting point/point in the character string image for character recognition can be input. a prediction unit that obtains terminal position information;
an output unit that outputs the prediction result of the prediction unit;
A character string image position prediction device comprising: A character position estimation device that estimates the position in the character string image of a partial character string of one or more characters obtained by performing character string image recognition to recognize characters from a character string image representing a character string,
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. The character string included in the character string image and the character string image to be recognized, which are obtained by inputting the character string image to be recognized into a learning model generated by learning using information. , and convert the obtained character string into a partial string based on the same conditions as those used to generate the partial string from the character string written on the character string image. A dividing part to be divided,
an estimating unit that estimates a range of the divided partial character strings in the character string image to be recognized, which includes the partial character strings divided by the dividing unit;
an output unit that outputs a range estimated by the estimation unit and a partial character string obtained by the division unit included within the range;
A character position estimation device comprising: The learning model learned by the learning unit is learned by deep learning using text sequence data,
The learning model learning device according to claim 1. In character string image recognition, which recognizes characters from a character string image representing a character string, the character string written in the character string image and the position of a partial character string containing one or more characters in the character string image are learned. A learning model learning method, comprising:
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. A learning model learning method characterized by generating a learning model by learning using information. Predicting the position of a character string written in a character string image and a partial character string containing one or more characters in the character string image in character string image recognition that recognizes characters from a character string image representing a character string. A method for predicting character string image position, the method comprising:
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. By inputting a character string image for character recognition into a learning model generated by learning using information, the character string included in the character string image and the starting point/point in the character string image for character recognition can be input. A character string image prediction method characterized by acquiring end position information as a prediction result. A character position estimation method for estimating the position in the character string image of a partial character string of one or more characters obtained by performing character string image recognition to recognize characters from a character string image representing a character string, the method comprising:
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. The character string included in the character string image and the character string image to be recognized, which are obtained by inputting the character string image to be recognized into a learning model generated by learning using information. , and convert the obtained character string into a partial string based on the same conditions as those used to generate the partial string from the character string written on the character string image. divide,
estimating the range of the divided partial character string in the character string image to be recognized, which includes the divided partial character string;
outputting the estimated range and the divided partial character strings included in the range, respectively;
A character position estimation method characterized by the following. In character string image recognition, which recognizes characters from a character string image representing a character string, the character string written in the character string image and the position of a partial character string containing one or more characters in the character string image are learned. to the computer,
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. A program that functions as a learning section that generates a learning model by learning using information. Predicting the position of a character string written in a character string image and a partial character string containing one or more characters in the character string image in character string image recognition that recognizes characters from a character string image representing a character string. to the computer,
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. By inputting a character string image to be recognized into a learning model that is generated by learning using information, the character string included in the character string image and the starting point/point in the character string image to be recognized are determined. a prediction unit that obtains terminal position information;
an output unit that outputs the prediction result of the prediction unit;
A program to function as A computer that estimates the position in the character string image of a partial character string of one or more characters obtained by performing character string image recognition that recognizes characters from a character string image representing a character string,
A character string image in which one line of character strings containing two or more of the above-mentioned sub-character strings is written, a character string written on the above-mentioned character string image, and the starting and ending positions of each sub-character string in the above-mentioned character string image. The character string included in the character string image and the character string image to be recognized, which are obtained by inputting the character string image to be recognized into a learning model generated by learning using information. , and convert the obtained character string into a partial string based on the same conditions as those used to generate the partial string from the character string written on the character string image. The dividing part to be divided,
an estimating unit that estimates a range of the divided partial character strings in the character string image to be recognized, which includes the partial character strings divided by the dividing unit;
an output unit that outputs the range estimated by the estimation unit and the partial character strings obtained by the division unit included within the range;
A program to function as

Images