JP2023043910A - Character string extraction device, character string extraction method and character string extraction program - Google Patents

Abstract

To highly accurately extract a line image region of a character string for each line.SOLUTION: A character string extraction device 10 comprises: a derivation unit 22; and an extraction unit 24. The derivation unit 22 derives a character string center region likelihood 62, a boundary region likelihood 64 and a background region likelihood 66 for each pixel region from an image 50 of a recording medium entered with characters by using a NNW 23. The extraction unit 24 extracts a line image region 60 of a character string 52 for each line included in the image 50 on the basis of the character string center region likelihood 62, the boundary region likelihood 64 and the background region likelihood 66.SELECTED DRAWING: Figure 2

Description

TECHNICAL FIELD Embodiments of the present invention relate to a character string extraction device, a character string extraction method, and a character string extraction program.

Techniques for recognizing characters written on a recording medium are known. For example, a technology is disclosed in which a character string included in an image of a recording medium in which characters are written is extracted for each line, and character recognition is performed for each extracted line. For example, by inputting an image into the learning model, an area obtained by reducing the line area of the character string included in the image by a predetermined reduction ratio is derived as the central area of the character string. A technique for extracting a region obtained by enlarging the derived center region by a predetermined ratio as a line image region of a character string for one line is disclosed (see, for example, Non-Patent Document 1).

However, in the prior art, when a plurality of lines of character strings are written close to each other or overlapped, there are cases where the lines of a plurality of character strings are erroneously identified as the line image area of the same line. That is, in the conventional technology, it is difficult to extract the line image area of the character string for each line from the image with high accuracy.

Wenhai Wang, et al. “Shape Robust Text Detection with Progressive Scale Expansion Network” 2019

The present invention has been made in view of the above, and provides a character string extraction device, a character string extraction method, and a character string extraction program capable of extracting a line image region of a character string for each line with high accuracy. intended to provide

A character string extraction device according to an embodiment includes a derivation unit and an extraction unit. The deriving unit uses a neural network to determine, for each pixel area of an image of a recording medium in which characters are written, the likeness of a character string central area within a character string area, the boundary between the character string area and the character string central area. Region-likeness and background-likeness are derived. The extraction unit extracts a line image area of the character string for each line included in the image based on the likeness of the character string center area, the likeness of the boundary area, and the likeness of the background area.

1 is a block diagram showing the configuration of a character string extraction device according to an embodiment; FIG. FIG. 4 is a schematic diagram showing the flow of processing by a derivation unit and an extraction unit; Explanatory drawing of a character string central area, a boundary area, and a background area. Schematic diagram of a row image area. Schematic diagram of a row image area. Explanatory drawing of learning of NNW. Explanatory drawing of learning of NNW. Explanatory drawing of learning of NNW. Explanatory drawing of learning of NNW. Explanatory diagram of learning to minimize the loss function. Schematic diagram of the boundary region. 4 is a flowchart showing the flow of information processing; FIG. 11 is a specific explanatory diagram of a conventional row image area; FIG. 4 is an explanatory diagram of extraction of a row image region according to the embodiment; FIG. 4 is an explanatory diagram showing a character string area reduced at a predetermined reduction ratio; FIG. 10 is an explanatory diagram of a character string area reduced by a second number of pixels; Hardware configuration diagram.

A character string extraction device, a character string extraction method, and a character string extraction program will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing an example of the configuration of a character string extraction device 10 of this embodiment.

The character string extraction device 10 is an information processing device that extracts a line image area from an image of a recording medium in which characters are written. Details of the row image area will be described later.

The character string extraction device 10 includes a storage unit 12 , a communication unit 14 , a UI (user interface) unit 16 and a control unit 20 . The storage unit 12, the communication unit 14, the UI unit 16, and the control unit 20 are communicably connected via a bus 18 or the like.

The storage unit 12 stores various data. The storage unit 12 is, for example, a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. Note that the storage unit 12 may be a storage device provided outside the character string extraction device 10 .

The communication unit 14 is a communication interface that communicates with an external information processing device via a network or the like.

The UI unit 16 has a reception function for receiving operation input by the user and a display function for displaying various information. The reception function is implemented by, for example, a pointing device such as a mouse, a keyboard, or the like. A display function is realized by, for example, a display. Note that the UI unit 16 may be a touch panel that integrates a reception function and a display function.

The control unit 20 executes various types of information processing in the character string extraction device 10 .

The control unit 20 includes a derivation unit 22 , an extraction unit 24 and a character string recognition unit 26 .

The derivation unit 22, the extraction unit 24, and the character string recognition unit 26 are realized by, for example, one or more processors. For example, each of the above units may be implemented by causing a processor such as a CPU (Central Processing Unit) to execute a program, that is, by software. Each of the above units may be implemented by a processor such as a dedicated IC, that is, by hardware. Each of the above units may be implemented using both software and hardware. When multiple processors are used, each processor may implement one of the units, or may implement two or more of the units.

Note that the control unit 20 of the character string extraction device 10 may be configured to include at least the derivation unit 22 and the extraction unit 24, and may not include the character string recognition unit 26. FIG. For example, the character string recognition unit 26 may be installed in an external information processing device communicably connected to the character string extraction device 10 .

FIG. 2 is a schematic diagram showing an example of the flow of processing by the derivation unit 22 and the extraction unit 24. As shown in FIG.

The deriving unit 22 derives a character string central region likelihood 62 , a boundary region likelihood 64 , and a background region likelihood 66 for each pixel region of the image 50 .

Image 50 is an image of a recording medium on which characters are written. FIG. 2 shows an image 50A as an example of the image 50. As shown in FIG.

The image 50A includes a character string 52 written on the recording medium by handwriting or the like. Character string 52 is a group of one or more characters written along the writing direction. FIG. 2 shows, as an example, a character string 52A consisting of characters "1234" and a character string 52B consisting of characters "567". Character string 52A and character string 52B are examples of character string 52 .

A pixel region is a region composed of one pixel or a plurality of continuous (adjacent) pixels.

The character string central area likeness 62 means the degree of character string central area 80 . The character string central area likeness 62 is represented by, for example, a score 61 that expresses the character string central area likeness 62 . The boundary area likelihood 64 means the degree of being the boundary area 82 . The border-region likeness 64 is represented by, for example, a score 61 representing the border-region likeness 64 . The background area likeness 66 means the degree of being the background area 84 . The background area likeness 66 is represented by, for example, a score 61 representing the background area likeness 66 or the like.

FIG. 3 is an explanatory diagram of an example of the character string center area 80, the border area 82, and the background area 84. As shown in FIG. FIG. 3 shows a schematic diagram of an image 50 in which a region including one line of character string 52 (for example, character string 52A) included in image 50A shown in FIG. 2 is enlarged.

A character string area 86 is an area of one line of character strings 52 included in the image 50 . The character string central region 80 is an area within the character string region 86 . An area within the character string area 86 means an area within the character string area 86 and having a size equal to or smaller than the character string area 86 .

For example, the character string center area 80 is an area obtained by reducing the character string area 86 by the second number of pixels toward a predetermined position within the character string area 86 . The predetermined position in the character string area 86 may be the position of the character string area 86, and may be the center of the character string area 86 or a position other than the center.

The second number of pixels may be one or more pixels, and may be determined in advance. Also, the second number of pixels may be changeable according to an operation instruction of the UI unit 16 by the user.

The second number of pixels is set to the number of pixels that does not eliminate the character string central region 80, which is the region obtained by reducing the character string region 86 by the second number of pixels, by the reduction by the second number of pixels. For example, the width Y of the shortest side of the character string central region 80 obtained by shrinking the character string region 86 by the second number of pixels is set to a width equal to or larger than the number of pixels that does not cause the character string central region 80 to disappear. 2 It suffices to determine the number of pixels. The minimum number of pixels in which the character string central region 80 does not disappear is, for example, 1 pixel.

Boundary region 82 is the region between character string region 86 and character string central region 80 in image 50 . In other words, border region 82 is the entire region between character string region 86 and character string center region 80 . That is, the boundary area 82 represents the boundary between the character string central area 80 of the character string area 86 of a certain character string 52 and the character string central area 80 of the character string area 86 of the other character string 52 or the background area 84. area. A background area 84 is an area other than the character string center area 80 , the boundary area 82 , and the character string area 86 in the image 50 .

Note that the boundary area 82 may be an area that includes an area between the character string area 86 and the character string center area 80, and may include an area protruding a predetermined number of pixels toward the outside of the character string area 86. good. In this case, the background area 84 may be an area obtained by excluding an area obtained by enlarging all the character string areas 86 by, for example, two pixels from the entire image area of the image 50 . The boundary area 82 may be an area other than the character string central area 80 and the background area 84 .

Returning to FIG. 2, the description is continued.

The deriving unit 22 derives a character string central region likelihood 62 , a boundary region likelihood 64 , and a background region likelihood 66 for each pixel region of the image 50 . The deriving unit 22 uses the NNW 23 to derive a score 61 representing each of the character string center area likelihood 62 , boundary area likelihood 64 , and background area likelihood 66 from the image 50 . Details of the derivation unit 22 will be described later.

The extraction unit 24 extracts the line image region 60 included in the image 50 based on the character string central region likelihood 62, boundary region likelihood 64, and background region likelihood 66 derived for each pixel region.

A line image area 60 is an area of one line of character strings 52 included in the image 50 .

The extracting unit 24 uses the character string central area likelihood 62, the boundary area likelihood 64, and the background area likelihood 66 to calculate the likelihood of the character string central area likelihood 62 for each pixel area. Specifically, the extracting unit 24 calculates a score 61 representing each of the character string central region likelihood 62, the boundary region likelihood 64, and the background region likelihood 66 derived for each pixel region, and the total value of these scores 61 is " Each likelihood is calculated by normalizing to 1″.

Then, the extraction unit 24 identifies an area in the image 50 in which the likelihood of the character string central area likelihood 62 is equal to or greater than the threshold as the character string central area 80 .

In FIG. 2, an area where the likelihood of the likelihood of the character string center area 62 is equal to or greater than the threshold is defined as an area 63A, an area where the likelihood of the likelihood of the boundary area 64 is equal to or greater than the threshold is defined as an area 65A, and the likelihood of the likelihood of the background area 66 is equal to or greater than the threshold. The above area is shown as area 67A. When the image 50A is used, the extraction unit 24 specifies the areas 63A1 and 63A2 as areas where the likelihood of the character string center area likelihood 62 is greater than or equal to the threshold. The area 63A1 and the area 63A2 are examples of the area 63A in which the likelihood of the character string center area likelihood 62 is equal to or greater than the threshold.

The extraction unit 24 identifies an area 63</b>A in the image 50 in which the likelihood of the character string central area likeness 62 is greater than or equal to a threshold as the character string central area 80 . In the example shown in FIG. 2, the extracting unit 24 identifies an area 63A1 in which the likelihood of the character string central area likelihood 62 is equal to or greater than the threshold as the character string central area 80A. In addition, the extraction unit 24 identifies an area 63A2 in which the likelihood of the character string central area likelihood 62 is equal to or greater than the threshold as the character string central area 80B. The character string central area 80A and the character string central area 80B are examples of the character string central area 80. FIG. The character string central area 80A is the character string central area 80 of the character string area 86A corresponding to the character string 52A. The character string central area 80B is the character string central area 80 of the character string area 86B corresponding to the character string 52B.

The extraction unit 24 extracts the identified character string central region 80 or an area obtained by enlarging the identified character string central region 80 by the first number of pixels as the line image region 60 . For this reason, in the case of the example shown in FIG. 2, for example, the extracting unit 24 extracts the character string central region 80A of the character string region 86A corresponding to the character string 52A, or the region obtained by enlarging the character string central region 80A by the first number of pixels. , are extracted as the row image area 60A. The extracting unit 24 also extracts a character string central region 80B of a character string region 86B corresponding to the character string 52B, or a region obtained by enlarging the character string central region 80B by the first number of pixels, as a row image region 60B. Line image area 60A and line image area 60B are examples of line image area 60 .

The first number of pixels may be a value representing the number of pixels of 1 or more. The first number of pixels is adjusted in advance to the number of pixels that does not exceed the outer frame of the character string area 86 including the character string central area 80 when the character string central area 80 is enlarged by the first number of pixels. Note that the first pixel count may be changeable within a range that satisfies these conditions according to an operation instruction of the UI unit 16 by the user or the like.

4A and 4B are schematic diagrams of an example of row image area 60. FIG.

As shown in FIG. 4A , for example, the extraction unit 24 expands the character string central region 80 by the first number of pixels to expand the character string central region 80 to a range that matches the outer frame of the character string region 86. Regions are extracted as line image regions 60 .

Further, as shown in FIG. 4B , the extraction unit 24 expands the character string central region 80 by the first number of pixels, and expands the character string central region 80 by the first number of pixels within the range of the character string region 86. may be extracted as row image region 60 .

Also, the extracting unit 24 may extract the character string central region 80 as the line image region 60 .

Returning to FIG. 2, the description is continued. The derivation unit 22 will be described in detail.

The derivation unit 22 includes an NNW (neural network) 23 . The deriving unit 22 uses the NNW 23 to derive a character string center area likelihood 62, a boundary area likelihood 64, and a background area likelihood 66 for each pixel area.

The NNW 23 is a learning model that receives the image 50 as an input and outputs a character string center area likelihood 62 , a boundary area likelihood 64 , and a background area likelihood 66 for each pixel area of the image 50 . The NNW 23 is a deep learning model (DNN) composed of a neural network such as a CNN (Convolutional Neural Network), a GCN (Graph Convolutional Network), or an RNN (Recurrent Neural Network).

The derivation unit 22 learns the NNW 23 in advance using a plurality of learning data 70 . Training data 70 are pairs of images 50 and scores 61 . The score 61 is a score representing each of character string central area likeness 62, boundary area likeness 64, and background area likeness 66 for each pixel area. A score 61 included in the learning data 70 corresponds to the correct data of the corresponding image 50 . For at least a part of the image 50 included in the learning data 70, if an image 50 including a character string 52 written obliquely, an image 50 in which a plurality of lines of character strings 52 are written closely or overlappingly, etc. good.

The deriving unit 22 derives a character string central area likeness 62, a boundary area likeness 64, and a background area likeness 66 in which the character string central areas 80 of the character strings 52 on different lines specified by the extracting unit 24 are not connected. It is preferable to learn the NNW 23 so that it does.

5A to 5D are explanatory diagrams of an example of learning of the NNW 23. FIG.

For example, there are cases where a plurality of character string areas 86 are written on the recording medium in a state in which parts of them overlap. Specifically, as shown in FIG. 5A, it is assumed that a character string region 86A and a character string region 86B included in image 50A overlap each other. Character string area 86A and character string area 86B are examples of character string area 86 . A character string area 86A is a character string area 86 corresponding to the character string 52A. A character string area 86B is a character string area 86 corresponding to the character string 52B.

In this case, the deriving unit 22 extracts the character string central region-likenesses 62, the boundary region-likenesses 64, and the background region-likenesses in which the character string central regions 80 of the character strings 52 on different lines specified by the extracting unit 24 are not connected. NNW 23 is preferably trained to derive .66.

That is, the NNW 23 is trained in advance so as to derive a character string center region likelihood 62, a boundary region likelihood 64, and a background region likelihood 66, in which the character string center regions 80 of the character strings 52 on different lines are not connected. It becomes

Specifically, as shown in FIGS. 5B and 5D, the NNW 23 calculates a score 61 such that the character string central region 80A and the character string central region 80B specified by the extraction unit 24 are not connected. is preferably learned in advance.

Through such learning, when the derivation unit 22 inputs the image 50A to the NNW 23, the extraction unit 24 identifies the non-connected character string central regions 80A and 80B. A score 61 of likeness 62, border-likeness 64, and background-likeness 66 is output.

The NNW 23 may output the score 61 such that the character string center regions 80 corresponding to the character strings 52 on different lines are not connected. Therefore, the shape of the boundary region 82A of the character string central region 80A and the boundary region 82B of the character string central region 80B may be, for example, any of the following shapes.

That is, as shown in FIGS. 5B and 5D, the shapes of the boundary areas 82A and 82B follow the contours of the respective character string areas 86A and 86B where the character string areas 86A and 86B do not overlap. It is a shape that follows. Further, the shape of the boundary area 82A and the boundary area 82B may be an area consisting of one or a plurality of straight lines passing through the overlapping area with respect to the area where the character string area 86A and the character string area 86B overlap (Fig. 5B, see FIG. 5C). In addition, the shape of the boundary area 82A and the boundary area 82B may be a shape along the outline of one of the character string areas 86 for the area where the character string area 86A and the character string area 86B overlap ( See Figure 5D).

Returning to FIG. 2, the description is continued.

The NNW 23 is preferably trained in advance so as to minimize a loss function in which the loss weighting factor of the boundary area likeness 64 is greater than the loss weighting factor of the character string central area likeness 62 and the background area likeness 66 .

FIG. 6 is an explanatory diagram of an example of learning for minimizing a loss function.

For example, it is assumed that there is an overlapping area D in which the character string 52A and the character string 52B included in the image 50A partially overlap. In this case, it may be difficult to identify this overlapping area D as the boundary area 82 .

Therefore, the NNW 23 is preliminarily learned so as to minimize a loss function in which the loss weighting factor of the boundary area likeness 64 is larger than the loss weighting factor of the character string central area likeness 62 and the background area likeness 66. preferable. Specifically, the NNW 23 is preferably trained to minimize the loss function represented by Equation (1) below.

L=w _f L _f + w _b L _b + w _e L _e Expression (1)

In Equation (1), L represents a loss function. L _f represents the loss of string central region likelihood 62 . w _f represents a weighting factor for the loss of text central region likeness 62 . L _b represents the loss of background area likeness 66 . _wb represents a weighting factor for the loss of background area likeness 66; L _e represents the loss of border-likeness 64 . w _e represents a weighting factor for the loss of border-likeness 64 .

In equation (1), the weighting factor w _e for the loss of the border area likeness 64 is greater than the weighting factor w _f for the loss of the character string central area likeness 62 and the weighting factor w _b for the loss of the background area likeness 66 . I wish I had.

The weighting factor w _e for the loss of the border area likeness 64 is larger than the weighting factor w _f for the loss of the character string central area likeness 62 and the weighting factor w _b for the loss of the background area likeness 66 , and the loss function L is minimized. NNW 23 is trained to This learning enables the derivation unit 22 to derive the boundary area likelihood 64 with higher accuracy. In other words, the derivation unit 22 can derive the border area likelihood 64 that enables the extraction unit 24 to extract the line image area 60 with higher accuracy.

Note that the derivation unit 22 derives one or more types of character string center region likenesses 62, one or more types of boundary region likenesses 64, and one or more types of background region likenesses 66 for each pixel region of the image 50. may

The type of character string central region likelihood 62 represents the label of each group obtained by classifying the character string central region 80 into a plurality of groups according to predetermined classification conditions. For example, the type of character string center region likeness 62 is represented by the attribute of the included character string 52, the writing direction of the included character string 52, the type of sentence represented by the included character string 52, and the included character string 52. character shapes, etc.

The attributes of the character string 52 are, for example, English, kanji, katakana, numerals, and the like. The writing direction of the character string 52 is, for example, vertical writing, horizontal writing, or the like. Types of sentences represented by the character string 52 are, for example, addresses, telephone numbers, postal codes, names, and the like. Sentence types are sometimes referred to as field types. Character shapes are, for example, handwriting, printed characters, and the like.

The type of boundary area-likeness 64 represents the label of each group in which the boundary area 82 is classified into a plurality of groups according to predetermined classification conditions. For example, the types of boundary area likeness 64 are classified into areas that do not overlap with the boundary area 82 of another character string central area 80 and areas that overlap with the boundary area 82 of another character string central area 80 .

FIG. 7 is a schematic diagram of an example of the boundary area 82. As shown in FIG. For example, the types of boundary area likelihood 64 include the boundary area likelihood 64 of a boundary area 82A that does not overlap with another character string central area 80, and the boundary area likelihood 64 of a boundary area 82B that overlaps with another character string central area 80. ,are categorized.

The type of background area-likeness 66 represents the label of each group in which the background area 84 is classified into a plurality of groups according to predetermined classification conditions. For example, the types of background area-likeness 66 include a table area representing a table, a diagram area representing a diagram, and other areas other than tables and diagrams.

The derivation unit 22 derives a plurality of types of character string central area likenesses 62 , border area likenesses 64 , and background area likenesses 66 for each pixel area of the image 50 . By this derivation process, the extraction unit 24 can specify the character string central region 80 with higher accuracy. Therefore, the extraction unit 24 can extract the row image region 60 with higher accuracy.

Returning to FIG. 1, the description continues.

The character string recognition unit 26 recognizes characters included in each line image region 60 extracted by the extraction unit 24, and outputs a character recognition result. A known method may be used for character recognition by the character string recognition unit 26 .

Next, an example of the flow of information processing executed by the character string extraction device 10 of this embodiment will be described.

FIG. 8 is a flowchart showing an example of the flow of information processing executed by the character string extraction device 10. As shown in FIG. Note that FIG. 8 shows an example of the flow of information processing when the character string extraction device 10 is configured to include the character string recognition unit 26 .

The deriving unit 22 derives a score 61 representing each of the character string central region likelihood 62, boundary region likelihood 64, and background region likelihood 66 for each pixel region from the image 50 (step S100).

The extraction unit 24 extracts the row image area 60 included in the image 50 based on the score 61 derived in step S100 (step S102).

The character string recognition unit 26 outputs the character recognition result of the row image area 60 extracted in step S102 (step S104).

Then, the routine ends.

As described above, the character string extraction device 10 of this embodiment includes the derivation unit 22 and the extraction unit 24 . Using the NNW 23, the derivation unit 22 derives a character string central region likelihood 62, a boundary region likelihood 64, and a background region likelihood 66 for each pixel region from the image 50 of the recording medium on which characters are written. The extraction unit 24 extracts a line image area 60 of the character string 52 for each line included in the image 50 based on the character string center area likeness 62 , the boundary area likeness 64 , and the background area likeness 66 .

Here, in the prior art, when lines of a plurality of character strings 52 are written close to each other or overlapped, the lines of a plurality of character strings 52 may be erroneously specified as the line image area of the same line.

FIG. 9A is an explanatory diagram of an example of conventional line image area identification. For example, in the method disclosed in Non-Patent Document 1, when lines of a plurality of character strings 52 are written closely or overlappingly, it is difficult to classify overlapping or close areas Q as background. Therefore, in the prior art, as shown in FIG. 9A, character strings 52A and 52B, which are character strings 52 on different lines, may be erroneously specified as the line image area 600 on the same line. That is, in the conventional technique, it is difficult to extract the line image area 60 of the character string 52 for each line from the image 50 with high precision.

On the other hand, in the character string extraction device 10 of the present embodiment, the extracting unit 24 extracts the character strings 52 for each line included in the image 50 based on the character string central area likeness 62, the boundary area likeness 64, and the background area likeness 66. , the row image area 60 is extracted. That is, the character string extracting apparatus 10 of the present embodiment extracts the line image region 60 by using not only the character string central region likelihood 62 and the background region likelihood 66 but also the boundary region likelihood 64 .

For this reason, as shown in FIG. 9B, in the character string extraction device 10 of the present embodiment, the character strings 52A and 52B, which are the character strings 52 of different lines, are extracted from the line image area 60A and line image area 60A of different lines. It can be extracted as each of the regions 60B.

That is, in the character string extracting apparatus 10 of the present embodiment, in addition to the character string central region likelihood 62 and the background region likelihood 66, the boundary region likelihood 64 is further used to calculate the character string central region 80 for each pixel region. can be calculated with high accuracy. Then, the character string extraction device 10 of the present embodiment can extract the line image area 60 with high accuracy by using the character string central area 80 specified based on the calculated likelihood.

Therefore, the character string extracting device 10 of the present embodiment can extract the line image area 60 of the character string 52 for each line with high accuracy.

Further, in the character string extracting device 10 of the present embodiment, the character string center region 80 is a region obtained by reducing the character string region 86 by the second number of pixels toward a predetermined position within the character string region 86 .

FIG. 10A is an explanatory diagram when the character string area 86 is reduced at a predetermined reduction ratio. As shown in FIG. 10A, when a character string central region 80 is formed by reducing a character string region 86 at a predetermined reduction ratio, part of the character string central region 80 may disappear.

FIG. 10B is an explanatory diagram when the character string area 86 is reduced by the second number of pixels. As shown in FIG. 10B , if the character string region 86 is reduced by the second number of pixels to be the character string central region 80 , it is possible to prevent part of the character string central region 80 from disappearing.

Therefore, in addition to the effects described above, the character string extraction device 10 of the present embodiment can extract the line image region 60 with high accuracy even when the character string region 86 has a complicated shape.

Further, in the character string extraction device 10 of the present embodiment, character recognition of the line image region 60 extracted with high accuracy is performed, so that in addition to the above effects, characters included in the image 50 can be recognized with high accuracy. be able to.

Next, the hardware configuration of the character string extraction device 10 of this embodiment will be described.

FIG. 11 is a hardware configuration diagram of an example of the character string extraction device 10 of this embodiment.

The character string extracting device 10 of the present embodiment includes a control device such as a CPU 91, a storage device such as a ROM (Read Only Memory) 92 and a RAM (Random Access Memory) 93, and a communication I/ F94 and a bus 95 connecting each part.

A program executed by the character string extracting device 10 of the present embodiment is preinstalled in the ROM 92 or the like and provided.

The program to be executed by the character string extraction device 10 of the present embodiment can be stored as an installable or executable file on a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), DVD (Digital Versatile Disk), or other computer-readable recording medium, and provided as a computer program product.

Furthermore, the program executed by the character string extraction device 10 of this embodiment may be stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Also, the program executed by the character string extracting device 10 of this embodiment may be provided or distributed via a network such as the Internet.

A program executed by the character string extraction device 10 of this embodiment can cause a computer to function as each part of the character string extraction device 10 of this embodiment. In this computer, the CPU 91 can read a program from a computer-readable storage medium onto the main storage device and execute it.

Although embodiments of the present invention have been described above, the embodiments are presented as examples and are not intended to limit the scope of the invention. The novel embodiments described above can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

10 character string extraction device 22 derivation unit 23 NNW
24 Extractor

A character string extraction device according to an embodiment includes a derivation unit and an extraction unit. The deriving unit uses a neural network to determine, for each pixel area of an image of a recording medium in which characters are written, the likeness of a character string central area within a character string area, the boundary between the character string area and the character string central area. Region-likeness and background-likeness are derived. The extraction unit extracts a line image area of the character string for each line included in the image based on the likeness of the character string center area, the likeness of the boundary area, and the likeness of the background area. The boundary area-likeness indicates the degree of being a boundary area, and the boundary area is the entire area between the character string area and the character string center area.

Claims (8)

Using a neural network, for each pixel area of an image of a recording medium in which characters are written, the likeness of a character string central area within a character string area, the likeness of a boundary area between the character string area and the character string central area, and , a derivation unit for deriving the likeness of the background region;
an extraction unit that extracts a line image area of a character string for each line included in the image based on the likeness of the central area of the character string, the likeness of the boundary area, and the likeness of the background area;
A string extractor with The extractor is
Identifying the character string central region included in the image based on the likelihood of the character string central region calculated for each pixel region using the character string central region likelihood, the boundary region likelihood, and the background region likelihood death,
extracting the character string center region or a region obtained by enlarging the character string center region by a first number of pixels as the line image region;
The character string extraction device according to claim 1. The character string central region is
An area obtained by reducing the character string area by a second number of pixels toward a predetermined position within the character string area,
3. The character string extraction device according to claim 1 or 2. The neural network is
It is learned in advance so as to derive the character string central region-likeness, the boundary region-likeness, and the background region-likeness in which the character string central regions of character strings on different lines are not connected,
The character string extraction device according to any one of claims 1 to 3. The neural network is
pre-learned so as to minimize a loss function in which the weighting factor of the loss of the likeness of the boundary area is greater than the weighting factor of the loss of the likeness of the character string central area and the likeness of the background area;
The character string extraction device according to any one of claims 1 to 4. The derivation unit is
For each pixel region of the image,
Deriving one or more types of the character string central region-likeness, one or more types of the boundary region-likeness, and one or more types of the background region-likeness;
The character string extraction device according to any one of claims 1 to 5. Using a neural network, for each pixel area of an image of a recording medium in which characters are written, the likeness of a character string central area within a character string area, the likeness of a boundary area between the character string area and the character string central area, and , background region-likeness, and
extracting a line image region of a character string for each line included in the image based on the likeness of the character string center region, the likeness of the border region, and the likeness of the background region;
String extraction method containing . Using a neural network, for each pixel area of an image of a recording medium in which characters are written, the likeness of a character string central area within a character string area, the likeness of a boundary area between the character string area and the character string central area, and , background region-likeness, and
extracting a line image region of a character string for each line included in the image based on the likeness of the character string center region, the likeness of the border region, and the likeness of the background region;
A string extraction program for executing on a computer.

Images