JP6003375B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Patent Document 1 discloses a character segmentation unit that gives a probability to a candidate for one character region from a handwritten character string that is written without a character frame, and a character region that is output from the character segmentation unit. 1 character recognition means for performing recognition on a candidate, giving a probability to a character code candidate and outputting it, and a term matching means for giving a probability to a term candidate from the output result of the 1 character recognition means, A first character area candidate output obtained by integrating the probability of one character area candidate output by the character cutout means and the probability of one character area candidate obtained based on the output result of the one character recognition means. A probability integration unit for one character, a probability of a character code candidate output by the one character recognition unit, and a probability of a character code candidate obtained on the basis of the output result of the term matching unit. Sure Second probability integration means for obtaining, and output determination means for outputting a character string code string as a final result based on the outputs of the first probability integration means and the second probability integration means, Characters characterized in that the outputs of the first probability integration means and the second probability integration means are fed back to the character cutout means and the one character recognition means, respectively, and the probability integration is repeated until a steady state is reached. A column input method is disclosed.

  In Patent Document 2, the number of character blocks is registered in a character pattern dictionary for the purpose of increasing the recognition rate of multi-size, undefined pitch Japanese documents. The recognition unit compares it with the character pattern dictionary, compares the number of recognized character blocks with the number of character pattern blocks in the block number reference unit, and if the number of blocks does not match, the character pattern is divided in the re-character segmentation unit. It is disclosed that recognition can be performed by a re-recognition unit, and a recognition result can be output, thereby improving separation accuracy and recognition rate of separated characters.

Japanese Patent Laid-Open No. 02-165287 JP 05-143778 A

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus and an image processing program that prevent a character string of a plurality of lines from being extracted when a character string of one line is extracted from an image. Yes.

The gist of the present invention for achieving the object lies in the inventions of the following items.
According to the first aspect of the present invention, a cutting means for cutting out a character string candidate area that is a character string candidate area from an image, and a determination means for determining whether or not the character string candidate area is a character string in one line. If the determination result is not a character string of one line, the character string candidate area includes a dividing unit that divides the character string candidate area in a direction perpendicular to the line direction, and the clipping unit is an area divided by the dividing unit. The division unit calculates a projection distribution in the character string direction, and performs division when the minimum value of the projection distribution exceeds a predetermined threshold value. It is a processing device.

  The invention of claim 2 further comprises a second cutout unit that cuts out a character candidate region that is a character candidate region from the character string candidate region, and the determination unit is based on a feature amount of the character candidate region. The image processing apparatus according to claim 1, wherein it is determined whether or not the character string candidate area is a character string of one line.

  According to a third aspect of the present invention, the determination means uses any one of the size and position of the character candidate region or the ratio of the size of the character string candidate region and the character candidate region as the feature amount of the character candidate region. The image processing apparatus according to claim 2, wherein two or more are used.

  The invention according to claim 4 further comprises character recognition means for recognizing characters for the character candidate area, and the determination means is a character in which the character string candidate area is one line based on a recognition result by the character recognition means. 4. The image processing apparatus according to claim 2, wherein it is determined whether or not it is a column.

The invention of claim 5, wherein the dividing means, on the basis of the character candidate region, claim claim 2, or dependent on claim 2, characterized in that reproduces an image of the divided character string candidate region The image processing apparatus according to claim 3 .

According to a sixth aspect of the present invention, the computer determines a character string candidate region that is a character string candidate region from the image, and determines whether or not the character string candidate region is a single character string. And when the determination result is not a character string of one line, the dividing means functions as a dividing means for dividing the character string candidate region in a direction perpendicular to the line direction, and the cutting means is divided by the dividing means. The divided region image is set as a target image, and the dividing unit calculates a projection distribution in a character string direction, and performs division when a minimum value of the projection distribution exceeds a predetermined threshold value. Is an image processing program.

According to the image processing apparatus of the first aspect, it is possible to prevent a plurality of lines of character strings from being extracted when a single line of character strings is extracted from the image. In addition, the division position can be determined based on the projection distribution in the row direction in the character string candidate region.

  According to the image processing apparatus of the second aspect, it is possible to determine whether or not the character string candidate area is a single line character string based on the feature amount of the character candidate area.

  According to the image processing apparatus of claim 3, as the feature amount of the character candidate area, any one or more of the size and position of the character candidate area or the ratio of the size of the character string candidate area and the character candidate area Can be used.

  According to the image processing apparatus of the fourth aspect, it is possible to determine whether or not the character string candidate area is a one-line character string based on the character recognition result.

According to the image processing device of the fifth aspect , it is possible to reproduce the image of the divided character string candidate region based on the character candidate region.

According to the image processing program of the sixth aspect , it is possible to prevent the extraction of a plurality of lines of character strings when a single line of character strings is extracted from the image. In addition, the division position can be determined based on the projection distribution in the row direction in the character string candidate region.

It is a conceptual module block diagram about the structural example of this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is explanatory drawing which shows the process example in a character string determination module. It is a flowchart which shows the process example by this Embodiment. It is explanatory drawing which shows the example of the character string candidate area | region made into object. It is explanatory drawing which shows the example which extracted the single character candidate area | region from the inside of a character string candidate area | region. It is a flowchart which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is explanatory drawing which shows the process example by this Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, the values may be different from each other, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

  The image processing apparatus according to the present embodiment extracts one line of character string from the image and recognizes the image of the character string of one line. As shown in the example of FIG. A cutout module 110, a character cutout module 120, a character recognition module 130, a character string determination module 140, and a character string division module 150 are included. In particular, when there is a skew (slope) in the document image, when the spacing between lines or columns is not constant in a handwritten document, or when the text line is written diagonally in a handwritten free entry frame, for example, This is for the case where there is a skew in (character string) itself.

The character string cutting module 110 is connected to the character cutting module 120 and the character string dividing module 150. The character string cutout module 110 cuts out a character string candidate area (character string image data 112), which is a candidate area for a character string, from an image (character image data 108). The character string cutout module 110 cuts out a character string candidate area (character string image data 112) using the image of the area divided by the character string division module 150 (divided image data 152) as a target image. Here, the character string is a sequence of characters and may be a character string in horizontal writing or a character string in vertical writing.
The character cutout module 120 is connected to the character string cutout module 110, the character recognition module 130, and the character string determination module 140. The character cutout module 120 cuts out a character candidate area that is a candidate for a character from the character string candidate area (character string image data 112) cut out by the character string cutout module 110, and converts the character candidate area into the character recognition module 130. The character string candidate area (character string image data 122 (equivalent to the character string image data 112)) is passed to the character string determination module 140.
The character recognition module 130 is connected to the character cutout module 120 and the character string determination module 140. The character recognition module 130 performs character recognition (processing for recognizing a character in the image and converting it into character code data) for the character candidate region cut out by the character cut-out module 120, and the recognition result 132 is converted into the character string determination module 140. To pass.

The character string determination module 140 is connected to the character cutout module 120, the character recognition module 130, and the character string division module 150. The character string determination module 140 determines whether or not the character string candidate area cut out by the character string cut-out module 110 is a one-line character string. Here, the line may be a horizontal line (character string) or a vertical line (character string). If the character string determination module 140 determines that the character string is not a single-line character string (a character string candidate area includes a plurality of lines of character strings), the character string image data 142 (character string image data) 112 or equivalent to the character string image data 122) is passed to the character string dividing module 150, and it is determined that the character string is one line (the character string candidate area includes only one line of character string). In this case, a final recognition result 144 that is a recognition result by the character recognition module 130 for the character string of one line is output. The final recognition result 144 is output, for example, displayed on a display device such as a display, printed on a printing device such as a printer, written as a document recognition result in a storage device such as a document database, a memory card, etc. It includes storing in a storage medium, passing to another information processing apparatus (for example, a translation processing apparatus), and the like.
Further, the character string determination module 140 may determine whether or not the character string candidate area is a single line character string based on the feature amount of the character candidate area. As the feature amount of the character candidate area, any one or more of the size and position of the character candidate area or the ratio of the size of the character candidate area and the character candidate area may be used.
The character string determination module 140 may determine whether or not the character string candidate area is a one-line character string based on the recognition result by the character recognition module 130.

The character string dividing module 150 is connected to the character string cutting module 110 and the character string determining module 140. The character string division module 150 divides the character string candidate area in a direction perpendicular to the line direction when the determination result by the character string determination module 140 is not a single line character string. Here, the vertical direction may be a line (including a straight line and a curve) for dividing the row direction of the character string candidate region, in addition to the case of being perpendicular to the row direction. For example, it may be a position determined in advance as a division position (for example, a position such as ½ in the line direction in the character string candidate region).
Further, the character string division module 150 may determine the division position based on the projection distribution in the row direction in the character string candidate region.
The character string division module 150 may reproduce the image of the divided character string candidate area based on the character candidate area. The image reproduction process will be described later with reference to FIG.
In the example of FIG. 1, there are the character cutout module 120 and the character recognition module 130, but these two may be omitted. That is, the character string cutout module 110 may be connected to the character string determination module 140. The character string determination module 140 determines whether or not the character string candidate area is a one-line character string based on the feature amount of the character string candidate area. As the feature amount of the character string candidate region, for example, there is a shape of the projection distribution of the character string candidate region. Specific examples will be described later.

  FIG. 2 is a flowchart showing a processing example (character recognition processing example) according to this embodiment. Note that the processing flow described below describes the processing flow related to a single character string. When processing a plurality of character strings, steps S202 to S214 described below are performed for the number of character strings. Try to repeat.

In step S202, the character string cutout module 110 sequentially cuts out candidate areas corresponding to the character strings in the character image data 108 as character string candidate areas (character string image data 112). The character string cutting process in the character string cutting module 110 may be an existing method using a conventional technique for dividing an area based on a projection distribution in the character string direction.
In step S204, the character cutout module 120 performs a character cutout process on the character string candidate image (character string image data 112) cut out by the character string cutout module 110, and cuts out a single character candidate area. The character cutout processing in the character cutout module 120 may be an existing method using a conventional technique for cutting out a pixel block. Here, the pixel block includes at least a pixel region that is continuous in four or eight connections, and also includes a set of these pixel regions. The set of these pixel areas means that there are a plurality of continuous pixel areas such as 4-connected, and the plurality of pixel areas are in the vicinity. Here, what is in the vicinity is, for example, an image area in which the pixel areas are close to each other in distance, an image area that is projected vertically or horizontally so as to cut out one character at a time from a line as a sentence, and cut out at a blank spot Or an image region cut out at regular intervals. In many cases, an image of one character is formed as one pixel block. However, it is not necessary that the pixel area is actually recognizable as a character by humans. There are a part of a character, a pixel region that does not form a character, and the like, and any pixel block may be used. The single character candidate area may be a circumscribed rectangle of these pixel blocks.

In step S <b> 206, the character recognition module 130 performs character recognition on the single character candidate region cut out by the character cut-out module 120 and outputs a character code as the recognition result 132.
In step S <b> 208, the character string determination module 140 determines, for example, that the character string candidate area is one line based on the position information of the single character candidate area clipped by the character cutout module 120 and the recognition result 132 obtained by the character recognition module 130. It is determined whether or not it is cut out as a character string or a region mixed with a plurality of lines. Details of character string determination in the character string determination module 140 will be described later.

In step S210, the character string determination module 140 determines whether or not the character string candidate image is a one-line character string image. If the character string candidate image is a one-line character string image, the process proceeds to step S214. If it is determined that a plurality of lines are mixed in the character string candidate area, the process proceeds to step S212.
In step S212, the character string division module 150 performs division processing. Details of the division processing in the character string division module 150 will be described later with reference to the flowchart example of FIG.
In step S214, the character string determination module 140 outputs the recognition result 132 as the final recognition result 144. Since it is determined that the character string candidate area is one line of character string, the character string determination module 140 outputs the recognition result 132 of the character recognition module 130 and ends the processing of the target character string candidate area.

Next, details of the determination process for the character string candidate region in the character string determination module 140 will be described.
Based on the information obtained by the character string cutout module 110, the character cutout module 120, and the character recognition module 130, the character string determination module 140 “characteristic” or “cleanness as a character string” of the character string candidate area. Is calculated as a numerical value, and determination processing is performed based on the numerical value. Here, “characteristic” or “cleanness as a character string” means that each single character candidate area in the character string candidate area is aligned, and the size of each single character candidate area is uniform. This means that the character recognition result of each single character candidate area is valid. Therefore, for example, the size (width, height, and area of the circumscribed rectangle) of each single character candidate area in the input character string candidate area is substantially uniform, and an equal distance along the center of the character string candidate area If the character recognition result of each single character candidate area is determined to be valid, the character string is assumed to be “character string” or “clean as a character string”. On the other hand, if each single character candidate area in the character string candidate area is scattered or the size of each single character candidate area is extremely different, the character string is `` not like a character string ''. Or, “It is not beautiful as a character string”. In other words, “character string-likeness” or “cleanness as a character string” is obtained by quantifying the variation in arrangement, size, or character recognition result of single-character candidate areas in the character string candidate area.

ここで、ｓ_ｉはｉ番目の文字矩形高さ、ｓ（なお、数式中ではｓのオーバーバー（ｏｖｅｒｂａｒ、上線）、以下同様）は文字列中の文字矩形高さの平均値、ｎは文字列中の文字矩形数をそれぞれ表す。文字矩形高さとは、単文字候補領域の高さである。なお、文字矩形高さのかわりに、文字矩形幅、文字矩形面積であってもよい。
２．（文字列中央値−文字中央値）の標準偏差：σ_ｃ

ここで、ｃ_ｉはｉ番目の文字矩形の重心のｙ座標値（又はｘ座標値）と文字矩形の重心のｙ座標値（又はｘ座標値）の差分の絶対値、ｃ（なお、数式中ではｃのオーバーバー、以下同様）はｃ_ｉの平均値、ｎは文字列中の文字矩形数をそれぞれ表す。
３．文字列高さ（あるいは文字列幅）対する平均文字矩形高さ：ｒ_ｓ

ここで、ｓ（なお、数式中ではｓのオーバーバー、以下同様）は文字列中の文字矩形高さの平均値、ｌｓは文字列高さをそれぞれ表す。なお、文字列高さのかわりに文字列幅であってもよい。文字矩形高さのかわりに文字矩形幅であってもよい。
４．認識確度平均値：ｃｆ（なお、数式中ではｃｆのオーバーバー、以下同様）

ここで、ｃｆ_ｉは文字列中のｉ番目の文字の認識確度値を表す。 Here, a specific example of a numerical value representing “character string likelihood” or “character string cleanliness” calculated by the character string determination module 140 will be shown.
1. Standard deviation of height of character rectangle in character string: σ _s

Here, s _i is the height of the i-th character rectangle, s (in the equation, s overbar (overbar), the same applies hereinafter) is the average value of the character rectangle height in the character string, and n is the character Represents the number of character rectangles in the column. The character rectangle height is the height of the single character candidate area. Instead of the character rectangle height, the character rectangle width and the character rectangle area may be used.
2. Standard deviation of (character string median−character median): σ _c

Here, c _i is the absolute value of the difference between the y-coordinate value (or x-coordinate value) of the centroid of the i-th character rectangle and the y-coordinate value (or x-coordinate value) of the centroid of the character rectangle, c ( in c the over bars, hereinafter the same) is the average value of c _i, n represents the number character rectangle in the character string, respectively.
3. String Height (or string width) against the average character rectangle Height: r _s

Here, s (in the formula, an overs of s, and so on) represents the average value of the height of the character rectangle in the character string, and ls represents the height of the character string. The character string width may be used instead of the character string height. The character rectangle width may be used instead of the character rectangle height.
4). Recognition accuracy average value: cf (in the formula, cf overbar, and so on)

Here, cf _i represents the recognition accuracy value of the i-th character in the character string.

Next, details of the determination process using the numerical value representing the calculated “character string likelihood” or “character string cleanliness” will be described.
FIG. 3 is a diagram illustrating an example of the flow of determination processing in the character string determination module 140. The character string determination module 140 compares the calculated numerical value indicating the “character string likelihood” or “character string cleanliness” with each predetermined threshold value. Here, the flow of the determination process will be described with reference to FIG.
1. First, a comparison determination process is performed between the value of the standard deviation σ _s of the calculated character rectangle height (or character rectangle width) in the character string and a predetermined threshold Th _σs (threshold determination 315). Here, when σ _s ≦ Th _σs , the standard deviation σ _s is equal to or smaller than the predetermined threshold Th _σs , so it is determined that the single character candidate areas have the same height, and the character string candidate image is assumed to be “character string”. The process proceeds to a determination process based on the standard deviation σ _c of (character string median−character median). If σ _s > Th _σs , it is determined that the heights of the single character candidate areas are not uniform, the character string candidate area is determined to be “not like a character string” as a multi-line mixed character string area 380, and the character The character string candidate area determination process by the column determination module 140 is terminated.

2. Next, a comparison determination process is performed between the calculated standard deviation σ _c of (character string median−character median) and a predetermined threshold Th _σc (threshold determination 325). Here, when σ _c ≦ Th _σc , it is determined that the centroids of the single character candidate regions are substantially matched, and the character string candidate image is assumed to be “character string”, and the character string height (or character string width) is determined. The process _{proceeds to} a determination process based on the average character rectangle height (or average character rectangle width) rs. If σ _c > Th _σc , it is determined that the centroids of the single character candidate areas are not aligned, and the character string candidate area is determined not to be a character string and is determined to be a multi-line mixed character string area 380. The character string candidate area determination process by the column determination module 140 is terminated.

3. Next, the comparison processing for determining whether the calculated text height (or string width) average character rectangle height for (or character rectangle width) r _s with a predetermined threshold value Th _rs (threshold determination 335). Here, when r _s ≧ Th _rs , it is determined that the single character candidate areas are arranged along the column direction of the character string candidate areas, and the character string candidate images are regarded as “character strings”, and the recognition accuracy average The process proceeds to determination processing using the value cf. If r _s <Th _rs , it is determined that the single character candidate areas are not arranged in the column direction, and the character string candidate area is determined not to be a character string and is determined to be a multi-line mixed character string area 380. Then, the determination process of the character string candidate area by the character string determination module 140 ends.

4). Finally, the character string determination module 140 performs a comparison determination process between the recognition accuracy average value cf and a predetermined threshold Th _cf (threshold determination 345). Here, if cf ≧ Th _cf , it is determined that the character recognition result of each single character candidate area is valid and character segmentation has been processed with high accuracy, and the character string candidate image is “like a character string”. A determination is made 390 as a row region. If cf <Th _cf , it is determined that the character recognition result of each single character candidate area is not valid and character segmentation has not been processed accurately, and the character string candidate area is “not likely to be a character string”. The mixed character string area is determined 380. In any case, the character string candidate area determination processing by the character string determination module 140 is terminated.

As described above, the character string determination module 140 compares each calculated numerical value representing “characteristic” or “cleanness of character string” with predetermined threshold values, and finally determines all the values. The character string candidate area determined as “character string” by comparison with the numerical value of is determined as a one-line character string area 390, and the recognition result in the character recognition module 130 is output. Conversely, the character string candidate area determined as “not like a character string” in comparison with any of the numerical values is determined as a mixed character string area 380 including a plurality of lines, and the character string candidate area is output to the subsequent character string dividing module 150. To do.
In the description so far, the standard deviation σ _s , the standard deviation σ _c , the height ratio r _s , and the recognition accuracy average value cf are determined in this order. Since it is evaluated whether or not it is determined as “character string” in the comparison result between each numerical value and a predetermined threshold value, the determination may be made from any numerical value regardless of the order of determination. However, it may be determined first that the amount of calculation is small, or it may be determined first what is sure to be “determined as a multi-line character string 380” as in the present embodiment. Good.

Next, the character string candidate area dividing process in the character string dividing module 150 will be described in detail.
FIG. 5A illustrates a specific example of the character string candidate area 510 that has been determined as “not like a character string” by the character string determination module 140 and in which a plurality of lines are extracted as one-line character string areas by the character string extraction module 110. It is a figure showing an example. The character string candidate area 510 is a specific example in which a two-line character string is cut out as a one-line character string area. As shown in FIG. 5 (a), for example, in the case of a document image written obliquely by handwriting in a free description frame, a technique for cutting out a character string region based on a black pixel projection distribution 520 in the character string direction is used. In this case, the black pixel projection distribution 520 is, for example, a distribution as shown in FIG. 5B, and a valley of a clear distribution that divides the character string region in one line does not appear, and as a result, the black pixel projection distribution 520 in FIG. Such two lines are cut out as a one-line character string area.

FIG. 6 is a diagram illustrating a specific example of a result of character extraction processing performed by the character extraction module 120 on the character string candidate region illustrated in FIG. Here, each rectangular area (single character candidate area 602 etc.) shown in FIG. 6 represents a single character candidate area.
As illustrated in the example of FIG. 6, when a plurality of lines as illustrated in the example of FIG. 5A is erroneously cut out as a one-line character string region, the character cutout module 120 inputs the input character string candidate region. As shown in FIG. 6, for example, “hand” and “2” lined up and down are cut out as single character candidate regions 602. As a result, the character recognition accuracy in the character recognition module 130 that inputs this single character candidate area is significantly reduced.
Therefore, in the character string dividing module 150, for example, two character strings as shown in FIG. 5A, which are determined as “not like a character string” in the character string determining module 140, are cut out as a one-line character string region. The character string candidate area 510 is divided to generate divided images, and the character string cutting process by the character string cutting module 110 and the character cutting process by the character cutting module 120 are performed again on each divided image.

Here, the flow of the character string candidate area dividing process in the character string dividing module 150 will be described with reference to the flowchart example shown in FIG. This process is performed in the case of NO in step S210 in the flowchart illustrated in FIG.
In step S402, the character string division module 150 searches for a division position for dividing the character string candidate area. Details of the division position search method in the character string division module 150 will be described later.
In step S404, the character string division module 150 divides the character string candidate region by a predetermined method based on the division position searched in step S402, and generates a plurality of divided images. Details of the divided image generation method in the character string division module 150 will also be described later. The divided image becomes character image data 108 in the next processing. However, although there are a plurality of lines in the divided image at this stage, there is a high possibility that the character string extraction module 110 will extract the character string candidate area of only one line.

In step S406, the character string cutout module 110 cuts out character string candidates in the divided image generated by dividing the character string candidate area.
In step S408, the character cutout module 120 performs a character cutout process on the character string candidate image cut out by the character string cutout module 110, and cuts out a single character candidate area.
In step S410, the character recognition module 130 performs character recognition on the single character candidate region cut out by the character cut-out module 120 and outputs a recognition result 132.

In step S412, the character string determination module 140 determines whether or not character extraction processing and character recognition processing have been performed on all character string candidate regions in the divided image. If processing has been performed, the process proceeds to step S414. In other cases (when an unprocessed character string candidate area exists in the divided image), the process returns to step S408, and character extraction processing and character recognition in steps S408 and S410 are performed on the unprocessed character string candidate area. Repeat the process.
In step S414, the character string determination module 140 determines whether or not all the divided images generated by dividing the original character string candidate region at the dividing position have been processed. The process proceeds to S214. In other cases (when an unprocessed divided image exists), the process returns to step S406, and the processes from step S406 to step S410 are repeated for the unprocessed divided image.

Next, details of the division position search in the character string division module 150 will be described with reference to FIGS.
FIG. 7 is a flowchart for explaining a flow of a processing example of division position search in the character string division module 150.
In step S702, the character string dividing module 150 sets the initial position s and the current position c to s = 0 and c = 1, respectively. Here, as shown in the example of FIG. 8, the initial position s and the current position c correspond to the position of the single character candidate area by the character cutout module 120 with respect to the character string candidate area 510 to be divided. In the example shown in FIG. 8, the single character area position is an index (0, 1, 2,...) Indicating the number of single character candidate areas, but the relative coordinates of the single character candidate area with respect to the character string candidate area. A value may be set. Eventually, the area between the initial position s and the current position c is divided. The left end of the divided image is the initial position s, and the right end is the current position c.

In step S704, the character string dividing module 150 calculates a projection distribution in the character string direction from the initial position s to the current position c. The character string direction is the horizontal direction for horizontal writing, the vertical direction for vertical writing, and the horizontal direction in the example of FIG.
In step S706, the character string dividing module 150 determines whether or not the lowest value (the deepest valley value) of the projection distribution exceeds a predetermined threshold (for example, 0), and if so, the process proceeds to step S712. In other cases (when the minimum value of the projection distribution is equal to or smaller than a predetermined threshold value), the process proceeds to step S708. Note that the threshold value here may be the same value as the threshold value used by the character string cutting module 110.
In step S708, since the minimum value of the projection distribution is equal to or smaller than the threshold value, the character string dividing module 150 adds 1 (increments) to the current position c. That is, when dividing at the current position c, the character string cutting module 110 can cut out one line of character string, and therefore the dividing position is moved to the right.

In step S710, the character string dividing module 150 determines whether or not (current position c <number of single character candidate areas). If (c <number of single character candidate areas), the process returns to step S704, and the same. In other cases (when the current position c is the same as the number of single character candidate areas), the division process in the character string division module 150 is terminated (step S799). Here, in this step, the current position c is compared with the number of single character candidate areas because the single character candidate area position is used as an index representing the number of single character candidate areas as described above. When the current position c is a relative coordinate value, it may be compared with the right end coordinate value of the character string candidate area.
In step S712, the character string division module 150 sets the division position d of the character string candidate region to d = c−1 because the minimum value of the projection distribution exceeds the predetermined threshold at the current position c.
In step S714, the character string dividing module 150 sets the initial position s = c−1 and resets the projection distribution so far. Then, the process returns to step S704, and the search for another division position of the character string candidate area is continued.

Next, the divided image generation processing in the character string dividing module 150 will be described.
FIG. 9 is a diagram for explaining an example of divided image generation in the character string division module 150.
For example, the division position search 902 is determined as shown in FIG. 9A by the division position search in the character string division module 150 described above. In the divided image generation processing in the character string dividing module 150, the input character string candidate area 510 is cut perpendicularly to the character string direction at the dividing position 902, and as shown in FIGS. 9B and 9C. As shown in the example, a divided image 910 and a divided image 920 are generated. That is, in the divided image 910 illustrated in the example of FIG. 9B, the black pixel projection distribution 915 is illustrated, and in the divided image 920 illustrated in the example of FIG. 9C, the black pixel projection distribution 925 is illustrated. This is a state where the value is equal to or less than a predetermined threshold. In the specific example shown in FIG. 9, two divided images are generated because the division position is one. For example, when there are two division positions, three divided images are generated. That is, the character string division module 150 generates (division position + 1) divided images.

  The example of FIG. 10A is the processing result of the character string extraction module 110 and the character extraction module 120 when the divided image 910 is targeted, and the example of FIG. 10B is the target of the divided image 920. This is a processing result of the character string cutout module 110 and the character cutout module 120. That is, the character string cutout module 110 performs character string cutout processing on the divided image 910 to extract the character string candidate area 1010 and the character string candidate area 1020 from the divided image 910. The character cutout module 120 The character extraction process is performed on the candidate area 1010 to extract the single character candidate areas 1011 to 1016, and the character extraction process is performed on the character string candidate area 1020 to extract the single character candidate areas 1021 to 1026. Similarly, the character string cutting module 110 performs character string cutting processing on the divided image 920 to extract the character string candidate area 1030 and the character string candidate area 1050 from the divided image 920, and the character cutting module 120 The character extraction process is performed on the character string candidate area 1030 to extract the single character candidate areas 1031 to 1042, and the character extraction process is performed on the character string candidate area 1050 to extract the single character candidate areas 1051 to 1057. ing.

FIG. 11 is a diagram illustrating another example of divided image generation processing in the character string division module 150. Here, the case of the division position 902 similar to the specific example of FIG. 9 will be described.
In this divided image generation processing, the single character candidate region 602 (hand, 2), single character candidate region 604 (hand, line), single character candidate region 606 (ki, Eye), single character candidate region 608 (sammo), single character candidate region 610 (re, ya), single character candidate region 612 (ta, tsu), and to the right of division position 902 in FIG. Single character candidate region 614 (sentence, pal), single character candidate region 616 (character, ri), single character candidate region 618 (column, tilt), single character candidate region 620 (k, i), single character candidate region 622 (て, te), single character candidate region 624 (diagonal), single character candidate region 626 (me, ru), single character candidate region 628 (ni), single character candidate region 630 (tilt), single character candidate region 632 (I), single character candidate region 634 (t), single character candidate region 636 In addition, the single character candidate area 638 (単) and the single character candidate area 640 (ru) are re-imaged (re-rasterized), respectively, thereby dividing the divided image 910 as shown in FIGS. 9B and 9C. Then, a divided image 920 is generated.
The reason why the image can be re-imaged is that the character string candidate area input as the division target of the character string dividing module 150 is already cut out by the character string cutting module 110, the character cutting module 120, and the character recognition module 130. The position information of the single character candidate area with respect to the character string candidate area (for example, the upper left coordinates of the rectangle) and the rectangle information (including the image in the rectangle in addition to the width and height of the rectangle) are obtained. Therefore, re-imaging is possible while maintaining the positional relationship of each single character candidate region with respect to the character string candidate region. That is, the position information and rectangular information of the single character candidate area are stored, and the reimaging process is performed using them.
In addition, since this divided image generation process generates a divided image by performing a re-imaging process, the processing load is higher than the above-described divided image generation process, but the following process can also be performed. It becomes like this.

For example, as shown in FIG. 12, a case where the character string division module 150 searches for the single character candidate area position n-1 as the division position 1248 of the character string candidate area will be described. When the circumscribed rectangles of the two single character candidate regions 1210 and 1220 sandwiching the division position 1248 overlap as shown in the example of FIG. 12, the character string candidate region is simply displayed at the division position 1248 in the above-described divided image generation method. Therefore, the single character candidate area “tsu” is divided, and the single character candidate area “tsu” cannot be correctly recognized, resulting in erroneous recognition.
However, in this divided image generation process, a divided image is generated by reimaging the left and right single character candidate region groups at the division position 1248. Therefore, the division of the single character candidate area as described above does not occur in principle, and it is possible to prevent erroneous recognition that occurs by dividing the single character candidate area.

  A hardware configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration shown in FIG. 13 is configured by a personal computer (PC), for example, and shows a hardware configuration example including a data reading unit 1317 such as a scanner and a data output unit 1318 such as a printer.

  A CPU (Central Processing Unit) 1301 includes various modules described in the above-described embodiments, that is, a character string extraction module 110, a character extraction module 120, a character recognition module 130, a character string determination module 140, and a character string division module 150. The control unit executes processing according to a computer program describing an execution sequence of each module.

  A ROM (Read Only Memory) 1302 stores programs used by the CPU 1301, calculation parameters, and the like. A RAM (Random Access Memory) 1303 stores programs used in the execution of the CPU 1301, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 1304 including a CPU bus.

  The host bus 1304 is connected to an external bus 1306 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1305.

  A keyboard 1308 and a pointing device 1309 such as a mouse are input devices operated by an operator. The display 1310 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 1311 has a built-in hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 1301 and information. The hard disk stores character image data 108, character string image data 112, recognition result 132, final recognition result 144, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 1312 reads data or a program recorded on a removable recording medium 1313 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and reads the data or program into an interface 1307 and an external bus 1306. , The bridge 1305, and the RAM 1303 connected via the host bus 1304. The removable recording medium 1313 can also be used as a data recording area similar to a hard disk.

  The connection port 1314 is a port for connecting the external connection device 1315 and has a connection unit such as USB and IEEE1394. The connection port 1314 is connected to the CPU 1301 and the like via the interface 1307, the external bus 1306, the bridge 1305, the host bus 1304, and the like. A communication unit 1316 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1317 is, for example, a scanner, and executes document reading processing. The data output unit 1318 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus illustrated in FIG. 13 shows one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 13, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line Alternatively, a plurality of systems shown in FIG. 13 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions of a scanner, a printer, a copying machine, a fax machine, etc.).

In addition, although demonstrated using a numerical formula, the thing equivalent to the numerical formula is contained in a numerical formula. The equivalent includes not only the mathematical formula itself, but also transformation of the mathematical formula to the extent that the final result is not affected, or solving the mathematical formula by an algorithmic solution.
Further, in the description of the above-described embodiment, “more than”, “less than”, “greater than”, and “less than (less than)” in a comparison with a predetermined value contradicts the combination. As long as the above does not occur, “larger”, “smaller (less than)”, “more”, and “less” may be used.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray Disc (registered trademark), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 110 ... Character string extraction module 120 ... Character extraction module 130 ... Character recognition module 140 ... Character string determination module 150 ... Character string division | segmentation module

Claims (6)

Clipping means for cutting out a character string candidate area that is a candidate area for a character string from an image;
Determination means for determining whether or not the character string candidate area is a character string of one line;
When the determination result is not a single-line character string, the character string candidate area includes a dividing unit that divides in a direction perpendicular to the line direction,
The clipping means sets the image of the area divided by the dividing means as a target image ,
The image processing apparatus according to claim 1, wherein the dividing unit calculates a projection distribution in a character string direction and performs division when a minimum value of the projection distribution exceeds a predetermined threshold . A second cutting means for cutting out a character candidate area that is a candidate area for a character from the character string candidate area;
The image processing apparatus according to claim 1, wherein the determination unit determines whether or not the character string candidate area is a character string in one line based on a feature amount of the character candidate area. The determining means uses, as the feature amount of the character candidate area, one or more of the size and position of the character candidate area or the ratio of the size of the character string candidate area and the character candidate area. The image processing apparatus according to claim 2. Comprising character recognition means for recognizing characters for the character candidate area;
The image processing according to claim 2 or 3, wherein the determination unit determines whether or not the character string candidate region is a character string of one line based on a recognition result by the character recognition unit. apparatus. The said division | segmentation means reproduces | regenerates the image of the said character string candidate area | region divided | segmented based on the said character candidate area | region. Either of Claim 3 or Claim 4 dependent on Claim 2 or Claim 4 characterized by the above-mentioned. An image processing apparatus according to claim 1. Computer
Clipping means for cutting out a character string candidate area that is a candidate area for a character string from an image;
Determination means for determining whether or not the character string candidate area is a character string of one line;
If the determination result is not a single-line character string, it functions as a dividing means for dividing the character string candidate area in a direction perpendicular to the line direction;
The clipping means sets the image of the area divided by the dividing means as a target image ,
The image processing program characterized in that the dividing means calculates a projection distribution in a character string direction and performs division when a minimum value of the projection distribution exceeds a predetermined threshold .

Images