JP5991076B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program.

  In Patent Document 1, a first storage unit that reads a character on a form and stores a binarized character pattern for a plurality of lines, and scans the first storage unit in the column direction to count black dots. Second storage means for storing the projection distribution of black dots in the line direction obtained, third storage means for storing the lower end address of the character for at least one line, and determining the upper end address of the character line based on the projection distribution , Scanning from one side in the column direction is performed on the first row direction scanning range from the upper end address to detect the first black point, and then performed on the second row direction scanning range moving according to the black point detection result. The lower end address of the character is sequentially detected, the maximum value of the lower end address at the time of detection is sequentially stored in the third storage means, and the content of the storage result is detected by scanning from the other in the column direction. Scan smaller value by comparing the updating the contents, it discloses a character line extracting method characterized by having a control means for cutting out a character row based on the updated results.

  Patent Document 2 reads in a meandering manner for the purpose of improving the recognition accuracy rate in character recognition by extracting the line to be read and correcting the deviation and inclination in the column direction of the read character image. By extracting characters that exist at or near the center of the column direction of document images of multiple lines, the lines to be read are extracted based on the characters, and the character images in the same line that have been read are extracted in the column direction. If there is a misalignment, find the misalignment based on the tilt between the characters, correct it according to the tilt, and if the read character image has a tilt in the column direction, the tilt It is disclosed that the inclination is corrected by performing a process that minimizes the inclination of.

  The method shown in Non-Patent Document 1 uses one connected component as a character string element. It is also assumed that the character string direction is horizontal. First, a pair of connected components is created from a condition that is the nearest neighbor and approximates the size. Next, the tracking is performed from the pair to the left and right. In tracking, two parallel lines are calculated from the upper and lower ends of the connected component, and the region sandwiched between these is observed as a prediction region. The parallel lines are calculated by the least squares method from a certain number of recently observed connected components. Non-Patent Document 1 shows a cutout accuracy that is robust against the inclination and curvature of a character string.

  The method shown in Non-Patent Document 2 uses one connected component as a character string element. First, the general graphic Voronoi diagram of the image is analyzed to create an adjacency graph between connected components. Next, a partial graph having a high probability of being a part of the character string is detected from the adjacent graph. Next, the connected component is traced starting from this subgraph. In tracking, an optimum one is observed from a plurality of adjacent connected components. In determining the optimum connected component, the shape of the character string being tracked is estimated and referred to. The method of Non-Patent Document 2 shows a cutout accuracy that is robust against variations in the thickness and direction of character strings.

  In the method shown in Non-Patent Document 3, a pixel run that is vertically connected to a binary image is used as a character string element. It is also assumed that the character string direction is horizontal. First, the document image is reduced. Due to the reduction, the character string area has a higher character pixel density than the other areas. Next, the reduced image is binarized. Next, the pixel run is traced to the right end (left end) starting from the left end (or right end) pixel run. In tracking, Kalman-Filter is used. The method of Non-Patent Document 3 shows robust cutout accuracy against the inclination / distortion of a character string.

Japanese Patent Laid-Open No. 62-262193 Japanese Patent Laid-Open No. 08-044819

Daniel M. Oliveira, Rafael D. et al. Lins, Gabriel Torreao, Jian Fan, Marcelo Thielo, “A New Method for Text-Line Segmentation for Warped Documents,” in Proc. of Int. Conf. on Image Analysis and Recognition, Povoa de Varzim, Portugal, pp. 398-408, 2010. Moto Iwata, Koichi Kise, Hiroyuki Matsumoto, “Extracting Character Strings from European Documents Using Neighboring Graphs” Information Processing Society Journal, Vol. 49, no. 8, pp. 3239-3248, Aug 1999. Aurelie Lemaitre, Jean Camillerapp, “Text Line Extraction in Handwritten Document with Kalman Filter Applied on Low Resolution Image,” 2nd Int. Conf. on Document Image Analysis for Libraries, Lyon, France, 2006. On Document Image Analysis for Libraries, Lyon, France, 2006.

  The present invention is suitable when the character string element is composed of a plurality of connected components, and an extra connected component is included in the area including the next character string element. Thus, an object of the present invention is to provide an image processing apparatus and an image processing program that prevent generation of an erroneous character string.

The gist of the present invention for achieving the object lies in the inventions of the following items.
According to the first aspect of the present invention, a character string element that can constitute a character string based on a combination of a connected component detected by the detection means and a detection means that detects a combination of the connected components in the region of the target image. A character string element candidate that creates a set of character string element candidates by generating a plurality of character string element candidates composed of a plurality of characters, and the character string element candidate image processing, wherein the set of strings component candidate created by the creation means, to be provided with a string element selection means for selecting a character string element candidates following the string elements that are selected in the past as a string element Device.
The invention according to claim 2 is the image processing apparatus according to claim 1, wherein the character string element candidate creating means uses an ellipse inscribed in a circumscribed rectangle of the connected component as the connected component.

The invention of claim 3 further comprises prediction means for predicting a region where a character string element following the character string element should be based on a character string element selected in the past, and the detection means includes the prediction means. the image processing apparatus according to claim 1 or claim 2, characterized in that to detect the combination of the connected components within the predicted region by.

The invention of claim 4 further comprises connected component selection means for selecting a connected component by removing a connected component inappropriate as a character string element from the combination of connected components detected by the detecting means, string element candidate creating means, on the basis of the combination of the selected connected component by connected component selection means, from claim 1, characterized in that to create a set of string element candidates in any one of 3 It is an image processing apparatus of description.

The invention of claim 5, wherein the predicting means, creating a plurality of prediction areas, said detection means, according to claim 3 or claim, characterized in that it is carried out combining the coupling components contained in the different prediction region 5. The image processing apparatus according to claim 4 , which is dependent on 3 .

According to a sixth aspect of the present invention, there is provided an output means for outputting the character string element selected by the character string element selection means as a character string, and a start character string element as a connected component is not detected from the image. the image processing apparatus according to any one of claims 1 5, characterized by comprising discriminating means for discriminating the completion.

The invention of claim 7, the output string by said output means, an image processing apparatus according to claim 6, further comprising a character recognition means for performing character recognition.

The invention according to claim 8 is a calculation unit that calculates a distortion amount based on the character string output by the output unit, and a distortion of the image including the character string based on the distortion amount calculated by the calculation unit. The image processing apparatus according to claim 6 , further comprising a correcting unit that corrects the error.

According to the ninth aspect of the present invention, the computer can configure the character string based on the combination of the connected component detected by the detecting unit detecting the combination of the connected component in the target image area and the detecting unit. A character string element candidate that is a character string element candidate that is a character string element candidate, and generates a set of character string element candidates by generating a plurality of character string element candidates composed of a plurality of characters, and the character image processing for the set of strings component candidate created by the column elements candidate creating unit, to function as a string element selection means for selecting a character string element candidates following the string elements that are selected in the past as a string element It is a program.

According to the image processing apparatus of the first aspect, the character string element is composed of a plurality of connected components, and there is an extra connected component for forming a character string in the area including the next character string element. In such a case, it is possible to prevent generation of an erroneous character string.
According to the image processing apparatus of claim 2, an ellipse inscribed in the circumscribed rectangle of the connected component can be used as the connected component.

According to the image processing device of the third aspect , based on the character string element selected in the past, it is possible to predict a region where the character string element following the character string element should be.

According to the image processing apparatus of the fourth aspect , it is possible to remove a connected component that is inappropriate as a character string element.

According to the image processing device of the fifth aspect , it is possible not to combine connected components included in different prediction regions.

According to the image processing apparatus of the sixth aspect , the character string element can be output as a character string.

According to the image processing apparatus of the seventh aspect , character recognition can be performed on the extracted character string.

According to the image processing apparatus of the eighth aspect , it is possible to correct the distortion of the image including the character string based on the extracted distortion amount of the character string.

According to the image processing program of claim 9 , the character string element is composed of a plurality of connected components, and an extra connected component is included to form a character string in a region including the next character string element. In such a case, it is possible to prevent generation of an erroneous character string.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is explanatory drawing which shows the example of a tracking type | mold character string cut-out. It is explanatory drawing which shows the example in case a prediction area | region does not contain the original character string element. It is explanatory drawing which shows the example at the time of selecting the incorrect character string element in a prediction area | region. It is explanatory drawing which shows the example of a prediction area | region. It is explanatory drawing which shows the example of a prediction area | region. It is explanatory drawing which shows the process example by this Embodiment. It is a flowchart which shows the process example by 1st Embodiment. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is a flowchart which shows the process example by 2nd Embodiment. It is explanatory drawing which shows the process example by 3rd Embodiment. It is a conceptual module block diagram about the structural example of 3rd Embodiment. It is a flowchart which shows the process example by 3rd Embodiment. It is explanatory drawing which shows the process example by 4th Embodiment. It is a flowchart which shows the process example by 4th Embodiment. It is a notional module block diagram about the structural example of 5th Embodiment. It is a notional module block diagram about the structural example of 6th Embodiment. It is a notional module block diagram about the structural example of 7th Embodiment. It is a notional module block diagram about the structural example of 8th Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

First, before explaining the present embodiment, a technique that is a premise thereof will be described. This description is intended to facilitate understanding of the present embodiment.
The embodiments described below belong to the character string segmentation technique. The character string cut-out technique is a technique for individually detecting character string regions in a document image (bitmap image, stroke image, vector image).
In the present embodiment, a character string is a set (set) of character string elements continuous in the character string direction. In the present embodiment, the character string element is a set of connected components. The connected component corresponds to a set of character pixels to be connected in the bitmap image. Moreover, in a stroke image, it corresponds to one stroke or a set of strokes that cover each other. The character string element is not necessarily a single character.
Note that a character pixel (pixel block) includes at least a pixel region that is continuous in four or eight connections, and includes a set of these pixel regions. The set of these pixel areas means that there are a plurality of continuous pixel areas such as 4-connected, and the plurality of pixel areas are in the vicinity. Here, what is in the vicinity is, for example, an image area in which the pixel areas are close to each other in distance, an image area that is projected vertically or horizontally so as to cut out one character at a time from a line as a sentence, and cut out at a blank spot Or an image region cut out at regular intervals.
In many cases, one pixel block is an image of one character. However, it is not necessary that the pixel area is actually recognizable as a character by humans. There are a part of a character, a pixel region that does not form a character, and the like, and any pixel block may be used.

In particular, the present embodiment belongs to a character string cutting technique by tracking.
Tracking is a process of repeatedly observing the following character string elements from the character string elements that have been observed so far. At this time, a region where a subsequent character string element should be predicted is predicted. Note that “observation” means adding a character string element to the character string being tracked.
A schematic diagram for explanation is shown in FIG. A solid line circle represents the character string element C _k , a broken line circle represents the prediction region R _k, and a rectangle represents the connected component c _{k, i} . Further, the bold line rectangle represents a linking component included in the prediction region R _k. Here, C _k-1 has already been selected as a character string element. In this state, the character string element C _k is observed (selected) for the prediction region R _k as a region where the character string element following C _k−1 should be.

The character string segmentation technique is an element technique for character recognition and distortion correction of a document image.
The character string segmentation method based on tracking performs detection individually for each character string, and therefore shows robust segmentation accuracy against the diversity of character string shapes and document layouts compared to other approaches. . In addition, the shape of a character string refers to thickness, direction, and curvature.

The preceding method of character string extraction by tracking fails to extract a character string when the prediction region does not include the following character string element. A schematic diagram for explanation is shown in FIG. Now, an attempt is made to observe a character string element C _k that follows the character string element C _k−1 . Therefore, although the prediction region R _k is created, the connected component {c _{k, 1} , c _{k, 2} } in which R _k is an element of C _k is not included. In this case, the character string extraction fails.

On the other hand, when the prediction region includes a connected component that is not included in the subsequent character string element, the character string extraction fails. A schematic diagram for explanation is shown in FIG. In the example shown in FIG. 4, R _k includes a connected component {c _{k, 4} , c _{k, 5} , c _{k, 6} } that is not an element of C _k . Note that it is not necessarily an error to observe _{ck, 6} here. This is because the purpose of character string extraction is to correctly detect a character string region, not to correctly detect a break between character string elements. However, the tracking predicts the character string element k that has been observed so far, the region where the subsequent character string element should be, and observes the character string element included in the prediction region. Therefore, observing a character string element that is not optimal for tracking and reducing the accuracy of the subsequent observation of the character string element may occur. Therefore, the accuracy of character string extraction is reduced.

Therefore, in the methods shown in Non-Patent Document 1, Non-Patent Document 2, Non-Patent Document 3, and Patent Document 2, in order to avoid erroneous observation, whether or not to observe from the size / angle of the character string element, etc. There is a device to determine whether. Further, the method of Non-Patent Document 2 has a device of observing an optimum character string element from a plurality of character string elements.
However, a character string element does not necessarily consist of one connected component. On the other hand, the methods of Non-Patent Document 1, Non-Patent Document 2, Non-Patent Document 3, and Patent Document 2 assume that a character string element is composed of one connected component. Therefore, in order to determine whether or not to observe every connected component, the character string segmentation fails under the situation of FIGS.

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, the values may be different from each other, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection) or the like, or one computer, hardware, device. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

  The image processing apparatus according to the first embodiment cuts out a character string. As shown in the example of FIG. 1, the prediction module 110, the connected component detection module 120, the character string element candidate creation module 130, the character It has a column element observation module 140, a connected component holding module 150, and a character string element holding module 160.

The prediction module 110 is connected to the connected component detection module 120 and the character string element observation module 140. The prediction module 110 predicts a prediction region 115 where a character string element following the character string element should be based on a character string element selected in the past (here, the start character string element 105 or the character string element 145). . That is, the prediction module 110 predicts a region where a subsequent character string element should be present from character string elements that have been observed so far.
The connected component detection module 120 is connected to the prediction module 110, the character string element candidate creation module 130, and the connected component holding module 150. The connected component detection module 120 detects a combination of connected components (connected component set 125) in the prediction region 115 predicted by the prediction module 110. That is, the connected component detection module 120 detects a set of connected components included in the prediction region. When no connected component is observed in the connected component detection module 120, the tracking is ended (an end signal 195 is output). Then, a set of character string elements held in the character string element holding module 160 is output as a character string.

The character string element candidate creation module 130 is connected to the connected component detection module 120 and the character string element observation module 140. The character string element candidate creation module 130, based on the combination of connected components detected by the connected component detection module 120, sets of character string element candidates that are candidates for character string elements that can form a character string (character string element set). 135). That is, the character string element candidate creation module 130 creates a character string element candidate set from a combination of connected component sets.
The character string element observation module 140 is connected to the character string element candidate creation module 130, the character string element holding module 160, and the prediction module 110. The character string element observation module 140 selects a character string element (character string element 145) that follows the character string element selected in the past from the set of character string element candidates created by the character string element candidate creation module 130. That is, the character string element observation module 140 selects a character string element suitable for the character string from the candidate set.

The connected component holding module 150 is connected to the connected component detection module 120. The connected component holding module 150 stores the connected components in the image in advance. For example, a connected component is extracted from the target image using an existing technique, stored in the connected component holding module 150, and supplied to the connected component detection module 120.
The character string element holding module 160 is connected to the character string element observation module 140. The character string element holding module 160 stores the character string element 145 output by the character string element observation module 140. The character string element 145 stored in the character string element holding module 160 is a character string in the target image.

  In the first embodiment, the prediction module 110 may be omitted. In that case, the connected component detection module 120 detects a combination of connected components in the region of the target image. That is, the connected component set 125 is detected based on the start character string element 105 or the character string element 145. In this case, the target area is, for example, the area on the right side of the start character string element 105 or the character string element 145 (in the case of horizontal writing from left to right) and the area on the left side (in the case of horizontal writing from right to left). ), Or a lower area (in the case of vertical writing).

Hereinafter, the processes of the prediction module 110, the character string element candidate creation module 130, and the character string element observation module 140 will be described in detail.
The prediction module 110 sets a prediction area. The character string element candidate creation module 130 creates a character string element candidate from a combination of connected components. Then, the character string element observation module 140 selects a suitable character string from the character string element candidates.
Schematic diagrams for explanation are shown in FIGS.
First, the character string element candidate creation module 130 acquires a set of connected components {c _{k, 1} ,..., C _{k, M} } included in the prediction region R _k . Next, a set {C _{k, 1} ,..., C _{k, N} } of subsequent character string elements is created from the combination of connected components (step 1).
Next, the character string element observation module 140 selects a character string element C _{k, n ′} suitable for the character string from the set of character string element candidates (step 2).

Note that the method of creating R _k performed by the prediction module 110 may be freely determined according to the purpose. For example, there are the following methods.
Now, let the circumscribed rectangle of the character string element C _{k be} r _k = (x _k , y _k , w _k , h _k ) ^T. In _addition, x _k, the center coordinates of the _{y k} is _{r k,} w _{k, h k} is the size of _{r k.} From the recently observed set of m string elements {C _k-1-m ,..., C _k-1 }, the region R _k that should be the subsequent set of string elements C _k−1 is _defined as a rectangle (x _k , y _k , w _k , h _k ) ^T is calculated as Equation 1, Equation 2, Equation 3, and Equation 4. Note that δ _k is the distance formed by C _k and C _k−1 and is calculated as in Equation 5. Θ _k is an angle formed by C _k and C _k−1 , and is calculated as in Expression 6. In the example shown here, an average of a set of circumscribed rectangles {r _k-1-m ,..., R _k-1 } of character string elements is used. The above situation is shown in FIG.

また、平均を利用するのではなく、２乗誤差最小法を用いてもよい。又は、Ｋａｌｍａｎ−Ｆｉｌｔｅｒを利用した逐次演算を行ってもよい。さらに、Ｒ_ｋの領域のサイズを、ある割合で拡大してもよい。 Further, instead of the circumscribed rectangle of C _k, on the basis of the ellipse formed by circumscribing rectangle of C _k, may be performed to calculate the R _k. At this time, the distance δ _k formed by C _k and C _k−1 may not include an elliptical region as shown in Equation 7. Note Radius _{(r k, θ k)} is calculated by the equation 10, the radius about the angle theta _k ellipse inscribing the rectangle _{r k.} The size of C _k may be the diameter of an ellipse calculated for the character string direction and the direction orthogonal to the character string. Τ _k and λ _k are calculated as Equation 8 and Equation 9, respectively. Each is calculated from a set of circumscribing rectangles {r _k-1−m ,... R _k−1 } of character string elements, and uses the average in the same manner as Equation 1, Equation 2, Equation 3, Equation 4, and Equation 5. calculate. This is shown in FIG.

Further, instead of using the average, the square error minimum method may be used. Or you may perform the sequential calculation using Kalman-Filter. Further, the size of the region of R _k may be enlarged at a certain rate.

また、連結成分の総数Ｎに対する文字列要素の候補の総数の最大値は、２^Ｎ（＝Ｍ）であるが、適用目的に応じて２^Ｎ以下の候補数へ限定してもよい。 The selection of the character string element C _{k, n ′} performed by the character string element observation module 140 may be freely determined according to the purpose.
For example, there are the following methods.
Now, let e _{k be} the feature quantity of the character string element. And using the evaluation function d representing a distance and if e _k, selects n 'satisfying Equation 11. Note that e ^m _{k, n} is a feature amount of an nth character string element candidate. Further, e ^p _{k, n} is a feature amount of a character string element serving as a reference. For example, the character string element feature amount e _k may be (x _k , y _k , w _k , h _k ) or (θ _k , δ _k , τ _k , λ _k ). Calculations are made as Equation 1, Equation 2, Equation 3, Equation 4, Equation 5, Equation 6, Equation 2, Equation 3, and Equation 4, respectively. As the evaluation function d, for example, Formula 12 can be considered. Now, the i-th feature amount of the character string element e is [e] _i . Note that ψ _i is a weight.
The selection of the character string element C _{k, n ′} suitable for the character string may be performed by a decision tree.

The maximum value of the total number of character string element candidates with respect to the total number N of connected components is 2 ^N (= M), but may be limited to 2 ^N or less candidates depending on the application purpose.

In this embodiment, a combination of connected components is used for tracking. As a result, the character string element is not necessarily composed of one connected component, and it is possible to cope with the case where the prediction region does not include only the correct character string element (when there is an excess or deficiency).
Further, there is a case where the distance between character string elements is constant in tracking. Distance to correspond to [delta] _k. At this time, observing the character string element at a suitable break in the character string direction improves the accuracy of clipping. Note this is not limited to the [delta] _k, same is true for the feature quantity of each other other string element.

FIG. 8 is a flowchart illustrating a processing example according to the first exemplary embodiment.
In step S <b> 802, the character string element C <b> ₁ is input to the prediction module 110.
In step S804, 1 is substituted into the variable k.
In step S806, k + 1 is substituted for variable k.
In step S808, the prediction module 110 generates a prediction region _{R k.}
In step S810, the connected component detection module 120 detects the set S of connected components included in R _k from within the connected component holding module 150.
In step S812, the connected component detection module 120 determines whether or not # {S}> 0. If # {S}> 0, the process proceeds to step S814; otherwise, the process proceeds to step S820. . Note that # {S} is the number of elements in the set S.

In step S814, character string component candidate creation module 130 creates a set E of combinations of elements of S in _{C k} candidates.
In step S816, the character string element observation module 140 selects an element C _k from E.
In step S818, the character string element holding module 160 holds C _k and returns to step S806.
In step S820, the connected component detection module 120 outputs {C ₁ ,..., C _k }.

FIG. 9 is a conceptual module configuration diagram of a configuration example according to the second embodiment.
The image processing apparatus according to the second embodiment selects a connected component and cuts out a character string. As shown in the example of FIG. 9, the prediction module 110, the connected component detection module 120, the connected component selection, and the like. A module 926, a character string element candidate creation module 130, a character string element observation module 140, a connected component holding module 150, and a character string element holding module 160 are included. In the second embodiment, a connected component selection module 926 is added to the first embodiment. In addition, the same code | symbol is attached | subjected to the site | part of the same kind as the above-mentioned embodiment, and the overlapping description is abbreviate | omitted (hereinafter the same).

The prediction module 110 is connected to the connected component detection module 120, the connected component selection module 926, and the character string element observation module 140. The prediction module 110 passes the prediction region 115 to the connected component detection module 120 and the connected component selection module 926.
The connected component detection module 120 is connected to the prediction module 110, the connected component selection module 926, and the connected component holding module 150. The connected component detection module 120 passes the connected component set 125 to the connected component selection module 926.
The connected component selection module 926 is connected to the prediction module 110, the connected component detection module 120, and the character string element candidate creation module 130. The connected component selection module 926 removes a connected component that is inappropriate as a character string element from the combination of connected components detected by the connected component detection module 120 in the prediction region 115 (connected component set 125). Select. The selected result is a character string element set 928. That is, the connected component selection module 926 removes an element that is inappropriate as a character string element from the connected component set 125 detected by the connected component detection module 120. Here, inappropriate means a connected component that has already been detected as another character string, or a connected component that is significantly different in size from the character string element that has been observed so far. Specifically, a statistical value (average value, median value, mode value, etc.) of the size of the character string element observed so far is calculated, and the difference between the statistical value and the size of the character string element is determined in advance. If the value is greater than or equal to the value, the character string element is determined to be an inappropriate element. This process corresponds to noise removal and is performed when the cutout accuracy is improved.
The character string element candidate creation module 130 is connected to the connected component selection module 926 and the character string element observation module 140. The character string element candidate creation module 130 creates a set of character string element candidates based on the combination of connected components selected by the connected component selection module 926 (character string element set 928).

FIG. 10 is a flowchart illustrating a processing example according to the second exemplary embodiment.
In step S <b> 1002, the character string element C <b> ₁ is input to the prediction module 110.
In step S1004, 1 is substituted into the variable k.
In step S1006, k + 1 is substituted for variable k.
In step S1008, the prediction module 110 generates a prediction region _{R k.}
In step S1010, the connected component detection module 120 detects a connected component set S ′ included in R _k .
In step S1012, the connected component selection module 926 removes inappropriate elements from S ′ and creates S.

In step S1014, the connected component selection module 926 determines whether or not # {S}> 0. If # {S}> 0, the process proceeds to step S1016. Otherwise, the process proceeds to step S1022. .
In step S1016, the string element candidate creation module 130 creates a set E of combinations of elements of S in _{C k} candidates.
In step S1018, the character string element observation module 140 selects an element C _k from E.
In step S1020, the character string element holding module 160 holds C _k and returns to step S1006.
In step S1022, the connected component detection module 120 outputs {C ₁ ,..., C _k }.

FIG. 11 is an explanatory diagram illustrating a processing example according to the third exemplary embodiment. The third embodiment targets a plurality of prediction regions.
First, the multiple prediction region creation module 1206 creates a plurality of prediction regions. Next, the character string element candidate creation module 130 generates a set of character string element candidates from combinations of connected components included in each prediction region. create. Next, the character string element observation module 140 selects a character string element suitable as a character string from the candidate set.

A schematic diagram for explanation is shown in FIG. First, the multiple prediction region creation module 1206 creates multiple prediction regions R _{k, 1} , R _{k, 2} , R _{k, 3} . {C _{k, 1} , c _{k, 2} , c _{k, 3} } is detected from R _{k, 1} . {C _{k, 3} } is detected from R _{k, 2} . {C _{k, 4} , c _{k, 5} } is detected from R _{k, 3} . Since combinations of connected components included in different prediction regions are not performed, it can be seen that the number of candidates is reduced as compared to the example of FIG.

FIG. 12 is a conceptual module configuration diagram of an exemplary configuration according to the third embodiment. In the third embodiment, as shown in the example of FIG. 12, a plurality of prediction region creation module 1206, a connected component detection module 120, a character string element candidate creation module 130, a character string element observation module 140, and a connected component holding module 150. And a character string element holding module 160.
The multiple prediction region creation module 1206 is connected to the connected component detection module 120 and the character string element observation module 140. The multiple prediction region creation module 1206 creates a plurality of prediction regions. For example, a plurality of prediction regions R are created by varying parameters (for example, θ and the like) by the method described above using the examples of FIGS.

The connected component detection module 120 is connected to the multiple prediction region creation module 1206, the character string element candidate creation module 130, and the connected component holding module 150. The connected component detection module 120 does not combine connected components included in different prediction regions. That is, a combination of connected components is detected for each prediction region created by the multiple prediction region creation module 1206.
The character string element candidate creation module 130 is connected to the connected component detection module 120 and the character string element observation module 140. The character string element candidate creation module 130 creates a candidate set of character string elements from combinations of sets of connected components for each prediction region.
The character string element observation module 140 is connected to the character string element candidate creation module 130, the character string element holding module 160, and the multiple prediction region creation module 1206. The character string element observation module 140 selects a character string element suitable as a character string from character string element candidates created for each prediction region.

FIG. 13 is a flowchart illustrating a processing example according to the third exemplary embodiment.
In step S1302, a prediction area set A = {R ₁ ,..., R _{# {A}} } is created.
In step S1304, ∞ (a value indicating ∞) is substituted for the variable min.
In step S1306, 1 is substituted into variable a.
In step S1308, for R _a , the connected component detection module 120 performs the connected component detection, the character string element candidate creation module 130 uses the character string element candidate creation, and the character string element observation module 140 uses a character string suitable as a character string. Element detection is performed to create C _a .

In step S1310, it is determined whether the min> d _{(C a),} if a min> d _{(C a)} the process proceeds to step S1312, otherwise, the process goes to step S1316. The function d (C) is an evaluation function that returns a smaller value as the character string element C is appropriate. For example, the function d (C) is implemented by Equation 12.
In step S1312, d (C _a ) is substituted for the variable min.
In step S1314, a is substituted for variable a ′.
In step S1316, it is determined whether or not a <# {A}. If a <# {A}, the process proceeds to step S1318; otherwise, the process ends (step S1399).
In step S1318, a + 1 is substituted for variable a, and the process returns to step S1308.

FIG. 14 is an explanatory diagram illustrating a processing example according to the fourth exemplary embodiment. In the fourth embodiment, combinations are performed with respect to a variable of interest.
As a modification of the character string element candidate creation module 130 and the character string element selection module 140, a combination having a variable of interest is targeted. A set of connected components is sorted with respect to a predetermined variable, and a start position and an end position of a set of connected components that are character string elements are selected.
FIG. 14 shows a schematic diagram for explanation. FIG. 15 shows a processing flow of the fourth embodiment.
First, the character string element candidate creation module 130 sorts the connected components included in the prediction region for the variable of interest. In the example of FIG. 14, the tracking direction is sorted in order of position. As the attention variable, an angle, a size, or the like may be used. Next, a set of candidate character string elements is created from the combination of the start position i and the end position j for the set of connected components. In the example of FIG. 14, a set of character string element candidates for c _{k, 2} , c _{k, 3} , c _{k, 4} between the start position i (= 2) and the end position j (= 4). Create
Then, the character string element observation module 140 selects a character string element suitable as a character string from the candidates.
This process, up to ^{2 #} number of candidates for a character string element in ^{S} is reduced to _{# {S} C 2.}

FIG. 15 is a flowchart illustrating a processing example according to the fourth exemplary embodiment.
In step S1502, the set S of connected components is sorted for the variable of interest.
In step S1504, ∞ is substituted for the variable min.
In step S1506, 1 is substituted into the variable i.
In step S1508, i is substituted into variable j.
In step _{S1510, min> d ({c,} ..., c j}) determines _{whether, min> d ({c i} , ..., c j}) if a flow proceeds to step S1512, Otherwise, the process proceeds to step S1518.
In step S1512, d ({c _i ,..., C _j }) is substituted for the variable min.

In step S1514, 1 is substituted into variable i ′.
In step S1516, i is substituted into variable j ′.
In step S1518, it is determined whether or not j <# {S}. If j <# {S}, the process proceeds to step S1520. Otherwise, the process proceeds to step S1522.
In step S1520, j + 1 is substituted for variable j, and the process returns to step S1512.
In step S1522, it is determined whether i <# {S}. If i <# {S}, the process proceeds to step S1524. Otherwise, the process proceeds to step S1526.
In step S1524, i + 1 is substituted for variable i, and the process returns to step S1508.
In step S1526, {c _{i ′} ,..., C _{j ′} } are output.

FIG. 16 is a conceptual module configuration diagram of a configuration example according to the fifth embodiment. In the fifth embodiment, a plurality of character strings are cut out. As shown in the example of FIG. 16, an end determination module 1610, a character string tracking module 1620, a character string output module 1630, and a connected component holding module 1640 are provided. Have.
The end determination module 1610 is connected to the character string tracking module 1620, the character string output module 1630, and the connected component holding module 1640. The end determination module 1610 determines that the character string element for start as a connected component is not detected from the image as end. Specifically, the end determination module 1610 creates the start character string element 1615 from the connected component holding module 1640, and ends the processing when there is no connected component for creating the start character string element in the connected component holding module 1640. To do. The created start character string element 1615 is passed to the character string tracking module 1620.

The character string tracking module 1620 is connected to the end determination module 1610, the character string output module 1630, and the connected component holding module 1640. The character string tracking module 1620 is the above-described embodiment or a combination thereof.
The character string output module 1630 is connected to the end determination module 1610, the character string tracking module 1620, and the connected component holding module 1640. The character string output module 1630 outputs the character string element selected by the character string element observation module 140 (character string element 145 stored in the character string element holding module 160) as a character string. That is, the character string output module 1630 outputs a set of character string elements held by the character string tracking module 1620 as a character string.
The connected component holding module 1640 is connected to the end determination module 1610, the character string tracking module 1620, and the character string output module 1630. The connected component holding module 1640 corresponds to the aforementioned connected component holding module 150. Then, the character string tracking module 1620 updates the information in the connected component holding module 1640 so as to add information indicating that it has already been detected to the connected components included in the detected character string.

FIG. 17 is a conceptual module configuration diagram of a configuration example according to the sixth embodiment. In the sixth embodiment, a character string of an image is detected.
In the sixth embodiment, a character string in a document image is detected using the above-described embodiment. As shown in the example of FIG. 17, a connected component creation module 1710 and a character string cut-out module 1720 are used. And a connected component holding module 1730.
The connected component creation module 1710 is connected to the character string cutout module 1720 and the connected component holding module 1730. The connected component creation module 1710 creates a connected component 1717 for the received document image 1705. When all connected components in the document image 1705 are detected, a start signal 1715 is output.
The character string clipping module 1720 is connected to the connected component creation module 1710 and the connected component holding module 1730. The character string cutout module 1720 corresponds to the embodiment shown in the example of FIG. For illustration, the connected component holding module 1730 is taken out and shown.
The connected component holding module 1730 is connected to the connected component creation module 1710 and the character string cutout module 1720.

FIG. 18 is a conceptual module configuration diagram of a configuration example according to the seventh embodiment. The seventh embodiment is a character recognition device. That is, the character recognition apparatus uses the output of the above-described embodiment as a character recognition target.
As shown in the example of FIG. 18, the seventh embodiment includes a character string processing device 1810 and a character string recognition module 1820.
The character string processing device 1810 is connected to the character string recognition module 1820. The character string processing device 1810 corresponds to the embodiment shown in the example of FIG.
The character string recognition module 1820 is connected to the character string processing device 1810. The character string recognition module 1820 performs character recognition on the character string 1815 (character string 1725 in FIG. 17) output by the character string processing device 1810, and outputs a character recognition result 1825. The character string recognition result 1825 includes the position and text code of each single character with respect to the character string.

FIG. 19 is a conceptual module configuration diagram of an exemplary configuration according to the eighth embodiment. The eighth embodiment is a document distortion estimation / correction apparatus. In other words, the above-described embodiment is used for correcting the distortion of the document image. The output (character string) according to the above-described embodiment is accepted, and the character string is horizontally or vertically corrected to correct the distortion of the document image. To do.
As shown in the example of FIG. 19, the eighth embodiment includes a character string processing device 1910, a distortion amount estimation module 1920, and a distortion correction module 1930.
The character string processing device 1910 is connected to a distortion amount estimation module 1920 and a distortion correction module 1930. The character string processing device 1910 corresponds to the embodiment shown in the example of FIG.

The distortion amount estimation module 1920 is connected to the character string processing device 1910 and the distortion correction module 1930. The distortion amount estimation module 1920 calculates a distortion amount 1925 based on the character string output by the character string processing device 1910 (character string 1725 output by the character string cutout module 1720). That is, the distortion amount estimation module 1920 estimates the distortion amount 1925 of the character string necessary for correction with respect to the received character string 1915. For example, the center line of each character string is extracted and its inclination is detected.
The distortion correction module 1930 is connected to the character string processing device 1910 and the distortion amount estimation module 1920. The distortion correction module 1930 corrects the distortion of the document image 1905 including the character string based on the distortion amount 1925 calculated by the distortion amount estimation module 1920. In order to consider the distortion of a plurality of character strings, the processing is started after receiving an end signal 1917 indicating that the character string processing device 1910 has detected all the character strings. Based on the estimated distortion amount 1925, the distortion of the document image 1905 is corrected and a distortion corrected document image 1935 is output. Specifically, the distortion is corrected by performing affine transformation such as correcting the inclination in the reverse direction.

  A hardware configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 20 is configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 2017 such as a scanner and a data output unit 2018 such as a printer.

  A CPU (Central Processing Unit) 2001 includes various modules described in the above-described embodiments, that is, a prediction module 110, a connected component detection module 120, a character string element candidate creation module 130, a character string element observation module 140, a connected component. Execution of modules such as a selection module 926, multiple prediction region creation module 1206, end determination module 1610, character string output module 1630, connected component creation module 1710, character string recognition module 1820, distortion amount estimation module 1920, distortion correction module 1930, etc. It is a control part which performs the process according to the computer program which described the sequence.

  A ROM (Read Only Memory) 2002 stores programs, calculation parameters, and the like used by the CPU 2001. A RAM (Random Access Memory) 2003 stores programs used in the execution of the CPU 2001, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 2004 including a CPU bus.

  The host bus 2004 is connected via a bridge 2005 to an external bus 2006 such as a PCI (Peripheral Component Interconnect / Interface) bus.

  A keyboard 2008 and a pointing device 2009 such as a mouse are input devices operated by an operator. The display 2010 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 2011 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 2001 and information. The hard disk stores a target image, a connected component, a character string element candidate, a character string element, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 2012 reads data or a program recorded in a removable recording medium 2013 such as a mounted magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read from the interface 2007 and an external bus 2006. , The bridge 2005, and the RAM 2003 connected via the host bus 2004. The removable recording medium 2013 can also be used as a data recording area similar to the hard disk.

  The connection port 2014 is a port for connecting the external connection device 2015, and has a connection unit such as USB, IEEE1394. The connection port 2014 is connected to the CPU 2001 and the like via the interface 2007, the external bus 2006, the bridge 2005, the host bus 2004, and the like. The communication unit 2016 is connected to a communication line and executes data communication processing with the outside. The data reading unit 2017 is a scanner, for example, and executes document reading processing. The data output unit 2018 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus shown in FIG. 20 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 20, and the modules described in this embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 20 may be connected to each other via a communication line so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions of a scanner, a printer, a copying machine, a fax machine, etc.).

  Note that the above-described embodiments may be combined. For example, the second embodiment and the third embodiment may be combined.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray Disc (registered trademark), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 110 ... Prediction module 120 ... Connected component detection module 130 ... Character string element candidate creation module 140 ... Character string element observation module 150 ... Connected component holding module 160 ... Character string element holding module 926 ... Connected component selection module 1206 ... Multiple prediction area creation Module 1610 ... End determination module 1620 ... Character string tracking module 1630 ... Character string output module 1640 ... Connected component holding module 1710 ... Connected component creation module 1720 ... Character string cutout module 1730 ... Connected component holding module 1820 ... Character string recognition module 1920 ... Distortion estimation module 1930 ... Distortion correction module

Claims (9)

Detecting means for detecting a combination of connected components in the region of the target image;
Generating a plurality of character string element candidates that are character string element candidates that are candidates for character string elements that can form a character string, based on a combination of connected components detected by the detecting means, by a string element candidate creating means for creating a set of string element candidates,
From a set of string element candidates created by the string element candidate creating unit, that comprises a string element selection means for selecting a character string element candidates following the string elements that are selected in the past as a string element A featured image processing apparatus.   The character string element candidate creation means uses an ellipse inscribed in a circumscribed rectangle of the connected component as the connected component.
The image processing apparatus according to claim 1. A predicting means for predicting a region where a character string element following the character string element should be based on a character string element selected in the past;
The detecting device, an image processing apparatus according to claim 1 or claim 2, characterized in that to detect the combination of the connected components within the predicted region by said predicting means. A connected component selecting means for selecting a connected component by removing a connected component that is inappropriate as a character string element from the combination of connected components detected by the detecting means;
The string element candidate creating unit, the connected component selection means based on a combination of the selected connected component by any one of claims 1, characterized in that to create a set of string element candidates 3 An image processing apparatus according to 1. The prediction means creates a plurality of prediction regions,
The detecting device, an image processing apparatus according to claim 4 dependent on claim 3 or claim 3, characterized in that it is carried out combining the coupling components contained in the different prediction region. Output means for outputting the character string element selected by the character string element selection means as a character string;
The image processing device according to claim 1, characterized by comprising determining means for determining that the string element for starting a connected component from the image is not detected as the end in any one of 5 . The image processing apparatus according to claim 6 , further comprising: a character recognition unit that performs character recognition on the character string output by the output unit. Calculation means for calculating a distortion amount based on the character string output by the output means;
The image processing apparatus according to claim 6 , further comprising: a correcting unit that corrects distortion of the image including the character string based on the distortion amount calculated by the calculating unit. Computer
Detecting means for detecting a combination of connected components in the region of the target image;
Generating a plurality of character string element candidates that are character string element candidates that are candidates for character string elements that can form a character string, based on a combination of connected components detected by the detecting means, by a string element candidate creating means for creating a set of string element candidates,
From a set of string element candidates created by the string element candidate creating unit, to function as a string element selection means for selecting a character string element candidates following the string elements that are selected in the past as a string element Image processing program.

Images