JP6671613B2 - Character recognition method and computer program - Google Patents

Description

The present invention recognizes individual characters in an image in which many colors are used, and in particular, recognizes a character in which multiple colors are used, a gradation is applied, or a hatched character. And a character recognition method that enables the same.

In a moving image such as a television image, characters are often overlaid on the image, and a function of extracting only characters may be required. Many of recent images are color images, and usually a plurality of colors are used for the background and the character, so it is not easy to extract only the character from the target color image and recognize the character. . Patent Document 1 proposes a “character recognition device and image processing program” in which a character string is extracted from an image having a background.

JP 2015-184691 A

The invention described in Patent Literature 1 is suitable for character recognition and text processing of subtitles and documents of televisions and movies in which characters having the same color and size are arranged. However, according to the present invention, when characters are sparsely arranged in the entire image, the size of the characters is various, or the colors of the individual characters constituting the character string are different, for example, in a television, It is not suitable for recognizing characters such as telops in variety programs.

  An object of the present invention is to extract characters with high precision from a color image including characters having different positions, sizes, and colors, in addition to subtitles and documents, and recognize the characters.

The present invention is a character recognition method for recognizing characters included in an image, comprising: obtaining a plurality of binary images generated from a target image; and extracting a connected component from each of the binary images. Determining whether a combination of adjacent connected components (hereinafter, “connected component group”) is a character candidate to be subjected to character recognition; and subjecting the connected component group determined as the character candidate to a neural network, to determine whether a character or non-character, if the judgment result is a character and a character code thereof likelihood consists of a step of obtaining a likelihood non-character as long as it is a non-character, the teacher data for the neural network character The same code is assigned to the same character regardless of the typeface, and the teacher data for non-characters in the neural network is processed by the fractal generation process. The fractal graphic generated as described above and data obtained by randomly combining a plurality of characters are included . The present invention can be applied to any image, and is particularly effective in character recognition for a color image including various types of colors. Even if it is not a color image, it is also useful for recognizing characters in a grayscale monochrome image.

According to the image processing method of the present invention, whether a character is a character or a non-character is determined by using a neural network. There are various types of neural networks. In the following embodiments, a convolutional neural network (Convolutional Neural Network; hereinafter, "CNN") is used. Since character recognition is performed on a plurality of binary images, even if the background and the character are each a color image including a plurality of colors, high accuracy of the character recognition result can be expected. For example, a connected component may be lost in some binary images and character recognition may not be possible, but character recognition may be possible in another binary image. Before applying the connected component group to the neural network, it is simply determined whether or not there is a possibility of a character, so that a quick processing speed can be realized.

In the present invention ,
By assigning the same code to the same character, the generalization capability of the neural network is enhanced, and even a handwritten character by a different person can be recognized. It can also handle slightly designed characters that are not available in existing fonts.
In addition, since the non-character teacher data includes a fractal figure generated by the fractal generation process and data obtained by randomly combining a plurality of characters, a large amount of non-character teacher data can be generated quickly. Therefore, it is easy to extract noise included in the binary image.

In the present invention, the N groups (N> = 3) are classified by the K-means method, and the classified N groups are divided into two groups. Pixels included in one of the groups are white, and pixels included in the other are white. It is desirable that the ^2N -2 binary images displayed in black be subjected to character recognition. Since these 2 ^N −2 binary images include pairs of binary images in which white and black are inverted from each other, it is also possible to recognize white characters, characters with borders around them, and the like. .

In the present invention, it is desirable to output the result of the neural network determination of the character candidates obtained from each binary image together with the position and size of the circumscribed rectangle for each character candidate.

  Characters included in the input color image can be extracted from the background and can be recognized with high accuracy. In particular, it is possible to recognize even a character in which a plurality of colors are used or a gradation is applied to one character, or a character which is isolated in an image. Even in the case of monochrome, even if the characters are included in a grayscale image and the luminance differs for each character or within one character, character recognition can be performed with high accuracy.

It is a functional block diagram showing the composition of the character recognition device concerning the embodiment of the present invention. It is a figure which illustrates the character data for learning which concerns on embodiment of this invention. FIG. 6 is a diagram illustrating a fractal figure generated for learning a non-character according to the embodiment of the present invention. It is a figure which illustrates non-character data similar to the character for learning concerning embodiment of this invention. FIG. 4 is a diagram illustrating the number of binary images according to the embodiment of the present invention. FIG. 4 is a diagram for explaining that character recognition according to the embodiment of the present invention requires a plurality of binary images. FIG. 4 is a diagram for explaining that character recognition according to the embodiment of the present invention requires a plurality of binary images. 5 is a flowchart showing a processing flow of character recognition according to the embodiment of the present invention. FIG. 6 is a diagram illustrating that a binary image is scanned and an estimated character area is extracted according to the embodiment of the present invention. It is a figure for explaining the meaning of the “connection component” concerning the embodiment of the present invention. It is a figure for explaining labeling processing of a connected component by expansion and contraction processing concerning an embodiment of the present invention. FIG. 4 is a diagram for describing connected components forming a character candidate and a circumscribed rectangle thereof according to the embodiment of the present invention. It is a figure for explaining a character candidate excluded from character judgment by CNN concerning an embodiment of the present invention. It is a figure which illustrates the determination result by CNN of the character candidate which concerns on embodiment of this invention. FIG. 9 is a diagram for explaining a character recognition result obtained from a plurality of binary images according to the embodiment of the present invention.

The character recognition processing according to the embodiment of the present invention will be described below with reference to the drawings.
<< 1. Functional block configuration of character recognition device >>
<< 2. Preprocessing by character recognition device (machine learning (generation of CNN discriminator 22)) >>
<< 3. Preprocessing by character recognition device (generation of multiple binary images) >>
<< 4. Main processing by character recognition device (recognition of each character included in original image) >>

<< 1. Functional block configuration of character recognition device >>
A configuration focusing on the function of a computer that executes the present embodiment (hereinafter, referred to as a “character recognition device”) will be described with reference to FIG.
The character recognition device 1 is realized by a computer such as a personal computer and a smartphone, and a computer program installed in the computer .
The character recognition device 1 includes a processing unit 2, a storage unit 3, and a communication interface unit 4. In addition to these, an input operation unit such as a mouse and a keyboard used by the operator during operation, an output unit such as a display and a printer, a camera, and the like are appropriately provided, but not shown.

The storage unit 3 stores an input image to be processed, a learning sample for character identification, parameters such as various threshold values, various intermediate processing results by the processing unit 2, and the like, and is stored in a storage device such as a memory or a hard disk. Is achieved.
The intermediate processing result includes a pixel group of the estimated character area, a connected component, a character candidate, a character recognition result for each binary image, and the like.
The storage unit 3 also includes programs for causing a computer to function as the character recognition device 1. These programs are read into a memory, and the read program code is executed by a CPU (not shown), so that the processing unit 2 Each part operates.
Next, the processing unit 2 will be described.

  The processing unit 2 includes a machine learning data acquisition unit 20, a machine learning unit 21, a CNN discriminator 22, a binary image acquisition unit 23, an estimated character area scanning unit 24, a connected component extraction unit 25, a character A candidate selection unit 26, a character candidate recognition unit 27, and a character recognition result output unit 28 are provided. Hereinafter, each of the units 20 to 28 will be described.

The machine learning data acquisition unit 20 acquires character data and non-character data for machine learning from an external communication network or an information processing device via the communication interface unit 4. Although a fractal figure is used for learning non-character data, the fractal figure may be obtained from the outside, or a fractal figure generation unit 20a may be provided inside the character recognition device 1. In this embodiment, a description will be given assuming that a fractal figure as non-character data is generated by the fractal figure generating unit 20a.

The machine learning unit 21 performs learning using the data for machine learning, and causes the CNN discriminator 22 to store parameters obtained as a result. In the present embodiment, the character candidate input by the character candidate recognition unit 27 is determined as a character or non-character (noise) by the function of the CNN mounted on the CNN discriminator 22, and the determination result is determined by the character candidate recognition unit 27. Is returned to
For the machine learning, see “2. Preprocessing by character recognition device (machine learning (generation of CNN discriminator 22)) >> will be described in detail.

The binary image acquisition unit 23 acquires binary image data to be processed from an external communication network or an information processing device via the communication interface unit 4. However, an original color image may be obtained from the outside, and a binary image may be generated by the image binarization processing unit 23a provided inside the character recognition device 1. In this embodiment, a description will be given assuming that a binary image is generated by the image binarization processing unit 23a.
The generation of the binary image will be described later in << 3. Preprocessing by character recognition device (generation of plural binary images) >> will be described in detail.

The estimated character area scanning unit 24 scans one binary image in the vertical and horizontal directions starting from the upper left vertex, and extracts an estimated character area in which one or more characters are estimated to be collected.

The connected component extraction unit 25 extracts a connected component from the estimated character area. Some pixels of different characters may be connected to each other depending on the resolution or the like. Therefore, expansion and contraction processing is appropriately performed, labeling is performed by a known method, and each connected component is extracted.

The character candidate selection unit 26 determines whether a connected component group in which the circumscribed rectangles partially overlap or a connected component group in which there is no overlap and the distance between the circumscribed rectangles is small is appropriate for character recognition. Then, if appropriate, it is a character candidate. Only this character candidate is to be determined by the CNN discriminator 22.

The character candidate recognition unit 27 uses the CNN discriminator 22 to determine whether one or more connected component groups selected as character candidates are characters or non-characters. If the determination result is a character, a character code and its likelihood are obtained as an output of the CNN.

The character recognition result output unit 28 outputs the character recognition result to a printer, a screen, or the like provided in the character recognition device 1, or outputs the data as input data for subsequent text processing or the like.

<< 2. Preprocessing by character recognition device (machine learning (generation of CNN discriminator 22)) >>
This is a process in which learning data is acquired from outside or generated internally, machine learning is performed, and parameters obtained by learning are stored in the CNN discriminator 22.

The learning data includes character data and non-character data.
The sample corresponding to the character data, that is, the character code may be provided with an image in which the character is drawn. However, it is assumed that an image having as much variety as possible is prepared to improve the recognition accuracy. For example, a character to which the same character code is assigned is drawn using a large number of fonts.
FIG. 2 shows an example of character data. As the arithmetic numeral "3", various fonts and handwritten characters are stored in association with the same character code. In this way, the same code is set for the same character regardless of the typeface or whether the character is handwritten or handwritten. This increases the generalization ability of the CNN. If a different code is set depending on the typeface or the like, a problem of so-called overfitting, which cannot be applied to unlearned data, is likely to occur.

Among the character candidates, there are two types of data (non-character data) determined to be non-character.
The first is a natural object that appears in a live-action image, which has been subjected to character recognition as a result of binarization. The second is an object in which multiple characters are lined up as a group for character recognition. is there.

As the learning data for the first non-character pattern, a fractal figure often used as a simulation of a natural object can be used. A terrain of a mountain area is randomly created by a fractal terrain generation method, a binary image is generated by dividing the terrain by contour lines, and data for learning is appropriately extracted from the binary image. FIGS. 3A, 3B, and 3C show diagrams in which randomly generated mountains are divided by contour lines in ascending order of altitude. A portion surrounded by a broken-line rectangle in the drawing is an example of data that is arbitrarily selected and registered as non-character data. A positive integer code is assigned to character data codes, while a negative integer code is assigned to non-character data. This is because whether a character or a non-character can be immediately determined only by the sign of the code.
The reason why a fractal figure is used as non-character data corresponding to a natural object is that many non-characters, that is, noises, resemble a fractal figure.

The second non-character pattern is noise that resembles characters. This can be achieved by randomly generating an image in which characters are arranged in a grid or a triangle as illustrated in FIGS. 4A to 4B.
There is a possibility that kanji combining complex radicals may also match this learning data. However, even if there is such a character, the learning data corresponding to the character should match with a higher likelihood. For example, the non-character data in FIG. 4 (2a) is similar to the character data in FIG. 4 (3a). However, since both the non-character data and the character data are learned in the CNN of the present embodiment, it is considered that the likelihood in FIG.

<< 3. Preprocessing by character recognition device (generation of multiple binary images) >>
In the present embodiment, it is assumed that characters are extracted from a color image. If the original image is a binary image (not necessarily monochrome) or a monochrome document, only one binary image is required, but if it is a color image or grayscale image, multiple binary images are required. You need an image. Next, a procedure for generating a binary image in the present embodiment will be briefly described.

All pixels in the original image are classified into N (N is 3 or more) groups by the K-means method. Divide the N groups into groups of white pixels and groups of black pixels. The number N of groups may be appropriately determined in consideration of the number of colors used in the original color image and the processing speed and accuracy of character recognition. As described above, in the present embodiment, a plurality of binary images can be generated at the same time by the same algorithm of processing by the K-means method. In the example of FIG. 5, a group number N = 3, there are colored separately how are two ^3. However, if all the groups are white or black, they are excluded from the processing and six binary images (2) to (7) are processed. Note that the black pixels are hereinafter referred to as “foreground pixels”.

In FIG. 5, for example, (2) and (7) are simply inverted in black and white, so it may seem sufficient to perform the character recognition process on either one of the binary images. However, as illustrated in FIG. 6A, the image also includes bordered characters and outline characters. In this embodiment, since only black foreground pixels are targeted for character recognition, the outlined character "Z" in FIG. 6A may be excluded from character recognition. This is because the surrounding area is merely surrounded by a narrow foreground pixel, and this foreground pixel may be determined as a non-character, or may not be recognized as a character candidate in the first place. However, if an inverted binary image is prepared as shown in FIG. 6B, characters that are outlined in the original image are also subjected to character recognition.

In the present embodiment, even a single character must be able to recognize a character in which a plurality of colors are used or a gradation is applied. Therefore, it is significant that there are a plurality of binary images. For example, FIG. 7 shows a state where a gradation-applied capital letter “K” is binarized. Although it is difficult to specify the capital letter “K” alone in each of (a), (b), and (C) of FIG. 7, if the information obtained from these three binary images is combined, the capital letter “K” is obtained. It is possible to recognize.

<< 4. Main processing by character recognition device (recognition of each character included in binary image) >>
A description will be given according to the processing flow of FIG.
First, J binary images are acquired (step S10), and an initial value 1 is set to an image counter variable j (j = 1 to an integer of J) (step S11).

For the target binary image, the binary image is scanned to extract an estimated character area (step S12).
As shown in FIG. 9, first, the image is scanned vertically downward from the upper left vertex of the image. A region R1 in which the foreground pixels are spread in the horizontal direction is found. However, when the vertical length of the circumscribed rectangle of the foreground pixel group is equal to or less than a predetermined threshold, it is determined that the noise is noise, and is not subjected to character recognition, and scanning downward is restarted. If the height and width of the circumscribed rectangle of the region R2 are equal to or larger than a predetermined threshold value, it is estimated that the region includes one or more characters, and becomes a processing target after step S13.
In this way, some noise can be removed at the time of scanning the image.

A connected component is extracted from the estimated character area extracted in step S12 (step S13).
Here, the terms “connection” and “connected component” will be described with reference to FIG. Incidentally, the terms “connection” and “connected component” in the present invention are obtained by modifying the concept of connectivity in the phase space so that the concept can be applied to a set of pixels that is a discrete set.

If B is a set of foreground pixels in a set U = [1, W] × [1, H] based on all pixels of the binary image, then B⊆U. The foreground pixel surrounded by the broken ellipse in the figure is a source of the set B. Here, the adjacent relationship between the pixels is an important concept. ) And the case where the diagonal is treated as an adjacent point (8 concatenation). This may be chosen arbitrarily.
In FIG. 10A, p and q are adjacent to the pixel p, q, r∈B, and q and r are adjacent. When arbitrary pixels can be reached by tracing adjacent pixels as described above, this is called “connected”, and a subset C of B based on only these pixels is called “connected component”. . Similarly, the subset D of B is also a “connected component” (in the figure, the pixels that are the elements of the sets C and D are surrounded by a chain line ellipse). A common part between connected components such as the set C and the set D is an empty set.
One character is composed of one or more connected components. The character “A” in FIG. 10B is composed of only one connected component, and the character “Dan” in FIG. 10D is composed of 11 connected components. Note that, in one connected component, the true subset is not a connected component. For example, FIG. 10C is a set obtained by extracting a part of the pixels in FIG. 10B, and is no longer a connected component, and is not a processing target of the present embodiment.

Next, a method of extracting a connected component will be described.
A connected component can be easily extracted from the estimated character area by labeling the foreground pixels according to the adjacent relation. However, a situation in which a plurality of characters share a pixel often occurs due to restrictions on the resolution of an image or the like. In order to solve this problem, expansion / shrinkage processing is used.
In the example of FIG. 11, as shown in FIG. 11A, adjacent characters “ta” and “ke” are connected by a portion indicated by a broken line (FIG. 11B is an enlarged view of the broken line). . Therefore, as shown in FIG. 11C, the image is subjected to a contraction process. This separates pixels that should not be adjacent to each other. Labeling is performed on the contracted image, and the pixels connected to the obtained connected components P1 and P2, which have been removed by the contraction processing in the vicinity thereof, are added again. NP2.
Note that there is also a secondary effect that fine dirt and whiskers derived from image noise can be eliminated by the expansion / contraction processing.

A character consists of one or more connected components. Therefore, a connected component group that can be a character candidate is extracted before performing the character determination process by the CNN (step S14).
A character candidate refers to a connected component group that is presumed to constitute one character, and has a meaning that is sufficient to be determined by the CNN.
The character “ta” at the left end of the character string illustrated in FIG. 12 includes three connected components Pa, Pb, and Pc. If the CNN is to perform determination processing on seven elements excluding the empty set of the power set of the set {Pa, Pb, Pc}, this is not desirable in terms of processing speed. Therefore, in the present embodiment, the circumscribed rectangle of the connected component is used as follows.

The character string in FIG. 12A is composed of connected components of Pa, Pb,..., Ph, and the circumscribed rectangle of each connected component is represented by rPa, rPb,. And Scanning starts from the X coordinate of the upper left vertex of the circumscribed rectangle rPa located at the left end. Since the circumscribed rectangle rPa overlaps the circumscribed rectangles rPb and rPc, the connected component group (Pa, Pb, Pc) inside the rectangle Rect1 including these three circumscribed rectangles is a character candidate (at this stage, a temporary It is just a character candidate).
The circumscribed rectangle rPd is on the right side of the circumscribed rectangles rPb and rPc, but since the X coordinates (x3 and x4) are far apart, Rect1 does not include the circumscribed rectangle rPd.
Subsequently, scanning is horizontally restarted rightward from the X coordinate x4 of the upper left vertex of the circumscribed rectangle rPd. The values of the X coordinates x5, x6, x7 and x8 of the upper right vertex of the circumscribed rectangle located on the right side from the starting point x4 are extracted. Since the width of the circumscribed rectangle rPd is narrow (x5−x4), the connected component group (Pd, Pe) inside the rectangle Rect2 that also includes the circumscribed rectangle rPe on the right is used as a temporary character candidate. Furthermore, the connected component group (Pd, Pe, Pf) inside the rectangle Rect3 that also includes the circumscribed rectangle rPf on the right may be used as a temporary character candidate. Since the upper right X coordinate x8 of the circumscribed rectangles rPg and rPh at the right end of the character string is too far from the X coordinate x4 of the start position, the connected component group (Pd, Pe, Pf, Pg, and Ph) are not temporary character candidates.

  Although only the comparison between the x-coordinates of the circumscribed rectangles has been described to avoid complexity, the comparison between the y-coordinates is also natural. For example, when focusing on the connected component Pg, since the circumscribed rectangles of the connected component Ph are vertically adjacent to each other, the connected component group (Pg, Ph) inside the rectangle Rect5 including the circumscribed rectangles rPg and rPh is also temporary. Character candidates.

  If a method of selecting a character candidate using such a circumscribed rectangle is adopted, there is a possibility that the recognition accuracy is affected by noise mixed in the rectangle. However, this embodiment does not pose a problem for the following reason. That is, first, due to the characteristics of the binarization method, even if noise is mixed in a certain binary image, in almost all cases, if almost the same rectangular portion of another image is extracted, a character without noise can be obtained. Because. Second, although CNN is used for character determination, it can be trained to be resistant to such noise as a feature thereof. If learning with high generalization ability has been achieved, even if only images containing noise are obtained as recognition target images, the recognition results will be slightly low likelihood and the quality of the final result will be significantly affected. It is not considered.

The above is the basic method of determining a character candidate.
However, among the character candidates obtained, there are many that need not be subjected to character identification by CNN, but can be determined only by simple determination and that a character is not formed. Therefore, before applying to CNN, character candidates to be subjected to character identification are selected (step S15). Thus, narrowing down the number of character candidates by a simple determination method is effective for speeding up the entire processing.
The following is an example of such a determination method.

(1) If the connected components that are in contact with any of the upper, lower, left, and right ends of the circumscribed rectangle are too small, they are excluded from the connected components constituting the character candidate (c1 in FIG. 13A is excluded, and c2 is excluded). Character candidates).
(2) The upper limit and the lower limit of the size of the circumscribed rectangle are set in advance, and character candidates exceeding the upper limit or below the lower limit are excluded (c3 and c4 in FIG. 13B).
(3) Exclude circumscribed rectangles having an extreme aspect ratio. For example, c6 in FIG. 13C has an aspect ratio of 1: 2, and it is highly likely that the character is not one character even when compared with the adjacent character candidate c5. However, some characters have an extreme aspect ratio (such as the Chinese character "one"), which is a balance with consideration for them. For example, in applications where recognition accuracy is more important than processing speed, the determination based on the aspect ratio may be omitted.
(4) Those containing too many connected components are excluded from character candidates (c7 in FIG. 13D).
(5) Exclude the case where the sum of the areas of the circumscribed rectangles of the respective connected components is too small with respect to the area of the entire circumscribed rectangle (c8 in FIG. 13E). Here, the reason why the determination is not made based on the ratio of the number of pixels is to prevent characters such as “mouth” from being excluded.

The above methods (1) to (5) for determining the suitability as a character candidate are merely examples. The point is that it is only necessary to be able to select character candidates to be subjected to character identification processing using CNN in consideration of the accuracy of character recognition and the processing speed.

Subsequently, the character candidate (one or more connected component groups) which is simply determined to be a character is applied to the CNN (step S16).
A character candidate is input to a CNN that has been trained with previously prepared character data and non-character data. If the CNN determines that the input data is a character, the CNN returns the character code and likelihood. If it is determined that the data is not a character, the CNN returns a determination result of “not a character” along with the non-character likelihood. In this embodiment, since the CNN learns both the character data and the non-character data, it is possible to simultaneously determine whether the character is a character or a non-character (= noise) with the likelihood.

FIG. 14A shows an output result when a character is determined, but the combination of the character code and the likelihood is not always one. Since it is difficult to obtain a single determination result from one binary image, it is sufficient that character code candidates can be acquired in the order of likelihood. FIG. 14B shows a determination result when it is estimated that the character is not a character.
These output results are stored in the storage unit 3 together with the (upper left) position and the vertical and horizontal sizes of the circumscribed rectangle of the character candidate, and are referred to in subsequent processing. Here, only the connected component group determined to be a character with high likelihood may be output.
Inconsistent judgments of characters and non-characters may be returned for one connected component group. However, since the recognition results of all the binary images are finally integrated, a proper judgment can be obtained. That is, depending on one binary image, it may be a character or a non-character, or in the case of a character, the character code may not be clearly determined.

If the processing has not been completed for all the estimated character areas included in one binary image (No in step S17), the process returns to step S12, and the image is scanned to extract the next estimated character area. . In the case of the example of FIG. 9, if the scanning in the vertical direction has been completed, the image is scanned in the horizontal direction from the upper left coordinates to the right. An area R3 in which foreground pixels are spread in the vertical direction is found, but if the horizontal length is equal to or less than a predetermined threshold, it is determined that the noise is noise and is not subjected to character recognition. The scanning in the horizontal direction is continued, and if the height and width of the circumscribed rectangle of the region R4 are equal to or larger than a predetermined threshold value, it is estimated that the region includes one or more characters, and the processes after step S13 are executed.
If the character recognition processing for one binary image has been completed (Yes in step S17), it is determined whether the processing for all J binary images has been completed. If the processing has not been completed (No in step S18), the variable j is incremented (step S19), and the process returns to step S12 to scan the j-th image to extract an estimated character area.

If the character recognition of the foreground pixel has been completed for each of all the binary images (Yes in step S18), the character recognition results for all the binary images are output to a screen or a printer, or another processing system. Output to the server (step S20). For example, text processing considering the context. This subsequent processing may be performed by another information processing device, or may be performed inside the character recognition device 1. FIG. 15 illustrates a recognition result of a connected component group at substantially the same position in a plurality of binary images. Although the judgment result of CNN differs depending on the binary image, how to use these judgment results depends on the subsequent processing.

Hereinabove, one embodiment of the present invention has been described. However, the present invention is not limited to this embodiment, and various embodiments can be considered without departing from the scope of the claims. For example, a plurality of binary images is assumed, but the present invention can of course be applied to one binary image. Further, in order to generate a binary image from one color image or grayscale image, it is not always necessary to use the k-means method. Further, the processing flow illustrated in FIG. 8 is merely an example, and it goes without saying that the determination result may be output each time character recognition is performed on one image.

  Characters included in color images can be recognized with high accuracy, and wide use is expected as a basic technology for extracting text from television telops, road traffic signs, signboards, and the like.

1: Character recognition device 2: Processing unit 20: Machine learning data acquisition unit 21: Machine learning unit 22: CNN discriminator 23: Binary image acquisition unit 24: Estimated character area scanning unit 25: Connected component extraction unit 26: Character Candidate selection unit 27: Character candidate recognition unit 28: Character recognition result output unit 3: Storage unit 4: Communication interface unit

Claims (2)

A character recognition method for recognizing characters included in an image, comprising: obtaining a plurality of binary images generated from a target image; extracting connected components from each of the binary images; Determining whether the combination of components (hereinafter, “connected component group”) is a character candidate to be subjected to character recognition; and subjecting the connected component group determined as the character candidate to a neural network to perform a character or non-character or by determining, if the judgment result is a character and a character code thereof likelihood consists of a step of obtaining a likelihood non-character as long as it is a non-character, the teacher data for the neural network character, typeface The same code is assigned to the same character regardless of the difference between them. The teacher data for non-characters in the neural network is generated by the fractal generation process. A character recognition method , comprising: a fractal pattern obtained by combining a plurality of characters ; It is classified into N (N> = 3) groups by the K-means method, and the classified N groups are divided into two groups. Pixels included in one of the groups are displayed in white, and pixels included in the other are displayed in black. 2. The character recognition method according to claim 1, wherein the ^2N −2 binary images are subjected to character recognition. 3.

Images