JP4995507B2 - Image processing system, character recognition system, and image processing program - Google Patents

Description

  The present invention relates to an image processing system, a character recognition system, and an image processing program.

As a technique for extracting character information by scanning a paper document, OCR (Optical Character Recognition) is used. Characters can be classified into handwritten characters and printed characters, but it is necessary to use a dedicated OCR method for obtaining high recognition accuracy. In particular, in order to recognize handwritten characters, it is necessary to designate an area in which characters are present in advance.
On the other hand, many documents handled by humans are a mixture of printed characters and handwritten characters. Various application forms and questionnaires are typical examples. In order to fully automate these processes from scanning to character information extraction, it is necessary to automatically distinguish between the printed characters and the handwritten characters.
In order to realize this, the conventional technique accurately cuts out each character from the image, creates a cut out circumscribed rectangle, measures the length of the vertical and horizontal sides, and based on this, calculates the vertical and horizontal directions. It was distinguished from type characters by evaluating the ratio of sides, the variance of side lengths, and the variation in distance between circumscribed rectangles.
However, these methods are based on the premise that type characters can surely create a circumscribed rectangle for each character, or the arrangement is uniform. However, there are many cases in which this assumption is not actually satisfied. In addition, these methods have strong language dependency.

As a technique related to these, for example, in Patent Document 1, focusing on the fact that handwritten characters vary in pitch as compared to printed characters, a technique for cutting out character lines and examining the pitch variation in the cut out character lines Is disclosed.
Further, for example, Patent Document 2 relates to a technique for determining by the aspect ratio of a circumscribed rectangle of a character, and determining by the arrangement mode of black pixels adjacent to white pixels (horizontal / vertical pixel ratio, curved / straight pixel ratio, etc.). Technology is disclosed.
Further, for example, Patent Document 3 discloses a technique for determining a characteristic parameter of a character and a distance obtained from the dictionary and determining an average variation in the distance.
Further, for example, Patent Document 4 focuses on the fact that handwritten characters have relatively few line segments extending in the horizontal direction, and a technique for determining based on the proportion of pixels in the horizontal direction component among the pixels constituting the character. It is disclosed.
Further, for example, Patent Document 5 discloses a technique for making a comprehensive determination based on the dispersion of various feature amounts (character height, pitch, area, etc.).
Further, for example, in Patent Document 6, it is determined whether or not the center of gravity of each label is aligned on a straight line with respect to a character line, and if they are aligned, the length of the circumscribed rectangle of each label in the long side direction is substantially equal. A technique for determining whether or not a character string is a character string is disclosed.
Further, for example, Patent Document 7 relates to extraction of a texture region, and a texture image is regularly scattered with spatial frequencies having superior power in the spatial frequency region. Focusing on the fact that the regularity of the dominant frequency is not seen in the power spectrum of the non-existing image, the change in the image feature amount of the texture image is examined and changed from the already extracted texture area to the outside. A technique is disclosed in which a one-dimensional power spectrum is adopted as an image feature amount by determining that the edge of the image is an edge portion.
In addition, for example, Patent Document 8 relates to determination of a character region. Generally, the periphery of a character is usually plain for easy viewing of the character, and in the case of a background pattern that is not a character. The image is divided into blocks of a predetermined size, orthogonal transform is performed to obtain a spatial frequency distribution, and blocks whose power is greater than a predetermined threshold value. If the power of the high spatial frequency region of a block in the vicinity of the candidate is smaller than a predetermined threshold and the powers of the low spatial frequency region are equal to each other, the character region candidate is selected as a character region candidate. A technique for determining an area is disclosed.
Japanese Patent Laid-Open No. 57-111679 JP 58-037775 A Japanese Patent Laid-Open No. 60-138692 JP-A 63-103389 Japanese Unexamined Patent Publication No. 01-014682 JP 2000-339404 A Japanese Patent Laid-Open No. 07-121699 Japanese Patent Laid-Open No. 09-186861

  The present invention has been made in the background of such a background art, so that a handwritten character region or a printed character region can be determined from an image that is unknown to be a handwritten character region or a printed character region. An object of the present invention is to provide an image processing system, a character recognition system, and an image processing program.

The gist of the present invention for achieving the object lies in the inventions of the following items.
[1] Feature extraction means for extracting a predetermined feature from an image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area;
In accordance with the partial space estimated by the partial space estimation means, comprising an area determination means for determining a handwritten character area or a printed character area on an actual image ,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing system, which is a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result .

[2] Feature extraction means for extracting predetermined features from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generating means, an index setting means for setting an index representing the handwriting character area-likeness or the print character area-likeness;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area according to the index set by the index setting means;
In accordance with the partial space estimated by the partial space estimation means, comprising an area determination means for determining a handwritten character area or a printed character area on an actual image ,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing system, which is a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result .

[ 3 ] Feature extraction means for extracting a predetermined feature from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area;
In accordance with the partial space estimated by the partial space estimating means, an area determining means for determining a handwritten character area or a printed character area on an actual image;
Using a handwritten character area or a printed character area determined by the area determination means, and comprising a layout analysis means for analyzing a layout from the image ,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing system, which is a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result .

[ 4 ] Feature extraction means for extracting predetermined features from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area;
In accordance with the partial space estimated by the partial space estimating means, an area determining means for determining a handwritten character area or a printed character area on an actual image;
Comprising layout analysis means for analyzing the layout from the image,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
It is the sum of the spectral intensities weighted by the distance from a predetermined axis in the frequency plane of the two-dimensional Fourier transform result,
The image processing system characterized in that the area determination means determines a handwritten character area or a printed character area using the layout analyzed by the layout analysis means.

[ 5 ] Handwritten character recognition means or typeface character recognition for recognizing a handwritten character area or a printed character area determined by the area determining means of the image processing system according to [1], [2], [3] or [4] A character recognition system comprising: means.

[ 6 ] The area determining means determines the handwritten character area or the printed character area using the character recognized by the handwritten character recognizing means or the probability of the character recognized by the printed character recognizing means. The character recognition system according to [ 5 ].

[ 7 ] To the computer,
A feature extraction function for extracting predetermined features from an image;
A texture space generation function for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction function;
In the texture space generated by the texture space generation function, a partial space estimation function that estimates a partial space that is a handwritten character region or a partial space that is a printed character region;
In accordance with the partial space estimated by the partial space estimation function, an area determination function for determining a handwritten character area or a printed character area on an actual image is realized ,
The feature amount extracted by the feature extraction function is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing program characterized by a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result .

[ 8 ] To the computer,
A feature extraction function for extracting predetermined features from an image;
A texture space generation function for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction function;
In the texture space generated by the texture space generation function, an index setting function for setting an index representing the handwriting character area-likeness or the typeface character area-likeness;
In the texture space generated by the texture space generation function, a partial space estimation function for estimating a partial space that is a handwritten character area or a partial space that is a printed character area according to an index set by the index setting function;
In accordance with the partial space estimated by the partial space estimation function, an area determination function for determining a handwritten character area or a printed character area on an actual image is realized ,
The feature amount extracted by the feature extraction function is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing program characterized by a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result .

According to the image processing system, the character recognition system, and the image processing program according to the present invention, any of the following effects can be achieved as compared with the case where the present configuration is not provided.
(1) A conventional character segmentation process is not required. Since it is not necessary to cut out each character accurately on the image, it does not depend on the character cut-out process, which is a factor that makes the determination difficult.
(2) Experience data is not always necessary. For example, pre-learning using a neural network and data prepared in advance such as a template are not required.
(3) Low language dependency. In order to handle fluctuations caused by human writing, it is not necessary to handle the characteristics of each language, and it is not necessary to ask the type of language.

First, an outline of the present embodiment will be described.
In the present embodiment, attention is paid to the feature that the line segments constituting the handwritten character have more “fluctuation” than the line segment in the printed character regardless of the language. We also devised a feature that accurately represents this fluctuation. This feature amount is obtained for each pixel or region of the image. A plurality of feature quantities may be used. A texture information space is generated using the obtained feature amount as a texture element. In the generated texture information space, a boundary of the space is obtained based on a predetermined index, and the texture information space is divided into a plurality of partial spaces based on the obtained boundary. To divide. A region on the actual image corresponding to a partial space that is considered to represent a handwritten character is determined as a handwritten character region. Alternatively, an area on the actual image corresponding to a partial space considered to represent a printed character is determined as a printed character area.
According to the present embodiment, since it is not necessary to accurately cut out individual characters, it is possible to determine a printed character region and a handwritten character region without depending on the character cutout process that is a factor that caused the determination to vary. It becomes. Moreover, experience data is not necessarily required. For example, pre-learning using a neural network and data prepared in advance such as a template are not required. Furthermore, since attention is paid to “line segment fluctuation” unique to handwritten characters, it is possible to deal with various languages (notation characters).
Note that texture is texture (texture), is quantified by a statistical index of pixel values, and is repeatedly arranged under a statistical property that is not completely periodic, for example. The pattern that can be done.

A document in which printed characters and handwritten characters targeted by the present embodiment are mixed will be described with reference to FIG. There are many documents used in business where a form is defined and the form is handwritten. For example, it is a document such as an estimate response shown in FIG. In this document, items to be entered in advance, such as “quantity / unit”, “unit price excluding tax (yen)”, “total amount excluding tax (yen)”, etc. are determined. It is filled in. If one sees this document, the difference between printed characters and handwritten characters is obvious. The reason for this at a glance is due to the fluctuation of characters. In other words, regardless of language, handwritten characters are characterized by many fluctuations caused by human writing. Character fluctuation refers to a deviation from a straight line or a smooth curve of a line segment, a non-uniformity in the size of each character, a deviation in arrangement, or the like.
Therefore, in the present embodiment, it is used that type characters have many straight line segments but handwritten characters have few.

The outline of the processing procedure of the present embodiment will be described with reference to FIG. The processing procedure consists of two modules. The first is a Fourier transform module 31 that performs a two-dimensional Fourier transform to analyze the texture of the input image (document data), and the second is the frequency output from the Fourier transform module 31. This is a handwritten character degree evaluation module 32 that calculates an evaluation amount of fluctuation for a region image (texture feature amount) and determines whether the character is a handwritten character or a printed character according to the evaluation amount.
As shown in FIG. 3, the Fourier transform module 31 inputs an input image 33 and creates a frequency domain image (texture feature amount) 34. Then, the handwritten character degree evaluation module 32 receives the frequency domain image (texture feature amount) 34 and finally outputs the handwritten character degree (evaluation value) E35.

The Fourier transform module 31 applies a two-dimensional Fourier transform to the image (document image data) to obtain a frequency domain image of the original data. A frequency domain image is an image that represents the spectral intensity of each frequency that constitutes a character, for example, when there is a character image, considering that the character is generated by collecting waves of various frequencies. .
The formula for defining the two-dimensional Fourier transform is shown in Equation 1.

  For example, the sizes of the image data of the input image 33 and the frequency domain image (texture feature amount) 34 are both 64 × 64 pixels. In the image data, the lower left of the image is the origin, the horizontal axis is the x axis, and the vertical axis is the y axis. The frequency domain image has a spectrum in the x-axis direction of the input image as the u-axis and a spectrum in the y-axis direction as the v-axis, and is centered at the origin where the respective frequencies are zero. The possible range of the spectrum intensity obtained by applying the two-dimensional Fourier transform to the image data is represented by a grayscale bitmap image, and is expressed as black when the spectrum intensity is high, and as white when the spectrum intensity is low (FIG. 3). Frequency domain image (texture feature amount) 34).

Ｅを手書き文字度と呼ぶことにする。
この抽出方法では、二次元フーリエ変換結果の周波数領域画像において、軸周りのピクセルのスペクトル強度を除き（重みを０にし）、その重み付け後の和を取っている。軸周りのピクセル強度を除くということは、元の文字画像データにおける水平、鉛直の直線成分に起因するスペクトル強度を除くということであり、それはすなわち手書き文字の揺らぎによって生じるスペクトル強度の非集中性を検出することができることを意味している。それにより漢字について手書き文字と活字文字を区別できることとなる。 The handwritten character degree evaluation module 32 calculates an evaluation amount of fluctuation from the frequency domain image, and distinguishes between handwritten characters and printed characters according to the evaluation amount.
The evaluation method for fluctuations peculiar to handwritten characters is as follows.
For example, when a human draws a straight line, it does not become a completely straight line, but a line that includes fluctuations. In the frequency domain image, a change in spectral intensity appears on the v-axis, but if fluctuation is included, the change in spectral intensity diffuses not only on the v-axis but also around it. Appear. In this example, extraction of components other than vertical and horizontal straight line components of image and document data will be described, targeting a Chinese character composed of horizontal and vertical line components. An example is shown in Equation 2.

Let E be called handwritten character degree.
In this extraction method, in the frequency domain image as a result of the two-dimensional Fourier transform, the spectral intensity of pixels around the axis is removed (weight is set to 0), and the weighted sum is taken. Excluding the pixel intensity around the axis means removing the spectral intensity caused by horizontal and vertical linear components in the original character image data, which means that the spectral intensity deconcentration caused by fluctuations in handwritten characters is eliminated. It means that it can be detected. As a result, handwritten characters and type characters can be distinguished for kanji.

An example of processing in which two-dimensional Fourier transform is performed on type characters and handwritten characters will be described with reference to FIG. FIG. 4A shows the result of performing a two-dimensional Fourier transform on the “wealth” type character (left side of FIG. 4A) (right side of FIG. 4A). FIG. 4C shows the result of performing two-dimensional Fourier transform on the handwritten character “wealth” (left side of FIG. 4C) (right side of FIG. 4C). From these, it can be seen that the printed characters have a spectrum concentrated on the U and V axes, and that the handwritten characters have a spectrum dispersed at positions away from the U and V axes.
Note that the word “wet” is composed of vertical and horizontal line segments, but the same result is obtained even for characters with diagonal line segments such as “ken” (FIG. 4B). FIG. 4D).

FIG. 1 is a conceptual module configuration diagram of an embodiment.
The module generally refers to a component such as software or hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a program but also a module in a hardware configuration. Therefore, the present embodiment also serves as an explanation of a program, a system, and a method. In addition, the modules correspond almost one-to-one with the functions. However, in mounting, one module may be composed of one program, or a plurality of modules may be composed of one program. A plurality of programs may be used. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Hereinafter, “connection” includes not only physical connection but also logical connection.
In addition, the system includes a configuration in which a plurality of computers, hardware, devices, and the like are connected via a network or the like, and includes a case where the system is realized by a single computer.

  In this embodiment, as shown in FIG. 1, an image inclination correction module 101, a resolution conversion module 102, a handwritten character feature extraction module 103, a texture space generation module 104, a subspace estimation module 105, a character type determination module 106, an image A binarization module 107, a layout analysis module 108, a handwritten character recognition module 109, and a type character recognition module 110 are provided. In FIG. 1, the flow of images is indicated by black arrows, and the flow of information is indicated by arrows. That is, the image inclination correction module 101 to the resolution conversion module 102, the image inclination correction module 101 to the image binarization module 107, the resolution conversion module 102 to the handwritten character feature extraction module 103, and the texture space generation module 104 to estimate the partial space. An image is passed from the image binarization module 107 to the layout analysis module 108, handwritten character recognition module 109, and type character recognition module 110 to the module 105. Information is passed between other modules.

  The image inclination correction module 101 is connected to the resolution conversion module 102 and the image binarization module 107. The image inclination correction module 101 receives document image data, detects the inclination of the document image data, and corrects the inclination by affine transformation. In general, when a document is input using a scanner, inclination or the like occurs. Since the inclination adversely affects the subsequent processing, it is corrected as preprocessing. The detection of the tilt includes, for example, a method of extracting a straight line existing in the document and rotating the entire image in a direction to make the straight line horizontal or vertical when the tilt is very small. The input image here is, for example, a color image, and one pixel is expressed by 24 bits. The image after the inclination correction processing is transferred to the resolution conversion module 102 and the image binarization module 107.

  The resolution conversion module 102 is connected to the image inclination correction module 101 and the handwritten character feature extraction module 103. That is, the document image data is received from the image inclination correction module 101, resolution conversion is performed, and the resulting image data is passed to the handwritten character feature extraction module 103. Here, resolution conversion (enlargement / reduction processing) is performed for the purpose of normalizing the target image. Image data having a resolution sufficient for feature extraction performed by the handwritten character feature extraction module 103 is passed.

  The handwritten character feature extraction module 103 is connected to the resolution conversion module 102 and the texture space generation module 104. In other words, the image data is received from the resolution conversion module 102, the feature amount (specifically, the handwritten character degree E calculated by the above-described equation 2) representing the fluctuation of the handwritten character is extracted and passed to the texture space generation module 104. .

  The texture space generation module 104 is connected to the handwritten character feature extraction module 103 and the partial space estimation module 105. That is, the feature amount of the handwritten character is received from the handwritten character feature extraction module 103, a texture space is generated, and the texture space is passed to the subspace estimation module 105. Here, the texture space generation module 104 maps the texture information of each pixel or each region into the texture space according to the feature extracted by the handwritten character feature extraction module 103. Then, the texture space in which the information of each pixel or each region is mapped is passed to the partial space estimation module 105.

  The partial space estimation module 105 is connected to the texture space generation module 104 and the character type determination module 106. That is, a texture space in which information of each pixel or each region is mapped is received from the texture space generation module 104, and a partial space that is a handwritten character region is estimated. Note that the estimation here is not a human brain process but a provisional determination that the handwriting character area is likely to be a partial space in the present embodiment. In this estimation, first, an index representing the likelihood of a handwritten character area is set, and a partial space that is a handwritten character area is estimated in the texture space generated by the texture space generation module 104 according to the set index. Then, the range information estimated to be highly likely to be a handwritten character area is passed to the character type determination module 106.

  The image binarization module 107 is connected to the image inclination correction module 101, the layout analysis module 108, the handwritten character recognition module 109, and the type character recognition module 110. That is, the binarization processing is performed on the document image data transferred from the image inclination correction module 101. This is because, in the processing of the layout analysis module 108, the handwritten character recognition module 109, and the printed character recognition module 110, a binary image is suitable instead of a multi-valued image. In the binarization processing, a threshold value is determined and a black pixel and a white pixel are determined. However, the threshold value may be adaptively determined. This binarized image (1 pixel is expressed by 1 bit) is passed to the layout analysis module 108, the handwritten character recognition module 109, and the type character recognition module 110.

  The layout analysis module 108 is connected to the image binarization module 107, the character type determination module 106, the handwritten character recognition module 109, and the type character recognition module 110. That is, a binarized image is received from the image binarization module 107, a document layout analysis process is performed on the received binarized image, and the result is passed to the handwritten character recognition module 109, the type character recognition module 110, or the character type determination module 106. . Layout analysis processing can be divided into a certain structure in a document, that is, character area, graphic area, image area, etc., and among them, approximate contents are estimated by the appearance of titles, paragraphs, notes, etc. is there. The area determined to be a character area by this layout analysis processing is passed to the handwritten character recognition module 109 or the type character recognition module 110. Further, by passing information related to the area determined to be a character area to the layout analysis module 108, it is possible to refer to the character type processing by the character type determination module 106. Conversely, by obtaining character type information in the area from the character type determination module 106, the information can be used as a reference for layout analysis processing. For example, the possibility that the character area is not a character area as a layout analysis process is not so high, but if it cannot be determined that it is clearly not a character area, the character type determination module 106 may determine that it is a handwritten character area. If it is found that the area is high, the area may be determined as a character area.

  The handwritten character recognition module 109 is connected to the image binarization module 107, the layout analysis module 108, and the character type determination module 106. That is, the binarized image is received from the image binarization module 107, the coordinates of the character area are received from the layout analysis module 108, and the coordinates of the handwritten character area are received from the character type determination module 106. And the character recognition process suitable for a handwritten character is performed with respect to the area | region, and the result (text code) is output as handwritten character information. Further, the probability (reliability) of the character recognition result may be output to the layout analysis module 108 or the character type determination module 106. For example, the reliability value of a character described in Japanese Patent No. 299979 can be applied.

  The type character recognition module 110 is connected to the image binarization module 107, the layout analysis module 108, and the character type determination module 106. That is, the binarized image is received from the image binarization module 107, the coordinates of the character area are received from the layout analysis module 108, and the coordinates of the printed character area are received from the character type determination module 106. And the character recognition process suitable for a printed character is performed with respect to the area | region, and the result (text code) is output as printed character information. Further, the probability (reliability) of the character recognition result may be output to the layout analysis module 108 or the character type determination module 106.

  The character type determination module 106 is connected to the partial space estimation module 105, the layout analysis module 108, the handwritten character recognition module 109, and the type character recognition module 110. That is, it receives information of a range estimated to have a high possibility of a handwritten character area from the subspace estimation module 105, receives the coordinates of the character area from the layout analysis module 108, and determines whether the area is a handwritten character area or a printed area. Determine if there is. Note that the likelihood of the result of character recognition is received from the handwritten character recognition module 109 and the typeface character recognition module 110, and the threshold value for determining whether the character is a handwritten character or type character is corrected using this information. Also good.

  With reference to FIG. 5, a hardware configuration example of the present embodiment will be described. The configuration illustrated in FIG. 5 is an image processing system configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 417 such as a scanner and a data output unit 418 such as a printer. .

  A CPU (Central Processing Unit) 401 includes various modules described in the above-described embodiments, that is, modules such as the image inclination correction module 101, the resolution conversion module 102, the handwritten character feature extraction module 103, and the texture space generation module 104. It is a control part which performs the process according to the computer program which described the execution sequence.

  A ROM (Read Only Memory) 402 stores programs used by the CPU 401, operation parameters, and the like. A RAM (Random Access Memory) 403 stores programs used in the execution of the CPU 401, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 404 including a CPU bus.

  The host bus 404 is connected to an external bus 406 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 405.

  A keyboard 408 and a pointing device 409 such as a mouse are input devices operated by an operator. The display 410 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text and image information.

  An HDD (Hard Disk Drive) 411 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 401 and information. The hard disk stores input images, binarized images, feature amounts, character recognition result data, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 412 reads data or a program recorded on a mounted removable recording medium 413 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read from the interface 407 and the external bus 406. , And supplied to the RAM 403 connected via the bridge 405 and the host bus 404. The removable recording medium 413 can also be used as a data recording area similar to a hard disk.

  The connection port 414 is a port for connecting the external connection device 415 and has a connection unit such as USB, IEEE1394. The connection port 414 is connected to the CPU 401 and the like via the interface 407, the external bus 406, the bridge 405, the host bus 404, and the like. The communication unit 416 is connected to a network and executes data communication processing with the outside. The data reading unit 417 is a scanner, for example, and executes document reading processing. The data output unit 418 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration illustrated in FIG. 5 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 5, and is configured to execute the modules described in the present embodiment. I just need it. For example, some modules may be configured by dedicated hardware (for example, ASIC), and some modules may be in an external system and connected via a communication line. A plurality of systems shown in FIG. 5 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (also called a multi-function copying machine, which has functions of a scanner, a printer, a copying machine, a fax machine, etc.).

Next, the function and operation (operation) will be described.
Processing for recognizing a document image will be described with reference to FIG.
In step S601, the image inclination correction module 101 corrects the inclination of the document image input. In addition to inclination correction, image processing as preprocessing for character recognition such as noise removal may be performed.
In step S602, the resolution conversion module 102 performs resolution conversion on the document image corrected in step S601.
In step S603, the handwritten character feature extraction module 103 extracts the fluctuation feature amount that is the feature of the handwritten character from the document image whose resolution has been converted in step S602.

In step S604, the texture space generation module 104 generates a texture space using the feature amount extracted in step S603.
In step S605, the partial space estimation module 105 estimates a partial space that is a handwritten character region in the texture space generated in step S604.
In step S606, the character type determination module 106 determines the handwritten character area estimated in step S605. However, if the result is No in step S611 or No in step S614, the process is repeated. In that case, the handwritten character region estimated in step S605 is corrected using the probability of the character recognition result (step S611, step S614) as a result of the layout analysis in step S608, and confirmed. To do.

In step S607, the image binarization module 107 performs binarization processing on the document image corrected in step S601.
In step S608, the layout analysis module 108 receives the image binarized in step S607 and performs layout analysis of the entire image. Here, the layout analysis is also performed using the character type determined in step S606. This can be performed with higher accuracy than the result of layout analysis alone.
In step S609, as a result of the layout analysis in step S608, it is determined whether the character area is a handwritten character area (may be a printed character area). In this determination, if Yes, the process proceeds to step S610, and if No, the process proceeds to step S613.

In step S610, the handwritten character recognition module 109 performs handwritten character recognition on the image of the area determined as the handwritten character area as a result of the layout analysis in step S608.
In step S611, it is determined whether the certainty is equal to or greater than a predetermined threshold as a result of handwritten character recognition in step S610. In this determination, if Yes, the process returns to Step S612, and if No, the process returns to Step S606.
In step S612, it is determined whether or not character recognition processing has been performed for all characters in the character region as a result of the layout analysis in step S608. In this determination, if yes, the process ends. If no, the process returns to step S609.

In step S613, the print character recognition module 110 performs print character recognition on an image in an area determined to be a print character area as a result of the layout analysis in step S608.
In step S614, it is determined whether or not the certainty is equal to or greater than a predetermined threshold as a result of the type character recognition in step S613. In this determination, if Yes, the process returns to Step S615, and if No, the process returns to Step S606.
In step S615, it is determined whether or not character recognition processing has been performed on all characters in the character region as a result of layout analysis in step S608. In this determination, if yes, the process ends. If no, the process returns to step S609.

In the above-described embodiment, a Chinese character is shown as an example, but a character in another language may be used.
Moreover, in the said embodiment, it showed that it was mainly judged as a handwritten character area | region using the fluctuation of a character. On the contrary, the fact that there is little fluctuation of the character means that it is a printed character. That is, when the calculated handwritten character degree is low with respect to the area determined as the character area, it can be determined that the area is a printed character area. Therefore, the determination of the type character area can be performed in the same manner as the handwritten character area.
In the above-described embodiment, fluctuations peculiar to handwritten characters are detected as horizontal and vertical line segments of the input image as a characteristic of the handwritten characters. However, fluctuations of diagonal line segments are further detected. Also good.

In addition, about the program demonstrated, it is also possible to store in a recording medium, and it can also be grasped | ascertained as the following invention, for example in that case.
On the computer,
A feature extraction function for extracting predetermined features from an image;
A texture space generation function for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction function;
In the texture space generated by the texture space generation function, a partial space estimation function that estimates a partial space that is a handwritten character region or a partial space that is a printed character region;
A computer readable recording of an image processing program that realizes an area determination function for determining a handwritten character area or a printed character area on an actual image according to the partial space estimated by the partial space estimation function Recording medium.

On the computer,
A feature extraction function for extracting predetermined features from an image;
A texture space generation function for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction function;
In the texture space generated by the texture space generation function, an index setting function for setting an index representing the handwriting character area-likeness or the typeface character area-likeness;
In the texture space generated by the texture space generation function, a partial space estimation function for estimating a partial space that is a handwritten character area or a partial space that is a printed character area according to an index set by the index setting function;
A computer readable recording of an image processing program that realizes an area determination function for determining a handwritten character area or a printed character area on an actual image according to the partial space estimated by the partial space estimation function Recording medium.

The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standards such as “DVD + R, DVD + RW, etc.”, compact discs (CDs), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc. MO), flexible disk (FD), magnetic tape, hard disk, read only memory (ROM), electrically erasable and rewritable read only memory (EEPROM), flash memory, random access memory (RAM), etc. It is.
The program or a part of the program can be recorded on the recording medium and stored or distributed. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It can be transmitted using a transmission medium such as a network or a combination of these, and can also be carried on a carrier wave.
Furthermore, the above program may be a part of another program, or may be recorded on a recording medium together with a separate program.

It is a block diagram which shows one structural example of embodiment of an image processing system. It is explanatory drawing which shows the document in which a print character and a handwritten character are mixed. It is a block diagram explaining the outline of embodiment. It is explanatory drawing which shows the example which carried out the Fourier-transform of the printed character image and the handwritten character image. And FIG. 11 is an explanatory diagram illustrating an example of a hardware configuration of a computer that implements an embodiment. It is a flowchart which shows the process which recognizes a document image.

Explanation of symbols

21 ... Handwritten characters 31 ... Fourier transform module 32 ... Handwritten character degree evaluation module 33 ... Input image 34 ... Frequency domain image (texture feature)
35 ... Handwritten character degree (evaluation value) E
DESCRIPTION OF SYMBOLS 101 ... Image inclination correction module 102 ... Resolution conversion module 103 ... Handwritten character feature extraction module 104 ... Texture space generation module 105 ... Subspace estimation module 106 ... Character type determination module 107 ... Image binarization module 108 ... Layout analysis module 109 ... Handwritten character recognition module 110 ... Printed character recognition module

Claims (8)

Feature extraction means for extracting predetermined features from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area;
In accordance with the partial space estimated by the partial space estimation means, comprising an area determination means for determining a handwritten character area or a printed character area on an actual image ,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing system, which is a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result . Feature extraction means for extracting predetermined features from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generating means, an index setting means for setting an index representing the handwriting character area-likeness or the print character area-likeness;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area according to the index set by the index setting means;
In accordance with the partial space estimated by the partial space estimation means, comprising an area determination means for determining a handwritten character area or a printed character area on an actual image ,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing system, which is a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result . Feature extraction means for extracting predetermined features from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area;
In accordance with the partial space estimated by the partial space estimating means, an area determining means for determining a handwritten character area or a printed character area on an actual image;
Using a handwritten character area or a printed character area determined by the area determination means, and comprising a layout analysis means for analyzing a layout from the image ,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing system, which is a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result . Feature extraction means for extracting predetermined features from the image;
Texture space generation means for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction means;
In the texture space generated by the texture space generation means, a partial space estimation means for estimating a partial space that is a handwritten character area or a partial space that is a printed character area;
In accordance with the partial space estimated by the partial space estimating means, an area determining means for determining a handwritten character area or a printed character area on an actual image;
Comprising layout analysis means for analyzing the layout from the image,
The feature amount extracted by the feature extraction means is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
It is the sum of the spectral intensities weighted by the distance from a predetermined axis in the frequency plane of the two-dimensional Fourier transform result,
The image processing system characterized in that the area determination means determines a handwritten character area or a printed character area using the layout analyzed by the layout analysis means. 5. A handwritten character recognition unit or a typeface character recognition unit for recognizing a handwritten character region or a typed character region determined by the region determination unit of the image processing system according to claim 1, 2, 3, or 4. Character recognition system. The region determination unit determines the handwritten character region or the printed character region using also the likelihood of the character recognized by the handwritten character recognition unit or the character recognized by the typed character recognition unit. Item 6. The character recognition system according to Item 5 . On the computer,
A feature extraction function for extracting predetermined features from an image;
A texture space generation function for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction function;
In the texture space generated by the texture space generation function, a partial space estimation function that estimates a partial space that is a handwritten character region or a partial space that is a printed character region;
In accordance with the partial space estimated by the partial space estimation function, an area determination function for determining a handwritten character area or a printed character area on an actual image is realized ,
The feature amount extracted by the feature extraction function is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing program characterized by a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result . On the computer,
A feature extraction function for extracting predetermined features from an image;
A texture space generation function for generating a texture space in which texture information of each pixel or each region is mapped to the texture space according to the feature extracted by the feature extraction function;
In the texture space generated by the texture space generation function, an index setting function for setting an index representing the handwriting character area-likeness or the typeface character area-likeness;
In the texture space generated by the texture space generation function, a partial space estimation function for estimating a partial space that is a handwritten character area or a partial space that is a printed character area according to an index set by the index setting function;
In accordance with the partial space estimated by the partial space estimation function, an area determination function for determining a handwritten character area or a printed character area on an actual image is realized ,
The feature amount extracted by the feature extraction function is a feature amount representing a handwritten character feature or a feature amount representing a type character feature,
An image processing program characterized by a sum of spectral intensities weighted by a distance from a predetermined axis on a frequency plane of a two-dimensional Fourier transform result .

Images