JP2021144526A - Image processing device, image reader device, determination method, program, and storage medium - Google Patents

Abstract

To provide an image processing device or the like capable of appropriately determining the orientation of an input image.SOLUTION: An image processing device is provided, comprising: an extraction unit configured to extract an image of an area including parts constituting a character from an input image as an area image; a detection unit configured to detect a partial image showing the parts constituting the character from the area image; and a determination unit configured to determine the orientation of the input image on the basis of the detected partial image.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing apparatus and the like.

In recent years, digital image processing systems have made remarkable progress, and the construction of digital image processing technology is progressing. For example, in the fields of copiers and multi-function printers (MFPs) that use electrophotographic or inkjet methods, a document document is read by a scanner and saved as a document file that is electronic data, and is a document file. Is compressed and sent by e-mail.

In recent years, multifunction devices equipped with an automatic seat feeder (ADF: Auto Document Feeder) have become widespread. By setting the document to be read in the ADF, the user can automatically sequentially convey the document to the scanner and have the image of the document read one side at a time. The multifunction device provided with the ADF can reduce the burden on the user, such as resetting the scanning surface on the platen every time the document is scanned.

On the other hand, when a user reads a plurality of documents at once on a multifunction device, there is a problem that the convenience of the obtained document file is reduced unless the directions of the documents are aligned in a direction that is easy for the user to read. .. If the orientation of the manuscript is not aligned in a direction that is easy for the user to read, the user must perform an operation to rotate the displayed manuscript image when the document file obtained from the multifunction device is displayed on a display device or the like. It doesn't become. On the other hand, it is troublesome for the user to manually align the directions of the documents to be set in the ADF. In addition, depending on the original to be read, it may be printed on both sides with long-edge binding, or it may be printed on both sides with short-edge binding. It is impossible to arrange them by hand.

In view of the above situation, for example, using optical character recognition (OCR) technology, the image of each character included in the input image (input image) is identified by scanning the original. However, a technique for determining the direction in which the identification accuracy is high is widely known as the direction of the input image. By incorporating software that realizes this technology into a multifunction device or the like, the direction can be automatically determined for each input image. Furthermore, by inputting and rotating the image according to the determined direction, the user recognizes that the input image has the correct page orientation, that is, the characters have not fallen (the characters are in the correct orientation). It is possible to align in the direction with the highest possibility.

For example, a technique has been proposed in which character recognition processing is performed on an input image in four directions of 0 °, 90 °, 180 °, and 270 °, and the direction in which the highest certainty is obtained is determined as the direction of the input image. (See, for example, Patent Document 1).

Further, as a technology that does not require OCR, a technology that divides black pixels judged to be characters in an input image into blocks, detects a vector indicating the directionality of the black pixels for each block, and determines the direction of the input image. Has been proposed (see, for example, Patent Document 2).

In addition, from the font elements extracted from the input image, the characteristics of the distribution of feature points that match a predetermined pixel pattern for each relative position are obtained, and the characteristics of the distribution and the characteristics of the distribution obtained in advance using the training image data are obtained. A technique for determining the direction of an input image by comparison has been proposed (see, for example, Patent Document 3).

Japanese Patent Application Laid-Open No. 11-120321 JP-A-2007-65864 JP 2009-020884

In order to improve the convenience of the user, the multifunction device needs to appropriately determine the direction of the input image. However, the technology disclosed in Patent Document 1 has a problem that it requires a cost for making OCR correspond to a language according to the area where the multifunction device is used by the user. The technique disclosed in Patent Document 2 requires character recognition by a character database, and has a problem that the scale of the database becomes enormous when it supports a wide range of languages. The technique disclosed in Patent Document 3 has a problem that it is difficult to specify the direction for Asian languages because there is little difference in the distribution characteristics for each direction in Chinese characters having a large number of strokes and a complicated shape.

In view of the above-mentioned problems, it is an object of the present invention to provide an image processing apparatus or the like capable of appropriately determining the direction of an input image.

In order to solve the above-mentioned problems, the image processing apparatus of the present invention
An extraction unit that extracts an image of the area containing elements that make up characters from the input image as an area image,
A detection unit that detects a partial image showing the elements that make up the character from the area image,
A determination unit that determines the direction of the input image based on the detected partial image,
It is characterized by having.

The image reading device of the present invention includes the above-mentioned image processing device and
An image input unit that reads an image and inputs the scanned image to the image processing device,
It is characterized by having.

The determination method of the present invention is
A step of extracting an image of an area containing elements constituting characters from an input image as an area image, and
A step of detecting a partial image showing an element constituting a character from the area image, and
A step of determining the direction of the input image based on the detected partial image, and
It is characterized by including.

The program of the present invention
On the computer
A function to extract an image of an area containing elements that make up characters from an input image as an area image,
A function to detect a partial image showing elements constituting characters from the area image, and
A function of determining the direction of the input image based on the detected partial image, and
It is characterized by realizing.

The recording medium of the present invention is a computer-readable recording medium on which the above program is recorded.

According to the present invention, it is possible to appropriately determine the direction of the input image.

It is a figure for demonstrating the functional structure of the image forming apparatus in 1st Embodiment. It is a figure for demonstrating the functional structure of the direction correction part in 1st Embodiment. It is a flow chart which shows the flow of the direction correction processing in 1st Embodiment. It is a flow chart which shows the flow of the direction analysis processing in 1st Embodiment. It is a figure which shows the operation example in 1st Embodiment. It is a figure which shows the operation example in 1st Embodiment. It is a figure which shows the operation example in 1st Embodiment. It is a figure which shows the operation example in 1st Embodiment. It is a figure which shows the operation example in 1st Embodiment. It is a figure which shows the operation example in 1st Embodiment. It is a figure which shows the operation example in 2nd Embodiment. It is a figure which shows the operation example in 2nd Embodiment. It is a flow chart which shows the flow of the direction correction processing in 3rd Embodiment. It is a flow chart which shows the flow of the punctuation mark direction analysis processing in 3rd Embodiment. It is a figure which shows the operation example in 3rd Embodiment. It is a figure which shows the operation example in 3rd Embodiment. It is a flow chart which shows the flow of the direction correction processing in 4th Embodiment. It is a figure for demonstrating the functional structure of the image reading apparatus in 5th Embodiment. It is a figure which shows the operation example in 6th Embodiment. It is a figure which shows the operation example in 7th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the present embodiment, as an example, an image forming apparatus provided with an image processing apparatus to which the present invention is applied will be described.

[1. First Embodiment]
[1.1 Functional configuration]
First, the functional configuration of the image forming apparatus 1 will be described with reference to FIG. As shown in FIG. 1, the image forming device 1 includes an image input device 10, an image processing device 20, an image output device 30, a transmission / reception device 40, an operation panel 50, a storage unit 60, and a control unit 70.

Various processes executed by the image forming apparatus 1 are executed by the control unit 70 controlling the image input device 10, the image processing device 20, the image output device 30, the transmission / reception device 40, the operation panel 50, and the storage unit 60. The control unit 70 is composed of, for example, a CPU (Central Processing Unit). Further, the control unit 70 performs data communication with a computer connected to the network, another digital multifunction device, or the like via a network card, a LAN (Local Area Network) cable, or the like.

[1.1.1 Image Reader]
The image input device 10 is a device that optically reads an image from a document. The image input device 10 is composed of, for example, a color scanner having a CCD (Charge Coupled Device). The image input device 10 reads the reflected light image from the document as an RGB (R: red, G: green, B: blue) analog signal using a CCD.

Further, the image processing device 20 is connected to the image input device 10. The image input device 10 outputs the read RGB analog signal to the image processing device 20.

[1.1.2 Image processing device]
The image processing device 20 processes the image data of the input image. Here, the input image is an image input to the image processing device 20, for example, an image read by the image input device 10 or an image received from an external device by the transmission / reception device 40. Further, the image processing device 20 generates a compressed file for transmission to an external device, or generates image data of an image formed (output) on a recording sheet or the like. The image processing device 20 may be connected to the storage unit 60 and store the processed image data in the storage unit 60.

The image processing device 20 is connected to an image output device 30 that outputs an image based on the image data generated by the image processing device 20 and a transmission / reception device 40 that transmits a compressed file generated by the image processing device 20 to an external device. There is.

Hereinafter, each processing unit in the image processing apparatus 20 will be described. The image processing device 20 has an A / D conversion unit 202, a shading correction unit 204, a document type determination unit 206, an ACS determination unit 208, and a direction correction unit 210 for an RGB analog signal input from the image input device 10. By executing image processing in the input gradation correction unit 212 and the area separation processing unit 214, image data composed of RGB digital signals (hereinafter referred to as RGB signals) is generated.

The A / D conversion unit 202 receives the RGB analog signal input from the image input device 10 to the image processing device 20, and performs a process of converting the RGB analog signal into an RGB digital signal (that is, an RGB signal). Further, the A / D conversion unit 202 outputs the converted RGB signal to the shading correction unit 204.

The shading correction unit 204 performs shading correction processing on the RGB signal input from the A / D conversion unit 202. As shading correction, the shading correction unit 204 performs, for example, a process of removing various distortions generated in the illumination system, the imaging system, and the imaging system of the image input device 10 for the RGB signal. Next, the shading correction unit 204 outputs the RGB signal from which the distortion has been removed to the document type determination unit 206.

The manuscript type determination unit 206 uses the RGB signal input from the shading correction unit 204 to perform manuscript type determination processing for determining the mode of the manuscript such as characters, photographs, or photographic paper. Further, the document type determination unit 206 outputs the input RGB signal to the input gradation correction unit 212. The processing result of the manuscript type determination processing is reflected in the image processing in the processing unit in the subsequent stage.

The ACS determination unit 208 determines whether to output the input image as a color image including chromatic colors or as a monochrome image not including chromatic colors from the RGB signal input from the document type determination unit 206. Further, the ACS determination unit 208 outputs the input RGB signal to the direction correction unit 210.

The direction correction unit 210 determines the direction of the input image based on the RGB signal input from the ACS determination unit 208, and executes the direction correction process for correcting the direction of the input image.

Here, the direction of the input image means the most appropriate direction as the direction in which the characters represented in the input image are facing. Further, the appropriate direction is, for example, the direction in which the characters appear most frequently as the direction of the characters represented in the input image. Specifically, when many of the characters represented in the input image appear as characters rotated 90 ° counterclockwise, the direction correction unit 210 indicates that the direction of the input image is rotated 90 ° counterclockwise. Is determined to be.

Further, the correction of the direction of the input image is to rotate the input image so that when the input image is displayed or output, the overturned character is recognized by the user as the least. It means the process to make it. For example, if the direction of the input image is rotated 90 ° counterclockwise, the direction correction unit 210 corrects the direction of the input image by rotating the input image 90 ° clockwise. By doing so, many characters displayed in the corrected input image will be displayed and output in a state where they have not fallen. The direction correction process will be described later.

The direction correction unit 210 outputs the input RGB signal or the RGB signal obtained by rotating the input RGB signal to the input gradation correction unit 212.

The input gradation correction unit 212 performs a process of correcting the gradation of the RGB signal input from the direction correction unit 210. The input gradation correction unit 212 performs processing such as color balance adjustment, background density removal, and contrast adjustment as gradation correction processing. The input gradation correction unit 212 outputs the processed RGB signal to the area separation processing unit 214.

The area separation processing unit 214 performs a process of separating each pixel in the image represented by the RGB signal input from the input gradation correction unit 212 into a character area, a halftone dot area, or a photographic area. Further, the area separation processing unit 214 outputs a region identification signal indicating which region each pixel belongs to based on the separation result, such as a black generation undercolor removing unit 218, a spatial filter processing unit 220, and a gradation reproduction processing unit. Output to 226. Further, the area separation processing unit 214 outputs the RGB signal input from the input gradation correction unit 212 to the color correction unit 216.

When the image data is finally output to the image output device 30, the color correction unit 216 converts the RGB signal input from the area separation processing unit 214 into a CMY digital signal (hereinafter referred to as a CMY signal) to reproduce colors. In order to realize the fidelity of the above, a process of removing color turbidity based on the spectral characteristics of the CMY color material containing an unnecessary absorbing component from the CMY signal is performed. Next, the color correction unit 216 outputs the color-corrected CMY signal to the black generation undercolor removal unit 218.

On the other hand, when the image data is finally output to the transmission / reception device 40, the color correction unit 216 converts the RGB signal input from the area separation processing unit 214 into a color-corrected RGB signal or a gray signal, and performs spatial filter processing. Output to unit 220.

The black generation undercolor removing unit 218 subtracts the black generation process for generating a black (K) signal from the CMY signal and the K signal obtained by black generation from the CMY signal based on the CMY signal input from the color correction unit 216. The process of generating a new CMY signal is performed. As a result, the CMY3 color digital signal is converted into a CMYK4 color digital signal (hereinafter referred to as a CMYK signal). Next, the black generation undercolor removing unit 218 outputs the CMYK signal converted from the CMY signal to the spatial filter processing unit 220.

Generally, a method of producing black by skeleton black is used for the black generation process. This method is a method of obtaining the output data C', M', Y', K'from the input data C, M, Y. Here, assuming that the input / output characteristics of the skeleton curve are y = f (x) and the UCR (Under Color Removal) rate is α (0 <α <1), the following equation (1) is used in the black generation undercolor removal process. ) ~ The CMY signal is converted into a CMYK signal by using the equation represented by the equation (4).
K'= f (min (C, M, Y)) ... (1)
C'= C-αK'… (2)
M'= M-αK'… (3)
Y'= Y-αK'… (4)

Here, the UCR rate α (0 <α <1) indicates how much the CMY is reduced by replacing the overlapping portion of the CMY with K. Equation (1) shows that a K signal is generated according to the lowest signal strength of each signal strength of CMY.

When the spatial filter processing unit 220 finally outputs the image data to the image output device 30, the spatial filter processing unit 220 inputs the image data of the CMYK signal input from the black generation undercolor removing unit 218 from the area separation processing unit 214. Spatial filter processing by a digital filter is performed based on the region identification signal, and the spatial frequency characteristics are corrected. This correction improves blurring or graininess deterioration of the image. For example, for the region separated into characters by the region separation processing unit 214, the spatial filter processing unit 220 uses a filter having a large emphasis on high-frequency components to perform spatial filter processing in order to improve the reproducibility of the characters. I do. Further, the spatial filter processing unit 220 performs low-pass filter processing for removing the input halftone dot component on the region separated into halftone dots by the region separation processing unit 214.

On the other hand, when the spatial filter processing unit 220 finally outputs the image data to the transmission / reception device 40, the area input from the area separation processing unit 214 with respect to the RGB signal or the gray signal input from the color correction unit 216. Spatial filter processing by a digital filter is performed based on the identification signal to correct the spatial frequency characteristics. This correction improves blurring or graininess deterioration of the image.

Next, when the spatial filter processing unit 220 finally outputs the image data to the image output device 30, the spatial filter processing unit 220 outputs the processed CMYK signal to the output gradation correction unit 224. On the other hand, when the image data is finally output to the transmission / reception device 40, the processed RGB signal is output to the resolution conversion processing unit 222.

The resolution conversion processing unit 222 performs resolution conversion processing so that the image data output to the transmission / reception device 40 becomes the image data of the resolution set by the operation panel 50. For example, when the resolution of the input image is 600 DPI × 300 DPI and the resolution set by the operation panel 50 is 300 DPI × 300 DPI, the average value is calculated for each two pixels in the main scanning direction, and this is used as the output value. By doing so, the resolution conversion from 600 DPI × 300 DPI to 300 DPI × 300 DPI is executed. The resolution conversion processing unit 222 outputs the processed RGB signal to the output gradation correction unit 224.

When the output gradation correction unit 224 finally outputs the image data to the image output device 30, the halftone dot area ratio, which is a characteristic of the image output device 30, is relative to the CMYK signal input from the spatial filter processing unit 220. The output gradation correction processing based on the above is performed, and the CMYK signal after the output gradation correction processing is output to the gradation reproduction processing unit 226.

On the other hand, when the output gradation correction unit 224 outputs the image data to the transmission / reception device 40, the background of the fog or highlight disappears or becomes lighter as necessary with respect to the RGB signal input from the resolution conversion processing unit 222. The output gradation is corrected so as to be, and the RGB signal after the output gradation correction processing is output to the gradation reproduction processing unit 226.

When the image data is finally output to the image output device 30, the gradation reproduction processing unit 226 identifies the area input from the area separation processing unit 214 with respect to the CMYK signal input from the output gradation correction unit 224. Based on the signal, halftone processing is performed according to the region. For example, for the region separated into characters by the region separation processing unit 214, the gradation reproduction processing unit 226 binarizes or multi-values with a high-resolution screen suitable for reproducing high-frequency components. Perform processing. Further, for the region separated into halftone dots by the region separation processing unit 214, the gradation reproduction processing unit 226 performs binarization or multi-value processing on a screen that emphasizes gradation reproducibility. .. Next, the gradation reproduction processing unit 226 outputs the processed image data to the image output device 30.

On the other hand, when the gradation reproduction processing unit 226 outputs the image data to the transmission / reception device 40, the RGB signal input from the output gradation correction unit 224 is used only when the monochrome binary value is selected on the operation panel 50. On the other hand, binarization processing is performed. Next, the gradation reproduction processing unit 226 inputs the RGB signal input from the output gradation correction unit 224 or the RGB signal that has been binarized to the compression processing unit 228.

The compression processing unit 228 uses JPEG (Joint Photographic Experts Group) or MMR as necessary for image data consisting of RGB signals, gray signals, and black-and-white binary signals according to the file format settings set in the operation panel 50. A compression process such as (Modified Modified Read) is performed to generate compressed data, an image file is generated, and the image file is output to the transmission / reception device 40. The compression processing unit 228 may generate a document file including a plurality of image data.

Although it has been described that the processing of each processing unit in the image processing device 20 is executed by controlling each processing unit by the control unit 70, each processing is performed by a computer including a processor such as a DSP (Digital Signal Processor). It may be executed by controlling the unit.

[1.1.3 Image output device]
The image output device 30 forms an image on a recording sheet and outputs it by a method such as thermal transfer, electrophotographic, or inkjet based on the image data input from the image processing device 20. The image output device 30 functions as an image forming means of the image forming device 1.

[1.1.4 Transmitter / receiver]
The transmission / reception device 40 can be connected to a communication network such as a public line network, LAN, or the Internet, and transmits a compressed file to the outside via the communication network by a communication method such as facsimile or e-mail. For example, when the Scan to e-mail mode is selected on the operation panel 50 by the user, the compressed file is attached to the e-mail by the transmission / reception device 40 using a network card, a modem, or the like, and the set transmission is performed. It will be sent first.

When transmitting by facsimile, the control unit 70 performs a communication procedure with the other party by the transmission / reception device 40 using a modem. Then, when the transmittable state is secured, the control unit 70 performs necessary processing such as changing the compression format on the compressed file, and then sequentially transmits the compressed file to the other party via the communication line. conduct.

Further, the transmission / reception device 40 may receive a compressed file from another device by a communication method such as facsimile. For example, when receiving a facsimile, the control unit 70 receives a compressed file transmitted from the other party and inputs it to the image processing device 20 while performing a communication procedure on the transmission / reception device 40. The image processing device 20 performs decompression processing on the received compressed file, and performs rotation processing and / or resolution conversion processing or the like on the image data of the image obtained by the decompression processing, if necessary. Further, in the image processing device 20, the output gradation correction unit 224 performs output gradation correction, and the gradation reproduction processing unit 226 performs gradation reproduction processing. The image processing device 20 outputs the image data subjected to various image processing to the image output device 30. Further, the image output device 30 forms an output image on the recording sheet based on the image data output from the image processing device 20.

[1.1.5 Operation panel]
The operation panel 50 includes an operation unit 52 composed of a setting button for the user to set an operation mode of the image forming apparatus 1 and hard keys such as a numeric keypad, a liquid crystal display, an organic EL (electro-luminescence) display, and the like. A display unit 54 composed of the above devices is provided. The operation panel 50 may be a touch panel in which the operation unit 52 and the display unit 54 are integrally formed. In this case, the method for detecting the input of the touch panel may be a general detection method such as a resistance film method, an infrared ray method, an electromagnetic induction method, or a capacitance method.

[1.1.6 Storage unit]
The storage unit 60 is a functional unit that stores various data such as image data and various programs. For example, a non-volatile storage device (for example, HDD (Hard Disk Drive)) or a semiconductor memory storage device (for example, SSD). (Solid State Drive)) etc.

[1.2 Direction correction processing]
Next, the direction correction process executed by the direction correction unit 210 will be described. First, the configuration of the direction correction unit 210 will be described with reference to FIG. As shown in FIG. 2, the direction correction unit 210 includes an extraction unit 2102, a direction analysis processing unit 2104, a determination processing unit 2106, and a rotation processing unit 2108, and RGB input from the ACS determination unit 208 in this order. The signal is processed and output to the input gradation correction unit 212.

The extraction unit 2102 extracts a character area indicating a character or a range of elements (parts) constituting the character from the input image, or a character string area composed of one or more character areas. Here, the character means a symbol used to describe a word. Further, the elements constituting the character are symbols and figures included as a part of the character, and include, for example, radicals, straight lines, curves, and points. The direction analysis processing unit 2104 detects a specific element among the elements constituting the character from the character area and the character string area, and calculates a score according to the detected direction. The determination processing unit 2106 determines the direction of the input image for each input image according to the score calculated over the entire input image. The rotation processing unit 2108 rotates the input image for each input image according to the direction of the input image determined by the determination processing unit 2106. For example, when the rotation processing unit 2108 rotates the input image determined by the determination processing unit 2106 in a direction other than the positive direction, the direction of the input image is corrected by rotating the input image in the positive direction. The positive direction means a direction in which the characters can be read correctly (a direction in which the angle of the characters is 0 °, a direction in which the characters are not overturned). For example, as will be described later, the direction in which the characters are arranged in the direction of FIG. 8A is the positive direction.

The direction correction process will be described with reference to FIG. First, the extraction unit 2102 extracts an area including an element constituting a character (step S102). The extraction unit 2102 extracts an area including an element constituting a character by, for example, the following method.

(1) Extraction of character area The extraction unit 2102 extracts a set of character-like pixels from the input image as a character area. The character-like pixels are, for example, pixels included in the foreground layer when the input image is separated into the foreground layer and the background layer, and are pixels including the pixels of the edge portion.
(2) Extraction of character string area The extraction unit 2102 extracts each group when the character area is divided into groups as a character string area based on the relationship between the positions and sizes of the character areas. At this time, the extraction unit 2102 may determine the character arrangement direction (character string direction) for each character string area based on the arrangement of the character areas constituting the character string area.

As a method for extracting the character region and a method for extracting the character string region, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2012-114744 can be used, and other known techniques can also be used. .. When the extracted character area is too large or too small as in the method disclosed in Japanese Patent Application Laid-Open No. 2012-114744, the extracted character area may be excluded as not being character-like, or a group. When the number of character areas included in is small, it is possible to exclude all the character areas included in the group, assuming that the character string does not constitute a sufficient number of characters. By doing so, it is possible to prevent edges included in non-character parts such as photographs from being mistakenly extracted as character areas.

Further, since the extraction of the character area is a preliminary step for extracting the character string area and determining the character string direction, it is not necessary to accurately extract each character. For example, each part such as "hen" or "making" may be separated and extracted, or one character area may be extracted across a plurality of characters.

Subsequently, the direction analysis processing unit 2104 sets a target area based on the character area and the character string area extracted by the extraction unit 2102, and extracts an image of one of the set target areas as an area image from the input image. (Step S104). The target area is an area to be targeted for the direction analysis processing described later, and is an area including pixels used for determining the direction of the input image. The area image is an image extracted from the input image based on the target area. The direction analysis processing unit 2104 may use the character string area itself as the target area, expand the character string area further outward by a predetermined number of pixels, or reduce the character string area by a predetermined number of pixels inside, or a plurality of one character string area. Each area divided into the above areas may be used as the target area. Further, the character area itself extracted by the extraction unit 2102 may be used as the target area.

When a plurality of character string areas and character areas are extracted from the input image by the extraction unit 2102, the direction analysis processing unit 2104 covers a part of the areas extracted by the extraction unit 2102 as a target area. May be. In this case, the direction analysis processing unit 2104 may use a region having a predetermined size or larger as the target region, a region existing at a specific position (for example, the upper half of the input image) as the target region, or a predetermined region. A number of randomly selected areas may be used as the target area.

Subsequently, the direction analysis processing unit 2104 executes a direction analysis process of detecting an image showing the target part from the area image and calculating a score (score for each direction) based on the detection result (step S106). Here, the target part means a part to be detected from the area image. The target part is a part defined before executing the direction analysis process, and is predetermined or selected by, for example, a user of the image forming apparatus 1, a designer, or the like. Parts suitable as target parts will be described later.

In the present embodiment, as the direction analysis process, the region image is rotated in a plurality of directions, the target parts in the positive direction are detected from the rotated images, and the score based on the detection result is calculated. The target part in the positive direction means the target part included in the characters in the positive direction.

Further, in the direction in which the area image is rotated (hereinafter referred to as "rotation direction of the area image"), there is an input direction which is the direction in which the image is read (the direction in which the rotation angle of the input image is 0 °). Add at least one of the following directions:
(1) Counterclockwise direction in which the input image is rotated 90 ° counterclockwise (2) Reverse (reverse) direction in which the input image is rotated 180 ° (3) 90 ° clockwise rotation of the input image ° (270 ° counterclockwise) Clockwise, which is the direction of rotation

The direction analysis processing unit 2104 may set the rotation direction of the region image to the four directions of the input direction, the counterclockwise direction, the reverse direction, and the clockwise direction, or set the rotation direction of the region image to the direction opposite to the input direction. It may be limited to two and only the top-bottom direction of the document may be determined. Further, the direction analysis processing unit 2104 may limit the rotation direction of the area image to two directions, an input direction and a counterclockwise direction (or a clockwise direction), and determine only a vertical document or a horizontal document. good. In this way, the rotation direction of the region image includes the direction in which the region image is in the positive direction.

The rotation direction of the region image may be selected by the user, or may be automatically determined by a simple method before the direction analysis processing unit 2104 executes the direction analysis process. Further, in the present embodiment, the number in the rotation direction of the region image is A.

The direction analysis process of this embodiment will be described with reference to FIG. First, the direction analysis processing unit 2104 substitutes 1 for the variable a indicating the rotation direction of the region image (step S122).

Subsequently, the direction analysis processing unit 2104 rotates the region image in the ath direction, and detects an image showing the target part in the positive direction (hereinafter, referred to as “partial image”).

Here, a part suitable as a target part will be described. In the present embodiment, the target part in the positive direction is detected from the region image rotated in the first direction a. It is desirable that the parts to be detected are, for example, parts that do not appear frequently in general documents or that are difficult to be confused with the part itself or other parts when the part itself is rotated.

The case where the part itself is rotated will be described with reference to FIG. In this embodiment, the parts are radicals of Chinese characters. Further, among the radicals of Chinese characters, an example of a radical that is appropriate as a target part is shown in FIG. 5 (a), and an example of an inappropriate radical is shown in FIG. 5 (b).

FIG. 5A is a diagram showing a part and a part when the part is rotated in four directions: a forward direction, a counterclockwise direction, a reverse direction, and a clockwise direction. In FIG. 5A, for example, the "kihen" shown on page 102 is confused with the "kihen" or other parts even if it is rotated in the forward direction, the counterclockwise direction, the reverse direction, and the clockwise direction, respectively. It is unlikely that you will. In addition, Chinese characters using "kihen" are frequently used in general documents, so it can be said that they are suitable as target parts. Other suitable parts as the target parts include "Takekanmuri" shown on P104, "Amekanmuri" shown on P106, and "Kanehen" shown on P108.

On the other hand, in FIG. 5B, for example, "Kusakanmuri" shown on page 112 and "Kurumahen" shown on page 114 are likely to be confused with the parts themselves when the parts themselves are rotated. Is. That is, in "Kusakanmuri" and "Kurumahen", the parts in the forward direction and the parts in the reverse direction have almost the same shape. When such a part is defined as a target part, the direction analysis processing unit 2104 is in the positive direction not only from the area image rotated in the positive direction but also from the area image rotated in the direction in which the characters are turned upside down. The target part is detected. As a result, the accuracy of the detection result is reduced.

Further, the "mehen" shown on P116 and the "brick" shown on P118 are parts that are likely to be confused with other parts when the parts are rotated. For example, rotating "Mehen" counterclockwise is likely to be confused with "Sara", and rotating "Brick" clockwise is likely to be confused with "Sanzui". If such a part is defined as a target part, the direction analysis processing unit 2104 detects the target part in the positive direction from characters that are not in the positive direction. As a result, the accuracy of the detection result is reduced. For example, if "Mehen" shown on page 116 is the target part, from the area image that includes the Chinese characters including "Sara" (for example, "Masu", "Bon", and "Sheng") rotated 90 ° clockwise. , Detects a positive "mehen".

In addition, "Shikahen" can be mentioned as another example that is inappropriate as a target part. "Shikahen" is hard to confuse with the part itself or other parts when the part itself is rotated, but it is said that the Chinese characters themselves using "Shikahen" are generally used frequently. It's hard to say, so it's not suitable as a part.

As for the method of detecting the partial image from the region image, the direction analysis processing unit 2104 can use a method using machine learning or pattern matching.

Here, a method using machine learning will be described. First, a detector for detecting a partial image from a region image is constructed in advance by learning. FIG. 6A is a diagram showing a combination (learning data) of the input data at the time of constructing the detector and the correct answer data (part name).

For example, when detecting a radical of a Chinese character such as "kihen" in the positive direction from a region image, a detector is constructed to detect a partial image indicating "kihen" in the positive direction from the region image. At this time, the detector can detect a partial image of the positive direction "kihen" by inputting an image showing the positive direction "kihen" as input data and giving "kihen" as the correct answer data. Generate a possible model. Then, by inputting the region image into the detector including the generated trained model, a partial image indicating a positive direction "kihen" is detected from the region image. In this way, the positive "kihen" is detected from the area image. The detector and the trained model are stored in the storage unit 60, for example.

As shown in P122 and P124 of FIG. 6A, even if the image shows "kihen" or "takekanmuri", the shape differs depending on the font, and even if the same font is used, the characters including the parts are included. Deformation such as a change in aspect ratio can be seen depending on the configuration. Therefore, in order to improve the detection accuracy, it is desirable to increase the learning data corresponding to one target part by using the input data in which the font and the aspect ratio are changed. Further, if the image quality of the input image is deteriorated, the detection accuracy may be lowered. In order to prevent a decrease in detection accuracy, input data whose image quality is intentionally deteriorated, such as a tilted image or an image to which noise is added, may be used.

The detector may be trained so that a target part other than the positive direction "kihen" can be detected. At this time, if there are variations in the size and shape of the target parts, such as elongated parts such as "kihen" in the forward direction and parts that are close to L-shape such as "shinnyo" in the forward direction, 1 It is difficult for one detector to detect all of these target parts. In this case, it is easy to detect by enlarging or reducing the input data in advance so that the input data will always be the same size, or by limiting all the input data to elongated parts such as "hen" and "crown". You may learn to become. Further, a detector may be constructed for each target part.

Further, as the input data, not the target part itself but the character itself including the target part in the positive direction may be learned as shown in P126 and P128 of FIG. 6B. In this case as well, the learning may be performed so as to cope with the difference in fonts and the deterioration of the image quality of the input image as described above. In addition, there may be a large number of characters including the part, such as "kihen", but since the purpose of this embodiment is not to identify the characters themselves, input data narrowed down to typical characters is used. It may be created to reduce the scale of the training data.

Although the case where the part name is given as the correct answer data is shown in FIG. 6, instead of giving the part name as the correct answer data, the classification result of two choices as to whether or not the part is the target part in the positive direction is used as the correct answer data. May be given.

The direction analysis processing unit 2104 rotates the region image obtained from the input image in the ath direction (step S124). Subsequently, the direction analysis processing unit 2104 detects the target part in the positive direction from the area image by detecting the partial image from the area image using the detector constructed from the learning data (step S126). The direction analysis processing unit 2104 adds 1 to a, and if a does not exceed A, returns to step S124 (step S128 → step S130; No → step S124). By doing so, the detection process of the target part in the positive direction is executed for each region image rotated in the predetermined direction.

Subsequently, the direction analysis processing unit 2104 adds the score calculated based on the direction rotated in step S124 and the detection result in step S126 to the direction-specific score (step S132). Here, the direction-specific score indicates a score obtained by adding up the scores calculated by a predetermined method for each rotation direction of the region image. For example, when the target part in the positive direction is detected only in the region image in one direction, the direction analysis processing unit 2104 detects the direction corresponding to the direction indicated by the region image in which the target part in the positive direction is detected. Choose as. When a target part in the positive direction is detected from the region images in a plurality of directions, the direction with the highest detection probability (likelihood, reliability) is selected as the detection direction. Then, the direction analysis processing unit 2104 adds a score to the corresponding direction-specific score by a predetermined method for the selected detection direction. By doing so, it can be shown that the higher the score, the higher the validity of correcting the input image in the positive direction by rotating in that direction.

Here, when the rotation direction of the region image includes the direction in which the region image is in the positive direction, the target part in the positive direction is detected from the region image rotated in the direction in which the region image is in the positive direction. , The score in the direction corresponding to the direction in which the area image is in the positive direction becomes high. Further, it is highly appropriate to set the direction in which the region image is rotated from the direction in which the score is high to the input direction as the direction of the input image. For example, when the direction in which the score is high is the clockwise direction, the direction in which the region image in the clockwise direction is rotated to become the region image in the input direction is the counterclockwise direction. Therefore, it is highly valid that the direction of the input image is the counterclockwise direction. Here, when the direction with a high score is the input direction (counterclockwise 0 °), the direction of the input image is the same as the input direction, and the direction with a high score is the counterclockwise direction (counterclockwise 90 °). In the case of, the direction of the input image is clockwise (270 ° counterclockwise), and when the direction with the higher score is the opposite direction (180 ° counterclockwise), the direction of the input image is the same as the reverse direction. When the direction in which the score is high is the clockwise direction (counterclockwise 270 °), the direction of the input image is the counterclockwise direction (counterclockwise 90 °).

As a predetermined method of adding the scores, a method of giving a constant in one selected direction may be used, or a method of giving a larger value as the above-mentioned certainty is larger may be used. Also, when a target part in the positive direction is detected from the area images in multiple directions, the score is not added only in that direction by selecting a single direction, but based on the magnitude of the above-mentioned certainty. The score may be added to each of the plurality of detected directions.

Returning to FIG. 3, the direction analysis processing unit 2104 determines whether or not the area image has been extracted from all the target areas, and if the area image has not been extracted from all the target areas, the process proceeds to step S104. Return (step S108; No → step S104). In this way, the direction analysis processing unit 2104 extracts a region image from all the target regions, detects a partial image from the extracted region image, and calculates a score for each direction. Further, the direction analysis processing unit 2104 can calculate the total score for each direction by sequentially adding the score for each direction calculated for each area image to the score for each direction in the corresponding direction.

After extracting the area images from all the target areas, the determination processing unit 2106 determines the direction of the input image based on the direction-specific score calculated by the direction analysis processing unit 2104 (step S108; Yes → step S110). Specifically, the determination processing unit 2106 determines as the direction of the input image the direction in which the input image rotated in the direction having the highest score among the directional scores is rotated in order to make the input image in the input direction. Just do it.

Subsequently, the rotation processing unit 2108 rotates the input image in the direction in which the input image is in the positive direction based on the direction of the input image determined by the determination processing unit 2106 in step S110 (step S112). For example, the rotation processing unit 2108 rotates the input image 90 ° counterclockwise if the direction of the input image is clockwise (270 ° counterclockwise), and the direction of the input image is opposite (counterclockwise). If it is 180 ° clockwise, the input image is rotated 180 °, and if the direction of the input image is counterclockwise (90 ° counterclockwise), the input image is rotated 90 ° clockwise. When the determination processing unit 2106 determines that the direction of the input image is the positive direction, the rotation processing unit 2108 does not rotate the input image (skips the process of rotating the input image).

Since the image data after rotating the input image is output from the direction correction unit 210 by the above-mentioned direction correction processing, the input image is rotated in an appropriate direction in the image file output via the compression processing unit 228. It becomes an image file that represents the image data. Further, the direction correction unit 210 executes the direction correction process for each image of the input original, so that even if there are a plurality of image data of the input original, each image data can be converted into image data in the correct direction. Can be rotated. As a result, the images included in the document file output via the compression processing unit 228 are aligned in the correct direction.

In addition to the machine learning method, template matching can also be used as the method for extracting the target parts. In this case, first, the template image of the target part is stored in the storage unit 60. For example, as in the example of the input data shown in FIG. 6A (for example, P122 and P124), a plurality of template images are stored according to the font and the aspect ratio. Then, the direction analysis processing unit 2104 performs template matching between the template image stored in the storage unit 60 and the area image. Processing time is required because the number of template matchings increases by the number of template images, but it can be realized on a relatively small scale because it does not require large-scale filtering such as machine learning. In addition, in order to reduce the amount of processing, the resolution of both the template image and the area image is lowered, and detailed information is intentionally omitted so that the target part can be detected by using the rough shape as a feature. It may be.

[1.3 Operation example]
An operation example of this embodiment will be described with reference to the figure. FIG. 7 is a diagram showing a specific example of extracting a character area and a character string area from an input image. FIG. 7A is a diagram showing an example of the input image E100, and it is assumed that the characters “sales promotion unit” appear in the area E102 in the input image. That is, the area E102 includes pixels constituting the characters "sales promotion unit".

The extraction unit 2102 extracts a character area from the input image. As a result of extracting the character area, as shown in FIG. 7B, each part constituting the character surrounded by the dotted line is extracted as the character area. As the character area, for example, an area including "anti" which is an element constituting the character of "sales" as shown in the area M100, and an area including a point forming the character of "sales" as shown in the area M102. Is extracted. In this way, the character area is extracted not as an individual character unit but as a unit of elements constituting a character such as a radical or a part of the radical.

Further, the extraction unit 2102 may determine the character string direction or extract the character string area based on the character area. For example, using the method disclosed in Japanese Patent Application Laid-Open No. 2012-114744, the extraction unit 2102 uses, for example, the number of consecutive flat pixel blocks in the horizontal character region and the continuous number of flat pixel blocks in the vertical character region. The character string direction is determined by comparing with the number. In the example of FIG. 7B, since the number of connections in the horizontal direction is larger than the number of connections in the vertical direction, the extraction unit 2102 assumes that the characters are arranged in the horizontal direction in the area E104 of FIG. 7C. As shown, the character string area whose character string direction is horizontal is extracted.

After extracting the character area, the extraction unit 2102 may integrate a plurality of character areas arranged in the line direction in one character string area. For example, under the assumption that a plurality of characters are not arranged in a line direction orthogonal to the character string direction in one character string area in principle, the extraction unit 2102 is arranged in the line direction in one character string area. A plurality of character areas may be integrated with each other. Further, in the character string direction, when the foreground pixels constituting the character are not connected to each other but the rectangles of the character area overlap between the plurality of character areas, the extraction unit 2102 sets 1 of these character areas to each other. It may be integrated as one character area.

FIG. 7D is a diagram showing an example in which the character area shown in FIG. 7B is redefined by integrating the character areas. By integrating the character areas in this way, an area containing individual character units as shown in the area M104 and an area including a group of units constituting one character as shown in the area M106 and the area M108 can be obtained. It can be a character area. As a result, the number of target areas is reduced as compared with the case where the character areas are not connected, and the process of detecting the partial image by the direction analysis processing unit 2104 can be reduced.

The direction analysis processing unit 2104 may set the character area (for example, M100 or M102) as shown in FIG. 7B as the target area, or the character string area (for example, E104) as shown in FIG. 7C. ) May be the target area, or the area after redefining the character area as shown in FIG. 7D (for example, M104, M106, M108) may be the target area. How the direction analysis processing unit 2104 sets the target area may be predetermined or may be determined by the user.

FIG. 8 is a diagram showing a region image when the direction of the input image is the positive direction and an extraction result when the target part is extracted from the region image. FIG. 8 shows a case where the character string area in which the characters “I forgot the pencil case at school” appears is the target area, and the target parts include “Takekanmuri” and “Kihen”. Further, in FIG. 8, the rotation direction of the region image is the input direction (counterclockwise 0 °) which is the first direction, the counterclockwise direction (counterclockwise 90 °) which is the second direction, and the second direction. It shows four cases of the reverse direction (180 ° counterclockwise) which is the direction 3 and the clockwise direction (270 ° counterclockwise) which is the fourth direction.

First, the direction analysis processing unit 2104 sets the region image to the input direction, which is the first direction. Specifically, the direction analysis processing unit 2104 extracts the image included in the target area as it is as the area image from the input image. FIG. 8A shows an image in which the area image is in the input direction. The area E110 in FIG. 8A shows a target area. The image in the area E110 is an area image. Further, the area E112 and the area E114 indicate the area where the partial image of "Takekanmuri" is detected in the image of the area E110, and the area E116 and the area E118 are the "kihen" part of the image of the area E110. Indicates the area where the image was extracted. It should be noted that the area E118 indicates that the partial image of "Kihen" was extracted from the image in which the character "box", which is the original radical of "Takekanmuri", was expressed. As described above, a partial image may be extracted from an element other than the original radical.

Subsequently, the direction analysis processing unit 2104 sets the region image in the counterclockwise direction, which is the second direction. FIG. 8B shows a region image in which the image in the region E110 is counterclockwise, and the region E120 indicates a target region. Similarly, the direction analysis processing unit 2104 sets the region image in the reverse direction which is the third direction and the clockwise direction which is the fourth direction. FIG. 8C shows an image in which the image in the area E110 is reversed, and the area E130 shows a target area. FIG. 8D shows an image in which the image in the area E110 is in the clockwise direction, and the area E140 shows a target area.

Further, the direction analysis processing unit 2104 extracts a partial image of the target part from the region image which is an image in each region of the region E120, the region E130, and the region E140. However, as shown in FIGS. 8 (b), 8 (c), and 8 (d), no partial image is detected from the region images of the region E120, the region E130, and the region E140.

FIG. 9 is a diagram showing an example in which the score is calculated for each of the input direction, the counterclockwise direction, the reverse direction, and the clockwise direction, which are the rotation directions of the region image. Specifically, FIG. 8 is a diagram showing an example. It is a figure which showed the specific example of the score in the example of. In the example of FIG. 8, the target part in the positive direction is detected only from the region image rotated in one direction (input direction). Based on this detection result, in FIG. 9, the direction when the direction analysis processing unit 2104 selects the input direction as the detection direction each time the target part is detected and adds 1 to the score of the corresponding direction (input direction). It shows another score. That is, since the direction analysis processing unit 2104 has extracted two positive "kihen" and two "takekanmuri" in the region image indicating the input direction, the score for each direction in the input direction is 4, and the other The direction-specific score in the direction of is 0.

As a result, the directional score in the input direction is higher than the directional score in the other directions (counterclockwise, counterclockwise, clockwise). Therefore, in the determination processing unit 2106, among the rotation directions of the region image, the input direction is the direction for rotating the input image so as to be the positive direction, that is, the direction of the input image is the positive direction. Can be determined. Further, since the determination processing unit 2106 determines that the input direction is the positive direction, the rotation processing unit 2108 does not rotate the input image.

In addition, the direction analysis processing unit 2104 sets a constraint for each target part in the detection of the target part, and ignores (does not detect) what is detected contrary to the constraint even if the certainty (or the degree of coincidence) is high. You may do so. For example, as shown in FIG. 10, in the character "螢" rotated by 180 °, a positive direction (0 ° counterclockwise) may be detected in the region E150, but "Takekanmuri" may be detected. Since "Takekanmuri" is originally located above the characters, the detection result is not appropriate. Therefore, if the positive direction "Takekanmuri" is detected as it is, the wrong direction may be determined as the direction of the input image. Here, as shown in FIG. 10, if another character area belonging to the same character string area exists above the position where "Takekanmuri" is detected, "Takekanmuri" is positioned on the lower side. You can see that it is doing. Therefore, by giving a constraint to "Takekanmuri" that "it is located on the upper side in the positive direction", the direction analysis processing unit 2104 violates the constraint even in the case shown in FIG. Deemed, detection can be canceled. Even if it is a target part other than "Takekanmuri", if the target part is detected from a specific position where it is not originally located, the detection result may be ignored (cancelled). In this way, it is possible to prevent a decrease in the accuracy of determining the direction of the input image.

In the above example, the target part is extracted by using the image of the original printed with the type as the input image, but the direction of the input image of the handwritten original may be determined. In this case, in constructing the detector, input data may be extracted from characters written by a person to create the detector. Further, the partial image of the character written by a person may be learned together with the partial image of other print characters, or the detector may be constructed only with the handwritten partial image. Further, by adding a process for detecting a handwritten document in the direction correction process, the direction correction process may be applied only to the document detected as the handwritten document.

As a method for detecting the handwritten manuscript, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2007-087196 can be used, and it may be realized by using other known techniques. By doing so, even for a handwritten document to which OCR is difficult to apply, it is possible to determine the direction of the input image and rotate it, which improves convenience.

In addition to the above description, the order of the steps may be changed or some steps may be omitted as long as there is no contradiction. For example, in the direction correction process, the area images of all the target areas may be first extracted, and in the direction analysis process, all the area images may be rotated to the ath position and then the partial image may be detected.

According to the present embodiment, only the main part is detected without distinguishing which character each character is, and the direction of the input image is based on the direction of the part when the part is detected. Can be determined. Since it is not necessary to recognize characters, a large-scale database is not required, and the direction of the input image can be determined by a simple process. Further, by defining a characteristic part such as a radical of a Chinese character as a target part, it is possible to correspond to an image including an image of a character having a large number of strokes and a complicated shape such as a Chinese character.

[2. Second Embodiment]
Next, the second embodiment will be described. The second embodiment is different from the first embodiment in that the direction of the input image is determined from the region image by rotating the partial image in a predetermined direction instead of rotating the region image.

In the present embodiment, as the direction analysis process, the area image is left as the input direction, the target part rotated in a predetermined direction is detected, and the score based on the detection result is calculated.

In the present embodiment, the number of directions in which the partial image is rotated is A. Further, in step S124 of the direction correction process described in the first embodiment, the direction analysis processing unit 2104 rotates a partial image showing the target part in the positive direction in the ath direction. Further, in step S126, the direction analysis processing unit 2104 detects a partial image rotated in the first direction from the region image in the input direction. For example, if there are four directions of the partial image to be detected, which are the forward direction, the counterclockwise direction, the reverse direction, and the clockwise direction, the direction analysis processing unit 2104 sets the partial image indicating the target part in the forward direction and the counterclockwise direction. It is rotated in the clockwise direction, the reverse direction, and the clockwise direction, respectively, and the rotated partial image is detected from the region image in the input direction.

Further, in step S132, the direction analysis processing unit 2104 selects the direction of the detected partial image as the detection direction, calculates the score, and adds the calculated score to the direction-specific score. That is, in the present embodiment, unlike the first embodiment, the score is calculated for each direction of the detected partial image. In this case, for example, if the direction of the input image is the positive direction, many partial images in the positive direction are detected, so that the score for each direction in the positive direction is high. Further, if the direction of the input image is the counterclockwise direction, many partial images in the counterclockwise direction are detected, so that the score for each direction in the counterclockwise direction becomes high. As described above, the direction-specific score of the present embodiment is different from that of the first embodiment, and the higher the score, the higher the validity that the direction is the direction of the input image. The direction in which the score based on the direction is high can be directly determined as the direction of the input image. Therefore, in step S110 of the direction correction process, the determination processing unit 2106 may determine the direction having the highest score among the direction-specific scores as the direction of the input image. When the direction analysis processing unit 2104 detects partial images in a plurality of directions from one area image, the direction analysis processing unit 2104 may calculate the score based on the certainty of detection (likelihood, reliability).

In the present embodiment, a detector for detecting a partial image is constructed by using learning data having direction information. An example of the learning data in this embodiment is shown in FIG. In the present embodiment, as shown in FIG. 11, the input data is rotated in association with the detection direction, and the correct answer data includes the part name and the rotated direction as learning data. As a pair of, a detector is constructed by learning.

An operation example in this embodiment is shown in FIG. In FIG. 12A, the direction analysis processing unit 2104 shows, from the image in the target area E200, a positive “take-top” in the area E202 and E204, and a positive “take-kanmuri” in the area E206 and E208. An example in which "kihen" is detected is shown. Further, FIG. 12B is a diagram showing a score when the scores corresponding to the directions of the detected partial images are added one by one. As shown in FIG. 12B, since the score in the positive direction is the highest, the determination processing unit 2106 can determine the positive direction as the direction of the input image.

When template matching is used instead of machine learning, the template image rotated in each detection direction may be stored in the storage unit 60 in advance as the template image of the target part. At this time, the direction analysis processing unit 2104 can detect the template image and the direction of the template image from the area image by performing template matching between the template image stored in the storage unit 60 and the area image. .. When using template matching, the template image in the positive direction is stored in the storage unit 60 in advance, and when the template matching is executed, the template image rotated in the direction in which the template image in the positive direction is detected is generated. You may. In this way, by generating a partial image rotated in a plurality of directions from the partial image in the positive direction, template matching is performed even when only the partial image in the positive direction is stored in the storage unit 60. be able to.

According to the present embodiment, it is possible to detect the target part in consideration of the direction without rotating the area image itself every time the input image is input to the direction correction unit 210, and the processing time in actual use is shortened. be able to.

[3. Third Embodiment]
Next, the third embodiment will be described. In the third embodiment, the directional analysis processing unit 2104 is provided with a plurality of directional analysis processing means, and the directional scores calculated by the respective directional analysis processing means are integrated into a finally obtained comprehensive directional score. Based on this, it is an embodiment in which the direction of the input image is determined for each page.

This embodiment is applicable to both the first embodiment and the second embodiment. Further, in the present embodiment, FIG. 3 relating to the direction correction processing in the first embodiment and the second embodiment is replaced with FIG. In this embodiment, the same functional parts and processes as those in the first embodiment and the second embodiment are designated by the same reference numerals, and the description thereof will be omitted.

In the present embodiment, when the direction analysis processing unit 2104 is provided with N (> 1) direction analysis processing means, they are referred to as a first direction analysis process, ..., And an Nth direction analysis process, respectively. Further, in the present embodiment, the directional score calculated by the k-direction analysis process (1 ≦ k ≦ N) is referred to as the k-direction score, and the directional score obtained by integrating the k-direction scores is obtained. It is called the overall score by direction. Further, in the first direction analysis process, the same process as the direction analysis processing unit 2104 described in the first embodiment is executed. The direction correction unit 210 in the present embodiment can have the same configuration as that in the first embodiment except for the direction analysis processing unit 2104, and the description thereof will be omitted.

The overall direction-specific score in the present embodiment is the score related to the direction for correcting the input image in the positive direction as described in the first embodiment or the score related to the direction of the input image as described in the second embodiment. Either. The type of the overall direction-specific score (whether the score is related to the direction for correcting the input image in the positive direction or the score is related to the direction of the input image) is predetermined, for example.

Here, if the k-direction score and the overall direction-specific score are of the same type, the direction analysis processing unit 2104 integrates the k-direction-specific score into the overall direction-specific score as it is (for example, addition). can. Therefore, by making the plurality of direction analysis processes a direction analysis process for calculating the same type of score, the direction analysis processing unit 2104 can efficiently calculate the overall score for each direction. On the other hand, if the k-direction score and the overall direction-specific score are different types of scores, the k-direction score is aligned with the score type indicated by the overall direction-specific score, and then the overall direction is adjusted. Integrate into another score.

For example, if the overall score for each direction is a score indicating the direction of the input image, and the score for the kth direction is a score indicating the direction for correcting the input image in the positive direction, the score for each k direction is input. After aligning the scores that indicate the direction of the image, integrate them into the overall score for each direction.

The score indicating the direction for correcting the input image in the positive direction is the score of the input image by regarding the score for each direction as the score corresponding to the direction of the input image determined if the score is the highest. It can be treated as a score indicating the direction.

The score indicating the direction of the input image represents the input image by regarding the score for each direction as the score corresponding to the direction in which the input image is rotated to correct it in the positive direction if the score is the highest. It can be treated as a score indicating the direction for correction in the positive direction.

FIG. 13 is a flow showing an example of a processing procedure of the direction correction process executed by the direction correction unit 210 in this embodiment. The direction correction unit 210 sets the total direction-specific score corresponding to the direction to be integrated into the comprehensive direction-specific score as the initial value of the total direction-specific score when the N direction analysis processing means are executed. Is initialized to zero.

In the present embodiment, the direction analysis processing unit 2104 extracts the area image from the target area of 1, and then calculates the score for each k direction by the k-direction analysis process with k = 1 as the initial value (step S302). (Step S304). Further, the direction analysis processing unit 2104 updates the total direction-specific score by adding the value of the k-th direction-specific score to the total direction-specific score (step S306).

Subsequently, the direction analysis processing unit 2104 adds 1 to k (step S308), and if k ≦ N, returns to step S304 based on the updated k (step S310; No → step S304). , The same process is repeated, and the overall score for each direction is updated until k> N. When k> N, the direction analysis processing unit 2104 ends the process of detecting the target part in the region image extracted in step S104. The direction analysis processing unit 2104 repeats the processes from step S304 to step S310 until the area image is extracted from all the target areas. In this way, the direction analysis processing unit 2104 updates the overall direction-specific score.

When the area image is extracted from all the target areas, the direction analysis processing unit 2104 inputs the overall direction-specific score to the subsequent determination processing unit 2106 as the direction-specific score corresponding to the input image (step S310). Yes). Although FIG. 13 shows a method of sequentially executing a plurality of directional analysis processing means, a plurality of directional analysis processing means may be executed in parallel. When the area image is extracted from all the target areas, the determination processing unit 2106 determines the direction of the input image based on the overall direction-specific score (step S108; Yes → step S312). When the overall direction-specific score is a score related to the direction for correcting the input image in the positive direction, the determination processing unit 2106 uses the input image rotated in the direction with the highest score as the input image in the input direction. The direction of rotation is determined as the direction of the input image. On the other hand, when the overall direction-specific score is a score indicating the direction of the input image, the determination processing unit 2106 determines the direction with the highest score as the direction of the input image.

In the direction correction process of the present embodiment, any method can be adopted as the method of detecting the parts executed by the k-direction analysis process (k> 1) and the method of calculating the score for each k-direction. For example, in addition to the method of detecting the target part described in the first embodiment and the second embodiment, a method of detecting a symbol used in a sentence and having a characteristic at the position where the symbol appears is used. Can be done.

As an example of the method of detecting the symbol, referring to FIG. 14, a method of detecting a punctuation mark as an example as a specific character included in the character string area and calculating a score for each direction based on the detection result (hereinafter, this book). This is called analysis processing). The specific characters in this analysis process are characters, symbols, etc. used to determine the direction. In this embodiment, punctuation marks are used as an example, but for example, they may be delimiters such as commas and periods, and for example, characters that often appear in the language to be analyzed (for example, "ha" in the case of Japanese). May be a specific character.

In the present analysis process shown in FIG. 14, the direction analysis processing unit 2104 detects the direction of the specific character according to the position including the specific character without rotating the area image, thereby that of the second embodiment. Similarly, the score regarding the direction of the input image is calculated. Therefore, when the present analysis process and the direction analysis process for detecting the target part are combined, the method of the second embodiment is used rather than the method of the first embodiment as the direction analysis process for detecting the target part. The analysis processing unit 2104 can efficiently calculate the overall score for each direction.

In this analysis process through the detection of punctuation marks, the direction analysis processing unit 2104 first detects punctuation marks that are specific characters from the character string area (step S322). Specifically, the direction analysis processing unit 2104 sets a target area based on the character string area extracted by the extraction unit 2102, extracts the area image, and detects punctuation marks from the area image. The method for detecting punctuation marks may be, for example, a method of detecting a partial image of punctuation marks by machine learning or pattern matching, similar to the processing of the direction analysis processing unit 2104 described in the first embodiment, or a known method. May be good.

For example, "," (reading point) has a difference in direction between the positive direction (0 ° counterclockwise) and the counterclockwise direction (90 ° counterclockwise), but the positive direction (counterclockwise). Since the difference in orientation cannot be seen between (0 °) and the opposite direction (180 ° counterclockwise), it is not necessary to try to detect all the directions to be integrated into the overall directional score, and either one of them. It is only necessary to know whether or not it was detected in the direction.

Also, although it is not a punctuation mark such as a dakuten or semi-voiced sound mark of kana characters, or a part of "Sanzuihen", a constraint is set to distinguish it from parts of similar characters, and if the constraint is not met, it is not a punctuation mark. You may cancel it. As an example of the constraint, for example, there is no other character pixel in the line direction from the area detected as a candidate for punctuation mark. Also, in many document layouts, the width between the punctuation mark and the next character is large, so the distance between the detected punctuation mark and the character area located before and after the punctuation mark in the character string area is calculated. Then, it is added as a condition of whether or not it is a punctuation mark that either one of the distances from the character areas adjacent to each other in the front direction and the character areas adjacent to each other in the rear direction is equal to or more than a predetermined value. You may.

With reference to FIG. 15, an example in which punctuation marks are detected in the extracted character area and character string area will be shown. FIG. 15A is a diagram showing that the range surrounded by the dotted line is each character area, and the shaded character area (area M300, area M302) is detected as a punctuation mark. In this case, as shown in the area E300, a dakuten is included in a plurality of characters such as the fifth character "de", but in many cases, another character pixel exists in the line direction, and the front and rear character areas are present. Since the distance to and is small, the detection as a punctuation mark is cancelled.

Next, when there is a punctuation mark whose detection direction has not been determined, the direction analysis processing unit 2104 selects a punctuation mark to be determined in the detection direction (step S324; Yes → step S326), and the selected punctuation mark and the front and back The positional relationship with the character area of is analyzed (step S328).

Regarding the analysis of the positional relationship, first, the direction analysis processing unit 2104 first describes the distance D1 between the punctuation mark and the character area adjacent to the front of the punctuation mark based on the character string direction in the character string area to which the punctuation mark belongs. , Calculate the distance D2 from the adjacent character area behind. When the character string direction is horizontal, the front of the punctuation mark is the left side of the punctuation mark, and the back of the punctuation mark is the right side of the punctuation mark. When the character string direction is vertical, the front of the punctuation mark is the upper side of the punctuation mark, and the back of the punctuation mark is the lower side of the punctuation mark. Then, let p be a positive predetermined coefficient, and set it as a positive coefficient.
D1 + p <D2 ... (Equation 1)
Suppose the punctuation marks have a trailing margin when satisfying. On the other hand, it does not satisfy Equation 1
D2 + p <D1 ... (Equation 2)
Suppose that the punctuation mark has a margin in front when the condition is satisfied. If neither Equation 1 nor Equation 2 is satisfied, it is assumed that the punctuation marks do not have sufficient margins at either the front or the back. If there is not enough margin in either the front or the back, the detected punctuation mark may be regarded as a detection error and the subsequent processing may be interrupted.

Further, the direction analysis processing unit 2104 analyzes the positional relationship between the punctuation marks and the character string region to which the punctuation marks belong (step S330). Specifically, the direction analysis processing unit 2104 sets the minimum value of the component corresponding to the row direction among the position coordinates indicating the detected punctuation mark and the character string area to be S1 and S2, and the maximum value to be E1 and E2. Further, the direction analysis processing unit 2104 defines the intermediate position in the line direction of the character string region as M2 = (S2 + E2) / 2. Then, the direction analysis processing unit 2104 determines that the punctuation marks are in the upper half (or left half) or the lower half (or right half) of the character string area due to the relationship between the punctuation marks positions S1 and E1 and the intermediate position M2 of the character string area. ), Or neither.

As an example of the determination method, the determination can be made based on, for example, the following conditions.
E1 ≤ M2 + β ... (Equation 3)
S1 ≧ M2-β ・ ・ ・ ・ ・ ・ (Equation 4)
When the equation 3 is satisfied, the direction analysis processing unit 2104 assumes that the punctuation marks are in the upper half (or left half) of the character string area. When the equation 4 is satisfied, the direction analysis processing unit 2104 assumes that the punctuation marks are in the lower half (or right half) of the character string area. If both equations 3 and 4 are satisfied, or vice versa, the direction analysis processing unit 2104 indicates that the punctuation marks are around the intermediate position or have a large width over the line direction of the character string area. Assuming that it is neither side. The coefficient β is a coefficient that adjusts the range that satisfies the conditions of Equations 3 and 4, and when it takes a positive value, it is relaxed as a condition for allowing even if it slightly exceeds the intermediate position, but it is negative. When taking a value, the condition becomes stricter.

FIG. 15B is a diagram showing an example of distances D1 and D2, positions S1, S2, E1, E2, and intermediate position M2. The character string area shown in FIG. 15B is a character string in the horizontal direction in which the character areas are arranged in the horizontal direction, and the line direction is the vertical direction. Therefore, the component corresponding to the line direction of the position is the Y component.

The direction analysis processing unit 2104 determines the detection direction for each punctuation mark from the analysis result of the positional relationship between the character area where the punctuation mark is detected and the character area before and after the punctuation mark and the character string area (step S332). Further, the direction analysis processing unit 2104 calculates and adds the score for each direction based on the determined detection direction (step S334). As an example of the method of calculating the score for each direction, for example, one of the most appropriate detection directions can be determined from the combination of the analysis results of the positional relationship described above, and the score for each direction corresponding to the determined detection direction can be added. ..

Subsequently, the direction analysis processing unit 2104 returns to step S324 and determines whether or not there is a punctuation mark whose determination direction is undetermined. At this time, if there are no punctuation marks for which the candidate determination has not been determined, that is, when the addition of the directional scores for all the detected punctuation marks is completed, the direction analysis processing unit 2104 ends the present analysis process (step S324; No).

An operation example of this analysis process will be described with reference to FIG. 16 (a) and 16 (b) are examples of a table showing the relationship between the combination of the analysis results of the positional relationship and the corresponding most appropriate detection direction. FIG. 16A shows a table when the character string direction is horizontal, and FIG. 16B shows a table when the character string direction is vertical.

A case where the character string direction is the horizontal direction will be described as an example with reference to FIG. 16A. When the punctuation mark is in the lower half of the character string area and there is a margin on the right side (backward) in relation to the character area before and after, the direction analysis processing unit 2104 sets the detection direction in the positive direction (0 ° counterclockwise). judge. When the punctuation mark is in the upper half of the character string area and there is a margin on the left side (front) in relation to the character area before and after, the direction analysis processing unit 2104 sets the detection direction in the opposite direction (180 ° counterclockwise). judge. When the punctuation mark is in the upper half of the character string area and there is a margin on the right side (backward) in relation to the previous and next character areas, the direction analysis processing unit 2104 sets the detection direction in the counterclockwise direction (90 ° counterclockwise). ) Is determined. When the punctuation mark is in the lower half of the character string area and there is a margin on the left side (front) in relation to the previous and next character areas, the direction analysis processing unit 2104 sets the detection direction clockwise (270 ° counterclockwise). To judge. If none of the above applies, it is determined that no appropriate detection direction is found, and the direction analysis processing unit 2104 determines that there is no detection (neither direction is the detection direction). When the direction analysis processing unit 2104 determines any of the detection directions, the direction analysis processing unit 2104 adds a direction-specific score corresponding to the detection direction. The score may be a constant or may be dynamically calculated based on the calculated distance and positional relationship.

Further, an example in the case where the character string direction is the vertical direction will be described with reference to FIG. 16 (b). The direction analysis processing unit 2104 has a detection direction when the punctuation mark is in the right half of the character string area when the character string direction is vertical and there is a margin on the lower side (backward) in relation to the previous and next character areas. To determine the positive direction (0 ° counterclockwise). When the punctuation mark is in the left half of the character string area and there is a margin on the upper side (front) in relation to the character area before and after, the direction analysis processing unit 2104 sets the detection direction in the opposite direction (180 ° counterclockwise). judge. When the punctuation mark is in the right half of the character string area and there is a margin on the upper side (front) in relation to the previous and next character areas, the direction analysis processing unit 2104 sets the detection direction in the counterclockwise direction (counterclockwise 90 °). ) Is determined. When the punctuation mark is in the left half of the character string area and there is a margin on the lower side (backward) in relation to the previous and next character areas, the direction analysis processing unit 2104 sets the detection direction in the clockwise direction (counterclockwise 270 °). ) Is determined. Even when the character string direction is the vertical direction, if none of the above applies, the direction analysis processing unit 2104 may determine that no detection is possible, assuming that an appropriate detection direction cannot be found. Further, when the direction analysis processing unit 2104 determines any of the detection directions, the direction-specific score corresponding to the detection direction is added. The score may be a constant or may be dynamically calculated based on the calculated distance and positional relationship.

The detection direction determined by the direction analysis processing unit 2104 when the character string direction is the horizontal direction will be described. FIG. 16C is a diagram showing a character string area E310 in which the character string direction is horizontal. The characters appearing in the character string area E310 have not fallen and indicate a positive direction. Further, the character string area E310 includes a reading point M310. The grid G310 shown in FIG. 16C is for explaining the position of the reading point M310. Here, the reading point M310 is located at the lower left of the grid G310, has a wide margin on the right side in relation to the front and rear character areas, and corresponds to the lower half of the character string area. Therefore, the direction analysis processing unit 2104 can determine that the detection direction of the reading point M310 is the same positive direction as the character direction of the character string area E310.

Another example will be described. FIG. 16D is a diagram showing a character string area E320 in which the character string direction is the horizontal direction. The characters appearing in the character string area E320 face counterclockwise. The character string area E320 includes a reading point M320 at the upper left position of the grid G320. That is, the reading point M320 has a wide margin on the right side in relation to the front and rear character areas, and is in the upper half of the character string area. Therefore, the direction analysis processing unit 2104 can determine that the detection direction of the reading point M320 is the same counterclockwise direction as the character direction of the character string area E320.

Next, the detection direction determined by the direction analysis processing unit 2104 when the character string direction is the vertical direction will be described. FIG. 16E is a diagram showing a character string area E330 in which the character string direction is the vertical direction. The characters appearing in the character string area E330 have not fallen and are facing in the positive direction. The character string area E330 includes a reading point M330 at a position on the upper right of the grid G330. That is, the comma M330 has a wide margin on the lower side in relation to the front and rear character areas, and is located in the right half of the character string area. Therefore, the direction analysis processing unit 2104 can determine that the detection direction of the reading point M330 is the same counterclockwise direction as the direction of the characters in the character string area E330.

Another example will be described. FIG. 16 (f) is a diagram showing a character string area E340 in which the character string direction is the vertical direction. The characters appearing in the character string area E340 are facing counterclockwise. The character string area E340 includes a reading point M340 at a position at the lower right of the grid G340. That is, the comma M340 has a wide margin on the upper side in relation to the front and rear character areas, and is located on the right half of the character string area. Therefore, the direction analysis processing unit 2104 can determine that the detection direction of the reading point M340 is the same counterclockwise direction as the character direction of the character string area E340.

In this way, the direction analysis processing unit 2104 executes the k-direction analysis process in step S304. Then, the direction analysis processing unit 2104 adds the final k-direction score obtained by the k-direction analysis process in step S304 to the overall direction-specific score in step S306, thereby performing another direction analysis process. Add to the directional score obtained by the means.

The k-th direction analysis process may be performed by another method different from the method described above. Further, in the k-direction analysis process, the same method as the direction analysis process described in the first embodiment may be adopted, one adopts part detection by machine learning and the other adopts parts detection by pattern matching. You may make a difference such as.

Further, the direction analysis processing unit 2104 makes a difference from the direction analysis processing by changing the setting method of the target area or the method of giving the score for each direction analysis processing, and the first embodiment. The processing method described in the above may be strengthened.

Further, the direction analysis processing unit 2104 may collectively execute the processes common to the plurality of direction analysis processes, or may share the commonly used data. For example, when the two direction analysis processes include the process of rotating the area image, the direction analysis processing unit 2104 temporarily stores the rotated area image in one direction analysis process and analyzes the other direction. The rotated region image may be reused in the process. By doing so, the direction analysis processing unit 2104 can efficiently execute a plurality of direction analysis processes.

By combining the direction analysis processing in this way, in the present embodiment, it is possible to improve the accuracy of determining the direction of the input image and to execute the direction analysis processing according to the input image.

[4. Fourth Embodiment]
Next, the fourth embodiment will be described. The fourth embodiment is an embodiment in which a language type in a document is determined from the characteristics of a character area extracted from an input image, and direction correction processing corresponding to the language type is executed. In this embodiment, FIG. 3 of the first embodiment is replaced with FIG. The same functional parts and processes are designated by the same reference numerals, and the description thereof will be omitted.

The direction correction unit 210 in the present embodiment corresponds to M (≧ 1) language types, and the direction analysis processing unit 2104 includes different direction analysis processing means for each language type. Here, one language type may be assigned to each language, or languages using similar characters and grammars may be grouped together as one language type. For convenience, the M language types are referred to as the first language type, ..., And the M language type, respectively.

FIG. 17 is a diagram showing a procedure of direction correction processing executed by the direction correction unit 210 in the present embodiment. In the present embodiment, after extracting the area including the elements constituting the characters, the direction analysis processing unit 2104 determines the language type of the input image (step S402). In the present embodiment, after the language type is determined, the number L (1 ≦ L ≦ M) corresponding to the determined language type is determined. This is called language type L.

A known method can be used as the language type determining means, and for example, the language family specified by the method described in Patent Document 3 can be used as the language type. In the method described in Patent Document 3, the language family is determined from the features based on the pixel pattern. For example, the Latin alphabet and the Asian language are determined by the number of features of the pixel pattern per character. Therefore, for example, the direction analysis processing unit 2104 may classify the Latin system as L = 1, the Asian system as L = 2, the other system as L = 3, and M = 3.

When the language type is determined, the direction analysis processing unit 2104 extracts the area image and executes the Lth direction analysis process corresponding to the language type number L for the extracted area image (step S404). As the direction analysis processing means of the direction analysis processing unit 2104, for example, the methods described in the first embodiment and the second embodiment are applied only to the target parts corresponding to the language type of the number L. The direction analysis processing unit 2104 may combine a plurality of different direction analysis processing means for each language type as in the third embodiment.

Further, for example, when a compatible language type is not found (in the above example, “other” of L = 3 corresponds to this case), the direction analysis processing unit 2104 limits the target parts according to the language type. You may set all the target parts as detection targets without doing so. Note that the direction analysis processing unit 2104 may output the input image as it is (counterclockwise at 0 °) without determining the direction of the input image and rotating it, assuming that the language is not supported. good.

According to the present embodiment, by determining the language type and switching the processing based on the language type, it is determined that the enormous target parts corresponding to a plurality of languages are included in the document without being targeted. Direction analysis processing can be executed only for the target parts corresponding to the language type. As a result, the amount of processing required for determining the direction of the input image can be shortened, and erroneous determination can be reduced by avoiding the detection of irrelevant target parts.

[5. Fifth Embodiment]
In the first to fourth embodiments, the configuration in which the image processing device is applied to the image processing device included in the image forming device has been described, but the present invention is not limited to this. Therefore, as a fifth embodiment, a case where the image processing apparatus described in the first to fourth embodiments is applied to an image processing apparatus included in an image reading apparatus such as a flatbed scanner will be described.

The present embodiment can be applied to any of the first to fourth embodiments, but a case where the present embodiment is applied to the first embodiment will be described. In this case, this embodiment replaces FIG. 1 of the first embodiment with FIG. The same functional parts and processes are designated by the same reference numerals, and the description thereof will be omitted.

FIG. 18 is a block diagram showing a configuration of an image reading device 2 including the image processing device 20 according to the present embodiment. As shown in FIG. 18, the image reading device 2 includes an image input device 10, an image processing device 20, a transmission / reception device 40, an operation panel 50, a storage unit 60, and a control unit 70. The image processing device 20 includes an A / D conversion unit 202, a shading correction unit 204, a document type determination unit 206, an ACS determination unit 208, a direction correction unit 210, an input gradation correction unit 212, an area separation processing unit 214, and a color correction unit. It includes 216, a spatial filter processing unit 220, a resolution conversion processing unit 222, an output gradation correction unit 224, a gradation reproduction processing unit 226, and a compression processing unit 228. When the direction correction unit 210 executes the direction correction process, the direction determination process and the rotation process of the input image are executed as in the first embodiment.

Various processes executed by the image reading device 2 are controlled by a control unit 70 (a computer including a processor such as a CPU or DSP) provided in the image reading device 2. In the present embodiment, the image reading device 2 is not limited to a scanner, and is, for example, a digital still camera, a document camera, or electronic devices equipped with a camera (for example, a mobile phone, a smartphone, a tablet terminal, etc.). It may be.

When applying this embodiment to the second to fourth embodiments, the image reading device 2 may be configured with an image processing device 20 that executes the direction correction process described in each embodiment. ..

As described above, according to the present embodiment, it is possible to determine the direction of the input image and correct the direction of the input image even in a device such as a scanner, a digital still camera, or an electronic device equipped with a camera. Become.

[6. 6th Embodiment]
In the first to fifth embodiments, the image processing device 20 that determines the direction of each input image for each image of the document read by the image input device 10 and outputs an image rotated according to the determined direction will be described. bottom. On the other hand, in the sixth embodiment, the image processing device 20 determines the direction of each input image by software in the receiving computer after transmitting the image file as an image file without determining the direction and rotating the input image. It is an embodiment to carry out. In the present embodiment, the user is made to judge whether or not to rotate by notifying the user of the possibility that the direction of the read image is incorrect (not in the positive direction).

FIG. 19 is software of the present embodiment, which determines a recommended direction for each input image (direction to rotate the input image in order to make it positive) and prompts the user to rotate the input image. Is an example of the screen W600 displayed when the above is executed. For the determination of the recommended direction for each input image, for example, among the direction correction processes executed by the direction correction unit 210 described in the first to fourth embodiments, steps S102 to S110 are executed. This can be achieved by making a judgment based on the obtained direction-specific scores.

In the example of FIG. 19, thumbnail images corresponding to the input images are displayed in a list in the area E600 on the left side of the screen W600 in association with the page number, and the thumbnail images corresponding to the page selected by the user are other pages. It is highlighted with a thicker border than the thumbnail image corresponding to. Further, the icon icons M600 and M602 are examples of a method for indicating that the direction of the determined input image is not the positive direction (0 ° counterclockwise). If the direction of the determined input image is not in the positive direction, the user cannot read the page in the correct direction (positive direction) by displaying a figure or character string that calls attention in association with the page number. You can know that there is a possibility.

Further, in the example of FIG. 19, in the area E602 on the right side of the screen W600, the input image corresponding to the page selected by the user and the input image based on the recommended direction determined by the software of the present embodiment are displayed. The rotated image is displayed side by side. Further, the right-pointing arrow A600 arranged between the two images is a process of updating the state in which the input image (original) is rotated based on the recommended direction as the latest state of the image on the page. It accepts instructions. Further, the arrow A602 pointing to the left receives an instruction of a process of returning from the state of being rotated by the above method to the state of the image in the input state, that is, canceling the rotation. The initial state of the image on each page is the state of the image at the time of input, and the latest state of the image is updated for each page by a method such as clicking the left and right arrows described above.

In FIG. 19, the latest state of the selected page is displayed by highlighting the image corresponding to the latest state among the input image and the image rotated based on the recommended direction with a thick frame. It makes it easier for the user to determine which is the case. For each selected page, the user can compare the input image with the image rotated based on the recommended direction, and switch which one to use as needed.

Each time the latest state of each image is switched, the image may be saved according to the latest state, or the selected page is saved at the timing when saving is instructed from the menu provided in the software of the present embodiment. You may. Further, until the software of the present embodiment is terminated, the latest changes of each page may be stored without being discarded, and may be collectively saved at the timing instructed to save from the menu. In addition, a function may be provided to instruct the pages whose recommended direction is not the positive direction (0 ° counterclockwise) to be rotated in the recommended direction at once.

According to the present embodiment, it is possible to determine the direction of the input image and correct the direction of the input image by software not in a device such as an image forming device or an image processing device but in a computer that receives the input image. Become.

[7. Seventh Embodiment]
In the first embodiment, the radical of the Chinese character is given as an example of the target part, but other character types and the parts constituting the character may be defined as the target part as long as they satisfy the criteria. In this embodiment, a case where a part other than the radical of the Chinese character is defined as a target part will be described.

For example, as the target part, Hangul may be used as shown in FIGS. 20 (a) and 20 (b), or hiragana such as “a” or “ki” may be used.

Further, in European languages, the target part may be configured not by a single character or a part of a character but by a word unit composed of a relatively small number of characters by utilizing the fact that a margin is inserted between words. At this time, it is assumed that the pair of character areas in which the distance between the character areas arranged in the character string direction is within a predetermined distance is in a concatenated relationship, and the set of the concatenated character areas is redefined as one character area. Therefore, the word can be treated as one character area. For example, FIGS. 20 (c), 20 (d), and 20 (e) are examples in which words are targeted parts. In addition to improving asymmetry by using words as parts, there is also the advantage that it is easy to separate one line of character string for each word in European languages, so a target area is provided for each separated word and the target part By performing the detection, the efficiency of the detection is also improved.

Further, as shown in FIGS. 20 (d) and 20 (e), it is useful to define a part in which the beginning is capitalized, because the frequency of appearance is reduced, but the frequency of false detection is also reduced.

[8. Modification example]
The present invention is not limited to the above-described embodiments, and various modifications can be made. That is, the technical scope of the present invention also includes embodiments obtained by combining technical means appropriately modified within a range that does not deviate from the gist of the present invention. Further, in the above-described embodiment, color image data is handled assuming that an RGB analog signal is input from the image input device 10 to the image processing device 20, but the present invention is not limited to this, and is not limited to black and white. It may be configured to handle image data.

In addition, although the above-described embodiments are described separately for convenience of explanation, it is needless to say that they may be combined and executed within a technically possible range. For example, the third embodiment and the fourth embodiment may be combined to execute a plurality of direction analysis processing means corresponding to the determined language type.

Further, the program that operates in each device in the embodiment is a program that controls a CPU or the like (a program that causes a computer to function) so as to realize the functions of the above-described embodiment. Then, the information handled by these devices is temporarily stored in a temporary storage device (for example, RAM) at the time of processing, and then stored in various storage devices such as ROM (Read Only Memory) and HDD, if necessary. Is read, modified, and written by the CPU.

Here, as the recording medium for storing the program, a semiconductor medium (for example, ROM, a non-volatile memory card, etc.), an optical recording medium / magneto-optical recording medium (for example, a DVD (Digital Versatile Disc), MO (Magneto Optical), etc.) Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray Disc) (registered trademark), etc.), magnetic recording medium (for example, magnetic tape, flexible disc, etc.) may be used. .. In addition, by executing the loaded program, not only the functions of the above-described embodiment are realized, but also by processing in collaboration with the operating system or other application programs based on the instructions of the program, the present invention In some cases, the functions of the invention are realized.

Further, in the case of distribution to the market, the program can be stored and distributed in a portable recording medium, or transferred to a server computer connected via a network such as the Internet. In this case, it goes without saying that the storage device of the server computer is also included in the present invention.

1 Image forming device 10 Image input device 20 Image processing device 202 A / D conversion unit 204 Shading correction unit 206 Manuscript type judgment unit 208 ACS judgment unit 210 Direction correction unit 2102 Extraction unit 2104 Direction analysis processing unit 2106 Judgment processing unit 2108 Rotation processing Unit 212 Input gradation correction unit 214 Area separation processing unit 216 Color correction unit 218 Black generation undercolor removal unit 220 Spatial filter processing unit 222 Resolution conversion processing unit 224 Output gradation correction unit 226 Gradation reproduction processing unit 228 Compression processing unit 30 Image output device 40 Transmission / output device 50 Operation panel 60 Storage unit 70 Control unit 2 Image reader

Claims (17)

An extraction unit that extracts an image of the area containing elements that make up characters from the input image as an area image,
A detection unit that detects a partial image showing the elements that make up the character from the area image,
A determination unit that determines the direction of the input image based on the detected partial image,
An image processing device comprising. The detection unit detects a partial image from an image obtained by rotating the region image in a plurality of directions.
The claim is characterized in that the determination unit calculates a score for each direction in which the region image is rotated based on the partial image detected by the detection unit, and determines the direction of the input image based on the calculated score. The image processing apparatus according to 1. The image processing apparatus according to claim 2, wherein the image rotated in a plurality of directions includes an image in at least a positive direction. The determination unit is characterized in that the number of partial images included in the region image detected by the detection unit is calculated as a score, and the direction of the input image is determined based on the direction of the region image having the highest score. The image processing apparatus according to claim 2 or 3. The detection unit detects the partial image rotated in a plurality of directions from the region image, and then detects the partial image.
The claim is characterized in that the determination unit calculates a score for each direction in which the partial image is rotated based on the partial image detected by the detection unit, and determines the direction of the input image based on the calculated score. The image processing apparatus according to 1. The image processing apparatus according to claim 5, further comprising a generation unit that generates a partial image rotated in a plurality of directions from the one partial image. 5. The detection unit is characterized in that it detects a partial image from the region image based on at least an image in the positive direction of the partial image and an image rotated in a direction different from the positive direction. The image processing apparatus according to 6. The determination unit calculates the number of partial images included in the region image detected by the detection unit as a score, and determines the direction of the partial image having the highest score as the direction of the input image. The image processing apparatus according to any one of claims 5 to 7. The image processing apparatus according to any one of claims 1 to 8, wherein the partial image is an image showing a radical of a Chinese character. The image processing apparatus according to any one of claims 1 to 9, wherein when the detection unit detects a partial image from a specific position of the region image, the detection is ignored. The detection unit further detects a specific character different from the partial image from the area image, and then detects the specific character.
The determination unit determines the direction of the input image based on the partial image detected by the detection unit and the specific character.
The image processing apparatus according to any one of claims 1 to 10. A language type determination unit for determining a language type from the input image is further provided.
The image processing apparatus according to claim 1, wherein the detection unit detects a partial image corresponding to the language type from the region image. Claims 1 to 12 further include a correction unit that corrects the direction of the input image to be the positive direction when the direction of the input image determined by the determination unit is other than the positive direction. The image processing apparatus according to any one of the above. The image processing apparatus according to any one of claims 1 to 13.
An image input unit that reads an image and inputs the scanned image to the image processing device,
An image reading device comprising. A step of extracting an image of an area containing elements constituting characters from an input image as an area image, and
A step of detecting a partial image showing an element constituting a character from the area image, and
A step of determining the direction of the input image based on the detected partial image, and
A determination method characterized by including. On the computer
A function to extract an image of an area containing elements that make up characters from an input image as an area image,
A function to detect a partial image showing elements constituting characters from the area image, and
A function of determining the direction of the input image based on the detected partial image, and
A program characterized by realizing. A computer-readable recording medium on which the program according to claim 16 is recorded.

Images