JP6635057B2 - Image reading apparatus, image reading method, image forming apparatus, and image reading program - Google Patents

Description

  The present invention relates to an image reading apparatus, and more particularly to an image reading apparatus capable of correcting the direction of a scanned document image.

  2. Description of the Related Art An image reading apparatus capable of reading an image of a document and correcting the orientation of the image by image processing has been proposed. Specifically, for example, Patent Document 1 proposes a technique of recognizing characters in a document image and rotating the document image according to the recognition result.

JP-A-8-202887

  However, the prior art is based on the premise that attention is paid to the direction of the document image and the direction of the entire document image is corrected. On the other hand, some originals are hand-written on a single original sheet by various users in various directions by brainstorming or other methods. Such a document contains characters in various directions, and thus cannot be dealt with by the related art.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a technique for aligning the directions of images displayed in a plurality of directions in a document image.

  The present invention provides an image reading apparatus that reads an image of a document and generates image data. The image reading device, a character recognition processing unit that recognizes a character represented by the generated image data, an image direction determination unit that determines the angle of the recognized character based on the recognition result, At least one most character area including the most characters having substantially the same angle, and a plurality of characters having substantially the same inclination angle with respect to the most character area and existing within a preset range. An image area extraction unit that extracts at least one inclined image area, and the image data is corrected so that each of the directions of the at least one inclined image area matches the direction of the most character area. An image processing unit having an image area rotating unit that aligns the direction of at least one inclined image area with the direction of the most character area.

  The present invention provides an image reading method for generating image data by reading an image of a document. The image reading method includes a character recognition processing step of recognizing a character represented by the generated image data, an image direction determination step of determining an angle of the recognized character based on the recognition result, At least one most character area including the most characters having substantially the same angle, and a plurality of characters having substantially the same inclination angle with respect to the most character area and existing within a preset range. An image area extracting step of extracting at least one inclined image area including: correcting the image data so that each of the directions of the at least one inclined image area matches the direction of the most character area. An image processing step having an image area rotating unit for aligning the direction of at least one inclined image area with the direction of the most character area.

  The present invention provides an image forming apparatus for forming an image on a print medium. The image forming apparatus includes the image reading device and an image forming unit that forms an image on a print medium based on image data in which the image directions of the plurality of originals are aligned.

  The present invention provides an image reading program for controlling an image reading apparatus that reads images of a plurality of documents and generates image data. The image reading program includes: a character recognition processing unit that recognizes a character represented by the generated image data; an image direction determination unit that determines an angle of the recognized character based on the recognition result; At least one majority character area including substantially the same maximum number of characters, and at least including a plurality of characters having substantially the same inclination angle with respect to the maximum number character area and existing within a preset range An image region extracting unit that extracts one inclined image region; and correcting the image data so that each of the directions of the at least one inclined image region matches the direction of the most character region. The image reading device functions as an image processing unit having an image region rotating unit that aligns the directions of the two inclined image regions with the direction of the most character region.

  According to the present invention, it is possible to align the directions of images displayed in a plurality of directions in a document image.

1 is a block diagram illustrating a functional configuration of an image forming apparatus according to an embodiment of the present disclosure. 6 is a flowchart illustrating the content of an image area alignment process according to one embodiment. FIG. 9 is an explanatory diagram illustrating the content of an image area alignment process according to an embodiment. It is a flowchart which shows the content of the grouping process which concerns on one Embodiment. FIG. 9 is an explanatory diagram illustrating the content of a grouping process according to an embodiment. FIG. 7 is an explanatory diagram illustrating details of a tilted image area rotation process and an image position adjustment process according to an embodiment.

  Hereinafter, embodiments for carrying out the present invention (hereinafter, referred to as “embodiments”) will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a functional configuration of an image forming apparatus 100 according to an embodiment of the present invention. The image forming apparatus 100 includes a control unit 110, an image forming unit 120, an operation display unit 130, a storage unit 140, and an automatic document feeder (ADF) 160. The control unit 110 includes an OCR processing unit 111 (also called a character recognition processing unit), an image direction determination unit 112, and an image area extraction unit 113. The image forming unit 120 includes an image reading unit 121 that reads an image of a document to generate image data, and an image processing unit 122, and forms an image on a print medium. The image processing unit 122 includes an image area rotation unit 122a and an image area adjustment unit 122b.

  The control unit 110 includes main storage means such as a RAM and a ROM, and control means such as an MPU (Micro Processing Unit) and a CPU (Central Processing Unit). The control unit 110 has a controller function related to various interfaces such as I / O, USB (Universal Serial Bus), bus, and other hardware, and controls the entire image forming apparatus 100.

  The storage unit 140 is a storage device including a non-temporary recording medium such as a hard disk drive and a flash memory, and stores a control program and data of a process executed by the control unit 110. The storage unit 140 stores a character recognition dictionary 141 and a character string dictionary 142. The character recognition dictionary 141 and the character string dictionary 142 may be stored in a server (not shown).

  The character recognition dictionary 141 is a character recognition dictionary used for estimating the direction of a character image, and stores feature vector information (also called a template) of each character. Here, the corresponding characters include hiragana, katakana, kanji, numerals and alphabets. The character string dictionary 142 is a dictionary that stores character strings as words composed of one or more characters.

  FIG. 2 is a flowchart illustrating the content of the image area alignment processing according to the embodiment. FIG. 3 is an explanatory diagram showing the contents of the image area alignment processing according to one embodiment. The image area alignment processing is processing for aligning the directions of images (image areas in this example) displayed in a plurality of directions in a document image. The image area alignment process is typically performed in a process of reading a document that has been handwritten on a single document sheet by various users from various directions by brainstorming or other methods.

  In step S10, the user operates the operation display unit 130 to set the operation mode to the image rotation scan mode. In step S20, the user presses a start button (not shown) and uses the image reading unit 121 to scan a document sheet. The image reading unit 121 reads an original sheet to generate image data.

  In step S30, the image direction determining unit 112 detects the angle of each character using the OCR processing unit 111. The OCR processing unit 111 performs a character recognition process (OCR process) using the character recognition dictionary 141. In the character recognition process, the OCR processing unit 111 recognizes a character assuming that the angle of each character represented by the image data is unknown.

  Specifically, the OCR processing unit 111 executes the character recognition process for each of a plurality of recognition directions, assuming, for example, a tilt at a preset angle (for example, every 2 degrees). The image direction determination unit determines an angle (recognition direction) at which the character is recognized and has the highest recognition accuracy as the angle of each character based on the recognition result by the OCR processing unit 111. The recognition accuracy is, for example, a value representing the coincidence between the feature vector information of each character stored in the character recognition dictionary 141 and the feature vector information obtained from the image data by a numerical value having a range of 0 to 1.0.

  In step S40, the image direction determination unit 112 executes the most character angle determination processing. In the most character angle determination process, the image direction determination unit 112 determines an angle at which the number of characters having substantially the same angle is the largest. In the document D1 of FIG. 3, the angle of 180 degrees (vertical inversion), which is the angle of the characters constituting the character group T1 composed of a plurality of characters, is determined as the maximum character angle.

  In this example, the most frequent character angle is determined only by the character group T1, but in the most frequent character angle determination process, the number of characters having the same angle may exist in a plurality of regions. In addition, the substantially same angle allowable range can be set by the user, and for example, an error of 5 degrees can be set as the allowable range.

  In step S50, the image area extraction unit 113 performs a grouping process. The grouping process is a process of collecting images assumed to be integrally drawn by a user into one group.

  FIG. 4 is a flowchart illustrating the content of the grouping process according to the embodiment. In step S51, the image area extraction unit 113 executes a nearby character detection process. In the nearby character detection processing, the image area extraction unit 113 detects a plurality of characters having substantially the same angle and existing within a preset range. In the document D1 shown in FIG. 3, a character group T2 composed of two characters, for example, "original" and "cause" is detected.

  FIG. 5 is an explanatory diagram illustrating the content of the grouping process according to the embodiment. FIG. 5A shows a method of processing the character group T2. In this example, the angle θ1 of the “original” character constituting the character group T2 is substantially the same as the angle θ2 of the “cause” character. The term “substantially the same” means that the setting can be made by the user and is within a preset error range (for example, within a 5-degree error range).

  In step S52, the image area extraction unit 113 performs a positional relationship determination process. In the positional relationship determination process, the image area extraction unit 113 uses the vertical difference δ1 with respect to the “original” and “cause” characters that constitute the character group T2, so that the two characters are located at substantially the same position. Check that it is located. Substantially the same means that it can be set by the user and is within a preset error range (for example, one fifth of the character height). Note that, in the example of the character group T3 of the original D1 in FIG. 3, both characters are represented using a horizontal difference δ (not shown) based on the characters “gender” and “noh” constituting the character group T3. Can be confirmed to be arranged at substantially the same position.

  In step S53, the image area extraction unit 113 executes a character string determination process. In the character string determination process, the image area extracting unit 113 combines the character group T2 as a character string candidate “cause”, and refers to the character string dictionary 142 for the character string of “cause”. When the “cause” as a character string candidate is registered in the character string dictionary 142, the image area extracting unit 113 determines that the character group T2 is a character string.

  In step S54, if it is determined that the character group T2 is a character string, the image region extraction unit 113 proceeds to step S55. If it is determined that the character group T2 is not a character string, the image region extraction unit 113 proceeds to step S55. Two image regions including each of “original” and “factor” are determined as each inclined image region, and the process proceeds to step S56. In this example, since it is determined that the character group T2 is a character string, the process proceeds to Step S55.

  In step S55, the image area extraction unit 113 executes a character string grouping process. In the character string grouping process, the image region extracting unit 113 extracts an image region of a region including the character group T2 (an ellipse in this example, but may be a rectangle, for example). At this time, the image area extraction unit 113 sets the angle of each character of the characters “original” and “cause” to the average angle of θ1 and θ2 (or any angle) in the character string, and determines the difference δ in the vertical direction. Can be adjusted so that is zero.

  The image area extraction unit 113 calculates the difference between the angle of the character string of “cause” and the most frequent character angle as the inclination angle. Thereby, the image area extracting unit 113 can extract the inclined image area RS1 (see the document D2 in FIG. 3) inclined at the inclination angle calculated with respect to the most frequent character angle.

  In step S56, the image area extraction unit 113 performs a nearby image detection process. In the nearby image detection processing, the image region extracting unit 113 extracts a non-character image that is not recognized as a character as a related image included in a part of the inclined image region. However, the image area extraction unit 113 performs extraction on the assumption that the angle of the non-character image is not different from the character group T2. The vicinity means within a range of a preset distance (for example, within a range of the size of one character) in an arbitrary direction (can be set by a user).

  In the example of the character group T2, a non-character image G1 exists near the character group T2. However, the non-character image G1 is provided with the direction symbol S2, whereby the angle is defined such that the direction indicated by the direction symbol S2 as an arrow is upward. Therefore, the angle of the non-character image G1 is different from the angle of the character group T2. The non-character image G1 is extracted as a tilted image region RS2 (see the document D2 in FIG. 3) tilted at the tilt angle calculated using the angle defined by the direction symbol S2.

  As described above, in the present embodiment, when intentional designation of the direction of the non-character image is desired, the inclined image area RS2 is specified by drawing the curve S1 surrounding the non-character image with a color pen of a predetermined color. By drawing the direction symbol S2 (arrow) so as to overlap the curve S1, the direction of the inclined image area RS2 can be designated.

  When the direction is not specified by the curve S1 and the direction symbol S2 in the non-character image, and a character related to the vicinity of the non-character image is not detected, an inquiry message is displayed on the operation display unit 130. Thus, the user may be able to specify the direction of the non-character image on the operation display unit 130.

  In step S57, the image area extraction unit 113 determines whether the non-character image G1 is a related image. If it is determined that the non-character image G1 is a related image, the image region extracting unit 113 proceeds to step S58. If it is determined that the non-character image G1 is not a related image, the image region extracting unit 113 proceeds to step S58. Proceed to S60 (see FIG. 2). In this example, since it is determined that the non-character image G1 is not a related image, the process proceeds to step S60.

  On the other hand, in the example of the character group T3, since the directions of the two non-character images G1 and G2 are not defined, the image area extracting unit 113 determines that the two non-character images G1 and G2 are related to the character group T3. It is determined that the image is an image. Further, the image area extracting unit 113 determines that the two non-character images G1 and G2 are also related images of the character group T4 for the character group T4 of the document D1 in FIG. Therefore, the image area extracting unit 113 can extract an image area including the two character groups T3 and T4 and the two non-character images G1 and G2 as the inclined image area RS3.

  In step S58, the image area extracting unit 113 extracts an area including the character group T1 as the most numerous character area RS0, and also extracts other inclined image areas RS4 and RS5. Through such processing, the image area extracting unit 113 extracts the most numerous character area RS0 and the five inclined image areas RS1 to RS5.

  In step S60, the image area rotation unit 122a performs an image rotation process. In the image rotation process, the image region rotation unit 122a rotates the entire image so that the angle of the most numerous character region RS0 matches the preset reference direction, and generates the image D3 shown in FIG. I do.

  FIG. 6 is an explanatory diagram illustrating the contents of the tilted image area rotation processing and the image position adjustment processing according to the embodiment. FIG. 6 shows an image D4 that has been subjected to the tilted image area rotation processing and an image D5 that has been subjected to the image position adjustment processing.

  In step S70, the image area rotation unit 122a performs a tilt image area rotation process. In the tilted image area rotation process, the image area rotating unit 122a rotates the five tilted image areas RS1 to RS5 by correcting the image data, and adjusts the maximum number of the tilted image areas RS1 to RS5 so as to match the direction of the most numerous character area RS0. Align in the direction of the character area RS0. Thus, the image area rotating unit 122a can generate image data representing the image D4 shown in FIG.

  In step S80, the image area adjustment unit 122b executes an arrangement position adjustment process. In the arrangement position adjustment processing, the image area adjustment unit 122b detects interference between the most frequent character area RS0 and the five inclined image areas RS1 to RS5 after rotation. In the example of the image D4, interference occurs between the tilt image region RS3 and the tilt image region RS5. The tilted image area RS5 further deviates from a preset image range.

  First, the image area adjusting unit 122b reduces the tilted image area RS3 and the tilted image area RS5, and attempts to eliminate interference between the tilted image area RS3 and the tilted image area RS5. When the interference can be eliminated within the range of a preset reduction ratio (for example, user setting is possible at 80% or more), at least one of the inclined image region RS3 and the inclined image region RS5 is provided. To shrink. In this example, the image area adjustment unit 122b attempts to change the position of at least one of the inclined image area RS3 and the inclined image area RS5 because the interference cannot be eliminated by the reduction processing.

  The image area adjustment unit 122b searches the image D4 for a blank area where either the tilted image area RS3 or the tilted image area RS5 can be placed, and the area where the tilted image area RS5 can be placed below the tilted image area RS3. Is detected. The image area adjustment unit 122b finely adjusts the positional relationship between the five inclined image areas RS1 to RS5 so that the inclined image area RS5 is moved below the inclined image area RS3 and excessive crowding does not occur.

  Thus, the image area adjustment unit 122b can eliminate the interference between the areas and generate image data representing the image D5 in which the arrangement balance of each image area is achieved. The image data can be formed on a print medium or transmitted to a user. At this time, the curve S1 and the direction symbol S2 drawn by the color pen are deleted. Further, the character can be replaced with a character image of the same size drawn by a font based on the recognition result. Further, the image area adjustment unit 122b can also adjust the size of the characters.

  As described above, the image forming apparatus 100 according to the embodiment can align the directions of the images displayed in a plurality of directions in the original image and replace the images with character images of the same size drawn by fonts. . As described above, the image forming apparatus 100 can automatically finish a document in which the directions of characters and images are aligned without having to copy the document. As a result, while allowing a plurality of users to write by hand on a single document sheet from various directions by brainstorming or other methods, the result can be automatically finished as an easy-to-read document.

  The present invention can be implemented not only in the above embodiment but also in the following modified examples.

  Modification Example 1 In the above embodiment, the directions and positions of the respective image regions are all automatically aligned, but it is not always necessary to automatically perform the processing. The image forming apparatus 100 may be configured such that, for example, the image D5 is preview-displayed on the operation display unit 130, and the user can adjust the direction and position of each image area on the operation display unit 130.

  Specifically, for example, in the above-described example, when the angle of the character group T3 and the angle of the character group T4 are different, the non-character images G1 and G2 are set as the related images of the character group T3 and the character group T4. May be confirmed by inquiring of the user, or a related image closer to the character group T3 and the character group T4 may be used.

  Further, the direction and position of each image area may be adjusted using a display of a personal computer, a tablet, or a smartphone connected via a network or close proximity wireless communication.

  Modification 2 In the above embodiment, the present invention is applied to an image forming apparatus, but can also be applied to an image reading apparatus.

100 image forming apparatus 110 control unit 111 OCR processing unit (character recognition processing unit)
112 Image direction determination unit 113 Image area extraction unit 120 Image formation unit 121 Image reading unit 122 Image processing unit 122a Image area rotation unit 122b Image area adjustment unit 130 Operation display unit 140 Storage unit 141 Character recognition dictionary 142 Character string dictionary 160 Automatic document Feeder (ADF)

Claims (10)

An image reading apparatus that reads an image of a document and generates image data,
A character recognition processing unit that recognizes a character represented by the generated image data,
An image direction determination unit which determines the angle of the pre-Symbol recognized characters,
At least one majority character area including the majority of characters having substantially the same angle, and a plurality of characters having substantially the same inclination angle with respect to the majority character area and existing within a preset range An image region extraction unit that extracts at least one inclined image region including:
Correcting the image data so that the direction of the at least one inclined image region is the direction of the most character region so that each of the directions of the at least one inclined image region matches the direction of the most character region; An image processing unit having an image area rotating unit that aligns
An image reading device comprising: The image reading device according to claim 1,
The image direction determination unit causes the character recognition processing unit to recognize a character represented by the generated image data from a plurality of recognition directions, and calculates recognition accuracy of the character for each of the plurality of recognition directions. An image reading device that determines the angle of the character based on the recognition direction having the highest recognition accuracy. The image reading device according to claim 1, further comprising:
The image processing unit detects interference in an area between the largest character area and a tilted image area aligned in the direction of the largest character area, and the largest character area so as to eliminate the interference. An image reading apparatus, comprising: an image area adjusting unit that reduces an area and at least one of the aligned inclined image areas. The image reading device according to claim 3,
The image area adjustment unit, when determining that the interference cannot be eliminated by simply reducing the inclined image area aligned with the majority character area and the aligned image area, reduces the interference so as to eliminate the interference. An image reading apparatus for moving at least one of a multi-character area and at least one of the aligned inclined image areas with or without reduction. The image reading device according to claim 3 or 4,
The image area adjustment unit determines whether the aligned inclined image area deviates from a preset image range, and determines the inclined character area aligned with the most character area so as to eliminate the deviation. An image reading device that moves at least one of the image areas while reducing or not reducing the size. The image reading device according to claim 1, wherein:
The image area extracting unit may include a non-character image that is not recognized as a character, and an image area that is closest to the non-character image among image areas including the most numerous character area and the at least one inclined image area. An image reading device that is included and extracted. The image reading device according to claim 1, wherein
The image area extracting unit may tilt a plurality of characters present in the preset range, wherein the plurality of characters are arranged at substantially the same position in a vertical direction or a horizontal direction based on the plurality of characters. An image reading device that extracts as an image area. An image reading apparatus according to claim 1,
An image forming unit that forms an image on a print medium based on image data in which the image directions of the plurality of documents are aligned,
An image forming apparatus comprising: An image reading method for reading an image of a document to generate image data,
A character recognition processing step of recognizing a character represented by the generated image data,
An image direction determination step of determining the angle of pre-Symbol recognized characters,
At least one majority character area including the majority of characters having substantially the same angle, and a plurality of characters having substantially the same inclination angle with respect to the majority character area and existing within a preset range An image area extracting step of extracting at least one tilted image area including:
Correcting the image data so that the direction of the at least one inclined image region is the direction of the most character region so that each of the directions of the at least one inclined image region matches the direction of the most character region; An image processing step to align
An image reading method comprising: An image reading program for controlling an image reading device that reads an image of a document and generates image data,
A character recognition processing unit that recognizes a character represented by the generated image data;
Image direction determination unit which determines the angle of the pre-Symbol recognized characters,
At least one majority character area including the majority of characters having substantially the same angle, and a plurality of characters having substantially the same inclination angle with respect to the majority character area and existing within a preset range An image region extraction unit that extracts at least one inclined image region including: and correcting the image data so that each of the directions of the at least one inclined image region matches the direction of the most numerous character region, An image reading program that causes the image reading device to function as an image processing unit having an image region rotating unit that aligns the direction of the at least one inclined image region with the direction of the most character region.