JP6676955B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing device and an image processing program.

  Patent Document 1 discloses that, when detecting a line that is to be treated separately from a body line group, such as a ruby / footnote line for a body line, the target ruby line can be detected anywhere in the manuscript (without setting a precondition). It is an object of the present invention to improve the extraction accuracy of a character string by using the detection result, and to select one of all line rectangles extracted from the target character string by a character circumscribed rectangle integration method as a reference line, Using half the line height as a threshold, each cut line is determined to be a ruby line or not (text line) based on the line height, and the reference line selection process is performed on a plurality of lines to be subjected to recognition processing. Apply the membership function to the row width and row height, calculate the sum of each evaluation value, select the row with the maximum sum, and determine whether the row is a ruby row or not. Delete the row data of the row, or check the row data with the It is disclosed that outputs to the processing unit.

  Patent Literature 2 has an object to enable extraction of a character line from a document in which the character line direction is unknown and the character size and pitch of the character and the region of the character line direction and the like having different character line directions are mixed. The line candidate generation unit connects the pixel connection rectangles generated from the image data to generate character line candidates, and the character rectangle generation unit sets the generated character line candidates as non-divided regions in the main direction and the sub direction as appropriate. In addition to extracting the character lines in the sub-direction and extracting the character rectangles in the sub-direction, the character rectangles are generated by integrating the basic rectangles in the character line candidates. It is possible to determine the configuration of the character line candidates, divide the character line candidates according to the configuration, and extract the character lines by connecting the character rectangles in the divided character line candidate regions at the character rectangle connection unit. It has been disclosed.

Japanese Patent Application Laid-Open No. 2004-094292 JP-A-10-031716

It has been practiced to delete ruby from images with ruby added to the text. For example, this is to improve the character recognition rate in character recognition of the text.
In the above-mentioned patent document, ruby is extracted by one judgment.
SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus and an image processing program capable of extracting a pixel block that was not ruby in the first extraction as ruby by using the result of the first ruby extraction. I have.

The gist of the present invention to achieve this object lies in the inventions in the following items.
The invention according to claim 1 includes a first extraction unit for extracting ruby added to a text from an image, a center line extraction unit for extracting a center line of the text, a pixel block in the image and a pixel block in the image. the distance between the center line adjacent, if the is ruby and the distance is greater than or more between the center lines adjacent to the ruby, have a second extraction means for extracting a該画pixel block as ruby, the The center line extracting unit is an image processing device that extrapolates the center line outside the center line group according to the statistical interval between the center lines already extracted and the position of the pixel block at the head of the center line .

  According to a second aspect of the present invention, there is provided a deleting unit for deleting ruby extracted by the first extracting unit and ruby extracted by the second extracting unit from the image, and deleting ruby from the image by the deleting unit. The image processing apparatus according to claim 1, further comprising an output unit that outputs the output image.

  The invention according to claim 3, wherein the distance between the ruby and the center line adjacent to the ruby is a distance between the center line and a corner close to the center line of a circumscribed rectangle of the ruby; 3. The image processing apparatus according to claim 1, wherein a distance between the center lines adjacent to the block is a distance from the center of a circumscribed rectangle of the pixel block to the center line. 4.

The invention according to claim 4 , wherein the first extracting means performs RLSA processing in a predetermined direction of the image, and a predetermined multiple of a first pixel block of a processing result by the RLSA means. A third extraction unit for extracting a second pixel block having a larger or larger size, wherein a distance between the first pixel block and the second pixel block is less than or less than a predetermined distance. In some cases, the first pixel block is extracted as ruby, or the first pixel block overlaps with the second pixel block, and the first pixel block is smaller than the second pixel block. And a ruby extracting means for extracting the first pixel block as ruby when the position of the first pixel block is within a predetermined area in the second pixel block. 3. The image processing device according to claim 3.

According to a fifth aspect of the present invention, there is provided a computer, comprising: a first extraction unit for extracting ruby added to a text from an image; a center line extraction unit for extracting a center line of the text; a pixel block in the image; When the distance between the center lines adjacent to the pixel block is greater than or equal to the distance between the ruby and the center line adjacent to the ruby, the pixel block functions as second extraction means for extracting the pixel block as ruby An image processing program that extrapolates the center line outside the center line group according to the statistical interval between the center lines already extracted and the position of a pixel block at the head of the center line. It is.

According to the image processing apparatus of the first aspect, a pixel block that is not ruby in the first extraction can be extracted as ruby using the result of the first ruby extraction. In addition, the center line can be extrapolated outside the center line group according to the statistical interval between the center lines already extracted and the position of the pixel block at the head of the center line.

  According to the image processing device of the second aspect, it is possible to output an image from which ruby has been deleted.

  According to the image processing apparatus of the third aspect, the distance between the ruby and the center line adjacent to the ruby is a distance between the center line of the circumscribed rectangle of the ruby and the distance from the center line, and the pixel block and the A process can be performed in which the distance between the center lines adjacent to the pixel block is the distance from the center of the circumscribed rectangle of the pixel block to the center line.

According to the image processing device of the fourth aspect , ruby can be extracted using the RLSA process.

According to the image processing program according to claim 5, by using the extraction result of the first ruby, a first extract may be extracted pixel block that has not been a ruby as ruby. In addition, the center line can be extrapolated outside the center line group according to the statistical interval between the center lines already extracted and the position of the pixel block at the head of the center line.

FIG. 2 is a conceptual module configuration diagram illustrating a configuration example according to the present embodiment. FIG. 2 is an explanatory diagram illustrating an example of a system configuration using the present embodiment. 5 is a flowchart illustrating a processing example according to the exemplary embodiment; 5 is a flowchart illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; 5 is a flowchart illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; 5 is a flowchart illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; 5 is a flowchart illustrating a processing example according to the exemplary embodiment; FIG. 9 is an explanatory diagram illustrating a processing example according to the exemplary embodiment; FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer that implements the exemplary embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the present embodiment.
Note that a module generally refers to a component such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is directed to a computer program (program for causing a computer to execute each procedure, a program for causing a computer to function as each means, And a description of the system and method. However, for the sake of explanation, the words “store”, “store”, and equivalent words are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored. This means that control is performed so as to be stored in the device. Also, the modules may correspond to the functions one-to-one, but in implementation, one module may be configured with one program, a plurality of modules may be configured with one program, and conversely, one module may be configured. May be composed of a plurality of programs. Further, a plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include another module. Hereinafter, the term “connection” is used not only for a physical connection but also for a logical connection (data transmission / reception, instruction, reference relationship between data, and the like). "Predetermined" means that the process is determined before the process of interest, not only before the process according to the present embodiment starts, but also after the process according to the present embodiment starts. Also, before the target processing, it is used in accordance with the situation / state at that time or with the intention of being determined according to the situation / state up to that time. When there are a plurality of “predetermined values”, the values may be different from each other, or two or more values (including all values, of course) may be the same. In addition, the description having the meaning of “if A, do B” is used to mean “determine whether or not A, and if it is A, perform B”. However, this does not apply to cases where it is not necessary to determine whether or not it is A.
In addition, a system or a device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one communication connection), and also includes one computer, hardware, and device. Etc. are also included. “Apparatus” and “system” are used as terms having the same meaning. Needless to say, the “system” does not include anything that is merely a social “mechanism” (social system) that is an artificial arrangement.
In addition, for each process performed by each module or when a plurality of processes are performed in a module, target information is read from a storage device, and after performing the process, a processing result is written to the storage device. is there. Therefore, the description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Note that the storage device here may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The image processing apparatus 100 according to the present embodiment extracts ruby from a document image. As shown in the example of FIG. 1, an image receiving module 110, a ruby extraction (A) module 120, a text center line extraction It has a module 130, a ruby extraction (B) module 140, a ruby deletion module 150, and a text image output module 160. The image receiving module 110, the ruby extraction (A) module 120, the text center line extraction module 130, the ruby extraction (B) module 140, the ruby deletion module 150, and the text image output module 160 are connected to each other.
In recognizing a document image, if the ruby is still attached, the recognition rate cannot be improved. For example, it is necessary to extract ruby when an old document is converted to text and used as a database. Note that ruby here generally means furigana. As for the reading of kanji, etc., it is usually written with characters smaller than the characters in the text (generally, hiragana, katakana, etc.) on the right of the kanji etc. in vertical writing and on the kanji etc. in horizontal writing . Further, in the present embodiment, ruby includes a side point to be emphasized for emphasis.
In particular, when ruby is close to the text, when there is much noise in the entire image, or when there is geometric distortion in the image, it is difficult to completely separate and extract ruby and text lines. In particular, in old documents, ruby is more similar to the main text than in modern sentences, and since the original document itself is old, the entire image contains more noise, and the document itself is valuable because it is valuable. When the image is read by a scanner, the image cannot be handled such as pressing against a platen glass, so that a geometrical distortion often occurs in the read image.
The description will be mainly given of vertical writing. The processing for horizontal writing can be handled by rotating by 90 degrees. That is, "scanning in the vertical direction" can be dealt with as "scanning in the horizontal direction", and "right side of the text" can be read as "top of the text" or the like. Also, in the case of horizontal writing, the image receiving module 110 performs a process equivalent to vertical writing by performing a 90-degree rotation to the right, and performs a 90-degree rotation to the left to return to the original state when outputting. Is also good.

  The image receiving module 110 receives an image and passes the image to another module (for example, the ruby extraction (A) module 120). Accepting an image includes, for example, reading an image with a scanner, a camera, or the like, receiving an image from an external device via a communication line by facsimile, etc., a hard disk (aside from a built-in computer, and a network) Read-out of an image stored in an image (e.g., an image or the like connected to an image). The image may be a binary image or a multi-value image (including a color image). The number of received images may be one or more. The image has ruby added to the text, and the content of the image may be an old document as described above, a document used for business, a pamphlet for advertising, or the like.

The ruby extraction (A) module 120 extracts ruby added to the text from the image received by the image receiving module 110. For example, a known technique described in Patent Literature 1, Patent Literature 2, or the like may be used.
The ruby extraction (A) module 120 may perform the following processing. Further, the processing result by the ruby extraction (A) module 120 may be used as the ruby extraction result by the image processing apparatus 100. In that case, the image processing apparatus 100 may be configured by the image receiving module 110, the ruby extraction (A) module 120, the ruby deletion module 150, and the text image output module 160.
(1) The RLSA process is performed in a predetermined direction of the image received by the image receiving module 110. Here, the “predetermined direction” is the vertical direction when the document image is a vertically written document image, and is the horizontal direction when the document image is a horizontally written document image.
(2) A second pixel block having a size that is greater than or greater than a predetermined multiple of the first pixel block as a result of the processing of (1) is extracted. The first pixel block may be any pixel block as long as it is a processing result of the RLSA process.
(3) When the distance between the first pixel block and the second pixel block is less than or less than a predetermined distance, the first pixel block is extracted as ruby, or the first pixel block is extracted. The chunk and its second pixel chunk overlap, the first pixel chunk is smaller than the second pixel chunk, and the location of the first pixel chunk is a predetermined area within the second pixel chunk. If it is within, the first pixel block is extracted as ruby.
The "predetermined multiple" may be any multiple (real number) of 1 or more, and includes, for example, 2 times and 2.5 times.
The “predetermined region” is a region in the direction in which ruby is applied when the second pixel block is set as the text. Specifically, when the vertical writing is performed, the second pixel block is used. Of the second pixel block in the case of horizontal writing.

The text center line extraction module 130 extracts a text center line in the image from which the ruby is extracted by the ruby extraction (A) module 120.
Further, the text center line extraction module 130 may extract the center line by connecting the centers of pixels having a shape similar to a square in the image.
Further, the text center line extraction module 130 may interpolate the center lines between the center lines according to the statistical intervals between the center lines already extracted. The “statistical interval” is a result of performing a statistical process on a distance between a plurality of center lines. For example, an average value, a median value, a mode value, and the like of the distance between the center lines are calculated. Say.
In addition, the text center line extraction module 130 extrapolates the center line outside the center line group according to the statistical interval between the center lines already extracted and the position of the pixel block at the head of the center line. Is also good.

If the distance between the pixel block in the image and the center line adjacent to the pixel block is greater than or equal to the distance between ruby and the center line adjacent to the ruby, the ruby extraction (B) module 140 determines that pixel Extract the mass as ruby. As the “center line adjacent to ruby”, for example, there is a center line closest to the ruby. Furthermore, in the case of vertical writing, the condition that the center line is on the left side of the ruby (the ruby is on the right side of the center line) may be added as a condition. In the case of horizontal writing, it may be added as a condition that there is a center line below the ruby (the ruby exists above the center line).
Similarly, the “center line adjacent to the pixel block” includes, for example, a center line closest to the pixel block. Furthermore, in the case of vertical writing, the condition that a center line is on the left side of the pixel block (the pixel block is on the right side of the center line) may be added as a condition. In the case of horizontal writing, a condition that a center line exists below the pixel block (the pixel block exists above the center line) may be added as a condition.
Here, “distance between ruby and a center line adjacent to the ruby” may be a distance between the center line and a corner of the circumscribed rectangle of the ruby that is close to the center line. Then, the “distance between the pixel block and the center line adjacent to the pixel block” may be the distance between the center line of the circumscribed rectangle of the pixel block and the center line.

The ruby deletion module 150 deletes the ruby extracted by the ruby extraction (A) module 120 and the ruby extracted by the ruby extraction (B) module 140 from the image. This generates an image of only the text.
The body image output module 160 outputs an image in which ruby has been deleted from the image by the ruby deletion module 150. Outputting an image here means, for example, printing with a printing device such as a printer, displaying on a display device such as a display, transmitting an image with an image transmitting device such as a fax, or an image storage device such as an image database. Writing an image to a storage medium such as a memory card, passing the image to another information processing device (for example, a character recognition device or the like), and the like.

FIG. 2 is an explanatory diagram showing a system configuration example using the present embodiment.
The example illustrated in FIG. 2A is a system configured by the image reading device 210, the image processing device 100, and the character recognition device 220. Specifically, a copier, a scanner, and a multifunction peripheral (scanner, printer, copier) Or an image processing device having at least two functions such as a machine and a facsimile.
The image reading device 210 is connected to the image processing device 100. The image reading device 210 reads a document in which ruby is added to the text and transfers it to the image processing device 100 as image data.
The image processing device 100 is connected to the image reading device 210 and the character recognition device 220. The image processing apparatus 100 deletes ruby from the image received from the image reading apparatus 210 and generates an image including only a text. The image processing apparatus 100 includes the image receiving module 110, the ruby extraction (A) module 120, the ruby deletion module 150, and the text image output module 160, as described above, in addition to the configuration shown in the example of FIG. It may be.
The character recognition device 220 is connected to the image processing device 100. The character recognition device 220 performs character recognition on the image received from the image processing device 100, and outputs a recognition result. For example, the information may be stored in a document database or the like in association with the image read by the image reading device 210.

In the example shown in FIG. 2B, the image processing device 100, the image reading device 210, the multifunction peripheral 215, the character recognition device 220, and the document DB device 230 are systems connected via a communication line 290, respectively. The communication line 290 may be wireless, wired, or a combination thereof, and may be, for example, the Internet or an intranet as a communication infrastructure. Further, the functions of the image processing device 100, the character recognition device 220, and the document DB device 230 may be realized as a cloud service.
The image reading device 210 and the multifunction peripheral 215 read a document and transmit the image to the image processing device 100 via the communication line 290.
The image processing apparatus 100 receives an image from the image reading apparatus 210 and the multifunction peripheral 215, and transmits a processing result (an image in which ruby has been deleted) to the character recognition apparatus 220 via the communication line 290.
The character recognition device 220 receives the image from the image processing device 100, performs a character recognition process, and transmits the result of the recognition process to the image processing device 100 via the communication line 290.

FIG. 3 is a flowchart illustrating a processing example according to the present embodiment.
In step S302, the image receiving module 110 receives an image.
In step S304, the image receiving module 110 performs preprocessing for ruby extraction. The detailed processing of step S304 will be described later using the flowchart shown in the example of FIG.
In step S306, the ruby extraction (A) module 120 performs first-stage ruby extraction. The detailed processing of step S306 will be described later with reference to the flowchart shown in the example of FIG.
In step S308, the text center line extraction module 130 extracts a center line in the text. Detailed processing of step S308 will be described later using a flowchart shown in the example of FIG.
In step S310, the ruby extraction (B) module 140 performs second-stage ruby extraction using the processing results of steps S306 and S308. The detailed process of step S310 will be described later using a flowchart shown in the example of FIG.
In step S312, the ruby deletion module 150 deletes ruby from the image.
In step S314, the body image output module 160 outputs an image from which ruby has been deleted. Thereafter, for example, character recognition processing is performed on the image of the body.

FIG. 4 is a flowchart illustrating a processing example (processing example in step S304) according to the present embodiment (image reception module 110).
In step S402, it is determined whether the received image is a binary image. If the image is a binary image, the process proceeds to step S406; otherwise, the process proceeds to step S404.
In step S404, a binarization process is performed on the received image. The binarization process here may employ an existing process.
In step S406, RLSA (Run Length Smoothing Algorithm) processing is performed in the vertical direction. Of course, in the case of a horizontally written document, the RLSA process is performed in the horizontal direction.
For example, a white run within two text characters is converted to black. The RLSA process is a process for realizing figure fusion by replacing short runs (length M or less) of white pixels sandwiched between black pixels with black pixels (KY Wong, R. Casey, and). FM Wahl.Document analysis system.IBM Journal of Research and Development, Vol.26, pp647-656, 1982). It should be noted that “two text characters” is described as an example of a predetermined value, and white runs of other lengths may be converted to black.
When ruby extraction is performed only by the RLSA process, there is a disadvantage that the ruby and the text are merged if the ruby and the text are close to each other, if there is dust (noise), or if the text is inclined. Therefore, it is difficult to extract ruby only by the RLSA process.
This will be described with reference to the example of FIG. FIG. 5 is an explanatory diagram illustrating a processing example according to the present embodiment.
The example shown in FIG. 5A is a binary image to be processed. Ruby is written vertically.
The example shown in FIG. 5B is obtained by performing RLSA processing on the binary image shown in FIG. The white run within two text characters is converted to black.
The example shown in FIG. 5C is an enlarged view of an area 500 which is the head of the RLSA-processed image shown in FIG. 5B. Since the ruby in the ruby RLSA result 510 is separated from the text in the text RLSA result 520, it can be extracted, and the ruby in the ruby RLSA result 532 is also separated from the text in the text RLSA result 540, so it is extracted. be able to. However, since the ruby in the ruby RLSA result 512 is close to the text in the text RLSA result 520, the ruby RLSA result 512 and the text RLSA result 520 are in contact with each other. Since the ruby in the ruby RLSA result 530 is close to the text in the text RLSA result 540, the ruby RLSA result 530 and the text RLSA result 540 are in contact with each other.

In step S408, a labeling process is performed. In other words, each of the black pixel blocks (Connection Component: CC) can be specified. A circumscribed rectangle is obtained by labeling the RLSA application image. In the example of FIG. 5C, the ruby RLSA result 510 and the body RLSA result 520 are given different labels, but the same label is given because the ruby RLSA result 512 and the body RLSA result 520 are in contact with each other. Will be. Note that a black pixel block is a pixel region in which black pixels are consecutively connected in four connections or eight connections.
In step S410, a noise removal process is performed. A black pixel block having a width that is too large to be considered as a line and a black pixel block that is too small to be considered as a part of a character (such as character deviation) are removed. For example, if the size (length, width, area, etc.) of each black pixel block is not within a predetermined range, it may be determined to be noise and removed.

FIG. 6 is a flowchart illustrating a processing example (processing example in step S306) according to the present embodiment (ruby extraction (A) module 120).
In step S602, a target black pixel block is selected. Any black pixel block that has been subjected to labeling processing may be used.

In step S604, a black pixel block (A) having a width of a circumscribed rectangle equal to or more than (width of a circumscribed rectangle of the target black pixel block) * 2 is extracted. Here, “2” is an example of a predetermined multiple, and may be another multiple, for example, “2.5”.
In step S606, the X-axis distance between the circumscribed rectangle of the target black pixel block and the circumscribed rectangle of the black pixel block (A) is calculated. The “X-axis distance” here is the distance between the X coordinate of the right end of the circumscribed rectangle of the black pixel block (A) and the X coordinate of the left end of the target black pixel block. As shown in the example of FIG. 7, the other black pixel block 720 corresponds to a circumscribed rectangle of the black pixel block (A), and the target black pixel block 710 corresponds to a circumscribed rectangle of the target black pixel block. The distance 725 corresponds to the X-axis distance. In the case of horizontal writing, the distance is the Y-axis distance, which is the distance between the X coordinate of the upper end of the circumscribed rectangle of the black pixel block (A) and the Y coordinate of the lower end of the target black pixel block.

  In step S608, it is determined whether or not “X-axis distance ≦ rubi_distance_max”. If “X-axis distance ≦ rubi_distance_max”, the process proceeds to step S612. Otherwise, the process proceeds to step S610. Note that “rubi_distance_max” is the maximum distance between the ruby and the text, and is a predetermined value. In the example of FIG. 7, when “distance 725 ≦ rubi_distance_max”, the target black pixel block 710 is extracted as ruby.

In step S610, the circumscribed rectangle of the target black pixel block and the circumscribed rectangular area of the black pixel block (A) overlap, the target black pixel block is smaller, and the X coordinate of the target black pixel block is the right half of the partner. It is determined whether or not it exists on the side. If it exists, the process proceeds to step S612; otherwise, the process proceeds to step S614. As “the circumscribed rectangle of the target black pixel block and the circumscribed rectangular region of the black pixel block (A) overlap”, all of the “circumscribed rectangle of the target black pixel block” are included in the “circumscribed rectangular region of the black pixel block (A)”. This includes the case where a part of the “circumscribed rectangle of the target black pixel block” is included in the “circumscribed rectangular area of the black pixel block (A)”. In the example of FIG. 8A, the entirety of the target black pixel block 810 is included in the other black pixel block 820, and in the example of FIG. 840, it is determined that “the circumscribed rectangle of the target black pixel block and the circumscribed rectangular region of the black pixel block (A) overlap”. Further, “the target black pixel block is smaller” means that, for example, the width, height, area, etc. of the circumscribed rectangle of the target black pixel block and the black pixel block (A) are compared, and the target black pixel block is black. This is the case where the pixel block is smaller than the pixel block (A). In the example of FIG. 8A, the target black pixel block 810 is smaller than the other black pixel block 820, and in the example of FIG. 8B, the target black pixel block 830 is smaller than the other black pixel block 840. Also in the example, it is determined that “the target black pixel block is smaller”. Further, “the X coordinate of the target black pixel block exists on the right half side of the other party” means that the position of the target black pixel block is on the right side of the center of the circumscribed rectangular area of the black pixel block (A). . In the example of FIG. 8A, the target black pixel block 810 exists on the right half side of the other black pixel block 820. In the example of FIG. 8B, the target black pixel block 830 is located on the right of the other black pixel block 840. It exists on the half side, and in each case, it is determined that “the X coordinate of the target black pixel block exists on the right half side of the other party”. Since these three conditions are satisfied, the target black pixel block 810 and the target black pixel block 830 are extracted as ruby.
In step S612, the target black pixel block is set to ruby.

In step S614, it is determined whether or not all the black pixel clusters have been targeted. If all the black pixel clusters have been targeted, the process ends (step S699), and otherwise, the process returns to step S602. For example, a process may be performed by attaching a flag indicating whether or not each black pixel block has been processed. More specifically, a flag indicating that processing has been performed is attached to the black pixel block that has been processed up to step S614, and a black pixel block that has not been flagged indicating that processing has been performed is added. The determination process of step S614 (if there is a black pixel block to which no flag indicating that processing has been performed is determined based on whether or not there is a black pixel block in step S614, a flag indicating that processing has been performed is added) If no black pixel block exists, “Y”) may be performed in step S614.
In the example of the flowchart shown in FIG. 6, both the determination processing in step S608 and the determination processing in step S610 are performed, but either one of the determination processing may be performed.
Further, it may be determined that the target pixel is a ruby on the condition that the width of the circumscribed rectangle of the target black pixel block is less than or less than a predetermined value (rubi_width_max).

FIG. 9 is a flowchart illustrating a processing example (processing example in step S308) according to the present embodiment (text center line extraction module 130).
In step S902, a labeling process is performed. The original binary image (not the image after the RLSA processing) is labeled to obtain a circumscribed rectangle.
In step S904, a noise removal process is performed. A black pixel block that is not a character is removed in consideration of the size. For example, if the size (length, width, area, etc.) of each black pixel block is not within a predetermined range, it may be determined to be noise and removed.
In step S906, ruby removal processing is performed. The ruby detected in the first stage of ruby detection (step S612 in the flowchart shown in the example of FIG. 6) is removed. Therefore, in the following processing, an image from which ruby has been deleted, that is, an image (circumscribed rectangle group) composed of only the text is targeted.

  In step S908, an overlapping synthesis process of black pixel blocks is performed. If the circumscribed rectangles overlap, combine them. The reason why the combining process is performed here is to increase the number of circumscribed rectangles that are as close to square as possible. This will be described with reference to the example of FIG. In the example of FIG. 10A, the character image of “ta” indicates that there are a black pixel block circumscribed rectangle 1010, a black pixel block circumscribed rectangle 1020, and a black pixel block circumscribed rectangle 1030 as circumscribed rectangles of the black pixel block. ing. There is an overlap between the black pixel block circumscribed rectangle 1010 and the black pixel block circumscribed rectangle 1020, and between the black pixel block circumscribed rectangle 1010 and the black pixel block circumscribed rectangle 1030. The black pixel block circumscribed rectangle 1010 and the black pixel block circumscribed rectangle 1020 are combined, and the black pixel block circumscribed rectangle 1010 and the black pixel block circumscribed rectangle 1030 are combined to form a composite black as shown in the example of FIG. A pixel block circumscribed rectangle 1050 is generated.

In step S910, a circumscribed rectangle having an aspect ratio (aspect ratio) close to 1 is picked up (extracted). In the case of Japanese, the fact that there are many characters whose circumscribed rectangle is close to a square is used. Note that “close to 1” may be determined based on, for example, whether the difference between the “vertical / horizontal” calculation result and “1” is less than or less than a predetermined value.
In step S912, the center coordinates of the circumscribed rectangle picked up in step S910 are collected for each row. Find a straight line that fits the collected center coordinates. This is the text centerline of the text. This will be described with reference to the example of FIG. In this example, white vertical lines (text center lines 1102 to 1146) are text center lines. The white rectangle is the circumscribed rectangle picked up in step S910. The text center line is a line segment passing through the center of the white rectangle. Therefore, two or more circumscribed rectangles are required to obtain the text center line. Note that the text center line is a vertical line, and a slope less than or equal to a predetermined value is allowed.

  In step S914, straight line interpolation processing and extrapolation processing are performed. In the processing up to step S912, if there are no two or more square rectangles in the same row, no straight line is obtained. In this case, interpolation or extrapolation is performed from the surrounding straight lines. The interpolation process will be described using the example of FIG. 12 and the extrapolation process will be described using the example of FIG.

  FIG. 12 is an explanatory diagram illustrating a processing example according to the present embodiment. In the interpolation of a straight line, an average interval is obtained from a group of straight lines within a predetermined interval, and the number of straight lines to be interpolated is calculated using the average interval. If N lines are required, the X coordinate of the effective straight line on both sides at y = 0 (upper end of the image) is equally divided into (N + 1), and a straight line having the average of the slopes of the straight lines on both sides as the text center line . In the example of FIG. 12, an average interval is calculated from four line segments of the text center lines 1204 to 1210. Since the interval between the text center line 1202 and the text center line 1204 is twice or more the average interval, the interpolation center line 1252 and the interpolation center line 1254 are generated at the interval. The inclination of the interpolation center line 1252 and the interpolation center line 1254 is the average of the inclinations of the body center line 1202 and the body center line 1204.

  FIG. 13 is an explanatory diagram illustrating a processing example according to the present embodiment. In the extrapolation of a straight line, first, an average head position of a row is extracted from a labeling image. Next, in the same way as in the case of interpolation, find the average interval of the straight line, at the place separated by the average interval of the end straight line, check whether there is a character-like black pixel block around the average head position, If so, place a straight line at that position. In this case, the slope of the straight line has the same value as the endmost straight line. In the example of FIG. 13, three line segments of the text center line 1302, the text center line 1304, and the text center line 1306 are extracted in step S912. Then, a circumscribed rectangle of the black pixel block at the head of each line segment (black pixel block circumscribed rectangle 1328, black pixel block circumscribed rectangle 1334, black pixel block circumscribed rectangle 1342) is extracted. Then, the head position 1370 which is the lower end of the circumscribed rectangle of many black pixel blocks is calculated. The average interval between the main text center line 1302, main text center line 1304, and main text center line 1306 is calculated, and the right side of the head (top position 1370) of the end main text center line 1302 (the left side of the front of the main text center line 1306) is calculated. Then, it is determined whether or not there is a black pixel block at the position (search range 1360) separated by the average interval. The search range 1360 includes a black pixel block circumscribed rectangle 1322 and a black pixel block circumscribed rectangle 1324, so that the extrapolation center line 1352 is inserted. The inclination of the extrapolation center line 1352 is the same as the body center line 1302. Note that the inclination of the extrapolation center line 1352 may be an average of the inclinations of the body center line 1302, the body center line 1304, and the body center line 1306.

FIG. 14 is a flowchart illustrating a processing example (processing example in step S310) according to the present embodiment (ruby extraction (B) module 140).
In step S1402, the distance between the upper-left coordinate of the circumscribed rectangle of the ruby extracted in the first-stage ruby extraction (the flowchart shown in the example of FIG. 6) and the nearest text center line is calculated, and the average value is calculated. In the example of FIG. 15, the distance 1512 that is the distance between the upper left coordinate of the target black pixel block circumscribed rectangle 1502 and the text center line 1504 is targeted.
In step S1404, a target black pixel block is selected. Any black pixel block may be used as long as it is a black pixel block.
In step S1406, the distance between the center of the circumscribed rectangle of the target black pixel block and the text center line closest to the target black pixel block is calculated. In the example of FIG. 15, the distance 1514 which is the distance between the coordinates of the center of the target black pixel block circumscribed rectangle 1502 and the text center line 1504 is targeted.

In step S1408, it is determined whether (distance calculated in step S1406)> (average value calculated in step S1402), and (distance calculated in step S1406)> (average value calculated in step S1402). If there is, the process proceeds to step S1410; otherwise, the process proceeds to step S1412.
In step S1410, the target black pixel block is set to ruby.
In step S1412, it is determined whether all the black pixel clusters have been targeted. If all the black pixel clusters have been targeted, the process ends (step S1499). Otherwise, the process returns to step S1404. For example, a process may be performed by attaching a flag indicating whether or not each black pixel block has been processed. More specifically, a flag indicating that processing has been performed is attached to the black pixel block that has been processed up to step S1412, and a black pixel block that has not been flagged indicating that processing has been performed is added. The determination process of step S1412 (if there is a black pixel block to which a flag indicating that processing has been performed is not performed, “N” is determined in step S1412, and a flag indicating that processing has been performed is determined based on whether or not there is a black pixel block. If there is no black pixel block, “Y”) may be performed in step S1412.

  The distance 1512 is the distance calculated at step S1402 (the upper left is the starting point), and the distance 1514 is the distance calculated at the step S1406 (the center is the starting point). Specifically, the distance calculated in step S1402 is shorter than the distance calculated in step S1406. This is to ensure that ruby is extracted. Note that the distance calculated in step S1402 and the distance calculated in step S1406 may be the same.

  An example of a hardware configuration of the image processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 16 is configured by, for example, a personal computer (PC) or the like, and illustrates a hardware configuration example including a data reading unit 1617 such as a scanner and a data output unit 1618 such as a printer.

  The CPU (Central Processing Unit) 1601 is composed of various modules described in the above embodiment, that is, the image receiving module 110, the ruby extraction (A) module 120, the text center line extraction module 130, and the ruby extraction (B) module 140 , A ruby deletion module 150, a body image output module 160, and the like.

  A ROM (Read Only Memory) 1602 stores a program used by the CPU 1601, an operation parameter, and the like. A RAM (Random Access Memory) 1603 stores programs used in the execution of the CPU 1601, parameters that change as appropriate in the execution, and the like. These are mutually connected by a host bus 1604 composed of a CPU bus and the like.

  The host bus 1604 is connected via a bridge 1605 to an external bus 1606 such as a PCI (Peripheral Component Interconnect / Interface) bus.

  A pointing device 1609 such as a keyboard 1608 and a mouse is a device operated by an operator. The display 1610 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various information as text or image information. Further, a touch screen or the like having both functions of the pointing device 1609 and the display 1610 may be used.

  An HDD (Hard Disk Drive) 1611 incorporates a hard disk (which may be a flash memory or the like), drives the hard disk, and records or reproduces a program or information executed by the CPU 1601. The hard disk stores an image to be processed, an image from which ruby has been deleted (image of only the text), a processing result of each module (including an intermediate result), and the like. Further, various other data, various computer programs, and the like are stored.

  The drive 1612 reads data or a program recorded on a removable recording medium 1613 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and reads the data or the program into the interface 1607 and the external bus 1606. , A bridge 1605, and a RAM 1603 connected via a host bus 1604. Note that the removable recording medium 1613 can also be used as a data recording area.

  The connection port 1614 is a port for connecting an externally connected device 1615, and has a connection unit such as USB, IEEE1394, or the like. The connection port 1614 is connected to the CPU 1601 and the like via the interface 1607, the external bus 1606, the bridge 1605, the host bus 1604, and the like. The communication unit 1616 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1617 is, for example, a scanner, and executes a document reading process. The data output unit 1618 is, for example, a printer, and executes a document data output process.

  Note that the hardware configuration of the image processing apparatus illustrated in FIG. 16 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 16 and executes the modules described in the present embodiment. Any configuration is possible. For example, some modules may be constituted by dedicated hardware (for example, an application-specific integrated circuit (ASIC) or the like), and some modules may be in an external system and connected by a communication line. Alternatively, a plurality of the systems shown in FIG. 16 may be connected to each other by a communication line so as to cooperate with each other. In particular, in addition to personal computers, portable information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc.), information appliances, robots, copiers, fax machines, scanners, printers, and multifunction devices (scanners, printers, An image processing apparatus having at least two functions such as a copying machine and a facsimile) may be incorporated.

  In the description of the above-described embodiment, in the comparison processing, “greater than”, “less than”, “greater than”, and “smaller (less than)” are used as long as there is no inconsistency in the combination. It may be “greater”, “smaller (less than)”, “greater than”, or “less than”.

The above-described program may be provided by being stored in a recording medium, or the program may be provided by a communication unit. In this case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium on which the program is recorded”.
The “computer-readable recording medium on which the program is recorded” refers to a computer-readable recording medium on which the program is recorded, which is used for installing, executing, and distributing the program.
The recording medium is, for example, a digital versatile disk (DVD), which is a standard defined by the DVD Forum, such as "DVD-R, DVD-RW, DVD-RAM, etc.", and DVD + RW. Blu-ray discs such as "DVD + R, DVD + RW, etc." which are standards, compact discs (CDs), such as read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc. Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark) )), Flash memory, random access memory (RAM) , SD (Secure Digital) memory card, and the like.
The whole or a part of the program may be recorded on the recording medium and stored or distributed. Further, the communication may be, for example, a wired network used for a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, an intranet, an extranet, or a wireless communication. The data may be transmitted using a transmission medium such as a network or a combination thereof, or may be transmitted on a carrier wave.
Further, the above-mentioned program may be a part or all of another program, or may be recorded on a recording medium together with a separate program. Further, the program may be divided and recorded on a plurality of recording media. Also, the program may be recorded in any form as long as it can be restored, such as compression or encryption.

REFERENCE SIGNS LIST 100 image processing device 110 image reception module 120 ruby extraction (A) module 130 text center line extraction module 140 ruby extraction (B) module 150 ruby deletion module 160 text image output module 210 image reading device 215 … MFP 220… Character recognition device 230… Document DB device 290… Communication line

Claims (5)

First extracting means for extracting ruby added to the text from the image;
Center line extracting means for extracting a center line of the text,
If the distance between the pixel block in the image and the center line adjacent to the pixel block is greater than or equal to the distance between the ruby and the center line adjacent to the ruby, the pixel block is extracted as ruby. second extracting means for possess,
The center line extracting unit extrapolates the center line outside the center line group according to the statistical interval between the center lines already extracted and the position of the pixel block at the head of the center line.
Image processing device. Deleting means for deleting the ruby extracted by the first extracting means and the ruby extracted by the second extracting means from the image;
The image processing apparatus according to claim 1, further comprising an output unit configured to output an image in which ruby has been deleted from the image by the deletion unit. As the distance between the ruby and the center line adjacent to the ruby, a distance between the center lines from an angle close to the center line of a circumscribed rectangle of the ruby;
As the distance between the pixel block and the center line adjacent to the pixel block, the distance between the center line from the center of the circumscribed rectangle of the pixel block,
The image processing device according to claim 1. The first extracting means includes:
RLSA means for performing RLSA processing in a predetermined direction of the image;
Third extraction means for extracting a second pixel block having a size larger than or greater than a predetermined multiple of the first pixel block as a processing result by the RLSA unit;
If the distance between the first pixel block and the second pixel block is less than or less than a predetermined distance, the first pixel block is extracted as ruby, or the first pixel block is extracted as ruby. And the second pixel block overlaps, the first pixel block is smaller than the second pixel block, and the position of the first pixel block is within a predetermined area in the second pixel block. If it is, ruby extracting means for extracting the first pixel block as ruby
including,
The image processing device according to claim 1 . Computer
First extracting means for extracting ruby added to the text from the image;
Center line extracting means for extracting a center line of the text,
If the distance between the pixel block in the image and the center line adjacent to the pixel block is greater than or equal to the distance between the ruby and the center line adjacent to the ruby, the pixel block is extracted as ruby. Function as a second extraction means ,
The center line extracting unit extrapolates the center line outside the center line group according to the statistical interval between the center lines already extracted and the position of the pixel block at the head of the center line.
Image processing program.

Images