JP2020025218A - Image processing apparatus and image processing program - Google Patents

Abstract

To make more user-friendly a technology to extract and process character strings included in a document image acquired by scanning a document.SOLUTION: An image processing apparatus comprises a control circuit operating as an image acquisition module that acquires a document image and marking images created by scanning a document including a text and markings for each of a plurality of lines; a line detection module that detects, in the document image, two lines at which both ends of the marking image in a direction perpendicular to the plurality of lines are located; a text extraction module that extracts texts included in an area surrounded by the two lines in the document image; a processing module that processes the texts included in the area according to a command defined by the color of the marking image; and a composite module that composites the processed texts.SELECTED DRAWING: Figure 3

Description

  The present disclosure relates to an image processing apparatus and an image processing program for extracting and processing a character string included in a document image obtained by scanning a document.

  2. Description of the Related Art There is known an image processing apparatus that extracts and processes a character string included in a document image obtained by scanning a document (Patent Document 1).

JP 2012-32979 A

  It is desirable that the technology for extracting and processing a character string included in a document image obtained by scanning a document is more user-friendly.

An image processing device according to an embodiment of the present disclosure includes:
An image acquisition module that acquires a document image and a marking image generated by scanning a document including text and markings for each of a plurality of lines,
A line detection module that detects two lines at both ends of the original image in a direction perpendicular to the plurality of lines of the marking image,
A text extraction module for extracting text included in an area surrounded by the two lines in the document image;
According to a command defined by the color of the marking image, a processing module that processes the text included in the area,
A control circuit that operates as a synthesis module that synthesizes the processed text.

  This allows the user to know the command defined by the color of the marking in advance, and draw the color marking that defines the command suitable for the user's intention at an appropriate position with respect to the text targeted by the command. For example, text can be processed with a command suitable for the user's intention.

The image acquisition module scans the document including the text and a plurality of markings of different colors to obtain the document image and the plurality of marking images,
The processing module processes the text included in the plurality of areas in different ways according to a command defined by the different colors of the plurality of marking images.

  With this, the user knows in advance a plurality of different commands defined by the colors of the marking, and assigns a color marking that defines a command suitable for the user's intention to the appropriate text for the command target text. Once in place, the text can be manipulated with different commands that suit the user's intentions.

  The processing module is defined by a command priority defined by a color of a marking image included in one area and a color of a marking image included in the other area when there is an overlapping area in which the plurality of areas overlap. Then, the text included in the overlapping area is processed according to a command defined by the color of the marking image having the higher priority.

  Thereby, the text included in the overlapping area is processed in accordance with the command defined by the color of the marking image with high priority, so that the text included in the overlapping area is converted into a command with high priority, that is, suitable for the intention of the user. It is possible to perform processing with a command that is likely to be a command that has been executed.

  The text extraction module extracts, as the text, a character code string obtained by performing optical character recognition on a character in the document image and converting the character code.

  By using a character code string as text in this way, text can be appropriately and easily processed.

The marking on the document is drawn with a fluorescent marker,
The image acquisition module acquires the document image generated by scanning the document for each of the plurality of lines while irradiating visible light, and irradiates the document with each of the plurality of lines while irradiating black light. Obtain the marking image generated by scanning.

  This makes it easy to accurately detect the marking by irradiating the marking drawn with the fluorescent marker with black light.

An image processing program according to an embodiment of the present disclosure includes:
An image acquisition module that acquires a document image and a marking image generated by scanning a document including text and markings for each of a plurality of lines,
A line detection module that detects two lines at both ends of the original image in a direction perpendicular to the plurality of lines of the marking image,
A text extraction module for extracting text included in an area surrounded by the two lines in the document image;
According to a command defined by the color of the marking image, a processing module that processes the text included in the area,
The control circuit of the image processing apparatus is operated as a synthesis module for synthesizing the processed text.

  According to the present fire, the technology of extracting and processing a character string included in a document image obtained by scanning a document can be made more user-friendly.

  Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

1 illustrates a hardware configuration of an image processing apparatus according to an embodiment of the present disclosure. 2 shows a functional configuration of the image processing apparatus. 4 shows an operation flow of the image processing apparatus. 4 shows an example of a document image and a marking image. 3 shows a line color command table. 4 shows an example of a composite image.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

  2. Hardware configuration of image processing device

  FIG. 1 illustrates a hardware configuration of an image processing apparatus according to an embodiment of the present disclosure.

  In the present embodiment, the image processing apparatus 10 is an MFP (Multifunction Peripheral), and is hereinafter referred to as the MFP 10.

  The MFP 10 includes a control circuit 11. The control circuit 11 includes a CPU 11a (Central Processing Unit) as a processor, a RAM 11b (Random Access Memory), a ROM 11c (Read Only Memory) as a memory, a dedicated hardware circuit, and the like, and controls the overall operation of the MFP 10. Govern. The CPU 11a loads the information processing program stored in the ROM 11c into the RAM 11b and executes it. The ROM 11c stores programs and data executed by the CPU 11a in a fixed manner. The ROM 11c is an example of a non-transitory computer-readable recording medium.

  The control circuit 11 includes an image reading unit 12 including a black light unit 12a, an image processing unit 14, an image memory 15, an image forming unit 16, an operation unit 17 including a display unit 17a, a storage device 18, a facsimile communication unit 19, and network communication. It is connected to the interface 13 and the like. The control circuit 11 controls the operation of each of the above-described units connected thereto, and transmits / receives signals or data to / from each unit.

  The image reading unit 12 is an image scanner. The image reading unit 12 generates a document image by scanning a document (FIG. 4) set on a platen or a feeder (not shown) for each of a plurality of lines while irradiating visible light.

  The black light unit 12a scans a document (FIG. 4) set on a platen or a feeder for each of a plurality of lines while irradiating the black light (ultraviolet light of a long wavelength) with respect to a plurality of lines. Generate a marking image that is an image.

  For example, when a document is set on the feeder, the image reading unit 12 irradiates the document with visible light for one line to obtain a document image, and then the black light unit 12a reads the document for the same line. Then, a marking image may be obtained by irradiating a black light on the image. Alternatively, the order may be reversed. This may be repeated for all of the plurality of lines. In short, scanning of one line by visible light and scanning of one line by black light may be alternately repeated.

  For example, when a document is set on a platen, first, the image reading unit 12 irradiates the document with visible light for each of a plurality of lines to acquire a document image, and then the black light unit 12a The marking image may be obtained by irradiating the original with black light every time. Alternatively, the order may be reversed. In short, it is sufficient to perform a total of two scans of scanning all lines by visible light and scanning all lines by black light. Alternatively, when the document is set on the platen, scanning may be performed in the same manner as when the document is set on the feeder.

  2. Functional configuration of image processing device

  FIG. 2 shows a functional configuration of the image processing apparatus.

  In the control circuit 11 of the MFP 10, the CPU 11 a loads the image processing program stored in the ROM 11 c into the RAM 11 b and executes the loaded image processing program, whereby the mode determination unit 101, the image acquisition module 102, the line detection module 103, the text extraction module 104, the processing module It operates as 105, the synthesis module 106 and the output module 107.

  The mode determination unit 101 determines whether the mode is the normal scan mode or the scan mode according to the present embodiment (scan mode for executing a command defined by the color of marking).

  The image acquisition module 102 acquires a document image and a marking image generated by scanning a document including text and marking for each of a plurality of lines.

  The line detection module 103 detects two lines in the document image at both ends in a direction perpendicular to the plurality of lines of the marking image.

  The text extraction module 104 extracts text included in an area between two lines in the document image.

  The processing module 105 processes text included in the area according to a command defined by the color of the marking image.

  The combining module 106 combines the text processed by the processing module 105 to generate a combined image.

  The output module 107 outputs (prints, saves, transmits, etc.) the composite image generated by the composite module 106.

  3. Operation flow of image processing device

  FIG. 3 shows an operation flow of the image processing apparatus.

  As a premise, the original to be scanned includes text (a typed and printed character string or a handwritten character string). Markings (a plurality in the present embodiment) are drawn on the original by a fluorescent marker by a user. The colors of the plurality of markings may be all different, partially the same and partially different, or all the same. For example, in this example, yellow marking (plural), pink marking (single), and blue marking (single) are drawn on the document. The user knows in advance the commands defined by the colors of the marking. The user draws color markings that define the command appropriate to the user's intent, at the appropriate locations relative to the text targeted by the command. The command will be described later with reference to FIG.

  Before starting scanning of a document, the mode determination unit 101 determines whether the mode is the normal scan mode or the scan mode according to the present embodiment (scan mode for executing a command defined by the color of marking). (Step S101). When the mode determination unit 101 determines that the mode is the normal scan mode (step S101, NO), the MFP 10 scans and outputs a document (print, data generation, etc.) (step S102). On the other hand, a process of the MFP 10 when the mode determination unit 101 determines that the mode is the scan mode according to the present embodiment (scan mode for executing a command defined by the color of marking) (step S101, YES).

  The image reading unit 12 generates a document image by scanning a document including text and marking for each of a plurality of lines while irradiating visible light. The black light unit 12a generates a marking image, which is an image of a marking drawn by a fluorescent marker, by scanning the document for each of a plurality of lines while irradiating the black light. The image acquisition module 102 acquires the generated document image and marking image (step S103).

  FIG. 4 shows an example of a document image and a marking image.

  The document image D is image data of the entire document including text (a typed and printed character string or a handwritten character string) and markings. The marking images M1, M2, M3 and M4 are image data of markings drawn by the user with fluorescent markers. In this example, the colors of the marking images M1 and M2 are yellow. The color of the marking image M3 is pink. The color of the marking image M4 is blue.

  The line detection module 103 detects two lines in the document image D where both ends in the direction perpendicular to the plurality of lines of each of the marking images M1, M2, M3, and M4 are located (step S104). In FIG. 4, a line direction (main scanning direction) is defined as an X direction, and a direction perpendicular to the line (sub scanning direction) is defined as a Y direction. Specifically, the line detection module 103 detects two lines L11 and L12 located at both ends in a direction (sub-scanning direction, Y direction) perpendicular to the line of the marking image M1. The line detection module 103 detects two lines L21 and L22 located at both ends in a direction (sub-scanning direction, Y direction) perpendicular to the line of the marking image M2. The line detection module 103 detects two lines L31 and L32 located at both ends in a direction (sub-scanning direction, Y direction) perpendicular to the line of the marking image M3. The line detection module 103 detects two lines L41 and L42 located at both ends in a direction (sub-scanning direction, Y direction) perpendicular to the line of the marking image M4.

  The text extraction module 104 extracts text contained in an area surrounded by each two lines in the document image D. Specifically, the text extraction module 104 extracts, as text, a character code string obtained by optically recognizing and converting a character included in an area surrounded by each two lines in the document image D (step S105). ). In this example, the text extraction module 104 extracts a character code string obtained by optically recognizing and converting a character included in an area A1 surrounded by two lines L11 and L12 in the document image D as a text T1. . The text extraction module 104 extracts a character code string obtained by optically recognizing and converting a character included in an area A2 surrounded by two lines L21 and L22 in the document image D as a text T2. The text extraction module 104 extracts a character code string obtained by optically recognizing and converting a character contained in an area A3 surrounded by two lines L31 and L32 in the document image D as a text T3. The text extraction module 104 extracts, as a text T4, a character code string obtained by optically recognizing and converting a character included in an area A4 surrounded by two lines L41 and L42 in the document image D.

  The processing module 105 refers to the line color command table 200 stored in the storage device 18 and processes the text included in the area according to the command defined by the color of the marking image. Specifically, the processing module 105 refers to the line color command table 200 and processes text included in a plurality of areas by a different method according to a command defined by a plurality of different colors of a plurality of marking images (step). S106).

  FIG. 5 shows a line color command table.

  The line color command table 200 stores a line color 201, a command 202, and a priority 203 in association with each other. The line color 201 indicates the color of the marking image. The command 202 is defined by the color of the marking image. Different commands 202 are associated with different line colors 201, respectively. The priority 203 indicates the priority of the plurality of commands 202. The line color command table 200 can be arbitrarily set as appropriate, for example, by a user inputting via the touch panel of the MFP 10. What is necessary is just to set by the start of an operation | movement flow.

  In this example, the command 202 defined by the line color 201 “yellow” is “change font to Meiryo UI”, and its priority 203 is “1”. The command 202 defined by the line color 201 “pink” is “change the text size to 20 points”, and its priority 203 is “2”. The command 202 defined by the line color 201 “blue” is “change the text background to yellow” (highlight), and its priority 203 is “3”. For example, the command 202 may be a single command by combining a plurality of commands, such as "change the font to Meiryo UI and change the text size to 20 points".

  The processing module 105 refers to the line color command table 200 and processes the text T1 included in the area A1 in accordance with the command 202 “change font to Meiryo UI” defined by the line color 201 “yellow” of the marking image M1. . The processing module 105 refers to the line color command table 200 and processes the text T2 included in the area A2 according to the command 202 “change font to Meiryo UI” defined by the line color 201 “yellow” of the marking image M2. . The processing module 105 refers to the line color command table 200, and according to the command 202 “change the text size to 20 points” defined by the line color 201 “pink” of the marking image M3, the text T3 included in the area A3. To process. The processing module 105 processes the text T4 included in the area A4 according to the command 202 “change the text background to yellow” defined by the line color 201 “blue” of the marking image M4 with reference to the line color command table 200. I do.

  By the way, when there is an overlapping area where the plurality of areas A2 and A3 overlap (an area from the line L31 to the line L22), the processing module 105 is defined by the line color 201 “yellow” of the marking image M2 included in one area A2. The priority 203 “1” of the command 202 to be executed is compared with the priority 203 “2” of the command 202 defined by the line color 201 “pink” of the marking image M3 included in the other area A3. The processing module 105 includes in the overlap area (the area from the line L31 to the line L22) according to the command 202 “Change font to Meiryo UI” defined by the line color 201 “yellow” of the marking image M2 having the higher priority 203. (However, in the example of FIG. 4, the text is not included in the overlapping area).

  The combining module 106 combines the texts processed by the processing module 105 to generate a combined image (Step S107).

  FIG. 6 shows an example of the composite image.

  In this example, the synthesis module 106 processes the text T1 according to the command 202 “Change the font to Meiryo UI” according to the command 202, and the text T2 according to the command 202 “Changes the font to Meiryo UI”. , And a processed text T13 obtained by processing the text T3 according to the command 202 “Change text size to 20 points” and a processed text T14 obtained by processing the text T4 according to the command 202 “Change text background to yellow”. An image D1 is generated.

  The output module 107 outputs (prints, saves, transmits, etc.) the composite image D1 generated by the composite module 106 (step S108).

  4. Modified example

  In the present embodiment, the combining module 106 converts the text (T5, T6 in FIG. 4) other than the text (T1, T2, T3, T4) included in the area surrounded by the two lines of the marking image into the combined image D1. Not included.

  Instead, the synthesis module 106 does not process texts (T5, T6 in FIG. 4) other than the texts (T1, T2, T3, T4) included in the area surrounded by the two lines of the marking image. Then, it may be included in the composite image D1 (not shown).

  In the present embodiment, a marking is drawn on a document to be scanned by a user with a fluorescent marker. The black light unit 12a generates a marking image, which is an image of a marking drawn by a fluorescent marker, by scanning the document for each of a plurality of lines while irradiating the black light. By irradiating a black light on the marking drawn with the fluorescent marker, it is easy to detect the marking accurately.

  Alternatively, the marking may be drawn on the document to be scanned with a marker or a colored pencil other than the fluorescent marker by the user. In this case, the image reading unit 12 does not have the black light unit 12a. The image reading unit 12 generates a document image by scanning a document including text and marking for each of a plurality of lines while irradiating visible light, and generates a marking image included in the document image by edge detection.

  The image processing apparatus need not be an MFP, and may be an information processing apparatus such as a computer, a personal computer, a tablet computer, and a smartphone as a server device for cloud computing. In this case, the image acquisition module 102 of the information processing device may receive the document image and the marking image from the MFP or the like.

  5. Conclusion

  According to the present embodiment, the image acquisition module 102 acquires a document image and a marking image generated by scanning a document including text and marking for each of a plurality of lines. The line detection module 103 detects two lines in the document image at both ends in a direction perpendicular to the plurality of lines of the marking image. The text extraction module 104 extracts text included in an area between two lines in the document image. The processing module 105 processes text included in the area according to a command defined by the color of the marking image.

  This allows the user to know the command defined by the color of the marking in advance, and draw the color marking that defines the command suitable for the user's intention at an appropriate position with respect to the text targeted by the command. For example, text can be processed with a command suitable for the user's intention.

  According to the present embodiment, the image acquisition module 102 scans a document including text and a plurality of markings of different colors to acquire a document image and a plurality of marking images, and the processing module 105 includes a plurality of marking images. According to a command defined by a plurality of different colors, texts included in a plurality of areas are processed in different ways.

  With this, the user knows in advance a plurality of different commands defined by the colors of the marking, and assigns a color marking that defines a command suitable for the user's intention to the appropriate text for the command target text. Once in place, the text can be manipulated with different commands that suit the user's intentions.

  According to the present embodiment, when there is an overlapping area in which a plurality of areas overlap, the processing module 105 determines the priority of the command defined by the color of the marking image included in one area and the marking included in the other area. The priority of the command defined by the color of the image is compared, and the text included in the overlapping area is processed according to the command defined by the color of the marking image having the higher priority.

  Thereby, the text included in the overlapping area is processed in accordance with the command defined by the color of the marking image with high priority, so that the text included in the overlapping area is converted into a command with high priority, that is, suitable for the intention of the user. It is possible to perform processing with a command that is likely to be a command that has been executed.

  According to the present embodiment, the text extraction module 104 extracts a character code string obtained by performing optical character recognition on a character in a document image and converting it as text.

  By using a character code string as text in this way, text can be appropriately and easily processed.

The markings on the manuscript are drawn with fluorescent markers,
The image acquisition module 102 acquires a document image generated by scanning the document for each of a plurality of lines while irradiating visible light, and generates the image by scanning the document for each of a plurality of lines while irradiating black light. Obtained marking image is obtained.

  This makes it easy to accurately detect the marking by irradiating the marking drawn with the fluorescent marker with black light.

  Although each embodiment and each modification of the present technology have been described above, the present technology is not limited to only the above-described embodiments, and it is possible to make various changes without departing from the gist of the present technology. Of course.

Image processing device (MFP) 10
Mode determination unit 101
Image acquisition module 102
Line detection module 103
Text extraction module 104
Processing module 105
Synthesis module 106
Output module 107

Claims (6)

An image acquisition module that acquires a document image and a marking image generated by scanning a document including text and markings for each of a plurality of lines,
A line detection module that detects two lines at both ends of the original image in a direction perpendicular to the plurality of lines of the marking image,
A text extraction module for extracting text included in an area surrounded by the two lines in the document image;
According to a command defined by the color of the marking image, a processing module that processes the text included in the area,
An image processing apparatus comprising: a control circuit that operates as a synthesis module that synthesizes the processed text. The image processing device according to claim 1,
The image acquisition module scans the document including the text and a plurality of markings of different colors to obtain the document image and the plurality of marking images,
The image processing device, wherein the processing module processes the text included in the plurality of areas by a different method according to a command defined by the plurality of different colors of the plurality of marking images. The image processing device according to claim 2,
The processing module is defined by a command priority defined by a color of a marking image included in one area and a color of a marking image included in the other area when there is an overlapping area in which the plurality of areas overlap. An image processing device that compares a priority of a command to be executed and processes text included in the overlapping area according to a command defined by a color of the marking image having the higher priority. The image processing device according to claim 1, wherein:
The image processing apparatus, wherein the text extraction module extracts, as the text, a character code string obtained by performing optical character recognition on a character in the document image and converting the character code string. The image processing apparatus according to claim 1, wherein:
The marking on the document is drawn with a fluorescent marker,
The image acquisition module acquires the document image generated by scanning the document for each of the plurality of lines while irradiating visible light, and irradiates the document with each of the plurality of lines while irradiating black light. An image processing device for acquiring the marking image generated by scanning. An image acquisition module that acquires a document image and a marking image generated by scanning a document including text and markings for each of a plurality of lines,
A line detection module that detects two lines at both ends of the original image in a direction perpendicular to the plurality of lines of the marking image,
A text extraction module for extracting text included in an area surrounded by the two lines in the document image;
According to a command defined by the color of the marking image, a processing module that processes the text included in the area,
An image processing program for operating a control circuit of an image processing apparatus as a synthesis module for synthesizing the processed text.