JP2017200121A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract the text of a marker place printed in a document and automatically edit the text.SOLUTION: An image processing apparatus of this invention comprises: an image reading part 10 which acquires image data; a marker detection part 105 which detects a marker of a regulated color printed in a document, on the basis of the image data; an OCR processing part 106 which analyzes the image data and extracts the text of the marker place of the regulated color in the document; and an editing part 107 which determines a line feed place in the text on the basis of a preset editing condition, feeds a line at the line feed place, and edits the text.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus that reads a document and extracts the text of the document, and more particularly to a technique for editing the extracted text.

  In Patent Document 1, a document is read, a marker marked on the document is detected, a region of the document surrounded by the marker is recognized, and the inside of the region is printed, or The outside is printed.

JP 2005-167875 A

  By the way, as another application of the technique for recognizing the area of the document surrounded by the markers as in Patent Document 1, the recognized area is subjected to OCR (Optical Character Recognition) processing, and the text in this area is extracted. Can be used. For example, if the extracted text is inserted and used as an e-mail document, the e-mail can be easily created.

  In addition to extracting the text at the marker location, if this text can be automatically edited, e-mail creation and the like can be proceeded quickly, improving convenience.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus capable of extracting and automatically editing text at a marker portion marked on a document.

  An image processing apparatus according to an aspect of the present invention includes an acquisition unit that acquires image data indicating an image of a document, a marker detection unit that detects a marker marked on the document based on the image data, and the image Analyzing the data, extracting the text at the marker location in the document, and determining a line break in the extracted text based on preset editing conditions, and cutting the text at the line break And an editing unit for editing.

  According to the present invention, it is possible to automatically extract and edit the text of the marker portion marked on the document.

1 is a perspective view showing an image forming apparatus to which an image processing apparatus according to an embodiment of the present invention is applied. 1 is a block diagram illustrating an internal configuration of an image forming apparatus to which an image processing apparatus according to an embodiment of the present invention is applied. It is explanatory drawing which shows the comparative example of the text extracted and recognized from the original by the image processing apparatus of this embodiment, and the text after an edit. It is explanatory drawing which shows the other comparative example of the text extracted and recognized from the original by the image processing apparatus of this embodiment, and the text after an edit. 6 is a flowchart showing a processing procedure for recognizing and editing a text of a document by the image processing apparatus of the present embodiment. FIG. 3 is a plan view showing an operation unit and a display unit of the image forming apparatus. It is explanatory drawing which shows another comparative example of the text extracted and recognized from the original by the image processing apparatus of this embodiment, and the edited text. It is explanatory drawing which shows another comparative example of the text extracted and recognized from the original by the image processing apparatus of this embodiment, and the edited text. It is explanatory drawing which shows another comparative example of the text extracted and recognized from the original by the image processing apparatus of this embodiment, and the edited text. It is explanatory drawing which shows another comparative example of the text extracted and recognized from the original by the image processing apparatus of this embodiment, and the edited text.

  An image processing apparatus according to an embodiment of the present invention will be described below with reference to the drawings.

  FIG. 1 is a perspective view showing an image forming apparatus to which an image processing apparatus according to an embodiment of the present invention is applied. FIG. 2 is a block diagram illustrating an internal configuration of the image forming apparatus. The image forming apparatus 10 is a multi-function machine having a plurality of functions such as a copy function, a printer function, a scanner function, and a facsimile function. The image forming apparatus 10 is generally configured by an apparatus main body 11, an image reading unit 110 disposed above the apparatus main body 11, and a connecting unit 12 provided between the image reading unit 110 and the apparatus main body 11.

  The apparatus main body 11 includes an image forming unit 120, a paper feeding unit 13, a discharge tray 14, and the like.

  When the image forming apparatus 10 performs an image reading operation, the image reading unit 110 (acquisition unit) reads a document placed on the document placing table 111 and controls an image of the document under the control of an operation control unit 102 described later. The image data shown is generated. This image data is stored (saved) in the storage unit 160. In the image forming apparatus 10, the communication unit 170 receives image data from another information processing apparatus such as a PC (Personal Computer) instead of the image reading unit 110 reading the bundle of documents to acquire image data. The image data indicating the original document may be acquired by.

  When the image forming apparatus 10 performs an image forming operation, the image forming unit 120 is fed from the paper feeding unit 13 under the control of the operation control unit 102 based on the image data stored in the storage unit 160. A toner image is formed on the recording paper. Thereafter, the toner image formed on the recording paper is thermally fixed by a fixing unit (not shown). The recording paper on which the fixing process has been completed is discharged to the discharge tray 14.

  An operation unit 150 and a display unit 130 are arranged on the front surface of the apparatus main body 11. The display unit 130 includes a liquid crystal display (LCD) or an organic EL (Organic Light-Emitting Diode) display.

  A touch panel 140 is disposed on the screen of the display unit 130. The touch panel 140 is a so-called resistive film type or capacitive type touch panel, and detects contact (touch) of the user's finger or the like with the touch position on the touch panel 140.

  The operation unit 150 includes, for example, a menu key for calling a menu, an arrow key for moving the focus in the GUI constituting the menu, a determination key for performing a confirmation operation on the GUI constituting the menu, a start key, and the like.

  The storage unit 160 is a large-capacity storage device such as an HDD.

  The communication unit 170 is a network interface composed of a communication module such as a wireless LAN board.

  The image forming apparatus 10 further includes a control unit 100. The control unit 100 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. The control unit 100 is configured such that the control unit 101, the operation control unit 102, the operation receiving unit 103, the communication control unit 104, the marker detection unit are executed by executing the program stored in the ROM or the storage unit 160 on the CPU. 105, functions as an OCR processing unit 106 and an editing unit 107. Note that each of the above-described configurations of the control unit 100 may be configured by a hardware circuit without depending on an operation based on a program.

  The control unit 101 governs overall operation control of the image forming apparatus 10.

  The operation receiving unit 103 has a function of receiving a user operation on the touch panel 140 based on a detection signal output from the touch panel 140. The operation receiving unit 103 has a function of receiving a user operation on the hard key of the operation unit 150.

  The communication control unit 104 has a function of controlling the communication operation by the communication unit 170. The communication unit 170 transmits / receives data to / from a PC (Personal Computer) or the like under the control of the communication control unit 104.

  The operation control unit 102 has a function of controlling an image reading operation by the image reading unit 110, a display operation by the display unit 130, and the like.

  The marker detection unit 105 has a function of analyzing image data indicating an image of a document acquired by the image reading unit 110 and detecting a marker portion marked by a preset color marker on the document.

  The OCR processing unit 106 (text extraction unit) has a function of extracting text included in the marker location detected by the marker detection unit 105.

  The editing unit 107 has a function of editing text included in the marker portion based on preset editing conditions.

  Here, in the present embodiment, a document is read, a marker of a preset color marked on the document is detected, and the text at the marker portion is recognized and extracted (also referred to as zone OCR). Based on the set editing condition, the text is edited, the edited document is inserted into the body field of the email, and the email is transmitted.

  For example, as the editing condition, a first condition is set in which a line break is set immediately after a punctuation mark “.” Included in the text of a marker position of a preset first color (for example, yellow). In this case, assuming that the document G1 in which the first color markers as shown in FIG. 3A are marked at three locations My1, My2, and My3 is a processing target, the first color marker locations My1 to My1. My3 is detected, and the texts “Typhoon”, “May land in western Japan”, and “It will rain” are detected and extracted from the marker locations My1 to My3 of the first color. Then, the texts of these marker locations My1 to My3 are joined together in the order of description of the texts, and this combined text “Typhoon may land in western Japan. It will rain. Is generated. Furthermore, based on the first condition that a line break is located immediately after the punctuation mark “.” Included in the text of the marker portion of the first color, the typhoon has landed in western Japan. 3) Edit each line after the punctuation “.” Of “Maybe it will rain” and the punctuation “.” After “It will rain.” As shown in FIG. A simple text T1 is generated and inserted into the body of the e-mail E.

  In addition, as the editing condition, together with the first condition, a line is generated before and after the text of the marker portion of the second color (for example, green) set in advance to generate a line composed of the text, and at the head of the line A second condition of adding a symbol “•” indicating a bullet is set. In this case, the first color marker as shown in FIG. 4 (a) is marked at two locations My1 and My2, and the second color marker is marked at five locations Mg1, Mg2, Mg3, Mg4, and Mg5. Assuming that the original G2 is a processing target, the marker locations My1 and My2 of the first color are detected, the marker locations Mg1 to Mg5 of the second color are detected, and the marker locations My1 and My2 of the first color are detected. Recognize and extract the text "My family is composed of" and "It consists of" and the texts "Father", "Mother", "Brother" Recognize and extract “I” and “Sister”. Then, the texts of these marker locations My1, My2, Mg1 to Mg5My1 to My3 are joined together in the description order of the respective texts, and this synthesized text “My family is made up of parents and siblings. Is generated. Further, based on the first condition, a line feed is made immediately after the phrase “.” That is included in the combined text. Also, based on the second condition, a line is generated before and after the text of the second color marker portion to generate a line composed of the text, and a symbol “•” indicating a bullet is added to the head of the line of the text. For each of the text “Father”, “Mother”, “Brother”, “I”, and “Sister” in the marker locations Mg1 to Mg5 of the second color, the text is inserted before and after the text, ”,“ Mother ”,“ Brother ”,“ I ”, and“ Sister ”are generated, and a symbol“ · ”indicating an itemized list is added to the beginning of each line of each text, and FIG. A text T2 as shown in FIG. 2 is generated and inserted into the body of the e-mail E.

  Next, the first color and second color markers marked on such an original are detected, the text of the marker locations of the first color and second color are recognized and extracted, and the first condition and the first color are detected. A processing procedure for editing the text based on the two conditions will be described with reference to the flowchart shown in FIG.

  First, it is assumed that a plurality of touch keys 21a to 21h and the like associated with each function are displayed on the screen of the display unit 130 of the image forming apparatus 10 as illustrated in FIG. In this state, when the user touches the touch key 21h corresponding to the editing and transmission of the document, the touch operation on the touch key 21h is detected by the touch panel 140, and this touch operation is received by the operation receiving unit 103, and the control unit 101 activates a function (application program) for editing and transmitting a document (step S101).

  Subsequently, the user operates the operation unit 150 to input a mail address indicating a document transmission partner (step S102). At this time, an e-mail address input screen may be displayed on the display unit 130 so that the user performs an input operation on the input screen.

  Further, the user sets a document on the image reading unit 110 and operates the start key of the operation unit 150. In the control unit 101, when the operation of the start key is received by the operation receiving unit 103 (step S103), the operation control unit 102 activates the image reading unit 110, the image reading unit 110 reads the original, and the image of the original Is stored in the storage unit 160 (step S104).

  In addition, the marker detection unit 105 analyzes the image data in the storage unit 160, sequentially detects the first color or second color marker marked on the document (step S105), and detects the first color or the first color on the document. The OCR processing unit 106 is instructed about the second color marker location. The OCR processing unit 106 analyzes image data indicating a document on which a marker of the first color or the second color is marked, and performs OCR processing on the marker portion for each marker portion of the first color or the second color. Then, the text of the marker portion of the first color or the second color is recognized and extracted, and this text is stored in the storage unit 160 (step S106). At this time, the storage unit 160 stores, for each text, the first color or the second color of the marker portion where the text is extracted in association with the text.

  The editing unit 107 combines all the texts of the marker portions of the first color and the second color that are recognized and extracted by the OCR processing unit 106 by combining them in the description order of the respective texts, and combines the combined texts. Is generated (step S107).

  Subsequently, the editing unit 107 refers to the association between the text stored in the storage unit 160 and the first color or the second color, and the combined text includes the text of the marker portion of the first color. Is determined (step S108), and if it is determined that the text of the marker portion of the first color is included (“Yes” in step S108), the text “.” Is included in the text of the marker portion of the first color. It is determined whether it is included (step S109). If the editing unit 107 determines that the punctuation mark “.” Is included (“Yes” in step S109), based on the first condition, the combined text is the text of the marker portion of the first color. A line break is started immediately after the punctuation mark “.” (Step S110). At this time, if the combined text includes a plurality of texts of the first color marker locations, it is determined whether or not the text “.” Is included for each text of the marker locations of the first color. , If it is determined that the punctuation mark “.” Is included, a line break is inserted in the synthesized text at a position immediately after the determined punctuation mark “.”.

  If the editing unit 107 determines that the combined text does not include the text of the marker portion of the first color (“No” in step S108), the editing unit 107 can perform the processes of steps S109 and S110. Absent. Alternatively, if the editing unit 107 determines that the phrase “.” Is not included in the text of the marker portion of the first color (“No” in step S109), the process of step S110 is not performed.

  Further, the editing unit 107 determines whether or not the text of the second color marker location is included in the combined single text (step S111), and determines that the text of the second color marker location is included. (“Yes” in step S111) Based on the second condition, a line is generated before and after the text of the marker portion of the second color to generate a line composed of the text, and a symbol indicating a bullet at the beginning of the line “·” Is added (step S112). At this time, if the combined single text includes a plurality of texts of the second color marker locations, a line break is made before and after the text for each text of the second color marker locations. And a symbol “·” indicating a bulleted list is added to the head of one line of the text.

  If the editing unit 107 determines that the combined text does not include the text of the marker portion of the second color (“No” in step S111), the editing unit 107 does not perform the process of step S112.

  Through the processing in steps S104 to S112 as described above, for example, the text T1 shown in FIG. 3B is generated from the original G1 shown in FIG. 3A, or the original G2 shown in FIG. The text T2 shown in 4 (b) is generated. Thereafter, the communication control unit 104 generates an e-mail E destined for the e-mail address input in step S102, and puts the text generated by the processes in steps S104 to S112 in the text field of the e-mail E. The electronic mail E is inserted and transmitted from the communication unit 170 to the network (step S113).

  As described above, in the present embodiment, a marker of a preset color marked on a document is detected, and the text at the marker portion is recognized and extracted, and the text is determined based on the first condition and the second condition. Is edited, the edited text is inserted into the body field of the e-mail, and the e-mail is transmitted. For this reason, e-mail creation and the like can be proceeded quickly, and convenience is improved.

  The present invention is not limited to the configuration of the above embodiment, and various modifications can be made.

  For example, in the first condition of the above embodiment, a line break is immediately after the punctuation mark “.” Included in the text of the marker portion of the first color. "May be used as a line break. Furthermore, a line feed point may be set immediately after or immediately before another symbol.

  In addition, in the second condition, a symbol “·” indicating bullets is added to the beginning of one line of text, but other symbols may be added. Alternatively, a new line is sequentially generated immediately after each punctuation mark “,” included in the text of the second color marker portion to generate a line of text, and a symbol “•” indicating a bulleted item is added at the beginning of the line of the text. May be added.

  Further, the first color and the second color may be set to other colors different from yellow and green.

  Alternatively, a symbol for a line break, a symbol indicating a bullet, a color of a marker location from which text is extracted, and the like may be selected by operating the operation unit 150. Further, various symbols and various color samples of the marker are displayed on the screen of the display unit 130, and any one of these symbols and color samples is selected through the touch panel 140, and the selected symbol or color sample is displayed. Colors may be selected for text editing.

  Moreover, although the first condition and the second condition are illustrated as the editing conditions, other conditions may be set. For example, you may set the 3rd condition that the space area | region or blank area | region of a marker location is made into a line feed location. In this case, assuming that the document G3 in which the first color markers as shown in FIG. 7A are marked at four locations My1, My2, My3, and My4 is the processing target, each marker of the first color. Detects locations My1 to My4 and recognizes the text “Typhoon”, “May land in western Japan”, “It will rain” in each marker location My1, My2, My4 of the first color. And extracting and recognizing the space area or blank area of each marker location My3 of the first color. Then, the text of each marker location My1, My2, My3, My4 and the space area or the blank area are connected and combined in the description order thereof, and based on the third condition, the combined text is combined into the space area or the blank area. A line break is made at a location, and a space area or blank area is deleted from the combined text to generate a text T3 as shown in FIG.

  Alternatively, if a document G4 as shown in FIG. 8A is a processing target, the first color marker portions My1 to My4 are detected, and the first color marker locations My1, My2, and My4 are converted into text. Recognize and extract, recognize and extract the space area or blank area of each marker location My3 of the first color, and describe the text and space area or blank area of the marker locations My1, My2, My3, My4 in their description order 8b, based on the third condition, the synthesized text is line-breaked at the space area or blank area, and the space area or blank area is deleted from the synthesized text. A text T3 as shown in FIG.

  In addition, the editing unit 107 identifies the text indicated in the marker location of a preset color (for example, blue) as the text to be repeatedly used, and the above-described position is set at the position of the space area or the blank area of the marker location of the set color. The process of inserting the specified text may be performed.

  In the example shown in FIG. 9A, the marker locations Mb1 and Mb2 are locations marked in blue. In this case, the editing unit 107 identifies the text “To West Japan” shown in the marker location Mb1 as the text to be repeatedly used. Also, the marker location Mb2 is extracted by the OCR processing unit 106 as a space area of a marker location of a preset color. Then, the editing unit 107 performs a process of inserting the specified text “in West Japan” at the position of the marker location Mb2. As a result, a text T4 as shown in FIG. 9B is generated.

  By performing the processing as described above, text data in which the same text is inserted at an arbitrary position can be generated based on a simple operation called marking.

  Further, the OCR processing unit 106 extracts the text of the marker part, determines whether or not a number is indicated at the position of the marker part or a position close to the marker part, and when the number is indicated. May extract the number. In this case, the editing unit 107 performs processing for changing the order of the texts in the order of the numbers extracted by the OCR processing unit 106.

  In the example shown in FIG. 10A, the number N1 is located near the marker location My1, the number N2 is located near the marker location My2, the number N3 is located near the marker location My3, and the marker location My4. A number N4 is entered by the user's handwriting in the adjacent position. The OCR processing unit 106 extracts these numbers.

  Then, the editing unit 107 corresponds to the text extracted by the OCR processing unit 106, the text corresponding to the number “1”, the text corresponding to the number “2”, the text corresponding to the number “3”, and the number “4”. Rearrange the text in order. As a result, a text T5 as shown in FIG. 10B is generated.

  Note that the OCR processing unit 106 displays characters (for example, “A, B, C...”, “I, B, C...”, Etc.) at the position of the marker location or a position close to the marker location. It may be determined whether or not the character is displayed. If a character is indicated, the character may be extracted. In this case, the editing unit 107 performs processing to change the order of the text in the order of characters extracted by the OCR processing unit 106 (for example, alphabetical order or 50-sound order).

  By performing the processing as described above, it is possible to generate a text in which the text shown at the marker location is rearranged in an arbitrary order based on a simple operation of entering a number by handwriting or the like.

DESCRIPTION OF SYMBOLS 10 Image forming apparatus 100 Control unit 101 Control part 102 Operation control part 103 Operation reception part 104 Communication control part 105 Marker detection part 106 OCR process part 107 Editing part 110 Image reading part 120 Image formation part 130 Display part 140 Touch panel 150 Operation part 160 Storage unit 170 Communication unit

Claims (8)

An acquisition unit that acquires image data indicating an image of a document;
A marker detection unit for detecting a marker marked on the document based on the image data;
A text extraction unit that analyzes the image data and extracts the text of the marker portion in the document;
An image processing apparatus comprising: an editing unit that determines a line break in the extracted text based on a preset editing condition, and edits the text by breaking the text at the line break. The text extraction unit extracts text at a plurality of marker locations,
The editing unit sequentially combines the texts of the plurality of marker locations, determines a line break location in the synthesized text based on the editing conditions, and edits the synthesized text with line breaks at the line feed locations. The image processing apparatus according to claim 1.   The editing unit according to claim 1 or 2, wherein the editing unit performs the editing process by using the editing condition as a condition in which a punctuation point or a preset symbol included in the text of the marker part is the line break part. Image processing device.   The editing section includes a condition for adding a line break before and after the text of a marker portion of a preset color, and a condition for adding a symbol indicating bullets to one line of the text of the marker portion generated by the line break as the editing condition The image processing apparatus according to claim 1, wherein the editing process is performed using. The text extraction unit extracts a space area or a blank area of the marker portion in the document,
5. The image processing apparatus according to claim 1, wherein the editing condition is a condition that a space area or a blank area of the marker location is set as the line feed location. 6. The editing unit further specifies the text shown in the marker portion of a preset color as the text to be used repeatedly,
The text extraction unit extracts a space area or a blank area of the marker location of the preset color,
The said editing part performs the process which inserts the said specified text in the position of the space area | region of the marker location of the preset color, or a blank area | region. Image processing device. The text extraction unit extracts the text of the marker location and determines whether or not a character or a number is indicated at the position of the marker location or a position close to the marker location. If so, extract the letters or numbers,
The image according to any one of claims 1 to 6, wherein the editing unit performs a process of changing the order of the text extracted by the text extraction unit in the order of characters or numbers extracted by the text extraction unit. Processing equipment. A communication unit for communicating e-mail;
8. The communication control unit according to claim 1, further comprising: a communication control unit that inserts text edited by the editing unit as a document of the electronic mail and transmits the electronic mail from the communication unit. An image processing apparatus according to 1.