JP2011004436A - Image processing apparatus, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the efficiency of processing an original, on which characters are described by handwriting.SOLUTION: An image processing apparatus (MFP) includes: an image reading unit 105 for reading an original and acquiring image data; a handwritten part extraction unit 151 for extracting a handwritten part, including a handwritten character, from the obtained image data; a handwritten part erase unit 153 for generating erase image data by erasing a plurality of handwritten parts extracted from the obtained image data; a cross-referencing unit 155 for storing the generated erase image data in association with the extracted handwritten parts; a compare and extraction unit 156 for comparing another image data obtained by reading any other original by the image reading unit 105 with the erase image data being stored in a data storage unit 108; and a handwritten part synthesis unit 157 for combining the handwritten part, stored in association with the erase image data determined to be matched with the other image data by the comparison and extraction unit 156, with the other image data.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program, and more particularly, to an image processing apparatus, an image processing method, and an image processing program for reading and processing a manuscript in which characters are handwritten.

  2. Description of the Related Art In recent years, when processing image data obtained by reading a document in which characters are added by handwriting on a paper imaged by an image forming apparatus such as a printer, a handwritten character portion and a non-handwritten printed character portion are used. A technique for separately processing is known (for example, Patent Document 1).

However, the conventional image processing apparatus has a problem in that when there are a plurality of handwritten character portions, the same processing is uniformly performed on the plurality of handwritten character portions. For example, even if you want to erase only some of the handwritten characters, all will be erased.
JP 2001-61061 A

  The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide an image processing apparatus, an image processing method, and an image processing program that can use handwritten characters entered in a document for another document.

  In order to achieve the above-described object, according to one aspect of the present invention, an image processing apparatus reads a document and acquires image data, and extracts a handwritten portion including handwritten characters from the acquired image data. Extraction means, erasure means for generating erased image data obtained by erasing the extracted handwritten part from the acquired image data, storage for storing the generated erased image data and the extracted handwritten part in association with each other Means for comparing another image data obtained by reading a document different from the document by the reading unit with the erased image data stored in the storage unit, and the comparison unit matches the other image data. A handwriting portion stored in association with the erased image data, and a combining means for combining the image data with another image data.

  According to this aspect, the acquired other image data is combined with the handwritten portion stored in association with the erased image data that is supposed to match, so that it is extracted from a manuscript with handwritten characters. The handwritten characters can be added to a manuscript that is not filled with handwritten characters. As a result, it is possible to provide an image processing apparatus that can use handwritten characters entered in a manuscript for another manuscript.

  Preferably, the image processing apparatus further includes display means for displaying a plurality of handwritten portions stored in association with erased image data determined to match another image data by the comparison means.

  According to another aspect of the present invention, an image processing method includes a step of reading a document and acquiring image data, a step of extracting a handwritten portion including handwritten characters from the acquired image data, and an extracted handwritten A step of generating erased image data in which a portion is erased from the acquired image data, a step of storing the generated erased image data and the extracted handwritten portion in association with each other, and reading a document different from the document Comparing the other image data acquired in the step with the erased image data stored in the storing step, the handwriting part stored in association with the erased image data determined to match the other image data in the comparison step, Synthesizing with another image data.

  According to this aspect, it is possible to provide an image processing method in which handwritten characters entered in a manuscript can be used for another manuscript.

  According to still another aspect of the present invention, an image processing program reads a document and acquires image data, a step of extracting a handwritten part including handwritten characters from the acquired image data, and an extracted hand A step of generating erased image data in which the handwritten portion is erased from the acquired image data, a step of storing the generated erased image data in association with the extracted handwritten portion, and a document different from the document Comparing another image data read and acquired with the erased image data stored in the storing step, and a handwritten part stored in association with the erased image data determined to match the other image data in the comparing step And a step of synthesizing the image data with another image data.

  According to this aspect, it is possible to provide an image processing program in which handwritten characters entered in a manuscript can be used for another manuscript.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  FIG. 1 is a functional block diagram showing an outline of the functions of an MFP according to one embodiment of the present invention. Referring to FIG. 1, MFP 100 includes a control unit (CPU) 101 that controls MFP 100 as a whole, an operation display unit 102 that serves as an interface with a user, and a ROM (Read) that stores programs to be executed by CPU 101. (Only Memory) 103, a RAM (Random Access Memory) 104 for loading a program to be executed and storing data during program execution, an image reading unit 105 for reading a document and outputting image data, An image processing unit 106 that executes predetermined processing on image data, an image forming unit 107 that forms an image on a recording medium such as paper based on the image data, and a CD-ROM (Compact Disc-Read Only Memory) 110A are mounted. External storage device Including the 110, a data storage unit 108 for storing data, and a communication unit 109 for connecting the MFP100 to the network.

  The image reading unit 105 includes a photoelectric conversion device such as a CCD (Charge Coupled Device), optically reads a document, and outputs image data as electronic data. The image processing unit 106 performs a predetermined process on the image data output from the image reading unit 105. The predetermined process will be described later. The image forming unit 107 is a laser printer, an inkjet printer, or the like, and visualizes image data on a recording medium such as paper.

  The operation display unit 102 includes a display unit 102A and an operation unit 102B. The display unit 102A is a display device such as a liquid crystal display (LCD) or an organic ELD (Electro Luminescence Display), and displays an instruction menu for the user, information about acquired image data, and the like. The operation unit 102 </ b> B includes a plurality of keys, and accepts input of various instructions, data such as characters and numbers by user operations corresponding to the keys. The operation unit 102B includes a touch panel made of a transparent member. By placing the touch panel on the display unit 102A, it is possible to detect an instruction of a button included in the screen displayed on the display unit 102A. Thereby, it becomes possible to accept input of various operations.

  The data storage unit 108 is a mass storage device such as a hard disk drive (HDD). The data storage unit 108 stores image data output by the image reading unit 105, image data output by the image processing unit 106, and the like.

  Communication unit 109 is a communication interface for connecting MFP 100 to a local area network (LAN). Therefore, the MFP 100 can communicate with other computers connected to the LAN using a communication protocol such as TCP (Transmission Control Protocol) or FTP (File Transfer Protocol). The connection form of the LAN to which the MFP 100 is connected may be wired or wireless. The network to which MFP 100 is connected is not limited to a LAN but may be a wide area network (WAN), the Internet, or the like. Further, the communication unit 109 can directly connect the MFP 100 to another computer via a serial interface or a parallel interface. Communication unit 109 communicates with communication protocol in accordance with the form in which MFP 100 is connected to another computer.

  The external storage device 110 is a CD-ROM driver to which a CD-ROM 110A is attached. The CD-ROM 110A stores an image processing program, and the CPU 101 can read the image processing program and execute the read image processing program to execute image processing to be described later.

  The recording medium for storing the image processing program is not limited to the CD-ROM 110A, but a flexible disk, a cassette tape, an optical disk (MO (Magnetic Optical Disc / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card (Including memory card), optical card, mask ROM, EPROM (Erasable Programmable ROM), semiconductor memory such as EEPROM (Electronically EPROM), etc. Furthermore, the CPU 101 downloads an image processing program from a computer connected to the LAN. Stored in the data storage unit 108, or the computer writes the image processing program into the data storage unit 108. In this manner, the image processing program stored in the data storage unit 108 may be loaded into the RAM 104 and executed by the CPU 101. The program here is not only a program that can be directly executed by the CPU 101, but also a source program. Programs, compressed programs, encrypted programs, and the like.

  FIG. 2 is a functional block diagram showing an outline of functions of the image processing unit together with data stored in the data storage unit. Referring to FIG. 2, the image processing unit 106 extracts a handwriting part extraction unit 151 that extracts a handwritten character part (handwriting part) from the image data output by the image reading unit 105, and a plurality of extracted handwriting parts. A list generating unit 152 that generates a list display screen in which images of handwritten parts are arranged, a handwritten part erasing unit 153 that generates erased image data in which the handwritten parts are erased from the image data, and a balloon image based on the handwritten parts A balloon image combining unit 154 that generates data and combines it with erased image data, and an association unit 155 that stores the erased image data and the handwritten portion in association with each other are included.

  The image data is bitmap data in which pixels are arranged two-dimensionally. The handwriting part extraction unit 151 extracts a character area in which characters are represented from the image data, and if the characters included in the character area are handwritten characters, the image data included in the character area is extracted as a handwriting part. . The technique for extracting the character area from the image data is a well-known technique and will not be described in detail. For example, the image data is divided into a plurality of blocks, and the image attribute is determined for each block. As a value for discriminating an image attribute, for example, a primary differential value calculated using an edge detection filter such as a primary differential filter can be used. This is because a block including a pixel having a large primary differential value is likely to include a line constituting a character. Then, adjacent blocks are collected from a plurality of blocks determined as character attributes, and a character area is specified.

  Further, the handwriting part extraction unit 151 compares the characters included in the character area image with the character information data such as a character font stored in advance, and the characters are included in the image of the character area that is not identical or similar to each other. The character to be read is determined as a handwritten character, and the image data of the character area including the character is extracted as a handwritten portion. The handwritten handwriting part extraction unit 151 determines that the character included in the image of the character region where the character included in the image of the character region and the character information data stored in advance match or similar is not a handwritten character, The image data of the character area including characters is not extracted as a handwritten part.

  The handwriting part extraction unit 151 combines two nearby handwriting parts into one handwriting part from the positions (two-dimensional coordinates) in each of the plurality of handwriting parts extracted from the image data. . This is because if the character spacing of the handwritten character string is wide, the character string may be divided into a plurality of parts and extracted as a plurality of handwritten parts, so that one character string becomes one handwritten part. The handwriting part extraction unit 151 extracts a plurality of handwriting parts from the image data if handwritten characters are described in a plurality of portions of the document. The handwritten part extracting unit 151 outputs a plurality of handwritten parts extracted from the image data to the list generating unit 152, the handwritten part erasing unit 153, the balloon image combining unit 154, and the association unit 155. The handwritten part extracted from the image data may be a part of the image data, or may be position information of the handwritten part in the image data.

  Upon receiving a plurality of handwritten parts from the handwritten part extracting unit 151, the list generating unit 152 generates a list display screen in which images of the plurality of handwritten parts extracted by the handwritten part extracting unit 151 are arranged. The list generation unit 152 outputs the generated list display screen to the display unit 102A, and causes the display unit 102A to display the list screen. Although details of the list display screen will be described later, processing instruction buttons corresponding to a plurality of handwritten character portions are arranged in the vicinity of each of the images of the plurality of handwritten portions displayed side by side. The processing instruction button is an instruction button for instructing the user to execute predetermined image processing only on the handwritten portion corresponding to the processing instruction button. The image processing includes an erasing process for erasing the handwritten part from the image data, and a balloon image combining process for generating balloon image data including the handwritten part and replacing the handwritten part with the balloon image data. Here, the instruction for instructing the erasing process is referred to as an erasing instruction, and the instruction for instructing the balloon image composition process is referred to as a balloon display instruction.

  When the user operates the operation unit 102B to instruct image processing to any of a plurality of instruction buttons respectively corresponding to the images of a plurality of handwriting parts, the image processing unit 106 performs a hand corresponding to the instructed instruction button. An instruction for image processing of a book part is received. When a plurality of instruction buttons are instructed, an instruction for image processing of each of a plurality of handwriting parts is accepted.

  The handwritten part erasing unit 153 receives image data from the image reading unit 105, receives a plurality of handwritten parts from the handwritten part extracting unit 151, and receives an erasing instruction from the operation unit 102B. The handwritten part erasing unit 153 erases the handwritten part specified by the erasing instruction from the image data. Erasing does not erase the image data itself, but changes a pixel value included in a handwritten portion instructed to be erased from a plurality of pixel values constituting the image data to a background pixel value. For example, if the background image is white, the pixel value of the handwritten part is changed to 255. When the handwriting part erasing unit 153 receives a plurality of erasing instructions, the handwriting part erasing unit 153 erases a plurality of handwriting parts respectively specified by the plurality of erasing instructions from the image data. The handwritten part erasing unit 153 outputs image data obtained by erasing the handwritten part from the image data in accordance with the erasing instruction to the image forming unit 107 and causes the image forming unit 107 to form an image. As a result, if the user causes the image reading unit 105 to read a manuscript in which a plurality of character strings are entered by handwriting, and inputs an instruction to erase a character string selected from the plurality of character strings, the user instructs the erasure. An image is formed by deleting only the handwritten character string from the document.

  The balloon image synthesis unit 154 receives image data from the image reading unit 105, receives a plurality of handwriting parts from the handwriting part extraction unit 151, and receives a balloon display instruction from the operation unit 102B. The balloon image combining unit 154 generates balloon image data including a handwriting part specified by the balloon display instruction, and generates balloon image data by combining the balloon image data with the handwriting part of the image data. In the composition, the handwritten part specified by the balloon display instruction is replaced with balloon image data. If the balloon image data is larger in size than the handwritten part, for example, it may be arranged so that the center of the handwritten part overlaps the center of the balloon image data. Upon receiving a plurality of balloon display instructions, the balloon image combining unit 154 replaces the handwritten part specified by each of the balloon display instructions with corresponding balloon image data. The balloon image combining unit 154 outputs combined image data obtained by combining the balloon image data to the image data in accordance with the balloon display instruction to the image forming unit 107 and causes the image forming unit 107 to form an image. Thus, when the user inputs a balloon display instruction for converting a character string selected from the plurality of character strings into a balloon display by causing the image reading unit 105 to read a document in which a plurality of character strings are entered by handwriting. An image in which only a handwritten character string is displayed in a balloon is formed.

  The association unit 155 receives image data from the image reading unit 105 and receives a plurality of handwritten parts from the handwritten part extracting unit 151. The association unit 155 erases all of the plurality of handwritten parts extracted from the image data by the handwritten part extracting unit 151 to generate erased image data. The associating unit 155 associates the erased image data with a plurality of handwriting parts and stores them in the data storage unit 108. As a result, the association data 108A in which the erased image data and the plurality of handwritten parts are associated with each other is stored in the data storage unit 108.

  The image processing unit 106 further includes a comparison / extraction unit 156 and a handwriting part synthesis unit 157. The comparison extraction unit 156 receives image data from the image reading unit 105. When receiving the image data, the comparison extraction unit 156 compares the image data with the erased image data of the associated data 108 </ b> A stored in the data storage unit 108. Then, association data 108A including erased image data that matches or is similar to the image data is extracted. If the print data excluding handwritten characters matches the image data and the associated data 108A stored in the data storage unit 108, the comparison / extraction unit 156 may determine whether the image data includes handwritten characters. Judge that both data are similar. Thus, it is possible to cope with a case where a document in which a part of the handwritten characters is erased with an eraser or the like is read from the document.

  The comparison extraction unit 156 outputs all of the handwriting parts included in the extracted association data 108A to the list generation unit 152. When the comparison / extraction unit 156 outputs the handwritten part, the original is the same as the original on which the handwritten characters read by the image reading unit 105 are written, but the original on which the handwritten characters are not written is imaged. This is a case where the reading unit 105 reads. The same manuscript includes a copied manuscript. For example, this is a case where the image reading unit 105 reads the original after erasing the handwritten characters previously written in the original read by the image reading unit 105 with an eraser or the like. In addition, when there are two copied originals, handwritten characters are written on one original, and handwritten characters are not written on the other original, the image reading unit 105 displays the characters of one handwritten. This is a case where after reading a manuscript in which is written, the manuscript in which the other handwritten character is not written is read.

  Upon receiving a plurality of handwriting parts from the comparison and extraction part 156, the list generating part 152 generates a history display screen in which images of a plurality of handwriting parts are arranged. The image of the handwritten part displayed on the history display screen is the handwritten part extracted from the image data output by the image reading unit 105 reading and outputting the original in which the letters of the handwritten character are entered. The list generation unit 152 outputs the generated history display screen to the display unit 102A, and causes the display unit 102A to display the history display screen. Although details of the history display screen will be described later, processing instruction buttons corresponding to a plurality of handwritten character portions are arranged in the vicinity of each of the images of the plurality of handwritten portions displayed side by side. The processing instruction button is an instruction button for instructing the user to execute predetermined image processing only on the handwriting portion corresponding to the processing instruction button. The image processing includes synthesis processing for synthesizing the handwritten portion with image data. Here, an instruction for instructing the synthesis process is referred to as a synthesis instruction.

  The handwriting part combining unit 157 receives image data from the image reading unit 105, receives a plurality of handwriting parts from the comparison and extraction unit 156, and receives a combining instruction from the operation unit 102B. The handwriting part combining unit 157 combines the handwriting part specified by the combining instruction with the image data to generate combined image data. In the composition, the handwritten part specified by the composition instruction is arranged at the same position as that of the original from which the handwritten part is extracted in the image data. When the handwriting part combining unit 157 receives a plurality of compositing instructions, the handwriting part combining unit 157 combines the handwriting part specified by each of the plurality of compositing instructions with the image data. The handwriting part synthesizing unit 157 outputs composite image data obtained by synthesizing the handwriting part in accordance with the compositing instruction to the image forming unit 107 and causes the image forming unit 107 to form an image. As a result, even if the user reads a document with handwritten characters erased with an eraser or the like, if the MFP 100 reads the document before erasing the handwritten characters, the document with the handwritten characters erased. An image in which handwritten characters are added to can be formed. Further, the handwriting part combining unit 157 may generate the blowout image data from the handwriting part specified by the combining instruction, and combine the blowing image data with the image data. . As a result, it is possible to form an image in which handwritten characters are blown out and displayed on a document from which handwritten characters are erased.

  In addition, although an example in which the deletion instruction and the balloon display instruction are separately indicated on the list screen has been shown, it is possible to instruct the deletion instruction to a part of a plurality of handwriting parts, and the other part It may be possible to instruct a balloon display instruction. As a result, it is possible to form an image in which a part of the plurality of handwritten characters entered in the document is erased and the other part is a blown-out image.

  FIG. 3 is a first flowchart showing an example of the flow of image forming processing executed by the MFP in the present embodiment. Referring to FIG. 3, MFP 100 reads a document and acquires image data (step S01). Then, a handwritten part is extracted from the image data (step S02). The handwritten part includes at least one character string. If handwritten character strings are entered in a plurality of areas in the document, a plurality of handwritten parts are extracted.

  Then, a list screen in which images of a plurality of handwritten parts extracted in step S02 are arranged is generated and displayed on the display unit 102A (step S03). A processing instruction corresponding to each of the plurality of handwriting parts is received (step S04), and the process proceeds to step S05. In step S05, a handwriting part to be processed is set from among a plurality of handwriting parts extracted in step S02. Then, it is determined whether or not the processing instruction corresponding to the handwritten portion set as the processing target is an erasure instruction (step S06). If the processing instruction is an erasure instruction, the process proceeds to step S07. If not, the process proceeds to step S08, skipping step S07. In step S07, the handwritten portion set as the processing target is deleted from the image data.

  Next, it is determined whether or not the processing instruction corresponding to the handwriting part set as the processing target is a balloon display instruction (step S08). If the process instruction is a balloon display instruction, the process proceeds to step S09; otherwise, the process proceeds to step S11. In step S09, balloon image data is generated based on the handwritten portion set as the processing target. Then, the handwritten part set as the processing target is erased from the image data, and the previously generated balloon image data is combined with the image data (step S10).

  In step S11, it is determined whether or not there is a handwriting part that should not be set yet as a processing target, and if such a handwriting part exists, the process proceeds to step S05. If not, the process proceeds to step S12.

  In step S12, the handwritten part is deleted in step S07, or the image data combined with the blown-out image data in step S10 is output to the image forming unit 107 to form an image. Then, from the image data acquired in step S01, erased image data is generated by erasing a plurality of handwritten parts extracted from the image data in step S02 (step S13), and the erased image data and the plurality of handwritten parts are obtained. The associated association data is generated (step S14), and the generated association data is stored in the data storage unit 108 (step S15).

  FIG. 4 is a first diagram illustrating an example of a document in which handwritten characters are entered. Referring to FIG. 4, the manuscript includes a handwritten character string “(Ox question)”, a character string “19:00 scheduled”, and a character string “the back of East Building 2F”. Including. FIG. 5 is a diagram illustrating an example of a list screen. The list screen shown in FIG. 5 includes a character string “(about the problem)”, a character string “scheduled for 19:00”, and a character string “the back side of East Building 2F” included in the document shown in FIG. Three images including are arranged side by side. The three images are images of three handwritten parts extracted from the image data obtained by reading the document shown in FIG. The list screen also displays a “menu” and includes a process selection button for selecting instruction contents. When the user designates a process selection button, a process instruction button for designating whether or not to delete the handwritten part from the image data is displayed on the right side of each of the three images.

  FIG. 6 is a first diagram illustrating an example of a list screen on which processing instruction buttons are displayed. Referring to FIG. 6, the list screen includes processing instruction buttons which are arranged on the right side of the three images and display “Do not delete”. Each time the user instructs the processing instruction button, the display of the processing instruction button is switched in the order of “Do not delete”, “Delete”, “Blowout display”, and “Do not delete”. The processing instruction button is a state instructing not to delete the corresponding handwritten portion from the image data in a state in which “Do not delete” is displayed, and is instructed from the image data in a state in which “Delete” is displayed. In the state instructing to erase the handwritten part to be erased, in a state in which “callout display” is displayed, instructing to erase the handwritten part from the image data and to synthesize the blowout image is there.

  FIG. 7 is a second diagram illustrating an example of a list screen on which processing instruction buttons are displayed. "Do not delete" is displayed on the processing instruction button arranged on the right side of the image including the uppermost character string "(XX problem)", and the processing instructions arranged on the right side of the other two images, respectively. “Erase” is displayed on the button. In this state, if the copy key indicating the characters “Copy” is designated, two deletion instructions for the character string “19:00 scheduled” and the character string “the back side of the east building 2F” of the image data are given to the MFP 100. Accept.

  FIG. 8 is a third diagram illustrating an example of a list screen on which processing instruction buttons are displayed. “Blowout display” is displayed on the processing instruction button arranged on the right side of the image including the uppermost character string “(XX problem)”, and the processing instructions arranged on the right side of the other two images, respectively. “Erase” is displayed on the button. In this state, if the copy key indicating the character “Copy” is indicated, a balloon display instruction for the handwritten portion including the character string “Ox question” of the image data and the character string “19:00 scheduled” In addition, MFP 100 receives an erasure instruction for each of the character strings “the back side of East Building 2F”.

  FIG. 9 is a first diagram illustrating an example of image data after image processing. The image data shown in FIG. 9 shows image data generated when a processing instruction is input according to the list screen shown in FIG. Referring to FIG. 9, as compared with the manuscript shown in FIG. 4, all the handwritten characters entered in the manuscript are erased, and the balloon image including the character string “Ox question” is synthesized. .

  FIG. 10 is a flowchart illustrating an example of the flow of handwriting part extraction processing. The handwriting part extraction process is a process executed in step S02 of FIG. Referring to FIG. 10, the image data is divided into a plurality of blocks (step S21). Then, the image attribute is discriminated for each block, and the block discriminated as the character attribute representing the character is extracted (step S22). Then, the plurality of extracted character attribute blocks are combined into a character region, and the character region is extracted from the image data (step S23). FIG. 11 is a diagram illustrating an example of a character region extracted from image data. FIG. 11 shows a character area extracted from image data obtained by reading the document shown in FIG. A region surrounded by one rectangle indicates a region extracted as one character region.

  Returning to FIG. 10, when a plurality of character areas are extracted, the processing from step S24 is executed for each character area. First, one of a plurality of character areas is set as a processing target, and characters included in the image of the character area are compared with character information data stored in advance in the data storage unit 108 (step S24). If they are not identical or similar, the character included in the character area image is determined to be a handwritten character, and the character area including the character is extracted and the process proceeds to step S25. It is determined that the characters included in the area are print characters that are not handwritten characters, and the process proceeds to step S27.

  In step S25, the relationship between the coordinate positions between the character areas is determined. Specifically, two character regions are included in the image data from the coordinate position in the image data of the character region determined to be a handwritten character and the coordinate position in the image data of another character region determined to be a handwritten character. It is determined whether or not they are adjacent to each other. If the two character areas are adjacent to each other in the image data, the process proceeds to step S26. If not, the process proceeds to step S27, skipping step S26. In step S26, two adjacent character areas are combined into one handwritten character portion. This is because if the character spacing of the handwritten character string is wide, the character string may be divided into a plurality of parts and extracted as a plurality of handwritten parts, so that one character string becomes one handwritten part. Then, it is determined whether or not there is a character area to be set next (step S27). If such a character area exists, the process returns to step S24, and if not, the process ends. .

  FIG. 12 is a diagram illustrating an example of a handwritten part extracted from image data. Referring to FIG. 12, the handwritten portion is indicated by a rectangular area with hatching.

  FIG. 13 is a second flowchart illustrating an example of the flow of image forming processing executed by the MFP according to the present embodiment. Referring to FIG. 13, MFP 100 reads a document and obtains image data (step S31). Then, the acquired image data is compared with the erased image data of the association data 108A stored in the data storage unit 108 (step S32). Then, it is determined whether or not the association data 108A including erased image data that matches or is similar to the image data has been extracted (step S33). For example, if only the handwriting part is different, it is determined that both are similar and the associated data 108A is extracted. If the associated data 108A is extracted, the process proceeds to step S34, and if not extracted, the process proceeds to step S41.

  In step S34, all of the plurality of handwritten parts included in the extracted association data 108A are acquired. And the log | history screen which arranged the image of several acquired handwritten parts is produced | generated, and it displays on 102 A of display parts (step S35). A processing instruction corresponding to each of the plurality of handwriting parts is received (step S36), and the process proceeds to step S37. In step S37, a handwriting part to be processed is set from among a plurality of handwriting parts acquired in step S34. Then, it is determined whether or not the processing instruction corresponding to the handwriting part set as the processing target is a compositing instruction (step S38). If the process instruction is a synthesis instruction, the process proceeds to step S39, and if not, the process proceeds to step S40, skipping step S39. In step S39, the handwritten portion set as the processing target is combined with the image data. Note that the blowout image data may be generated from the handwritten portion, and the blowout image data may be combined with the image data.

  In step S40, it is determined whether or not there is a handwriting part that should not be set as a processing target yet and should be set as a processing target next. If such a handwriting part exists, the process proceeds to step S37. If it does not exist, the process proceeds to step S41. When the process proceeds from step S33 to step S41, the image data obtained in step S31 is formed as it is, but when the process proceeds from step S40 to step S41, the handwritten part designated by the accepted compositing instruction is imaged. The image data combined with the data is imaged.

  FIG. 14 is a second diagram illustrating an example of a document. Referring to FIG. 14, the document includes only print characters, and the print characters are the same as the document shown in FIG. Therefore, there is a difference between the original shown in FIG. 14 and the original shown in FIG. 4 whether or not handwritten characters are entered. As described above, when the MFP 100 reads the document shown in FIG. 4, the handwriting part including the handwritten character string “(Ox question)” and the character string “19:00 scheduled” are read. The data storage unit 108 stores association data 108 </ b> A in which the handwriting part including the handwriting part including the character string “the back of the east building 2 </ b> F” and the erased image data from which the handwriting part is erased are associated.

  FIG. 15 is a diagram illustrating an example of a history screen. Since the image data obtained by reading the original shown in FIG. 14 matches the erased image data generated from the image data obtained by reading the original shown in FIG. 4, it is associated with the erased image data by the association data 108A. In addition, a handwriting part including the character string “(○ × problem)”, a handwriting part including the character string “scheduled for 19:00” and a handwriting part including the character string “the back side of the east building 2F” A history screen is generated from The history screen shown in FIG. 15 displays the character string “(about the problem)”, the character string “scheduled for 19:00” and the character string “the back side of the east building 2F” included in the document shown in FIG. Three images including are arranged side by side. Further, the history list screen has a character string “Example!” Extracted from image data obtained by reading a document in which the character string “Example!” Is written by handwriting and is the same as the document shown in FIG. ”Is included. In the history list screen, a processing instruction button for designating whether or not the handwriting part is to be combined with the image data is displayed on the right side of the four images. Each time the user instructs the process instruction button, the display of the process instruction button is switched in the order of “Do not stick”, “Paste”, and “Do not stick”. The processing instruction button is a state instructing not to synthesize the corresponding handwritten portion with the image data in a state in which “Do not paste” is displayed, and in the state in which “Paste” is displayed, This is a state in which it is instructed to synthesize a portion with image data.

  FIG. 16 is a second diagram illustrating an example of image data after image processing. The image data illustrated in FIG. 16 indicates image data generated when a processing instruction is input according to the history screen illustrated in FIG. Referring to FIG. 16, an image including a character string “Ox question” is synthesized as compared with the document shown in FIG. 4.

  As described above, portable MFP 100 in the present embodiment extracts a plurality of handwritten parts including handwritten characters from image data obtained by reading a document, and displays a list screen on which the images of the plurality of extracted handwritten parts are arranged. To do. When one of the images displayed on the list screen is selected, the handwritten part corresponding to the selected image is erased from the image data, or the balloon image data generated from the handwritten part is image data. And synthesize. For this reason, it is possible to selectively delete a plurality of handwritten characters written in the original on the original or to compose them as a balloon image.

  Further, MFP 100 extracts a plurality of handwritten parts including handwritten characters from image data obtained by reading a document, generates erased image data in which the extracted handwritten parts are erased from the image data, and creates erased image data and Data associated with the handwriting part is generated and stored in the data storage unit 108. Thereafter, another image data obtained by reading another document is compared with the erased image data stored in the data storage unit 108, and is stored in association with the erased image data determined to match the other image data. The handwritten part and another image data are combined. For this reason, handwritten characters extracted from a manuscript with handwritten characters can be added to a manuscript without handwritten characters.

  In the embodiment described above, MFP 100 has been described. Needless to say, the invention can be understood as an image processing method or an image processing program for causing MFP 100 to execute the image processing shown in FIG. 3 or FIG. Absent.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

<Appendix>
(1) The image processing means generates blowing image data including the handwritten part stored in association with the erased image data determined to match the other image data by the comparing means; ,
The image processing apparatus according to claim 1, further comprising blowing image data combining means for combining the generated blowing image data with the other image data.

FIG. 2 is a functional block diagram showing an outline of functions of an MFP according to one embodiment of the present invention. It is a functional block diagram which shows the outline | summary of the function of an image process part with the data memorize | stored in a data storage part. 6 is a first flowchart illustrating an example of a flow of image forming processing executed by the MFP according to the present embodiment. It is a 1st figure which shows an example of the original document in which the handwritten character was written. It is a figure which shows an example of a list screen. It is a 1st figure which shows an example of the list screen on which the process instruction button was displayed. It is a 2nd figure which shows an example of the list screen on which the process instruction button was displayed. It is a 3rd figure which shows an example of the list screen on which the process instruction button was displayed. It is a 1st figure which shows an example of the image data after an image process. It is a flowchart which shows an example of the flow of a handwritten part extraction process. It is a figure which shows an example of the character area extracted from image data. It is a figure which shows an example of the handwritten part extracted from image data. 10 is a second flowchart illustrating an example of the flow of image forming processing executed by the MFP according to the present embodiment. FIG. 10 is a second diagram illustrating an example of a document. It is a figure which shows an example of a log | history screen. It is a 2nd figure which shows an example of the image data after an image process.

100 MFP, 101 control unit, 102 operation display unit, 102A display unit, 102B operation unit, 103 ROM, 104 RAM, 105 image reading unit, 106 image processing unit, 107 image forming unit, 108 data storage unit, 108A associated data 109 communication unit, 110 external storage device, 110A CD-ROM, 151 handwriting part extraction unit, 152 list generation unit, 153 handwriting part deletion unit, 154 balloon image composition unit, 155 association unit, 156 comparison extraction unit, 157 Handwriting part composition part.

Claims (4)

Reading means for reading an original and acquiring image data;
Extracting means for extracting a handwritten portion including handwritten characters from the acquired image data;
An erasing unit for generating erased image data in which the extracted handwritten part is erased from the acquired image data;
Storage means for storing the generated erased image data in association with the extracted handwritten part;
Comparing means for comparing the image data acquired by the reading means by reading a document different from the document with erased image data stored in the storage means;
Image processing further comprising: a combining unit that combines the handwritten portion stored in association with the erased image data determined to match the other image data by the comparison unit and the other image data. apparatus.   The image processing apparatus according to claim 1, further comprising: a display unit configured to display a plurality of the handwritten parts stored in association with the erased image data determined to match the other image data by the comparison unit. Scanning the document to obtain image data;
Extracting a handwritten part including handwritten characters from the acquired image data;
Generating erased image data in which the extracted handwritten portion is erased from the acquired image data;
Storing the generated erased image data in association with the extracted handwritten part;
Comparing another image data obtained by reading a document different from the document with the erased image data stored by the storing step;
An image processing method comprising: synthesizing the handwritten portion stored in association with the erased image data determined to match the other image data by the comparing step and the other image data. Scanning the document to obtain image data;
Extracting a handwritten part including handwritten characters from the acquired image data;
Generating erased image data in which the extracted handwritten portion is erased from the acquired image data;
Storing the generated erased image data in association with the extracted handwritten part;
Comparing another image data obtained by reading a document different from the document with the erased image data stored in the storing step;
A step of combining the handwritten portion stored in association with the erased image data determined to match the other image data by the comparison step and the other image data, and an image processing program to be executed by a computer.

Images