JP4808661B2 - Image processing apparatus, program, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, a program, and an image processing method.

  In recent years, a scanner that reads a document image, a copying machine that prints a document image read by the scanner, a printer or facsimile machine that prints image data input from an external device, or a so-called MFP (Multi Function Peripheral) that combines these functions ) Is used.

  In this type of multifunction device, scanned copies, facsimile originals, and image data input from external devices are stored in an HDD (Hard Disk Drive) and re-outputted or retransmitted as necessary. Some of them have a document box function. According to such a document box function, work efficiency can be greatly improved and paperless can be realized.

  By the way, when proofreading a document in the document box function of the multifunction device, the image data in the HDD is output to paper for document proofreading, and the proofreading contents are added to the paper by hand. Become. In this case, the proofreading contents added on paper are reflected manually on the original image data. Usability differs between information printed on paper and information stored as electronic data. The information written on the paper can be referred to and added and corrected in many places if there are paper and writing tools, with few restrictions on the place of use. However, in the case of electronic data, an electronic device for handling it is necessary. In some cases, a power source may be required, and the place of use may be limited. However, the functions of correction and editing are excellent. However, complex editing may be difficult if you are not familiar with the electronic equipment you are using.

  Therefore, in order to solve such a problem, information to be added or replaced by the user from the contents added to the paper manuscript is reflected in the image data of the reading source, thereby adding information to the image data. Techniques that can improve convenience have been proposed (see, for example, Patent Document 1 and Patent Document 2).

  In the technique described in Patent Document 1, image data to be a document is displayed in order to facilitate editing work, and handwriting input is performed on the screen with a stylus pen or the like to perform editing work. More specifically, Patent Document 1 describes a technology that recognizes a command close to a pre-registered edit symbol as a command and allows an additional part to be input by kana-kanji conversion. As described above, according to Patent Document 1, it is possible to work on an edit symbol that has been conventionally used in a paper document by using a display screen of an electronic device.

  The technique described in Patent Document 2 adds a predetermined proofreading symbol to document data in order to enable quick editing processing, and performs editing processing on the document data based on the proofreading symbol. is there. More specifically, in Patent Document 2, an area graphic designated by graphic data of a calibration symbol is detected, the detected area graphic, the calibration content predetermined for the graphic data, and the graphic data A technique for editing document data in accordance with corresponding character information is described. As described above, according to Patent Document 2, editing of document data is realized by inputting a preset calibration symbol using an input unit of an electronic device.

Japanese Patent No. 3398981

  However, according to the technique described in Patent Document 1, since editing is performed by handwriting input with a stylus pen or the like on a screen on which image data serving as a document is displayed, the edit target area is accurately indicated. It is difficult. Here, the problem is whether to edit the contents to be added to which edit target area when a plurality of edit target areas specified by handwriting input are specified.

  The present invention has been made in view of the above, and improves the convenience of adding or deleting information from a paper document to the original document image information when the original document image information is calibrated on the paper document. An object is to provide an image processing apparatus, a program, and an image processing method.

  In addition, the present invention provides an image processing apparatus, a program, and an image processing method capable of improving the convenience of adding or deleting information to / from original image information when calibrating the original image information by acquiring additional information. The purpose is to provide.

In order to solve the above-described problems and achieve the object, an image processing apparatus according to a first aspect of the present invention relates to a document on which original document image information and link information indicating a storage position of the original document image information are printed. Document image information generating means for generating document image information, original document information acquiring means for acquiring the original document image information according to the link information of the document image information, and an image defining an image configuration of the original document image information an image content analyzing means for analyzing the building blocks, and revision information acquisition means for acquiring revision information by taking the difference between the original document image information and the original image information, the position of the revision information on the original image information At least one of the with identifying the image building blocks, designated area and determine that the specified image configuration block the revision information indicates based on the Is intended to, if said image construction block is more specific, depending on the proportion of the image building blocks for the attribute value due to the revision information, at least one of the identified plurality of images constituting the block and determining designated area determining means for one and the instruction region, the the process target block selection means for one or more selected the image building blocks to be processed based on the instruction region, the selected images building block wherein according to the respective contents, and the the processing instruction selection means for selecting a processing instruction in accordance with the relative position of the selected image configuration block and the revision information, the selected processing instruction and the revision information and Image processing means for processing original document image information.

The invention according to claim 2, in the image processing apparatus according to claim 1, wherein the instruction area determining means, the instruction of the highest percentage the image building blocks occupied for the attribute value due to the revision information It is determined as an area.

The invention according to claim 3, in the image processing apparatus according to claim 1, wherein the instruction area determining means, the ratio for the attribute value due to the revision information is higher than a predetermined threshold value the image The constituent block is determined as the indicated area.

According to a fourth aspect of the present invention, there is provided a program for generating document image information for a document on which original document image information and link information indicating a storage position of the original document image information are printed; An original document information acquisition function for acquiring the original document image information in accordance with the link information of the document image information, an image content analysis function for analyzing an image configuration block defining an image configuration of the original document image information, and the document image a revision information obtaining function for obtaining the revision information by taking the difference between the information and the original document image information, together with identifying at least one of the image building blocks based on the position of the revision information on the original image information , which the revision information the specified image building blocks is determined that instruction area for instructing said image building blocks If it is a specific number, depending on the proportion of the image building blocks for the attribute value due to the revision information, instruction determines that at least one of the instruction area of the identified plurality of images constituting the block the contents of each of the region determination function, the process target block selection function for one or more selected the image building blocks to be processed on the basis of the instruction region, and the revision information and the selected images building blocks, and relative to the processing instruction selection function of selecting a processing instruction in accordance with the position, the image processing function for processing the original document image information according to the selected processing instruction and the selected image configuration block and the revision information And make the computer execute.

According to a fifth aspect of the present invention, in the program according to the fourth aspect , the instruction area determination function sets the image composition block having the highest ratio to the attribute value caused by the additional information as the instruction area. judge.

According to a sixth aspect of the present invention, in the program according to the fourth aspect of the invention, the instruction area determination function is configured such that the proportion of the attribute value attributed to the attribute information is higher than a predetermined threshold value. Are determined as the indicated area.

According to a seventh aspect of the present invention, there is provided an image processing method for generating document image information for a document on which original document image information and link information indicating a storage position of the original document image information are printed. An original document information acquisition step for acquiring the original document image information according to the link information of the document image information, an image content analysis step for analyzing an image configuration block that defines an image configuration of the original document image information, identification and revision information acquisition step of acquiring revision information by taking the difference between the original image information with the original document image information, at least one of the image building blocks based on the position of the revision information on the original image information as well as, which the revision information the specified image building blocks is determined that instruction area for instructing said image building blocks If it is more specific, depending on the proportion of the image building blocks for the attribute value due to the revision information, instruction determines that at least one of the instruction area of the identified plurality of images constituting the block area determination process and the contents of each of said the process target block selection step of the to one or more selected images building blocks to be processed based on the instruction region, the revision information and the selected images building blocks, and image processing step of processing the original document image information and processing instruction selection step of selecting a processing instruction in accordance with the relative position between said selected image construction block and the revision information according to the selected processing instruction And have.

According to an eighth aspect of the present invention, in the image processing method according to the seventh aspect , in the instruction area determination step, the instruction component determination unit determines the image composition block having the highest ratio to the attribute value caused by the additional information. It is determined as an area.

The invention according to claim 9 is the image processing method according to claim 7 , wherein the instruction area determination step has a ratio that occupies with respect to the attribute value caused by the writing information is higher than a predetermined threshold value. The constituent block is determined as the indicated area.

According to the inventions according to claims 1, 4 , and 7 , at least one image constituent block is specified based on the position of the additional information on the document image information, and the additional information indicates the specified image constituent block In the case where a plurality of image constituent blocks are specified, at least one of the specified plurality of image constituent blocks is determined according to the ratio of the image constituent block to the attribute value caused by the writing information. One of the image composition blocks (characters, morphemes, sentences, paragraphs, figures, photos, etc.) to be processed based on the designated area and the contents of each of the additional information , and the relative processing instruction in accordance with the position of the selected image configuration block and revision information ( "delete", "highlight", "expansion", etc.) is selected, By processing the original document image information according to the selected processing instruction, when the original document image information is calibrated on the paper document, the additional information does not correctly indicate the image configuration block to be processed. In addition, by selecting the image composition block to be processed and the processing instruction according to the relative positions of the image composition block to be processed and the writing information, the user can use the editing information on the paper manuscript as a command. Can be reflected in the original document image information, so that the convenience of adding or deleting information to the original document image information can be improved . According to the first, fourth, and seventh aspects of the present invention, when the rewriting information with respect to the document image information is not accurately drawn or when the user who is not familiar with the handwriting input means such as the stylus pen inputs the rewriting information Even so, since an image constituent block that is highly likely to be processed is selected, the convenience of adding or deleting information with respect to the original document image information can be improved.

  Exemplary embodiments of an image processing apparatus, a program, and an image processing method according to the present invention are explained in detail below with reference to the accompanying drawings.

  An embodiment of the present invention will be described with reference to FIGS. In this embodiment, as an image processing apparatus, a copy function, a facsimile (FAX) function, a print function, a scanner function, an input image (a document image read by a scanner function, an image input by a printer or a FAX function), etc. This is an example in which a so-called MFP (Multi Function Peripheral) color digital multi-function peripheral is applied.

  FIG. 1 is a block diagram schematically showing a color digital multi-function peripheral 1000 according to an embodiment of the present invention. A color digital multi-function peripheral 1000 according to the present embodiment includes a finisher 100 that is a post-processing device, a scanner unit 200 that is an image reading device, and a printer unit 300 that is an image printing device.

  The color digital multi-function peripheral 1000 according to the present embodiment can sequentially select and select a document box function, a copy function, a printer function, and a facsimile function by using an application switching key of the operation unit 400 (see FIG. 2). It has become. The document box mode is selected when the document box function is selected, the copy mode is selected when the copy function is selected, the printer mode is selected when the printer function is selected, and the facsimile mode is selected when the facsimile mode is selected. Here, an example of the flow of image formation in the copy mode will be described with reference to FIG.

  First, the scanner unit 200 of the color digital multi-function peripheral 1000 will be described. The scanner unit 200 generally includes an automatic document feeder (hereinafter referred to as ADF (Auto Document Feeder)) 1 and a reading unit 50.

  A document bundle placed on the document table 2 of the ADF 1 with the image surface of the document facing up, when a print key (not shown) on the operation unit 400 is pressed, starts with the feed roller 3 from the bottom document. The sheet is fed to a predetermined position on the contact glass 6 by the feeding belt 4. The color digital multi-function peripheral 1000 has a counting function that counts up the number of documents every time a document is fed to a predetermined position on the contact glass 6.

  A document fed to a predetermined position on the contact glass 6 is read by the reading unit 50 as image data.

  Here, the reading unit 50 will be described in detail. The reading unit 50 includes a contact glass 6 on which an original is placed and an optical scanning system. The optical scanning system includes an exposure lamp 51, which is an illumination means, a first mirror 52, a lens 53, a CCD image sensor 54, and the like. The exposure lamp 51 and the first mirror 52 are fixed on a first carriage (not shown), and the second mirror 55 and the third mirror 56 are fixed on a first carriage (not shown). This optical scanning system is driven by a scanner drive motor. The reading unit 50 of the present embodiment turns on the exposure lamp 51 when a document is mounted on the contact glass 6, and moves and scans the first carriage and the second carriage, which are traveling bodies, to the right by the scanner drive motor. A reading method for reading a document and a reading method for reading a document conveyed by the ADF 1 can be selected while the exposure lamp 51 is turned on and the first carriage and the second carriage are stopped. In the reading method in which the first carriage and the second carriage are moved and scanned in the right direction by the scanner driving motor to read the original, when reading the original image, the first carriage and the second carriage are not changed so that the optical path length does not change. Are mechanically scanned in the sub-scanning direction at a relative speed of 2: 1. The document image is read by the CCD image sensor 54, converted into an electrical signal, and output. An output signal from the CCD image sensor 54 is converted into digital data (image data) by an AD converter.

  The document image information converted into digital data is sent to, for example, the printer unit 300 to output the image information as a print output, or sent to and stored in a storage device (for example, the HDD 1118 shown in FIG. 2). The information is used as information of each scanner unit 200. The document image information stored in the storage device (for example, the HDD 1118 shown in FIG. 2) can be re-outputted in the document box mode. According to such a document box function, work efficiency can be greatly improved and paperless can be realized.

  The document whose image data has been read by the reading unit 50 is discharged by the feeding belt 4 and the discharge roller 5.

  Further, when it is detected by the document set detection 7 that the next document is present on the document table 2, the next document is fed onto the contact glass 6 in the same manner as the previous document.

  The feeding roller 3, the feeding belt 4, and the discharging roller 5 described above are each driven by a conveyance motor.

  Next, the printer unit 300 of the color digital multifunction peripheral 1000 will be described. The printer unit 300 generally includes an image forming station 70, a fixing unit 17, a paper feeding unit 80, and a double-sided paper feeding unit 111.

  The image forming station 70 forms an image by an electrophotographic method, and includes a writing unit 57, four drum-shaped photosensitive members 15 of a tandem method, and cyan (C) and magenta provided for each photosensitive member 15. The developing unit 19 corresponding to each color of (M), yellow (Y), and black (K) and the image formed on each photoconductor 15 are sequentially transferred onto the transfer paper P at each transfer position. A belt-like conveyance belt 16 that conveys a sheet is used as a main component.

  The paper feed unit 80 includes a first tray 8, a second tray 9, a third tray 10, a first paper feeder 11, a second paper feeder 12, a third paper feeder 13, and a vertical transport unit 14. ing. The transfer sheets P stacked on the first tray 8, the second tray 9, and the third tray 10 are fed by the first sheet feeder 11, the second sheet feeder 12, and the third sheet feeder 13, respectively. The sheet is conveyed by the conveyance unit 14 to a position where it abuts on the photoconductor 15 located upstream in the conveyance direction.

  The image data read by the reading unit 50 is written on each photoconductor 15 by a laser beam output from the writing unit 57, and a toner image is formed by passing through each developing unit 19. The writing unit 57 is composed of a laser output unit 58, an imaging lens 59, and a mirror 60. Inside the laser output unit 58, a polygon mirror (polygon) that rotates at a constant high speed by a laser diode and a motor as a laser light source. Mirror). Although not particularly illustrated, a beam sensor that generates a main scanning synchronization signal is disposed at a position where a laser beam is irradiated near one end of each photoconductor 15.

  Each toner image on each photoconductor 15 is formed on the transfer paper P as a color image by being transferred onto the transfer paper P conveyed by the conveyance belt 16 at the same speed as the rotation of each photoconductor 15. . Thereafter, the transfer paper P that has been transported to the fixing unit 17 and on which the image has been fixed is discharged by the paper discharge unit 18 to the finisher 100 that is a post-processing device.

  The finisher 100 of the post-processing apparatus can guide the transfer paper P conveyed by the paper discharge roller 18a of the paper discharge unit 18 by switching between the normal paper discharge roller 102 direction and the staple processing unit direction. More specifically, the finisher 100 as a post-processing device can discharge the transfer paper P to the normal paper discharge tray 104 side via the transport roller 103 by switching the switching plate 101 upward. By switching the switching plate 101 downward, the transfer paper P can be conveyed to the staple table 108 via the conveying rollers 105 and 107.

  The transfer paper P loaded on the staple table 108 is aligned by the paper jogger 109 each time it is discharged, and is bound by the stapler 106 upon completion of a partial copy. The group of transfer sheets P bound by the stapler 106 is stored in the staple completion discharge tray 110 by its own weight.

  On the other hand, the normal paper discharge tray 104 of the finisher 100 is a paper discharge tray that can move back and forth. The paper discharge tray 104 that can be moved back and forth moves forward and backward for each original document or for each copy section sorted by the image memory, and sorts copy paper that is simply discharged.

  The color digital multifunction peripheral 1000 according to the present embodiment can form images on both sides of the transfer paper P. When images are formed on both sides of the transfer paper P, the path of the paper discharge unit 18 is switched without guiding the transfer paper P fed from each of the paper feed trays 8 to 10 to the paper discharge tray 104 side. By setting the branching claw 112 for the upper side to the upper side, the paper is once stocked in the duplex feeding unit 111. Thereafter, the transfer paper P stocked on the double-sided paper feeding unit 111 is re-fed from the double-sided paper feeding unit 111 in a reversed state in order to transfer the toner image formed on the photosensitive member 15 again, The paper is guided to the paper discharge tray 104 via the branching claw 112 set on the side. As described above, the duplex feeding unit 111 is used when images are created on both sides of the transfer sheet P. Further, the front and back sides of the transfer paper P can be changed using the double-sided paper feeding unit 111 when printing on the back surface of the transfer paper P on which an image is placed.

  FIG. 2 is a block diagram illustrating a hardware configuration of the digital multi-function peripheral 1000. As shown in FIG. 2, the digital multi-function peripheral 1000 has a configuration in which a controller 1101, a printer unit 300, and a scanner unit 200 are connected by a PCI (Peripheral Component Interconnect) bus. A controller 1101 is a controller that controls the entire digital multi-function peripheral 1000 and controls drawing, communication, and input from the operation display unit 400. The printer unit 300 or the scanner unit 200 includes image processing parts such as error diffusion and gamma conversion.

  The operation display unit 400 displays the document image information of the document read by the scanner unit 200, and the touch panel 400a is brought into contact with the tip of the pen to perform the same operation as pressing the mouse button. It has a stylus pen 400c as handwriting input means that can be performed, and a keyboard 400b that accepts a user's character input or editing operation by keystroke. If the display means for displaying the document image information or the like is not a touch panel, the handwriting can be input by operating the operation means such as a mouse to move the cursor on the screen. Further, the touch panel 400a may be directly operated by a user instruction or the like without using the stylus pen 400c.

  The controller 1101 includes a CPU (Central Processing Unit) 1111, a system memory (MEM-P) 1112, a north bridge (NB) 1113, a south bridge (SB) 1114, and an ASIC (Application Specific Integrated). Circuit) 1116, local memory (MEM-C) 1117, and hard disk drive (HDD) 1118, and NB 1113 and ASIC 1116 are connected by an AGP (Accelerated Graphics Port) bus 1115. The MEM-P 1112 further includes a ROM (Read Only Memory) 1112a and a RAM (Random Access Memory) 1112b.

  The CPU 1111 performs overall control of the color digital multi-function peripheral 1000, has a chip set including the NB 1113, the MEM-P 1112, and the SB 1114, and is connected to other devices via this chip set.

  The NB 1113 is a bridge for connecting the CPU 1111, the MEM-P 1112, the SB 1114, and the AGP bus 1115, and includes a memory controller that controls read / write to the MEM-P 1112, a PCI master, and an AGP target.

  The MEM-P 1112 is a system memory used as a memory for storing programs and data, a memory for developing programs and data, a memory for drawing a printer, and the like, and includes a ROM 1112a and a RAM 1112b. The ROM 1112a is a read-only memory used as a memory for storing programs and data for controlling the operation of the CPU 1111. The RAM 1112b is a writable and readable memory used as a program and data development memory, a printer drawing memory, and the like. It is.

  The SB 1114 is a bridge for connecting the NB 1113 to a PCI device and peripheral devices. The SB 1114 is connected to the NB 1113 via a PCI bus, and a network interface (I / F) unit 1104 and the like are also connected to the PCI bus.

  The ASIC 1116 is an IC (Integrated Circuit) for image processing applications having hardware elements for image processing, and has a role of a bridge for connecting the AGP bus 1115, the PCI bus, the HDD 1118, and the MEM-C 1117. The ASIC 1116 includes a PCI target and an AGP master, an arbiter (ARB) that forms the core of the ASIC 1116, a memory controller that controls the MEM-C 1117, and a plurality of DMACs (Direct Memory) that perform rotation of image data by hardware logic and the like. Access Controller) and a PCI unit that performs data transfer between the printer unit 300 and the scanner unit 200 via the PCI bus. The ASIC 1116 is connected to an FCU (Fax Control Unit) 1121, a USB (Universal Serial Bus) 1122, and an IEEE 1394 (the Institute of Electrical and Electronics Engineers 1394) interface 1123 via a PCI bus.

  A MEM-C 1117 is a local memory used as an image buffer for copying and a code buffer, and an HDD 1118 is a storage for storing image data, storing programs for controlling the operation of the CPU 1111, storing font data, and storing forms. It is.

  The AGP bus 1115 is a bus interface for a graphics accelerator card proposed for speeding up graphics processing. The AGP bus 1115 speeds up the graphics accelerator card by directly accessing the MEM-P 1112 with high throughput. .

  Next, an image processing process at the time of selecting a document box function which is a characteristic function realized by the controller 1101 of the digital multi-function peripheral 1000 according to the present embodiment according to a program will be described.

  Here, FIG. 3 is a block diagram showing a functional configuration related to the image processing when the document box function is selected, and FIG. 4 is a flowchart showing the flow of the image processing. As illustrated in FIG. 3, the digital multifunction peripheral 1000 includes a document image information generation unit 21, an original document information acquisition unit 22, an image content analysis unit 23, a writing information acquisition unit 24, according to a program executed by the CPU 1111. The processing target block selection unit 25, the processing instruction selection unit 26, the image processing unit 27, and the instruction area determination unit 28 are realized.

  The document image information generating unit 21 generates document image information based on the document read by operating the scanner unit 200 (step S1 shown in FIG. 4). When the document box function is selected, the document image information generated in this way may be stored in the HDD 1118 or may be used for retouching the document image information stored in the HDD 1118. Here, it is assumed that the document image information has been generated in order to modify the document image information stored in the HDD 1118.

  Here, FIG. 5 shows an example of a document used for rewriting correction of document image information stored in the HDD 1118. As shown in FIG. 5, the document D is entirely formed in a sheet shape, and for example, an A4 size document is used. On these originals D, various original image information stored in the HDD 1118 of the digital multi-function peripheral 1000 is printed in advance by the printer unit 300 in units of one screen. Link information indicating that it is document image information in the HDD 1118 of the machine 1000 is printed in the form of a two-dimensional code C.

  The link information (two-dimensional code C) printed on the document D will be described. The two-dimensional code C indicates that the original image information printed on the original D is the original image information in the HDD 1118 as described above. For example, there are a document name, a page number, a device name, its IP address, etc., and the document has a directory structure such as “¥¥ device name (IP address of device) ¥ drive name ¥ folder name ¥ document name ¥ page”. It is sufficient that the location (information accumulation location) stored (minimum is for one page) can be uniquely identified.

  The code symbol for storing the link information may be a one-dimensional barcode or the like, but it is necessary to classify a wide variety of documents for the document ID, and the number of classifications is insufficient for the one-dimensional barcode or the like. In view of this, it is desirable to use a two-dimensional code C having a remarkably large capacity as in the present embodiment.

  The original document information acquisition unit 22 acquires original document image information (original document image information) from the HDD 1118 according to the link information given to the document image information generated by the document image information generation unit 21 (see FIG. 4). Step S2). More specifically, the original document information obtaining unit 22 obtains link information by decoding the two-dimensional code C given to a predetermined position of the read document image information, and the original document information is obtained according to the obtained link information. Original image information is acquired from the HDD 1118.

  The image content analysis unit 23 analyzes the content of the original document image information acquired by the original document information acquisition unit 22 (step S3 shown in FIG. 4). More specifically, by performing an area extraction process of the original document image information acquired by the original document information acquisition unit 22, the hierarchical structure of the image configuration block that defines the image configuration of the original document image information is obtained. Here, the image configuration block is a character, morpheme, sentence, paragraph, figure, photograph, or the like on the document (document image information) D. The area extraction processing is performed after various image analysis values such as OCR processing for performing optical character recognition on original document image information are executed to calculate various image characteristic values representing image attributes of the document image. The image constituent block is extracted based on the various image characteristic values. FIG. 6 shows an example of extraction of image constituent blocks in the original document image information shown in FIG. Note that the content of the original document image information (image configuration block) may be included in the original document image information as meta information.

  The rewriting information acquisition means 24 takes the difference between the original image information generated by the original image information generation means 21 and the original original image information acquired by the original original information acquisition means 22 and uses this difference as additional information ( Step S4) shown in FIG. The additional information in the present embodiment is a figure, a line segment, a closed curve, or the like represented by “◯” or “×”. Although details will be described later, these figures, line segments, closed curves, and the like become commands for correcting correction of document image information stored in the HDD 1118.

  The instruction area determination unit 28 specifies an image constituent block based on the position on the original image information of the additional information such as the line segment and the closed curve acquired by the additional information acquisition unit 24, and the length of the acquired line segment. An image composition block having a high ratio of the various attribute values resulting from the writing information, such as the area of the region surrounded by the closed curve, is determined as a region indicated by the writing information (hereinafter referred to as an instruction region) (FIG. 4). Step S5).

Here, when the retouching information acquired by the retouching information acquisition unit 24 is a line segment, the instruction area determination unit 28 specifies an image constituent block that is close to or intersects with the line segment, and the length of the line segment. An image composing block that occupies a high percentage of the image is determined as an instruction area. When the retouching information acquired by the retouching information acquisition unit 24 is a closed curve, the instruction area determination unit 28 includes an image configuration block included in the region surrounded by the closed curve and an image configuration block that intersects the closed curve. The image constituting block that is specified and has a high ratio to the area of the area surrounded by the closed curve is determined as the designated area. In the following, a process for determining an image constituent block specified based on the writing information acquired by the writing information acquisition unit 24 as an instruction area will be described as an example.
(1) The instruction area determination unit 28 specifies an image configuration block based on the position of the writing information acquired by the writing information acquisition unit 24 on the document image information, and determines the specified image configuration block as the instruction area. . For example, when a line segment is acquired as the retouching information by the retouching information acquisition unit 24, the instruction area determination unit 28 specifies an image constituent block that is close to or intersects with the added line segment, and uses the specified image constituent block. It is determined as an instruction area. When the closed curve is acquired as the additional information by the additional information acquisition unit 24, the indication area determination unit 28 selects an image configuration block included in the region surrounded by the additional closed curve or an image configuration block that intersects the closed curve. The identified image constituent block is determined as the designated area.
(2) The instruction area determination unit 28 specifies an image constituent block based on the position of the writing information acquired by the writing information acquisition unit 24 on the document image information, and when a plurality of image constituent blocks are specified, the writing information An image composition block that occupies a high ratio with respect to various attribute values resulting from is determined as an instruction area. Specifically, when a plurality of image constituent blocks are specified based on the position on the original image information of the line segment acquired as the additional information, the image constituent block having the highest ratio to the length of the line segment Are determined to be designated areas. In addition, when a plurality of image constituent blocks are specified based on the position of the closed curve obtained as the additional information on the original image information, the image constituent block having the highest ratio to the area of the area surrounded by the closed curve is indicated. It is determined as an area. Thus, even if the image composition block is not accurately surrounded by the closed curve, or even if the line segment is not accurately drawn under or on the image composition block, The designated area can be accurately determined.
(3) The instruction area determination unit 28 specifies an image configuration block based on the position of the writing information acquired by the writing information acquisition unit 24 on the document image information, and when a plurality of image configuration blocks are specified, the writing information An image constituent block in which the ratio of the various attribute values resulting from is equal to or greater than a predetermined threshold is determined as an instruction area. Specifically, when a plurality of image constituent blocks are specified based on the position on the document image information of the line segment acquired as the writing information, a ratio that occupies the length of the line segment is determined in advance. The above image composition block is determined as the designated area. In addition, when a plurality of image constituent blocks are specified based on the position of the closed curve acquired as the additional information on the document image information, the ratio of the area surrounded by the closed curve is equal to or greater than a predetermined threshold. The image constituent block is determined as the designated area. Thus, even if the image composition block is not accurately surrounded by the closed curve, or even if the line segment is not accurately drawn under or on the image composition block, The designated area can be accurately determined.

  Here, an example of processing in the designated area determination unit 28 will be described.

  Here, FIG. 7 is a front view showing an example of the writing information written in the character line. The example illustrated in FIG. 7A is a diagram illustrating a case where, for example, when the image configuration block is a character line, writing x is performed on a plain portion between the lines. In this case, when the phrase close to the writing x is specified as the image constituent block, the indication area determination unit 28 selects the phrase B1, the phrase B2, the phrase B4, and the phrase B5 based on the position of the writing x on the document image information. Can be identified. However, in the phrase B1, since only the character A1 constituting the phrase B1 is close to the writing x, the ratio of the image constituent block of the phrase B1 to the length of the writing x is low. Therefore, there is a low possibility that the phrase B1 is indicated by the writing x. On the other hand, in the phrase B2, since the characters A2 to A5 are close to the writing x, the ratio of the image constituent block of the phrase B2 to the length of the writing x is high. Therefore, there is a high possibility that the phrase B2 is indicated by the writing x. Therefore, the instruction area determination unit 28 determines that the image constituent block of the phrase B2 is the instruction area. Further, when there is a writing x between the lines, the writing x may indicate the clause B4 or the clause B5 belonging to the lower row, so the determination is similarly made for the clause 4 and the clause B5. Note that, as in the example shown in FIG. 7B, when the image composition block is a character line, when writing y is performed on the character, only the line where the writing y is written may be indicated. Since it is high, it is not necessary to specify the image constituent block belonging to the lower row. The process in which the instruction area determination unit 28 determines any of the image constituent blocks of the phrase B1 or the phrase B2 as the instruction area is the same as the process described in the example illustrated in FIG.

  The example shown in FIG. 7C is an example when the closed curve z is written. In the example shown in FIG. 7C, the indication area determination unit 28 identifies the image constituent blocks of the phrase B1, the phrase B2, and the phrase B4 that include the area surrounded by the closed curve z or intersects the closed curve x, and the closed curve z An image constituent block that has the highest ratio to the area of the area surrounded by or that is equal to or larger than a predetermined threshold is determined as the designated area.

  The processing target block selecting unit 25 selects an image constituent block to be processed from the image constituent blocks analyzed by the image content analyzing unit 23 based on the determination result of the instruction area determining unit 28 (step S6 shown in FIG. 4). ).

The processing instruction selection means 26 includes the contents of one or a plurality of processing target image configuration blocks selected by the processing target block selection means 25 and the retouching information acquired by the retouching information acquisition means 24, and the image configuration blocks. A processing instruction is selected according to the relative position of the writing information and the writing information (step S6 shown in FIG. 4). In the following, the selection of processing instructions according to the contents of the image composition block and the additional information and the relative positions of the image constituent block and the additional information will be described as an example.
(1) The processing instruction selection unit 26 selects a processing instruction according to whether or not the additional information intersects with the region of the image constituent block to be processed. Specifically, the image constituent block on the original document image information is deleted or highlighted depending on whether the added line segment, closed curve, or the like intersects with the image constituent block. When a figure (image configuration block) is drawn on a part of the document, and the figure part and a large amount of additional information intersect, the user can predict that this figure is desired to be operated. In addition, since the range of the corresponding figure can be obtained by the image content analysis means 23, the user can select the target figure accurately without drawing so as to surround the target figure accurately. A range can be selected for a character string in a sentence by surrounding the corresponding character string.
(2) The processing instruction selection unit 26 selects a different processing instruction according to the size (for example, the size of “circle”, the thickness of the line, etc.) that is the feature information of the image included in the writing information. Specifically, the enhancement level of the image of the selected image configuration block is changed according to the size of the added “circle” or the like. As a result, the emphasis level of the image composition block can be changed by a simple operation, and when the paper document is digitized, the image composition block is emphasized according to the added content, and the convenience of adding information to the original document image information is improved. .
(3) The processing instruction selection unit 26 selects different processing instructions according to the multiplicity (superimposition of line segments and superimposition of closed curves) that is the feature information of the image included in the writing information. Specifically, the multiplicity of images is a double circle or the like. Specifically, the enhancement level of the image of the selected image configuration block is changed according to the multiplicity (double circle, triple circle, etc.) of the added “circle” or the like. As a result, the emphasis level of the image composition block can be changed by a simple operation, and when the paper document is digitized, the image composition block is emphasized according to the added content, and the convenience of adding information to the original document image information is improved. .
(4) The processing instruction selection unit 26 determines whether the image constituent block to be processed is a character, a photograph, or a figure, and selects a different processing instruction according to the determination result. For example, by changing the processing of added lines and circles according to the image composition block (characters, morphemes, sentences, paragraphs, figures, photos, etc.) to be processed, line segments will not appear on the added lines on the figure. Since different processing instructions can be selected such that deletion is performed on the added line on the character and deletion is performed, the convenience of adding and deleting information to the original document image information is improved according to the addition target.
(5) The processing instruction selection means 26 deletes the original document image information, changes the display method of the original document image information (font enlargement / reduction, highlighting), and enlarges / reduces the image of the original document image information. Select either reduction or trimming.

  The image processing means 27 processes the original document image information in the HDD 1118 acquired based on the link information in accordance with the processing instruction selected by the processing instruction selection means 26, and stores it as the original in the HDD 1118 (step shown in FIG. 4). S7).

  Here, an example of processing in the processing instruction selection unit 26 and the image processing unit 27 will be described.

  Here, FIG. 8 is a front view showing an example of the writing information, and FIG. 9 is a front view showing the processing result. In the example shown in FIG. 8, for example, when the image composition block is a character line, writing is performed on a plain portion between the lines (X in FIG. 8), and when a line is drawn on the character (in FIG. 8). Y). In these cases, the user's intention is different. The former often emphasizes the vicinity and makes it an index target, while the latter often is a deletion target. Accordingly, the processing instruction selection means 26 selects a processing instruction according to whether or not the additional information intersects with the area of the processing target image constituent block. That is, as indicated by X in FIG. 8, when the retouching information does not intersect the region of the image composition block to be processed, the processing instruction selecting unit 26 selects a processing instruction for emphasizing the vicinity of the retouching information. The image processing means 27 highlights the image constituent blocks as indicated by Z in FIG. On the other hand, as indicated by Y in FIG. 8, when the retouching information intersects the region of the image constituent block to be processed, the processing instruction selection means 26 selects the deletion processing instruction, and the image processing means 27 As shown in FIG. 9, the image constituent block instructed to delete is deleted.

  In other words, in the present embodiment, even if the added locus is drawn in an ambiguous manner on the document (original image information) D, the positional relationship between the added locus and the image constituent block is analyzed. As a result, the designated area to be edited can be accurately determined. In addition, by enclosing a lot of the periphery of the chart or itself, it is possible to feel the intention to select the chart. For this reason, the intention of the processing method of the writing information can be inferred depending on which image configuration block on the original (original image information) D the handwritten additional information passes.

  As described above, according to the present embodiment, the instruction area indicated by the additional information on the document image information is determined based on the image configuration block, and becomes one or a plurality of processing targets selected based on the instruction area. Processing instructions ("Delete", "Highlight") according to the content of each image composition block (character, morpheme, sentence, paragraph, figure, photo, etc.) and the relative position of the image composition block and the additional information , “Enlarge”, etc.) and the original document image information is processed according to the selected processing instruction. As a result, when the original document image information is calibrated on a paper document, even if the retouching information does not correctly indicate the image composing block to be processed, The user's intention can be reflected in the original manuscript image information by using the information added to the paper manuscript as a command by selecting the image constituent block to be processed and the processing instruction according to the relative position of the manuscript. .

  Next, image processing at the time of selecting a document box function, which is a characteristic function realized by the controller 1101 of the digital multi-function peripheral 1000 according to the present embodiment according to a program, the touch panel 400a and the stylus that the operation display unit 400 has. A description will be given of a process in which retouching information is input using the pen 400c and image processing is performed on the document image information using the input retouching information.

  Here, FIG. 10 is a block diagram showing a functional configuration related to the image processing when the document box function is selected, and FIG. 11 is a flowchart showing the flow of the image processing. As shown in FIG. 10, the digital multi-function peripheral 1000 includes an image content analysis unit 23, a writing information acquisition unit 24, a processing target block selection unit 25, a processing instruction selection unit 26, an image, and the like according to a program executed by the CPU 1111. A processing unit 27, an instruction area determination unit 28, a document image information acquisition unit 29, and a writing information input unit 29 are realized. The specific processing realized by the image content analysis unit 23, the processing target block selection unit 25, the processing instruction selection unit 26, the image processing unit 27, and the instruction area determination unit 28 is the same as that of the above-described embodiment. .

  The document image information acquisition unit 28 acquires document image information generated based on a document read by operating the scanner unit 200 or document image information stored in advance in the HDD 1118 by selecting a document box function (FIG. 11). Step S9). The document image information obtained in this way is stored in the HDD 1118 when the document box function is selected, and is used to correct the document image information generated by the scanner unit 200 or the document image information stored in the HDD 1118. It may be used. Here, it is assumed that the document image information has been acquired for the correction of the document image information generated based on the document read by the scanner unit 200 or the document image information stored in the HDD 1118. The document image information is displayed on the touch panel 400a. FIG. 12 is an example of document image information acquired by the document image information acquisition unit 21 and displayed on the touch panel 400a. The user can input handwriting by touching the touch panel 400a with the stylus pen 400c. The document image information acquisition unit 21 can also acquire document image information from the HDD 1118 according to link information such as a two-dimensional barcode attached to a document read by operating the scanner unit 200.

  The image content analysis unit 23 analyzes the content of the document image information acquired by the document image information acquisition unit 28 (step S10 shown in FIG. 11). Specific processing is the same as that in the above-described embodiment.

  The retouching information acquisition means 24 acquires retouching information for the original image information input by the user touching with the stylus pen 400c on the touch panel 400a on which the original image information is displayed (step S11 shown in FIG. 11). The additional information in the present embodiment is a figure represented by “◯” or “×”. Although details will be described later, this figure becomes a command for rewriting correction of the document image information stored in the HDD 1118.

  The instruction area determination unit 28 specifies an image constituent block based on the position on the original image information of the additional information such as the line segment and the closed curve acquired by the additional information acquisition unit 24, and the length of the acquired line segment. An image composition block having a high ratio of the various attribute values resulting from the writing information, such as the area of the region surrounded by the closed curve, is determined as the designated region indicated by the writing information (step S12 shown in FIG. 11).

  The processing target block selecting unit 25 selects an image constituent block to be processed from the image constituent blocks analyzed by the image content analyzing unit 23 based on the determination result of the instruction area determining unit 28 (step S13 shown in FIG. 11). ).

  The processing instruction selection means 26 includes the contents of one or a plurality of processing target image configuration blocks selected by the processing target block selection means 25 and the retouching information acquired by the retouching information acquisition means 24, and the image configuration blocks. A processing instruction is selected according to the relative position of the writing information and the writing information (step S14 shown in FIG. 11).

  The image processing means 27 processes the document image information in the HDD 1118 acquired by the document image information acquisition means 28 in accordance with the processing instruction selected by the processing instruction selection means 26, and stores it in the HDD 1118 (step S15 shown in FIG. 11). ).

  In other words, in the present embodiment, even if the rewritten trajectory is drawn ambiguously on the document image information, the positional relationship between the rewritten trajectory and the image constituent block to be processed is analyzed. As a result, the image constituent block to be edited can be accurately selected. In addition, by enclosing a lot of the periphery of the chart or itself, it is possible to feel the intention to select the chart. For this reason, the intention of the processing method of the writing information can be estimated depending on which image configuration block on the document image information the handwritten writing information passes.

  As described above, according to the present embodiment, when input information on original image information is input by handwriting input means such as a stylus pen and the original image information is calibrated, the additional information indicates on the original image information. The instruction area is determined based on the image composition block, and each of the image composition blocks (characters, morphemes, sentences, paragraphs, figures, photographs, etc.) to be processed based on the instruction area is selected. Select processing instructions (“Delete”, “Highlight”, “Enlarge”, etc.) according to the content and the relative position of the image composition block and additional information, and process the document image information according to the selected processing instructions To do. As a result, even if a user who has not used handwriting input means such as a stylus pen cannot correctly specify the image composition block to be processed, the relative processing between the image composition block to be processed and the additional information is performed. The user's intention can be reflected in the manuscript image information by using the input information as a command by selecting the image constituent block to be processed and the processing instruction according to the position.

  In the present embodiment, the document image information read by the scanner unit 200 provided in the color digital multifunction peripheral 1000 or the document image information in the HDD 1118 is processed. However, the present invention is not limited to this. Document image information in a storage unit such as an HDD provided in an external device connected to the digital multi-function peripheral 1000 via the network I / F unit 1104, USB 1122, IEEE 1394 interface 1123, etc. can be processed.

  Note that the program executed in the color digital multifunction peripheral 1000 of the present embodiment is an installable or executable file in a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). ) Or the like may be recorded and provided on a computer-readable recording medium.

  Further, the program executed in the color digital multifunction peripheral 1000 according to the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the program executed by the color digital multifunction peripheral 1000 according to the present embodiment may be provided or distributed via a network such as the Internet.

  Furthermore, in the present embodiment, the color digital multifunction peripheral 1000 called MFP is applied as the image processing apparatus, but the present invention is not limited to this. For example, as shown in FIG. 13, a printer device 1202 that is an image printing device and a scanner device 1204 that is an image reading device are connected to a personal computer 1201 via a cable 1203, and a personal computer 1201 that is an image processing device is connected. By installing a predetermined program in a storage device (not shown) such as an HDD and operating a CPU (not shown) of the personal computer 1201 according to this program, the same effects as the above-described various effects can be obtained. An effect can be obtained.

1 is a configuration diagram schematically illustrating a digital multifunction peripheral according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a hardware configuration of a digital multifunction peripheral. FIG. It is a block diagram which shows the function structure concerning an image process. It is a flowchart which shows the flow of an image process. It is a front view which shows an example of the original document used for correction correction of original image information. It is a schematic diagram which shows the example of extraction of an image structure block. It is a front view which shows an example of the additional information written in the character line. It is a front view which shows an example of writing information. It is a front view which shows a process result. It is a block diagram which shows the function structure concerning an image process. It is a flowchart which shows the flow of an image process. It is a front view which shows an example of the original image information used for correction correction. 1 is a block diagram illustrating a schematic configuration example of an image processing system.

Explanation of symbols

21 Document image information generation means 22 Original document information acquisition means 23 Image content analysis means 24 Writing information acquisition means 25 Processing target block selection means 26 Processing instruction selection means 27 Image processing means 28 Instruction area determination means 29 Document image information acquisition means 1000, 1201 Image processing apparatus C Link information D Document

Claims (9)

Document image information generating means for generating document image information for a document on which original document image information and link information indicating a storage position of the original document image information are printed;
Original document information acquisition means for acquiring the original document image information according to the link information of the document image information;
Image content analysis means for analyzing an image configuration block defining an image configuration of the original document image information;
Rewriting information acquisition means for acquiring rewriting information by taking a difference between the original image information and the original original image information;
Wherein with identifying at least one of the image building blocks based on the position of the revision information on original image information, which the revision information the specified image building blocks is determined that instruction area for instructing said image When a plurality of constituent blocks are specified, at least one of the specified plurality of image constituent blocks is defined as the instruction area according to a ratio of the image constituent blocks to the attribute value caused by the additional information. Instruction area determination means for determining;
Processing target block selecting means for selecting one or a plurality of the image constituent blocks to be processed based on the designated area;
And their contents, and processing instruction selection means for selecting a processing instruction in accordance with the relative position between said selected image construction block and the revision information of the revision information and the selected images building blocks,
An image processing means for processing the original document image information according to the selected processing instruction,
An image processing apparatus comprising:   The indication area determination means includes
  Determining the image configuration block having the highest ratio of the attribute value attributed to the writing information as the designated area;
The image processing apparatus according to claim 1.   The indication area determination means includes
  Determining the image configuration block that is higher than a predetermined threshold as a percentage of the attribute value attributed to the writing information as the designated area;
The image processing apparatus according to claim 1.   A document image information generation function for generating document image information for a document on which original document image information and link information indicating the storage position of the original document image information are printed;
  An original document information acquisition function for acquiring the original document image information according to the link information of the document image information;
  An image content analysis function for analyzing an image configuration block defining an image configuration of the original document image information;
  A rewriting information acquisition function for acquiring rewriting information by taking a difference between the original image information and the original original image information;
  The at least one image constituent block is specified based on the position of the additional information on the original image information, and the specified image constituent block is determined as an instruction area indicated by the additional information. When a plurality of constituent blocks are specified, at least one of the specified plurality of image constituent blocks is defined as the instruction area according to a ratio of the image constituent blocks to the attribute value caused by the additional information. An instruction area determination function for determining;
  A processing target block selection function for selecting one or a plurality of the image constituent blocks to be processed based on the designated area;
  A processing instruction selection function for selecting a processing instruction according to the contents of each of the selected image composition block and the retouching information and a relative position between the selected image composition block and the retouching information;
  An image processing function for processing the original document image information in accordance with the selected processing instruction;
A program that causes a computer to execute.   The indication area determination function is:
  Determining the image configuration block having the highest ratio of the attribute value attributed to the writing information as the designated area;
The program according to claim 4.   The indication area determination function is:
  Determining the image configuration block that is higher than a predetermined threshold as a percentage of the attribute value attributed to the writing information as the designated area;
The program according to claim 4.   A document image information generation step of generating document image information for a document on which original document image information and link information indicating a storage position of the original document image information are printed;
  An original document information acquisition step of acquiring the original document image information according to the link information of the document image information;
  An image content analysis step of analyzing an image configuration block defining an image configuration of the original document image information;
  A rewriting information acquisition step of acquiring rewriting information by taking a difference between the original image information and the original original image information;
  The at least one image constituent block is specified based on the position of the additional information on the original image information, and the specified image constituent block is determined as an instruction area indicated by the additional information. When a plurality of constituent blocks are specified, at least one of the specified plurality of image constituent blocks is defined as the instruction area according to a ratio of the image constituent blocks to the attribute value caused by the additional information. An instruction area determination step for determining;
  A processing target block selection step of selecting one or a plurality of the image constituent blocks to be processed based on the designated area;
  A process instruction selection step of selecting a process instruction according to the contents of each of the selected image composition block and the retouching information, and a relative position between the selected image composition block and the retouching information;
  An image processing step of processing the original document image information in accordance with the selected processing instruction;
An image processing method comprising:   The indicated area determination step includes
  Determining the image configuration block having the highest ratio of the attribute value attributed to the writing information as the designated area;
The image processing method according to claim 7.   The indicated area determination step includes
  Determining the image configuration block that is higher than a predetermined threshold as a percentage of the attribute value attributed to the writing information as the designated area;
The image processing method according to claim 7.

Images