JP2006261772A - Image forming apparatus and image forming method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image forming apparatus capable of sharing correction of the original document by converting the handwritten correction added to a paper document to electronic information, and an image forming method. <P>SOLUTION: The image forming apparatus includes an image input means for inputting an image, an original information acquisition means for acquiring original electronic information made to relate to the image, a vector data conversion means for converting the image to vector data, and a difference extraction means for preparing differential vector information from the original electronic information and the image. The extracted differential vector information is registered after making it relate to the original electronic data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image forming apparatus and an image forming method in which an image corrected by handwriting or image synthesis is compared with original image data, and the correction is vectorized and registered in a server.

  In recent years, an electronic document is often created using a document creation application on a word processor or a personal computer. As a method for sending this electronic document to another person, the content of the document is printed on a sheet by a printer and handed over to the other party in the same manner as a conventional circulation sheet. Document information is stored in a floppy (registered trademark). There is a method of recording in a storage medium such as a disk or further transmitting through a network.

  The recipient shares the information through a process such as printing the document information read from the storage medium or sent via the network on a sheet of paper or displaying it on an electronic display. . The recipient uses a document utilization cycle by storing it as knowledge for the future, processing it into new document information, or answering the sender with new document information.

  In order to correct the received document, it is conceivable to add the handwritten information directly to the printed paper document, or to edit the original document and send it by e-mail.

  However, since the document to be viewed is expressed by electronic means other than directly overwriting the printed matter on which the document is printed, the problem is that the visibility is poor compared to the printed matter on paper and the visual interface characteristics to the user are poor. There is. In a meeting or the like, the distribution document is often printed paper.

For this reason, a document processing device for reading a written document based on information on paper with good visibility, adding information by handwriting, transmitting it, and performing document editing / correction, etc. An invention as disclosed in Patent Document 1 has been made to provide a method.
Japanese Patent No. 3454273

  However, although Patent Document 1 shows a difference method in which an original image and a handwritten image are compared in bit units, it cannot cope with enlargement / reduction of the original image or layout change. Further, in Patent Document 1, handwriting only specifies a correction instruction, and important handwritten correction contents cannot be shared as data. In addition, contents modified by a plurality of people cannot be merged.

  The present invention was devised to solve such a conventional problem, and an image forming apparatus capable of converting a handwritten correction added to a paper document into electronic information and sharing the correction of the original document. An object is to provide an image forming method.

  In order to solve the above-described problems, in the present invention, first, an image input means for inputting an image, an original information acquisition means for acquiring original electronic information associated with the image, and a vector for converting the image into vector data An image forming apparatus including data conversion means and difference extraction means for creating difference vector information from original electronic information and an image, wherein the extracted difference vector information is registered in association with the original electronic data. Configure.

  Second, in the first image forming apparatus, the image input means for inputting an image is a document reading means for optically reading a document to obtain image information.

  Third, the first or second image forming apparatus includes an image output unit, and further outputs a difference to the output image when the difference vector information associated with the original electronic data is combined with the original electronic data and output. The vector information and the original electronic data are associated with each other.

  Fourth, an image input step for inputting an image, an original information acquisition step for acquiring original electronic information associated with the image, a vector data conversion step for converting the image into vector data, a difference from the original electronic information and the image In an image forming method including a difference extraction step of creating vector information, the extracted difference vector information is registered in association with original electronic data.

  Fifth, according to a fourth image forming method, the image input step for inputting an image is a document reading step for optically reading a document to obtain image information.

  Sixth, in the fourth or fifth image forming method, an image output step is included, and when the difference vector information associated with the original electronic data is combined with the original electronic data and output, the difference is output to the output image. The vector information and the original electronic data are associated with each other.

  According to the first or fourth configuration of the present invention, it is possible to acquire an original document from an input image and register the difference in the server. This makes it easy to link handwritten corrections with the original document.

  According to the second or fifth configuration of the present invention, the user inputs an image using a scanner for reading an original, and can easily input an image that is directly handwritten on paper.

  According to the third or sixth configuration of the present invention, the user can output the registered corrected image in various formats, and can share the correction by handwriting.

  The best mode for carrying out the invention will be described in detail with reference to examples.

  Next, a first embodiment of an image processing method according to the present invention will be described with reference to the drawings.

[Image processing system]
In FIG. 1, an image processing system that implements the first embodiment of the image processing method according to the present invention is used in an environment in which an office 10 and an office 20 are connected via the Internet 104.

  A LAN 107 constructed in the office 10 includes an MFP (multifunction multifunction peripheral) 100 as a recording device, a management PC 101 that controls the MFP 100, a client PC (external storage means) 102, a document management server 106, and a document management server 106. A database 105 is connected.

  A LAN 108 is constructed in the office 20, and a document management server 106 and a database 105 for the document management server 106 are connected to the LAN 108.

  A proxy server 103 is connected to the LANs 107 and 108, and the LANs 107 and 108 are connected to the Internet via the proxy server 103.

  The MFP 100 is in charge of part of the image processing for the input image read from the document, and the image data as the processing result is input to the management PC 101 via the LAN 109. The management PC 101 is a computer including an image storage unit, an image processing unit, a display unit, an input unit, and the like. Functionally, some of these components are integrated with the MFP 100 and become components of the image processing system. Yes. In this embodiment, the management PC 101 executes the search process described below, but all the processes performed by the management PC 101 may be executed by the MFP.

  Further, the MFP 100 is directly connected to the management PC 101 via the LAN 109.

[MFP configuration]
In FIG. 2, the MFP 100 includes an image reading unit 110 having an auto document feeder (ADF). The image reading unit 110 irradiates an image of a bundle or one original with a light source, and forms a reflected image on a solid-state image sensor with a lens. The solid-state imaging device generates an image reading signal having a predetermined resolution (for example, 600 dpi) and a predetermined density level (for example, 8 bits), and image data including raster data is configured from the image reading signal.

  The MFP 100 includes a storage device 111 and a recording device 112 having a printing function. When executing the copying function, the image data output from the image reading unit 110 is subjected to image processing by the data processing device 115 and converted into a recording signal, which is output to the recording device 112. In the case of copying a plurality of sheets, a recording signal for one page is temporarily stored in the storage device 111 and then sequentially output to the recording device 112 to form a recorded image on the recording paper.

  The MFP 100 includes a network IF 114 for connection with the LAN 107, and the image data output from the client PC 102 can be recorded by the recording device 112. The image data output from the client PC 102 is converted into a recording signal that can be recorded by the data processing device 115 from the LAN 107 via the network IF 114 and then recorded as a recorded image on the recording paper in the MFP 100.

  The storage device 111 can be directly controlled from the management PC 101 via the network IF 117.

  The MFP 100 includes a display device 116, and the display device 116 can display an operation input state and image data to be processed.

  The MFP 100 is operated through an input device 113 including a key operation unit provided in the MFP 100, a touch panel using the display device 116, or an input device such as a keyboard and a pointing device of the management PC 101. For these operations, the data processing apparatus 115 performs predetermined control by an internal control unit (not shown).

  The LAN 109 is used for data exchange and control signal exchange between the MFP 100 and the management PC 101.

[Overall flow of image processing method]
The first embodiment of the image processing method according to the present invention is executed by each step of FIG.

  In step S <b> 301, the image reading unit 110 of the MFP 100 is operated to scan a single original in a raster shape, and obtain an image reading signal having a predetermined resolution and a predetermined density level. The image reading signal is preprocessed by the data processing unit 115 and stored in the storage device 111 as image data for one page of the input image. Next, the process proceeds to step S302.

  In step S302 (block selection (area division) step), the management PC 101 divides the image data area stored in the storage device 111 into a character / line drawing area including characters or line drawings, a halftone photographic area, and an indefinite shape. Divide into image area and others. Further, with respect to the character / line drawing area, a character area mainly including characters and a line drawing area mainly including tables and figures are separated, and the line drawing area is separated into a table area and a graphic area. In this embodiment, connected pixels are detected and divided into regions for each attribute using the shape, size, pixel density, etc. of the circumscribed rectangular region of the connected pixels, but other region dividing methods are used. It doesn't matter.

  For the character area, the clustered blocks of each character paragraph are segmented into rectangular blocks (character area rectangular blocks), and in the line drawing area, individual objects such as tables and figures (table area rectangular block, line drawing area rectangular block) Segment into rectangular blocks every time.

  The photograph area expressed in halftone is segmented into rectangular blocks for each object such as an image area rectangular block and a background area rectangular block. Information on these rectangular blocks is referred to as “region division information”.

  In step S303, the region division information obtained in the block selection step and the input image are combined and displayed on the operation screen on the display device 116 of the MFP 100.

  An example of the operation screen displayed on the display device 116 is shown in FIG.

  On the operation screen, the input image itself is displayed as an image 41 on the left half, and an image 42 indicating region division information is displayed on the right half by a rectangular block.

  In the image 42, the input image is simply displayed and each rectangular block is indicated by a rectangle.

  In order to facilitate understanding of the concept of the rectangular block, each rectangular block is provided with TEXT, PICTURE, LINE, and TABLE indicating the attribute, but the attribute information is not actually displayed and divided. The rectangular area is displayed as a frame line.

  TEXT, PICTURE, LINE, and TABLE are attribute displays for a character area rectangular block, a drawing area rectangular block, a line figure area rectangular block, and a table area rectangular block. PHOTO indicating a photograph area rectangular block and other attribute displays are used.

  In FIG. 4, the input image and the area division information are displayed side by side, but various display forms are possible, such as displaying a rectangular block divided on the input image by overlapping them.

  In step S304 (specific area specifying step), a rectangular block including a handwritten part is specified as the specific area for the area division information displayed in step S303. As a specific area designating method, for example, the user designates one or a plurality of rectangular blocks in the image 42 by using a pointing device. Various methods such as a method in which the operation screen 116 is a touch panel and the user designates a desired rectangular block with a finger may be employed.

  Alternatively, all the displayed rectangles may be targeted without designating the area.

  In step S305, the specific area specified in step S304 is cut out as image data, and the following processing is performed on the cut out image data.

  In step S306, OCR and OMR processing is performed to determine whether or not pointer information of the original data file is embedded in the document image.

  A two-dimensional barcode recorded as additional information in a document image or an object corresponding to a URL is detected, the URL is character-recognized by OCR, the two-dimensional barcode is decoded by OMR, and an original data file of the input image is obtained. Pointer information in the stored storage device is detected.

  The means for adding pointer information is not limited to a two-dimensional barcode, but a method of embedding information as a change in adjacent character string spacing, a method of embedding in a halftone image, or the like, a so-called digital watermarking method that is not directly visible There is.

  In step S307 (pointer information extraction step), pointer information is extracted from the OCR, OMR, or digital watermark information in step S306.

  In step S308, it is determined whether pointer information has been acquired. When the pointer information is acquired, the process proceeds to step S309 to directly access the original data file.

  In step S309, the original data file is searched. The original file is stored in the form of an electronic file in the hard disk of the client PC 102 in FIG. 1, the database 105 in the document management server 106, the storage device 111 provided in the MFP 100, and the like obtained in step S308. These storage devices are searched according to address information (pointer information). If the electronic file search is successful, the process proceeds to step S311. Otherwise, the process is terminated.

  In step S311, it is determined whether the original data file extracted in step S310 is a raster data file or an image data file encoded with raster data represented by BMP and tiff. When the extracted data file is an image data file, the process proceeds to step S311.

  If the extracted data file is a data file that can be easily reused, such as a character code or vector data, the process proceeds to step S312.

  In step S312 (vectorization step), the process proceeds to a vectorization process, and the image data of each specific area is converted into vector data by the vectorization process.

  The vectorization methods include the following (a) to (f).

  (A) When the specific area is a character area, further font conversion is performed on the character image by OCR, or the character size, style, and font are recognized, and the font is visually faithful to the character obtained by scanning the document. Convert to data.

  (B) When the specific area is a character area and cannot be recognized by OCR, the outline of the character is traced, and the outline information (outline) is converted into a format that represents the connection of line segments.

  (C) When the specific area is a graphic area, the outline of the graphic object is tracked and converted into a format in which the outline information is expressed as a connection of line segments.

  (D) The outline information in the line segment format of (b) and (c) described above is fitted with a Bezier function or the like and converted into function information.

  (E) The shape of the figure is recognized from the outline information of the figure object in (c) described above, and converted into figure definition information such as a circle, a rectangle, and a polygon.

  (F) If the specific area is a graphic area and is a tabular object in the specific area, ruled lines and frame lines are recognized and converted into form format information of a predetermined format.

  In addition to the above methods, various vectorization processes can be considered in which image data is replaced with command definition information such as code information, graphic information, and function information.

  In step S312, the difference between the vectorized electronic file obtained in step S309 or the vectorized data of the original image obtained in step S311 and the file obtained by reading the image is extracted. The part from which the difference is extracted is only the area designated in step S304.

  In step S313, the difference vector information extracted in step S312 is registered in the server as an attribute of the original file.

  With the above configuration, correction by handwriting or the like can be converted into vector data from only an area desired by the user and registered in the server.

  Next, the main steps of FIG. 3 will be described in detail.

[Block selection step]
In step S302 (block selection step), as shown in the image 42 in the right half of FIG. 4, the input image is divided into rectangular blocks for each attribute. As described above, the attributes of the rectangular block include character (TEXT) / drawing (PICTURE) / line drawing (Line) / table (Table) / photograph (PHOTO).
In the block selection step, first, the input image is binarized into black and white, and a pixel block surrounded by a black pixel outline is extracted.

  Further, the size of the black pixel block extracted in this way is evaluated, and the contour tracking is performed for the white pixel block inside the black pixel block whose size is a predetermined value or more. As long as the internal pixel block is equal to or greater than a predetermined value, such as size evaluation for the white pixel block and tracking of the internal black pixel block, the internal pixel block is extracted recursively and the contour is traced.

  The size of the pixel block is evaluated by, for example, the area of the pixel block.

  A rectangular block circumscribing the pixel block thus obtained is generated, and attributes are determined based on the size and shape of the rectangular block.

  For example, a rectangular block whose aspect ratio is close to 1 and whose size is constant is a character-corresponding block that may be a character area rectangular block. When adjacent character-corresponding blocks are regularly aligned, these characters are equivalent. A new rectangular block in which the blocks are collected is generated, and the new rectangular block is set as a character area rectangular block.

  In addition, flat pixel blocks are line drawing area rectangular blocks, black pixel blocks that are larger than a certain size and square white pixel blocks are arranged in a well-aligned manner, front area rectangular blocks, and regions where irregular pixel blocks are scattered are photo areas. A rectangular block and other irregular pixel blocks are defined as a photographic area rectangular block.

  In the block selection step, block information such as attributes and input file information shown in FIG. 5 are generated for each of the rectangular blocks generated in this way.

  In FIG. 5, the block information includes attributes of each block, position coordinates X, coordinates Y, width W, height H, and OCR information. The attribute is given by a numerical value of 1 to 5, where 1 is a character area rectangular block, 2 is a drawing area rectangular block, 3 is a table area rectangular block, 4 is a line drawing area rectangular block, and 5 is a photographic area rectangular block. The coordinates X and Y are the X and Y coordinates (upper left corner coordinates) of the start point of each rectangular block in the input image. The width W and the height H are the width in the X coordinate direction and the height in the Y coordinate direction of the rectangular block. The OCR information indicates the presence or absence of pointer information in the input image.

  Furthermore, the total number N of blocks indicating the number of rectangular blocks is included as input file information.

  The block information for each rectangular block is used for vectorization of a specific area.

[Pointer information extraction step]
Step S307 (pointer information extraction step) is executed by each step of FIG. FIG. 7 shows a document 710 to be processed in the pointer information extraction step, and the document 710 is stored in the data processing device 115. In the document 710, character area rectangular blocks 702 and 703, a drawing area rectangular block 704, and a two-dimensional barcode (QR code) symbol 701 are recorded.

  In step S601, the CPU of the management PC 101 scans the input image of the document 710 stored in the storage device 111, and detects the position of the two-dimensional barcode symbol 701 from the processing result of the block selection step. The QR code symbol is provided with specific position detection element patterns at three of the four corners, and the QR code symbol can be detected by detecting the position detection element pattern.

  In step S602, the format information adjacent to the position detection pattern is restored, and the error correction level and mask pattern applied to the symbol are obtained.

  In step S603, the symbol model number (model) is determined.

  In step S604, the mask pattern obtained from the format information in step S602 is used to XOR the encoded area bit pattern of the QR code symbol, and the mask process of the QR code symbol is cancelled.

  In step S605, an arrangement rule is acquired based on the model obtained in step S603. Based on the arrangement rule, a symbol character is read, and message data and an error correction code word are restored.

  In step S606, it is detected whether there is an error in the restored message based on the error correction code word. If an error is detected, the process branches to step S607 for correction.

  In step S607, the restored message is corrected.

  In step S608, the data code word is divided into segments based on the mode indicator and the character number indicator from the error-corrected data, and the data code word is restored.

  In step S609, the data code character is decoded based on the detected specification mode, and the result is output.

  The data incorporated in the two-dimensional barcode represents pointer information of the original data file, and is composed of path information including a file server name and a file name, for example. Alternatively, it consists of a URI / URL to the corresponding file.

  In the present embodiment, the document 710 provided with pointer information in the form of a two-dimensional barcode has been described, but various pointer information recording forms may be employed.

  For example, pointer information may be directly recorded as a character string according to a character string according to a predetermined rule, and a rectangular block of the character string may be detected by a block selection step. The pointer information can be acquired by recognizing the detected character string.

  Alternatively, in the character area rectangular block 702 and the character area rectangular block 703, modulation that is difficult to visually recognize is added to the interval between adjacent character strings, and the pointer information can be expressed by the information on the character string interval modulation. Such watermark information can be detected by detecting an interval between characters when performing character recognition processing described later, and pointer information can be acquired. It is also possible to add pointer information as a digital watermark in the drawing area rectangular block 704.

  Regarding the pointer information extraction as described above, the pointer information can be acquired quickly and reliably by designating a specific area to be vectorized as in the present embodiment. That is, when the user designates a rectangular block in which a barcode, digital watermark or the like is embedded, efficient pointer information extraction can be performed.

[Vectorization step]
Step S311 (vectorization step) is executed by each step of FIG.

In step S801, it is determined whether or not the specific area is a character area rectangular block. If the specific area is a character area rectangular block, the process proceeds to step S802 and the subsequent steps, recognition is performed using one method of pattern matching, and the corresponding character code is determined. Get. If the specific area is not a character area rectangular block, the process proceeds to step S812.
In step S802, in order to perform horizontal writing and vertical writing determination (assembly direction determination) on the specific area, a horizontal / vertical projection is performed on the pixel value within the specific area.

  In step S803, the variance of the projection in step S802 is evaluated. If the horizontal projection variance is large, it is determined as horizontal writing, and if the vertical projection variance is large, it is determined as vertical writing.

  In step S804, the composition direction is determined based on the evaluation result in step S803, the line is cut out, and then the character is cut out to obtain a character image.

  In the case of horizontal writing, character strings and characters are cut out using horizontal projection, and characters are cut out from the projection in the vertical direction with respect to the cut lines. For vertically written character areas, the process is reversed for horizontal and vertical. When cutting out lines and characters, the size of characters can also be detected.

  In step S805, for each character extracted in step S804, an observation feature vector is generated by converting the feature obtained from the character image into a numerical sequence of several tens of dimensions. There are various known methods for extracting feature vectors. For example, there is a method in which characters are divided into meshes, and a mesh number-dimensional vector obtained by counting character lines in each mesh as line elements according to directions is used as a feature vector.

  In step S806, the observation feature vector obtained in step S805 is compared with the dictionary feature vector obtained for each character type in advance, and the distance between the observation feature vector and the dictionary feature vector is calculated.

  In step S807, the distance calculated in step S806 is evaluated, and the character type with the closest distance is used as the recognition result.

  In step S808, it is determined whether or not the shortest distance is larger than a predetermined value in the distance evaluation in step S807. When the similarity is equal to or greater than a predetermined value, there is a high possibility that the dictionary feature vector is erroneously recognized as another character having a similar shape. Therefore, if the similarity is greater than or equal to a predetermined value, the recognition result in step S807 is not adopted, and the process proceeds to step S811. When the similarity is lower (smaller) than the predetermined value, the recognition result in step S807 is adopted, and the process proceeds to step S809.

  In step S809 (font recognition step), a plurality of dictionary feature vectors for the number of character types used for character recognition are prepared for the character shape type, that is, the font type, and the character code and the font are used for pattern matching. The character font can be recognized by outputting the seed.

  In step S810, each character is converted into vector data using outline data prepared in advance using the character code and font information obtained by character recognition and font recognition. When the input image is color, the color of each character is extracted from the color image and recorded together with vector data.

  In step S811, the character is handled in the same manner as a general line drawing, and the character is outlined. In other words, outline vector data that is visually faithful to the image data is generated for characters that are likely to cause erroneous recognition.

  In step S812, when the specific area is not a character area rectangular block, vectorization processing is executed based on the contour of the image.

  Through the above processing, the image information belonging to the character area rectangular block can be converted into vector data that is substantially faithful in shape, size, and color.

[Vectorization of areas other than character areas]
If it is determined in step S812 that the area is other than the character area rectangular block, that is, the drawing area rectangular block, the line drawing area rectangular block, the table area rectangular block, or the like, the outline of the black pixel block extracted in the specific area is used as vector data. Convert.

  In vectorization of regions other than character regions, first, “corners” that divide a curve into a plurality of sections (pixel columns) are detected in order to represent a line drawing or the like as a combination of straight lines and / or curves. A corner is a point at which the curvature is maximized, and whether or not the pixel Pi on the curve in FIG. 9 is a corner is determined as follows.

  That is, pixels Pi-k and Pi + k that are separated by a predetermined pixel (k pixels) in both directions from Pi along the curve are connected by line segment L. When the distance between the pixels Pi-k and Pi + k is d1, the distance between the line segment L and the pixel Pi is d2, and the arc length between the curved pixels Pi-k and Pi + k is A, and d2 becomes a maximum. Alternatively, when the ratio (d1 / A) is equal to or less than the threshold value, the pixel Pi is determined to be a corner.

  The pixel row divided by the corner is approximated by a straight line or a curve. The approximation to a straight line is executed by the least square method or the like, and the approximation to a curve uses a cubic spline function or the like. The pixel at the corner that divides the pixel row is the approximate line or the start or end of the approximate line.

  Furthermore, it is judged whether or not the inner contour of the white pixel block exists in the vectorized contour, and when there is an inner contour, the contour is vectorized and recursively inverted as the inner contour of the inner contour. The inner contour of the pixel is vectorized.

  As described above, the outline of a figure having an arbitrary shape can be vectorized by using the contour line approximation. If the original document is in color, the figure color is extracted from the color image and recorded together with vector data.

  As shown in FIG. 10, when the outer contour PRj and the inner contour PRj + 1 or another outer contour are close to each other in a certain section of interest, two or a plurality of contour lines are grouped together and expressed as a thick line. can do. For example, the distance PiQi from each pixel Pi of the contour Pj + 1 to the pixel Qi that is the shortest distance on the contour PRj is calculated. It can be approximated by a straight line or curve along the point sequence. The thickness of the approximate line and the approximate curve is, for example, an average value of the distance PiQi.

  A table ruled line, which is a line or a set of lines, can be efficiently expressed as a vector by using a set of lines having a thickness.

  Note that the photographic area rectangular block is not vectorized and remains as image data.

[Correction part extraction processing]
Next, details of the corrected portion extraction process shown in step S312 of FIG. 3 will be described with reference to the flowchart of FIG.

  Here, it is assumed that the image 41 of FIG. 4 is read, and each rectangular block and input image data extracted as a result of step S302 include the block information and input file information shown in FIG. Assume that information is retained.

  Further, it is assumed that the original block information and vector information of the read original are obtained by the processing up to step S311.

  In step S1101, one of the uncompared blocks is selected from the blocks designated by the user in step S304.

  In step S1102, the block information of the selected block is compared with the block information of the original document. For comparison, the position and size of the block are used. If there are blocks of different sizes at the same position, the larger block is further divided into a plurality of blocks for comparison. As a result, it is possible to prevent comparisons from being made due to block recognition errors. When the block on the read original side is divided, the blocks other than those to be compared among the divided blocks are added to the uncompared block list.

  In step S1103, the result of the comparison performed in step S1102 is determined. When the position and size of the block are within the preset error range, it is determined that there is an approximate block, and the process proceeds to step S1104. Otherwise, the block itself is an additionally modified part. It is determined that there is, and the process proceeds to step S1109.

  In step S1109, vectorization is performed to set the washed block as a difference vector. Then, the obtained vector information is added to the difference vector list.

  In step S1104, the vector information in the approximate block is compared. If the block to be compared is a graphic area, the images are compared to create a difference. Then, the image difference is vectorized. When the block area to be compared is a character area, the comparison is performed using a character code. If the attributes of the areas are different, the area is converted into a graphic area and an image difference is acquired.

  In step S1105, it is determined whether the difference vector has been acquired as a result of the comparison in step S1105. If the difference vector has been acquired, the process proceeds to step S1106; otherwise, the process proceeds to step S1107.

  In step S1106, the difference vector information obtained in step S1104 is added to the difference vector list.

  In step S1107, it is checked whether there is an unselected block among the blocks designated by the user in step S304. If all block comparisons have been completed, the process proceeds to step S1108; otherwise, the process proceeds to step S1101.

  In step S1108, the difference vector information obtained in the above step is grouped for each graphic object. The grouping of figures will be described in detail with reference to the flowchart of FIG.

[Figure recognition]
After the outlines of the above line figures and the like are vectorized, the vectorized dividing lines are grouped for each graphic object.

  Each step of FIG. 12 shows a process of grouping vector data for each graphic object.

  In step S1201, first, the start point and end point of each vector data are calculated.

  In step S1202, a graphic element is detected using the start point and end point information obtained in step S1201. A graphic element is a closed graphic formed by a dividing line, and a vector is connected at a common corner pixel serving as a start point and an end point for detection. That is, the principle that each vector constituting the closed shape has a vector connected to each of both ends thereof is applied.

  First, unnecessary vectors that are not connected to both ends are removed from the vector data, and a vector constituting the closed figure is extracted. Next, for the vectors constituting the closed figure, the vectors are searched in order in a certain direction, for example, clockwise, with the end point (start point or end point) of one of the vectors as the start point. That is, the end point of another vector is searched at the other end point, and the closest end point within a predetermined distance is set as the end point of the connected vector. When the vector constituting the closed figure is rotated once and returned to the starting point, all the passed vectors are grouped as a closed figure constituting one graphic element. In addition, all closed graphic constituent vectors inside the closed graphic are also grouped. Further, the same processing is repeated with the starting point of a vector not yet grouped as a starting point. Finally, among the unnecessary vectors removed in step S1201, a vector whose end point is close to the vector grouped as a closed graphic in step S1202 is detected and grouped as one graphic element.

In step S1203, next, other graphic elements or division lines existing in the graphic element are grouped into one graphic object. If there is no other graphic element or dividing line in the graphic element, the graphic element is set as a graphic object.
With the above processing, a graphic block can be handled as a reusable individual graphic object.

  By this processing, the difference vector information is grouped as a graphic object.

  As a result of the above processing, if the original image of the document shown in FIG. 4 is 131 in FIG. 13, a graphic object can be obtained as 132.

  As described above, information added to the original document is obtained as a graphic object made up of vector information by handwriting or copy processing, and is registered in the server.

  FIG. 14 is a diagram showing the structure of a document stored in the database 105. In this example, a document a 1302 and a document b: 1305 are stored in the database 105. In the document a: 1302, handwritten information 1: 1030 and handwritten information 2: 1304 are registered as correction information. Similarly, handwritten information 3: 1306 and handwritten information 4307 are registered in the document b: 1305 as correction information. In the handwritten information, vector information of the corrected figure, date and time of addition, and information of the adder are recorded.

  The printing of a document registered in this way will be described. For example, when printing the document a 1302 registered in the database 105, not only the method of printing the original of the document a 1302, but also the document a 1302 and the handwritten information 1: 1303 are combined and printed out. be able to.

  First, the document a: 1301 in the database 105 is selected using the display device 116 and the input device 113. Next, the user is caused to specify correction information to be synthesized. At this time, there are (g) to (k) and the like in the correction information synthesizing method.

  (G) Restore and synthesize the read state.

  (H) The correction information is synthesized with a color different from that of the original image.

  (I) When the original image and the correction information overlap, or when two or more correction information are specified and they overlap, they are moved to a position where they do not overlap and are combined.

  (J) When the original image and the correction information overlap, or when two or more correction information are specified and they overlap, the correction information is reduced and combined.

  (K) Synthesize with the above combination.

  In these combining methods, correction information is stored as vector data and can be moved, enlarged / reduced, and changed in color.

  In the first embodiment, when a user designates a specific area to be vectorized, one or more of rectangular blocks automatically generated by a BS (block selection process) are designated by an input means or the like. However, it goes without saying that the present invention is not limited to this.

  Instead of generating a rectangular block by BS (block selection processing), the input image itself may be displayed on the operation unit screen of MFP 100, and an area desired by the user may be designated by start coordinates and end coordinates. For example, the specific area is designated by the start coordinate and the end coordinate. As a method for designating the start coordinate and the end coordinate, there are a method in which the operation screen is used as a touch panel, a method in which the touch screen is used, a method in which coordinates are input by a numerical key, and the like.

In the first embodiment, the original image is obtained from the pointer information included in the document image. However, the correction information may be added to the pointer information.
For example, it is assumed that the document a: 1302 in FIG. 13 is printed with the handwritten information 1 added. In this case, the information of the document a 1302 on the database 105 and the information of handwritten information 1: 1030 are also added as pointer information. Thus, when a correction is made to the printed image and the correction is registered, it is possible to register a difference from the image obtained by adding the handwritten information 1: 10303 to the document a 1302. That is, it is possible to prevent the handwritten information 1: 10303 from being registered twice.

  The above is an embodiment according to the present invention.

  In the above embodiment, processing of an input image input from the image reading device 110 or the like has been described. However, raster data stored in a storage medium or image data composed of encoded data thereof, supplied by communication means. The present invention is also effective for image data other than the input image, such as image data.

  Means for carrying out the image processing method according to the present invention is not limited to the image processing system described in FIGS. 1 and 2, and various means such as a dedicated image processing apparatus and a general-purpose computer may be employed.

  When executed on a general-purpose computer, a computer-executable program including program code for causing the general-purpose computer to execute each step of the image processing method is read into the general-purpose computer.

  A program for the general-purpose computer to execute image processing is read from a ROM built in the general-purpose computer, a storage medium readable by the general-purpose computer, or read from a server or the like through a network.

  Further, it is understood by those skilled in the art that the spirit and scope of the present invention are not limited to the specific description and drawings of the present invention, but cover various modifications and changes in the contents described in the claims of the present application. It will be.

  According to the present invention, it is possible to convert information that has been corrected by handwriting or synthesis of a print image into vector data, associate it with the original data, and register it. And handwriting correction information by a plurality of people can be shared, and the fusion of paper and electronic data can be facilitated.

1 is a block diagram showing an image processing system for implementing a first embodiment of an image processing method according to the present invention. Block diagram showing the MFP in FIG. 1 is a flowchart showing a first embodiment of an image processing method according to the present invention; The figure which shows the manuscript processed with the image processing method Table showing block information and input file information obtained by block selection processing A flowchart showing processing for extracting pointer information from an image of a document The figure which shows the example of an image of the original document containing pointer information Flowchart showing character region vectorization processing Diagram showing corner extraction processing in vectorization processing The figure which shows the process of the outline summary in the process of vectorization Flowchart showing difference vector information extraction processing Flow chart showing processing for grouping vector data The figure which shows the original image of the reading original of FIG. 4, and the extracted difference image Schematic diagram of the structure of documents registered in the database

Explanation of symbols

10 Office 20 Office 100 MFP
101 Management PC
102 Client PC
103 Proxy Server 104 Internet 105 Database 106 Document Management Server 107 LAN
108 LAN
109 LAN

Claims (6)

An image input means for inputting an image;
Original information acquisition means for acquiring original electronic information associated with the image;
Vector data conversion means for converting an image into vector data;
Difference extraction means for creating difference vector information from the original electronic information and the image;
In an image forming apparatus including:
An image forming apparatus, wherein the extracted difference vector information is registered in association with original electronic data. The image forming apparatus according to claim 1,
Image input means for inputting an image is
An image forming apparatus comprising: a document reading unit that optically reads a document and obtains image information. The image forming apparatus according to claim 1, wherein:
Including image output means,
Furthermore, when the difference vector information associated with the original electronic data is combined with the original electronic data and output,
An image forming apparatus, wherein difference vector information and original electronic data are associated with an output image. An image input step for inputting an image;
An original information acquisition step of acquiring original electronic information associated with the image;
A vector data conversion step for converting the image into vector data;
A difference extraction step of creating difference vector information from the original electronic information and the image;
In an image forming method including:
An image forming method, wherein the extracted difference vector information is registered in association with original electronic data. The image forming method according to claim 4,
The image input step to input an image is
An image forming method comprising: an original reading step for optically reading an original to obtain image information. The image forming method according to claim 4 or 5, wherein:
Including an image output step,
Furthermore, when the difference vector information associated with the original electronic data is combined with the original electronic data and output,
An image forming method comprising associating difference vector information and original electronic data with an output image.