JP2006134231A - Image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing system for improving through-put. <P>SOLUTION: The image data of an original read by scanning an original are outputted to a recording medium, and simultaneously an electronic file on a server is retrieved and detected from the image data obtained by scanning the original, and the electronic file data are stored in an HDD at an equipment side, and when all the electronic file data are stored in the HDD at the equipment side, the data to be outputted are switched from the image data obtained by scanning the original to the electronic file data stored in the HDD. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing system that converts image data read by an image processing apparatus such as a copying machine into vector data that can be reused by so-called document creation application software.

  In recent years, paperless offices are rapidly becoming paperless as environmental problems are screamed. That is, it is possible to construct a document management system by reading a paper document that has been conventionally stored in a binder or the like with a scanner, converting it into a portable document format (hereinafter referred to as PDF), and storing it as a database in an image storage device. On the other hand, in the MFP with the expanded function, when recording an image in advance, the pointer information in the image storage device in which the image file exists is recorded as additional information on the cover or description information of the document, and is stored again. When the document is copied or reused, the storage location of the original electronic file is detected from the pointer information, and the original information of the electronic file is directly used, thereby reducing the storage of the entire paper document.

  On the other hand, image data read from a scanner or image data obtained by expanding PDL data sent from a host computer via a network into a bitmap is temporarily stored in a memory such as a hard disk, and image data of an arbitrary document is stored from the memory. A digital copying machine having a function of repeatedly reading and printing out is known.

Another conventional example is an image processing method in which image data read by an image processing apparatus such as a copying machine is converted into vector data that can be reused by so-called document creation application software such as Word, and these are easily used. (See, for example, Patent Document 1).
JP 2004-252843 A

  However, since the storage location of the original electronic file is detected from the pointer information and the electronic file is output to the device via the network, there is a drawback that it takes time to output.

In order to solve the above problem, the invention described in claim 1 outputs a reading means for reading and scanning a document, a number specifying means for specifying the number of copies to be printed, and image data of the document obtained by the reading means. From the image data output means, the image information obtained by the reading means, a specifying means for specifying the electronic data of the original, a storage means for storing the electronic data on the device side, and the electronic means of the original by the specifying means When the data is specified and the electronic data is acquired, the electronic data is stored in the storage means in the storage device on the device side, and the storage device is stored from the output of the image data when the storage is completed And switching means for switching to the output of the electronic data.

  According to a second aspect of the present invention, in the image processing system according to the first aspect, the means for identifying the electronic file recognizes additional information indicating the storage location of the electronic file additionally recorded on the document. So you can easily identify the original electronic file from the image information.

  The invention described in claim 3 is the image processing system according to claim 1, wherein the electronic file specifying means has means for searching for the specific information described in the document from the files stored in the storage means, As a result of the search, by specifying the electronic file by matching the specific information, the original electronic file can be easily specified even for a document in which no additional information is recorded.

  According to a fourth aspect of the present invention, in the image processing system according to the second aspect, the file specifying means is obtained by searching for specific information described in the original from the electronic file of the original obtained from the pointer information. Since the electronic file is specified only in some cases, it is possible to specify with higher accuracy than the case where the electronic file is simply specified from the pointer information.

  According to a fifth aspect of the present invention, in the image processing system according to the first aspect, when the accumulation ends, the switching means switches from the output of the image data to the output of the electronic data in the storage device, It has a selection means for selecting whether to output only the image data without switching.

  According to a sixth aspect of the present invention, there is provided reading means for reading and scanning an original, specifying means for specifying electronic data of the original from image information obtained by the reading means, and storage for storing the electronic data on a device side. When the electronic data of the document is specified by the means and the specifying means and the electronic data is acquired, the electronic data is stored in the storage means in the storage device on the device side, and output is possible Electronic data output control means for outputting the electronic data stored in the storage device at the time.

  According to a seventh aspect of the present invention, in the image processing system according to the sixth aspect, the means for identifying the electronic file is a means for recognizing additional information indicating a storage location of the electronic file additionally recorded on the document. So you can easily identify the original electronic file from the image information.

  The invention according to claim 8 is the image processing system according to claim 6, wherein the electronic file specifying means has means for searching for specific information described in the document from files stored in the storage means, As a result of the search, by specifying the electronic file by matching the specific information, the original electronic file can be easily specified even for a document in which no additional information is recorded.

  According to a ninth aspect of the present invention, in the image processing system according to the seventh aspect, the file specifying means is obtained by retrieving specific information described in the original from the electronic file of the original obtained from the pointer information. Since the electronic file is specified only in some cases, it is possible to specify with higher accuracy than the case where the electronic file is simply specified from the pointer information.

  The image data of the original scanned by scanning the original is output to the recording medium, and at the same time, the electronic file on the server is searched from the image data obtained by the scan, and the electronic file data is stored in the HDD on the device side. When all the electronic file data is stored in the HDD on the device side, the output data is switched from the image data obtained by scanning the original to the electronic file data stored in the HDD. Improvement is planned.

  In addition, when the printer cannot output due to some cause, the document is scanned in advance, the electronic file on the server is searched, detected, and the electronic file data is stored in the HDD on the device side, Since the electronic file data stored in the HDD on the device side is output when the output of the printer becomes possible, the effect of improving the throughput can be obtained.

  Embodiments of the present invention will be described. FIG. 10 is a block diagram showing a configuration example of an image processing system according to the present invention. This image processing system is realized in an environment where the office 10 and the office 20 are connected by the Internet 104. A LAN 107 constructed in the office 10 is connected to the MFP 100, a management PC 101 that controls the MFP 100, a client PC (external storage means) 102, a document management server 106, its database 105, and a proxy server 103. The LAN 107 and the LAN 108 in the office 20 are connected to the Internet 104 via the proxy server 13. In the present invention, the MFP 100 is in charge of an image reading unit of a paper document and a part of image processing for the read image signal, and the image signal is input to the management PC 101 using the LAN 109. The management PC is a normal PC and includes an image storage unit, an image processing unit, a display unit, and an input unit. A part of the management PC is integrated with the MFP 100.

  FIG. 2 is a configuration diagram of the MFP 100. In FIG. 2, an image reading unit 110 including an auto document feeder (hereinafter referred to as ADF) irradiates a bundle or one original image with a light source (not shown), and forms an original reflection image on a solid-state image sensor with a lens. A raster-like image reading signal is obtained as image information having a density of 600 DPI from the solid-state imaging device. In the normal copying function, the image signal is processed into a recording signal by the data processing unit 115. In the case of copying every plural number, recording data for one page is temporarily stored in the recording device 111 and then sequentially output to the recording device 112. Then, an image is formed on the paper.

  On the other hand, the print data output from the client PC 102 is converted into raster data that can be recorded by the data processing device 115 from the LAN 107 via the network IF 114, and then formed as a recorded image on paper by the recording device.

  An operator's instruction to the MFP 100 is performed from a key operation unit provided in the MFP and an input device 113 including a keyboard and a mouse that are input to the management PC. These series of operations are performed by a control unit (not shown) in the data processing device 115. It is controlled by.

On the other hand, the status display of the operation input and the display of the image data being processed are performed on the display device 116. The storage device 111 is also controlled by the management PC, and data exchange and control between the MFP and the management PC are performed using the network IF 117 and the directly connected LAN 109.
(Outline of processing)
Next, an overview of the entire image processing according to the present invention will be described with reference to FIG.

  In FIG. 3, first, the image reading unit 110 of the MFP 100 is operated to scan one original in a raster pattern, and an image information input process 120 obtains a 600 DPI-8-bit image signal. The image signal is preprocessed by the data processing unit 115 and stored in the storage device 111 as image data for one page. The CPU of the management PC 101 first separates the area from the stored image signal into a character / line image portion and a halftone image portion, and the character portion is further divided into blocks or a line or a line. Separated into organized tables and figures, each segmented. On the other hand, the image portion expressed in halftone is divided into so-called independent objects for each block, such as an image portion of a block separated into rectangles, a background portion, and the like (step 121).

  At this time, a two-dimensional barcode recorded as additional information in the original image or an object corresponding to the URL is detected, and the URL recognizes the character by OCR. If the two-dimensional barcode is used, the mark is decoded (step 122). Pointer information in the storage device storing the original electronic file of the original is detected (step 123). As other means for adding pointer information, there are a so-called digital watermarking method that is not directly visualized, such as a method of embedding information in a character interval and a method of embedding in a halftone image.

  If the pointer information is detected, the process branches to step 125, and the original electronic file is searched from the address indicated by the pointer. In FIG. 10, the electronic file is stored in either the hard disk in the client PC, the database 105 in the document management server 105 connected to the office 10 or 20 LAN, or the storage device 111 of the MFP 100 itself. , The storage device is searched according to the address information obtained in step 123. If the electronic file is not found in step 125, if it is found but is a so-called image file represented by PDF or tiff, or if pointer information itself does not exist, the process branches to step 126.

  Step 126 is a so-called document search processing routine.

  First, in step 122, a word is extracted from the result of OCR performed on each character block and a full text search is performed, or a so-called layout search is performed from the array of each object and the attribute of each object. If an electronic file with a high degree of similarity is found as a result of the search, a thumbnail or the like is displayed (step 127), and if the operator needs to be selected from a plurality of files, the file is specified by the operator's input operation. If the candidate is one file, the process automatically branches from step 128 to step 134 to notify the storage address. If the electronic file is not found in the search process of step 126, or if it is found but is a so-called image file represented by PDF or tiff, the process branches to step 129.

  Step 129 is a conversion processing unit from image data to vector data, which converts the image data into an electronic file close to the original electronic file. First, for the character block that has been OCR in step 122, the character size, style, and font are further recognized and converted into font data that is visually faithful to the character obtained by scanning the document. Tables and graphic blocks composed of one line are outlined. The image block is processed as an individual JPEG file as image data. These vectorization processes are performed for each object, and further, layout information of each object is stored, converted into, for example, rtf (step 130), and stored in the storage device 111 as an electronic file (step 131).

  First, in step 132, index information for search is generated and added to the search index file so that the vectorized document image can be directly searched as an electronic file when the same processing is performed thereafter. Further, if it is determined in step 136 that the process that the operator wants to perform is recording, the process branches to step 133 to add pointer information to the file as image data. Similarly, when the electronic file can be specified by the search process, the process branches from step 128 to step 134 in order to directly specify the electronic file. In the case where the storage address is notified to the operator and recorded on the current paper, Similarly, pointer information is added to the electronic file. If the electronic file can be specified from the pointer information in step 125, if the electronic file can be specified by the search process, or converted into the electronic file by vectorization, the storage address of the electronic file is notified to the operator in step 134. To do.

  Note that, for example, the document can be processed, stored, transmitted, and recorded in step 135 by using the electronic file itself obtained by the present invention. These processes reduce the amount of information, increase the storage efficiency, shorten the transmission time, and are very advantageous as high-quality data when recording and displaying, compared to the case of using image data.

  Hereinafter, each processing block will be described in detail.

  First, the block selection process shown in step 121 will be described.

(Block selection processing)
In the block selection process, the image data of one page read in step 120 shown on the right in FIG. 4 is recognized as a block for each object, as shown on the left, and each block is represented as a character / drawing / photo / line / This is a process of determining an attribute such as a table and dividing it into areas having different attributes.

  An example of the block selection process will be described below.

  First, the input image is binarized into black and white, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. For a black pixel block with a large area, contour tracing is also performed for white pixels inside, and a white pixel block is extracted, and the black pixel block is recursively extracted from the white pixel block with a certain area or more. Extract lumps.

  The black pixel blocks obtained in this way are classified by size and shape, and are classified into regions having different attributes. For example, if the aspect ratio is close to 1 and the size is within a certain range, the pixel block corresponding to the character is used, the portion where the adjacent characters can be grouped in a well-aligned manner is the character region, and the flat pixel block is the line region. The area occupied by the black pixel block that is larger than the size and contains the square white pixel block in a well-aligned manner is the table region, the region where the irregular pixel block is scattered is the photo region, and the pixel block of any other shape is used. A drawing area, etc.

  FIG. 5 shows block information for each block obtained by the block selection process.

  Information for each block is used as information for vectorization or search described below.

(Detection of pointer information)
Next, the OCR / OMR process for extracting the file storage position shown in step 122 from the image information will be described.

  FIG. 6 is a flowchart showing a process of decoding a two-dimensional barcode (QR code symbol) added to a document image and outputting a data character string. An example of a document 310 to which a two-dimensional barcode is added is shown in FIG.

  First, an image representing the original 310 stored in the page memory in the data processing device 115 is scanned by a CPU (not shown), and a predetermined two-dimensional barcode symbol 311 is obtained from the result of the block selection process described above. Detect position. The QR code position detection pattern is composed of the same position detection element patterns arranged in three of the four corners of the symbol (step 300).

  Next, the format information adjacent to the position detection pattern is restored, and the error correction level and mask pattern applied to the symbol are obtained (step 301).

  After determining the symbol model number (step 302), the mask process is canceled by performing an XOR operation on the encoded area bit pattern using the mask pattern obtained from the format information (step 303).

  The symbol character is read in accordance with the arrangement rule corresponding to the model, and the message data and the error correction code word are restored (step 304).

  It is detected whether or not there is an error on the restored code (step 305). If an error is detected, the process branches to step 306 and is corrected.

  Based on the mode indicator and the character number indicator, the data code word is divided into segments from the error-corrected data (step 307).

  Finally, the data character is decoded based on the specification mode, and the result is output (step 308).

  The data incorporated in the two-dimensional bar code represents the address information of the corresponding file, and is composed of path information including a file server name and a file name, for example. Alternatively, it consists of a URL to the corresponding file.

  In the present embodiment, the document 310 to which pointer information is assigned using a two-dimensional barcode has been described. However, when pointer information is directly recorded as a character string, a block of a character string according to a predetermined rule is replaced with the previous block. It is possible to directly obtain the address information of the original file by detecting each character of the character string indicating the pointer information by detecting the selection process.

  Alternatively, pointer information can also be obtained by adding a modulation that is difficult to visually recognize to the character block 312 or 313 character string of the document 310 in FIG. Can be granted. As for the so-called watermark information, pointer information can be obtained by detecting an interval between characters when performing character recognition processing described later. It is also possible to add pointer information as a digital watermark in the natural image 314.

(File search based on pointer information)
Next, retrieval of an electronic file from pointer information shown in steps 125 and 128 described above with reference to FIG. 3 will be described using the flowchart of FIG.

First, the file server is specified based on the address included in the pointer information. (Step 400)
Here, the file server refers to the client PC 102, the document management server 106 including the database 105, and the MFP 100 itself including the storage device 111.

  Here, the address is URL or path information including a server name and a file name.

  If the file server can be identified, the address is transferred to the file server. (Step 401) Upon receiving the address, the file server searches for the corresponding file. (Step 402) If the file does not exist (Step 403-N), the MFP is notified of the fact.

If the file exists (step 403-Y), as described in FIG. 3, the file address is notified (step 134), and if the processing desired by the user is acquisition of image file data, the MFP Transfer files to. (Step 408)
(File search process)
Next, details of the file search process shown in step 126 of FIG. 3 will be described with reference to FIGS.

  As described above, the process of step 126 is performed when there is no pointer information in the input document (input file) in step 124, or when the pointer information is present but the electronic file is not found, or when the electronic file is not found. Performed when the file is an image file.

  Here, it is assumed that each block and input file extracted as a result of step 122 includes information (block information and input file information) shown in FIG. Examples of information contents include attributes, coordinate positions, width and height sizes, and presence / absence of OCR information. Attributes are classified into characters, lines, photographs, pictures, tables and others. For the sake of simple explanation, the blocks are arranged in the order of the smallest coordinate X, that is, in the order of (for example, X1 <X2 <X3 <X4 <X5 <X6), block 1, block 2, block 3, block 4, block 5, block 6 Named. The total number of blocks is the total number of blocks in the input file. In the case of FIG. FIG. 9 shows a flowchart for performing a layout search for a file similar to an input file from the database using these pieces of information. Here, it is assumed that the database file includes the same information as in FIG.

  The flow of the flowchart is to sequentially compare the input file and the file in the database. First, in step 510, initialization such as a similarity rate described later is performed. Next, in step 511, the total number of blocks is compared. If true, the information of the blocks in the file is further sequentially compared. In block information comparison, an attribute similarity rate, a size similarity rate, and an OCR similarity rate are calculated in steps 513, 515, and 518, respectively, and an overall similarity rate is calculated based on them in step 522. About the calculation method of each similarity rate, since a well-known technique is used, description is abbreviate | omitted. In step 523, if the overall similarity is higher than a preset threshold Th, the file is listed as a similarity candidate in step 524. However, N, W, and H in the figure are the total number of blocks of the input file, each block width, and each block height, and ΔN, ΔW, and ΔH consider errors based on the block information of the input file. n, w, and h are the total number of blocks in the database file, each block width, and each block height. Further, although not shown, the position information XY may be compared at the time of size comparison in step 514.

  As described above, as a result of the search, the similarity is higher than the threshold value Th, and the database file (step 524) stored as a candidate is displayed as a thumbnail or the like (step 127). If the operator needs to be selected from a plurality of files, the file is specified by the operator's input operation.

(Vectorization processing)
If the original file does not exist in the file server, the image data shown in FIG. 4 is vectorized for each block. Next, the vectorization shown in step 129 will be described in detail. First, character recognition processing is performed on each character for a character block.

(Character recognition)
The character recognition unit recognizes an image cut out in character units using a pattern matching method, and obtains a corresponding character code. This recognition process recognizes the character type with the closest distance by comparing the observed feature vector obtained by converting the feature obtained from the character image into a numerical sequence of several tens of dimensions and the dictionary feature vector obtained for each character type in advance. The resulting process. There are various known methods for extracting a feature vector. For example, there is a method characterized by dividing a character into meshes, and using a mesh number-dimensional vector obtained by counting character lines in each mesh as line elements according to directions.

  When character recognition is performed on the character area extracted in block selection (step 121), first, horizontal writing and vertical writing are determined for the corresponding area, lines are cut out in the corresponding directions, and then characters are cut out. Get a character image. Horizontal / vertical writing can be determined by taking a horizontal / vertical projection of the pixel value in the corresponding area, and determining that the horizontal projection area is large when the horizontal projection variance is large, and vertical writing area when the vertical projection variance is large. . For horizontal writing, character strings and characters are decomposed by cutting out lines using horizontal projection, and then cutting out characters from the vertical projection of the cut lines. For vertically written character areas, horizontal and vertical may be reversed. At this time, the character size can be detected.

(Font recognition)
Multiple character feature vectors used for character recognition are prepared for the character shape type, that is, font type, and the font type is output along with the character code when matching. I can do it.

(Character vectorization)
Using the character code and font information obtained by the character recognition and font recognition, the character portion information is converted into vector data using outline data prepared in advance. When the original document is color, the color of each character is extracted from the color image and recorded together with vector data.

  Through the above processing, the image information belonging to the character block can be converted into vector data that is almost faithful in shape, size, and color.

(Vectorization of parts other than characters)
In the block selection process (step 121), the outline of the pixel block extracted therein is converted into vector data for a region that is a drawing, line, or table region. Specifically, a point sequence of pixels forming an outline is divided by points regarded as corners, and each section is approximated by a partial straight line or curve. An angle is a point where the curvature is maximum, and the point where the curvature is maximum is that a string is placed between k points Pi-k and Pi + k that are separated from the arbitrary point Pi as shown in FIG. It is obtained as a point where the distance between the string and Pi becomes maximum when drawn. Furthermore, let R be the chord length / arc length between Pi−k and Pi + k, and a point where the value of R is equal to or less than a threshold value can be regarded as a corner. Each section after being divided by the corners can be vectorized by using a least square method for a straight line and a curve using a cubic spline function.

  Further, when the target has an inner contour, it is similarly approximated by a partial straight line or a curve using the point sequence of the white pixel contour extracted by block selection.

  As described above, the outline of a figure having an arbitrary shape can be vectorized by using the contour line approximation. If the original document is in color, the figure color is extracted from the color image and recorded together with vector data.

  Furthermore, as shown in FIG. 11, when an outer contour and an inner contour or another outer contour are close to each other in a certain section, two contour lines can be combined and expressed as a line having a thickness. . Specifically, when a line is drawn from each point Pi of a certain contour to a point Qi having the shortest distance on another contour, and each distance PQi is on average less than or equal to a certain length, the target section has a PQi midpoint as a point sequence The line is approximated by a straight line or a curve, and its thickness is the average value of PQi. A table ruled line that is a line or a set of lines can be efficiently expressed as a set of lines having the above-described thickness.

  The vectorization using the character recognition process for the character block has been described above. As a result of the character recognition process, the character having the closest distance from the dictionary is used as the recognition result, but this distance is not less than a predetermined value. Are not necessarily identical to the original characters and are often erroneously recognized as characters having similar shapes. Therefore, in the present invention, as described above, such characters are handled in the same manner as general line drawings, and the characters are outlined. That is, conventionally, even a character that is erroneously recognized in the character recognition process is not vectorized into an erroneous character, and can be vectorized by an outline that is visually faithful to image data.

  In the present invention, since it is not possible to vectorize a block determined to be a photograph, it is left as image data.

(Figure recognition)
A process of grouping the vectorized dividing lines for each graphic object after vectorizing the outline of the figure of an arbitrary shape as described above will be described.

  FIG. 12 shows a flowchart until the vector data is grouped for each graphic object. First, the start point and end point of each vector data are calculated (700). Next, a graphic element is detected using the start point and end point information of each vector (701). The detection of a graphic element is to detect a closed graphic formed by a dividing line. In detection, the detection is performed by applying the principle that each vector constituting the closed shape has vectors connected to both ends thereof. Next, other graphic elements or dividing lines existing in the graphic element are grouped to form one graphic object (702). If there is no other graphic element or dividing line in the graphic element, the graphic element is set as a graphic object.

  FIG. 13 shows a flowchart for detecting graphic elements. First, unnecessary vectors not connected to both ends are removed from the vector data, and a closed graphic component vector is extracted (710). Next, from the closed figure constituent vectors, the starting point of the vector is used as a starting point, and the vectors are sequentially followed in the clockwise direction. The process is repeated until the start point is returned, and all the passed vectors are grouped as a closed graphic constituting one graphic element (711). In addition, all closed graphic constituent vectors inside the closed graphic are also grouped. Further, the same processing is repeated with the starting point of a vector not yet grouped as a starting point. Finally, among the unnecessary vectors removed at 710, those joined to the vector grouped as a closed figure at 711 are detected and grouped as one figure element (712).

  As described above, the graphic block can be handled as an individual graphic object that can be reused individually.

(Conversion to application data)
By the way, the image data for one page is subjected to block selection processing (121), and the result of vectorization processing (129) is converted as an intermediate data format file as shown in FIG. It is called Document Analysis Output Format (DAOF).

  FIG. 14 shows the data structure of DAOF.

  In FIG. 14, reference numeral 791 denotes a header, which holds information relating to document image data to be processed. In the layout description data portion 792, attributes such as TEXT (character), TITLE (title), CAPTION (caption), LINEART (line drawing), EPICTURE (natural image), FRAME (frame), TABLE (table), etc. in the document image data. The attribute information of each block recognized every time and its rectangular address information are held. The character recognition description data portion 793 holds character recognition results obtained by character recognition of TEXT blocks such as TEXT, TITLE, and CAPTION. The table description data portion 794 stores details of the structure of the TABLE block. The image description data portion 795 cuts out image data of blocks such as PICTURE and LINEART from the document image data and holds them.

  Such a DAOF is not only used as intermediate data but may be stored as a file itself. In this file state, it is not possible to reuse individual objects in a so-called general document creation application. I can't. Therefore, a process 130 for converting the DAOF into application data (for example, data that can be used by MS-Word of Microsoft Corporation) will be described in detail.

  FIG. 15 is an overall schematic flow.

  In 8000, DAOF data is input.

  8002 generates a document structure tree that is the source of application data.

  In step 8004, actual data in the DAOF is poured based on the document structure tree to generate actual application data.

  FIG. 16 is a detailed flow of the 8002 document structure tree generation unit, and FIG. 17 is an explanatory diagram of the document structure tree. As a basic rule of overall control, the flow of processing shifts from a micro block (single block) to a macro block (an aggregate of blocks).

Hereinafter, the block refers to the micro block and the entire macro block.
8100 performs regrouping based on the relevance in the vertical direction in units of blocks. Immediately after the start, the judgment is made in units of micro blocks.

  Here, the relevance can be defined by the fact that the distance is close and the block width (height in the horizontal direction) is substantially the same.

  Information such as distance, width, and height is extracted with reference to GAOF.

  FIG. 17A shows an actual page configuration, and FIG. 17B shows its document structure tree. As a result of 8100, T3, T4, and T5 are generated as one group V1, and T6 and T7 are generated as one group V2 in the same hierarchy.

  8102 checks whether or not there is a separator in the vertical direction. For example, the separator is physically an object having a line attribute in the DAOF. Also, logically, it is an element that explicitly divides a block in the application. If a separator is detected here, it is subdivided at the same level.

  8104 uses the group length to determine whether there are no more divisions.

  If the group length in the vertical direction is the page height, the document structure tree generation ends.

  In the case of FIG. 17, since there is no separator and the group height is not the page height, the process proceeds to 8106.

  In step 8106, regrouping is performed based on the relevance in the horizontal direction in units of blocks. Here too, the first time immediately after the start is determined in units of microblocks.

  The definition of the relevance and the determination information is the same as in the vertical direction.

  In the case of FIG. 17, H1 at T1 and T2 and H2 at V1 and V2 are generated as a group in the same hierarchy one above V1 and V2.

  8108 checks for the presence of a horizontal separator.

  In FIG. 17, since there is S1, this is registered in the tree, and a hierarchy of H1, S1, and H2 is generated.

  8110 uses the group length to determine whether there are no more divisions.

  If the horizontal group length is the page width, the document structure tree generation ends.

  If not, the process returns to 8102, and the relevance check in the vertical direction is repeated again at the next higher level.

  In the case of FIG. 17, since the division width is the page width, the process ends here, and finally V0 of the highest hierarchy representing the entire page is added to the document structure tree.

  After the document structure tree is completed, application data is generated in 8006 based on the information.

  Specifically, in the case of FIG.

  That is, since there are two blocks T1 and T2 in the horizontal direction, H1 has two columns, and after T1 internal information (refer to DAOF, text of character recognition result, image, etc.) is output, the column is changed and the internal of T2 Information is output, and then S1 is output.

  Since H2 has two blocks V1 and V2 in the horizontal direction, it outputs as 2 columns, V1 outputs its internal information in the order of T3, T4, T5, then changes the column and outputs the internal information of T2, T7 of V2 To do.

  As described above, conversion processing to application data can be performed.

(Add pointer information)
Next, the pointer information addition process shown in step 133 will be described.

  When the document to be processed is specified by the search process, or when the original file can be reproduced by vectorization, when the document is recorded, pointer information is added when recording on paper. Thus, the original file data can be easily obtained when various processes are performed again using this document.

  FIG. 18 is a flowchart showing a process of encoding a data character string as pointer information with a two-dimensional barcode (QR code symbol: JIS X0510) 311 and adding it to an image.

  The data to be incorporated in the two-dimensional barcode represents the address information of the corresponding file, and is composed of, for example, path information including a file server name and a file name. Alternatively, the URL includes a URL to the corresponding file, a file ID managed in the database 105 in which the corresponding file is stored, or a storage device included in the MFP 100 itself.

  First, an input data string is analyzed in order to identify different characters to be encoded. Further, an error detection and error correction level is selected, and a minimum model number that can accommodate input data is selected. (Step 900).

  Next, the input data string is converted into a predetermined bit string, and an indicator indicating a data mode (numeric, alphanumeric, 8-bit byte, kanji, etc.) and a termination pattern are added as necessary. Further, it is converted into a predetermined bit code word. (Step 901).

  At this time, in order to perform error correction, the code word string is divided into a predetermined number of blocks according to the model number and the error correction level, an error correction code word is generated for each block, and is added after the data code word string ( Step 902).

  The data code words of each block obtained in step 902 are connected, and the error correction code word of each block and, if necessary, the remainder code word are followed. (Step 903).

  Next, the code word module is arranged in the matrix together with the position detection pattern, separation pattern, timing pattern, alignment pattern, and the like. (Step 904).

  Further, an optimal mask pattern is selected for the symbol coding region, and the mask processing pattern is converted into a module obtained in step 904 by an XOR operation. (Step 905).

  Finally, format information and model number information are generated in the module obtained in step 905 to complete a two-dimensional code symbol. (Step 906).

  The above-described two-dimensional barcode with address information embedded can be recorded in the data processing apparatus 115 when, for example, an electronic file is formed as print data from the client PC 102 on the recording apparatus 112 as a recorded image on paper. After being converted into raster data, it is added to a predetermined location on the raster data to form an image. The user who has distributed the image-formed paper here can read the image reading unit 110 to detect the storage location of the original electronic file from the pointer information in step 123 described above.

  In addition to the two-dimensional barcode described in this embodiment, the means for adding additional information for the same purpose is, for example, a method of adding pointer information directly to a document as a character string, a character string in a document, A method generally referred to as a digital watermark, such as a method of embedding information by modulating a character spacing and a method of embedding in a halftone image in a document, can be applied.

(Example of output data switching)
First, the user designates the number of copies to be printed from the operation unit, scans the document, outputs the image data of the document read to the recording medium, and at the same time, based on the image data obtained by the scan, Alternatively, an electronic file is searched from a database or the like in a document management server connected to the LAN, detected, and the electronic file data is stored in the HDD on the device side. When all the electronic file data is stored in the HDD on the device side, the output data is switched from the image data obtained by scanning the document to the electronic file data stored in the HDD.

  In addition, the user can designate in advance from the operation unit whether to switch from outputting image data read from a document to outputting the electronic file data stored in the storage device or only outputting image data without switching. Is possible.

(Example of electronic data reservation output)
In the MFP, other jobs, that is, received prints such as PDL print, FAX, and e-mail, are being performed, so there are cases where output cannot be performed or output is awaited because preparation of the printer engine has not been completed. In such a case, scanning is performed in advance, the original is read, and the electronic data is obtained from the hard disk in the client PC or the database in the document management server connected to the LAN based on the image data obtained by the scanning. The file is searched and acquired, and the electronic file data is stored in the HDD on the device side. The throughput is improved by outputting the electronic file data stored in the HDD when the output device is ready for output.

  In addition, the user can specify in advance from the operation unit whether to output the electronic file data in the storage device accumulated when the output device is ready to output or to output the image data read by scanning as it is. Is possible.

The block diagram which shows the image processing system structural example concerning this invention Configuration diagram of MFP Outline of processing Block selection processing Block information A flowchart showing a process of decoding a two-dimensional barcode added to a document image and outputting a data character string An example of a document with a two-dimensional barcode Flowchart explaining retrieval of electronic file from pointer information Flowchart for searching the layout of files similar to the input file Diagram showing the point of maximum curvature A figure that combines two contour lines together and expresses them as thick lines Flow chart for grouping vector data for each graphic object Flow chart for detecting graphic elements Diagram showing the data structure of DAOF Outline flowchart of processing to convert to application data Detailed flow of document structure tree generator Illustration of the document structure tree A flowchart showing a process of encoding a data character string as pointer information with a two-dimensional barcode and adding it to an image

Claims (10)

  From the copy number designating means for designating the number of copies to be output, the reading means for reading and scanning the original, the image data output means for outputting the image data of the original obtained by the reading means, and the image data obtained by the reading means Specifying means for specifying electronic data of the document, storage means for storing the electronic data on the device side, and when the electronic data of the document is specified by the specifying means and the electronic data is acquired, the device side Switching means for storing the electronic data in the storage means and switching the output of the image data to the output of the stored electronic data in the storage device when the storage is completed. Image processing system.   2. The image processing system according to claim 1, wherein the specifying means includes means for recognizing a storage location of electronic data additionally recorded on a document.   2. The image processing system according to claim 1, wherein the specifying means includes means for searching for specific information described in the document from a file stored in the storage means.   The specifying means specifies the electronic data only when specific information described in the original is retrieved from the electronic data of the original obtained by the reading means. The image processing system according to item.   Selection means for selecting whether to switch from the output of the image data to the output of the electronic data in the storage device accumulated by the switching means at the time when the storage is completed or only to output the image data without switching. The image processing system according to claim 1, further comprising:   The reading means for reading and scanning the original, the specifying means for specifying the electronic data of the original from the image data obtained by the reading means, the storage means for storing the electronic data on the device side, and the specifying means When the electronic data of the original is specified and the electronic data is acquired, the electronic data is stored in the storage means in the storage device on the device side, and the storage device stores the electronic data stored when the output becomes possible And an electronic data output control means for outputting the electronic data.   7. An image processing system according to claim 6, wherein said specifying means includes means for recognizing a storage location of electronic data additionally recorded on a document.   7. The image processing system according to claim 6, wherein the specifying means includes means for searching for specific information described in the document from a file stored in the storage means.   The specifying means specifies the electronic data only when specific information described in the original is retrieved from the electronic data of the original obtained by the reading means. The image processing system according to item.   7. The apparatus according to claim 6, further comprising designation means for designating whether to output the electronic data in the storage device accumulated when the output device is ready to output or to output the image data. Image processing system.

Images