JP2008084127A - Image forming device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine, whether to copy or transmit information in a portal sheet contained in a document bundle, or apply the processing to the portal sheet itself. <P>SOLUTION: Information showing a storage place of one or more electronic files additionally recorded in the document is recognized from image information obtained by scanning, and when the storage place information of electronic files additionally recorded in the document cannot be detected, the image information is converted to vector data. When an output image is formed, an operator is inquired to select whether to process the read page. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image forming apparatus such as a copying machine having an image input function, an image processing function, and an image output function.

  In recent years, paperless offices are rapidly becoming paperless as environmental problems are screamed. That is, it is possible to construct a document management system by reading a paper document that has been conventionally stored in a binder or the like with a scanner, converting it into a portable document format (hereinafter referred to as PDF), and storing it as a database in an image storage device. On the other hand, in the MFP with the expanded function, when recording an image in advance, the pointer information in the image storage device in which the image file exists is recorded as additional information on the cover or description information of the document, and is stored again. When the document is copied and reused, the storage location of the original electronic file is detected from the pointer information, and the original information of the electronic file is directly used, thereby reducing the storage of the entire paper document.

In such a background, by using the method of converting the image information obtained by the image reading scanning means into vector data by the vectorizing means, and converting the converted vector data as information described in the paper document An image processing system that can be used as a reusable electronic file for any paper document has been proposed. In an image processing system having vectorization means, page information of all pages specified or generated through scanning, file search, vectorization, that is, thumbnail, title, summary, keyword, ID, date, author, and processing An index sheet (hereinafter referred to as a portal sheet) including pointer information to the result is generated and output.
JP 2004-252843 A

  A portal sheet is a collection of a plurality of document information on a single sheet. When the operator includes a portal sheet in a bundle of documents to be copied or transmitted, the portal sheet includes the original document described on the portal sheet. There is a problem in that it is not possible to judge whether to copy or transmit information and to copy or transmit the portal sheet itself, and output and send an unintended erroneous manuscript.

  The present invention has been made in view of the above-described problems, and the purpose of the present invention is to obtain an output result intended by an operator of a document in which a portal sheet and other ordinary documents are mixed. An object of the present invention is to provide an image forming apparatus capable of reducing paper and communication costs by automatically discriminating and inquiring an operator and outputting and sending only necessary information.

  In order to solve the above problem, the image forming apparatus according to the present invention recognizes information indicating a storage location of one or more electronic files additionally recorded on the document from image information obtained by scanning the document. A file specifying means for converting the image information into vector data when the storage location information of the electronic file additionally recorded on the original cannot be detected; an output image forming means for forming an output image; Output data selection means for inquiring an operator to select whether or not to process the read page itself when the output image is formed.

In the image forming apparatus, a file specifying unit that recognizes information indicating a storage location of one or more electronic files additionally recorded on the document from a scanning unit that reads and scans the document and image information obtained by the scanning unit Means for converting the image information into vector data by the vectorization means when the storage location information of the electronic file additionally recorded on the original cannot be detected, and performing predetermined image processing on the read image information. Output image forming means for forming an output image, and output data selection means for inquiring an operator whether or not to process the read page itself when forming the output image,
When processing a bundle of documents in which an index sheet that holds information on one or more documents and other ordinary documents are processed, only necessary information is selectively output / sent for paper and communication. The effect of enabling cost reduction is obtained.

  Embodiments of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of an image processing system according to the present invention. This image processing system is realized in an environment where the office 10 and the office 20 are connected by the Internet 104. A LAN 107 constructed in the office 10 is connected to the MFP 100, a management PC 101 that controls the MFP 100, a client PC (external storage means) 102, a document management server 106, its database 105, and a proxy server 103. The LAN 107 and the LAN 108 in the office 20 are connected to the Internet 104 via the proxy server 13. In the present invention, the MFP 100 is in charge of an image reading unit of a paper document and a part of image processing for the read image signal, and the image signal is input to the management PC 101 using the LAN 109. The management PC is a normal PC and includes an image storage unit, an image processing unit, a display unit, and an input unit. A part of the management PC is integrated with the MFP 100.

  FIG. 2 is a configuration diagram of the MFP 100. In FIG. 2, an image reading unit 110 including an auto document feeder (hereinafter referred to as ADF) irradiates a bundle or one original image with a light source (not shown), and forms an original reflection image on a solid-state image sensor with a lens. A raster-like image reading signal is obtained as image information having a density of 600 DPI from the solid-state imaging device. In the normal copying function, the image signal is processed into a recording signal by the data processing unit 115. In the case of copying every plural number, recording data for one page is temporarily stored in the recording device 111 and then sequentially output to the recording device 112. Then, an image is formed on the paper.

  On the other hand, the print data output from the client PC 102 is converted into raster data that can be recorded by the data processing device 115 from the LAN 107 via the network IF 114, and then formed as a recorded image on paper by the recording device.

  An operator's instruction to the MFP 100 is performed from a key operation unit provided in the MFP and an input device 113 including a keyboard and a mouse that are input to the management PC. These series of operations are performed by a control unit (not shown) in the data processing device 115. It is controlled by.

  On the other hand, the status display of the operation input and the display of the image data being processed are performed on the display device 116. The storage device 111 is also controlled by the management PC, and data exchange and control between the MFP and the management PC are performed using the network IF 117 and the directly connected LAN 109.

[Outline of processing]
Next, an overview of the entire image processing according to the present invention will be described with reference to FIG.

  In FIG. 3, first, the image reading unit 110 of the MFP 100 is operated to scan one original in a raster pattern, and an image information input process 120 obtains a 600 DPI-8-bit image signal. The image signal is preprocessed by the data processing unit 115 and stored in the storage device 111 as image data for one page. The CPU of the management PC 101 first separates the area from the stored image signal into a character / line image portion and a halftone image portion, and the character portion is further divided into blocks or a block or a line. Separated into organized tables and figures and segmented. On the other hand, the image portion expressed in halftone is divided into independent objects for each so-called block such as an image portion of a block separated into rectangles, a background portion, and the like (step 121).

  At this time, a two-dimensional barcode recorded as additional information in the original image or an object corresponding to the URL is detected, and the URL recognizes the character by OCR. If the two-dimensional barcode is used, the mark is decoded (step 122). Pointer information in the storage device storing the original electronic file of the original is detected (step 123). As other means for adding pointer information, there are a so-called digital watermark method that is not directly visualized, such as a method of embedding information in the space between characters and a method of embedding in a halftone image.

  If the pointer information is detected, the process branches to step 125, and the original electronic file is searched from the address indicated by the pointer. In FIG. 1, the electronic file is stored in either the hard disk in the client PC, the database 105 in the document management server 105 connected to the office 10 or 20 LAN, or the storage device 111 of the MFP 100 itself. , The storage device is searched according to the address information obtained in step 123. If the electronic file is not found in step 125, if it is found but is a so-called image file represented by PDF or tiff, or if the pointer information itself does not exist, the process branches to step 126.

  Step 126 is a so-called document search processing routine.

  First, in step 122, a word is extracted from the result of OCR performed on each character block and a full text search is performed, or a so-called layout search is performed from the array of each object and the attribute of each object. If an electronic file with a high degree of similarity is found as a result of the search, a thumbnail, a version, a time stamp of the file, etc. are displayed (step 127). Identification and output format identification. If the candidate is one file, the process automatically branches from step 128 to step 133 to notify the storage address. If the electronic file is not found in the search process of step 126, or if it is found but is a so-called image file represented by PDF or tiff, the process branches to step 129.

  Step 129 is a conversion processing unit from image data to vector data, which converts the image data into an electronic file close to the original electronic file. First, for the character block that has been OCR in step 122, the character size, style, and font are further recognized and converted into font data that is visually faithful to the character obtained by scanning the document. Tables and graphic blocks composed of one line are outlined. The image block is processed as an individual JPEG file as image data. These vectorization processes are performed for each object, and further, the layout information of each object is stored, converted into, for example, rtf (step 130), and an inquiry is made as to whether or not to store in the storage device 111 as an electronic file (step 131). .

  When storing a vectorized document image, index information for search is first generated and added to the search index file in step 132 so that it can be directly searched as an electronic file when the same processing is performed thereafter. . Similarly, when the electronic file can be specified by the search process, the process branches from step 128 to step 132 in order to directly specify the electronic file, and index information for search is generated and added to the index file for search. In step 133, the storage address is notified to the MFP 100. In step 134, the MFP 100 determines whether to record or transmit on the current sheet. For example, if it is the last page or the operator indicates that there are no more documents due to accumulation from the pressure plate, the process proceeds to step 135. If there are more pages, go to step 120 to process the next page.

  If the electronic file can be specified from the pointer information in step 125 or the electronic file can be specified by the search process, the storage address of the electronic file is notified to MFP 100 in step 133.

  Note that, using the electronic file itself obtained by the present invention as described above, for example, processing, storage, transmission and recording of a document can be performed in step 135. These processes reduce the amount of information, increase the storage efficiency, shorten the transmission time, and are very advantageous as high-quality data when recording and displaying, compared to the case of using image data.

  Hereinafter, each processing block will be described in detail.

  First, the block selection process shown in step 121 will be described.

[Block selection processing]
In the block selection process, the image data of one page read in step 120 shown on the right in FIG. 11 is recognized as a block for each object, as shown on the left, and each block is represented as a character / drawing / photo / line / This is a process of determining an attribute such as a table and dividing it into areas having different attributes.

  An example of the block selection process will be described below.

  First, the input image is binarized into black and white, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. For a black pixel block with a large area, contour tracing is also performed for white pixels inside, and a white pixel block is extracted, and the black pixel block is recursively extracted from the white pixel block with a certain area or more. Extract lumps.

  The black pixel blocks obtained in this way are classified by size and shape, and are classified into regions having different attributes. For example, if the aspect ratio is close to 1 and the size is within a certain range, the pixel block corresponding to the character is used, the portion where the adjacent characters can be grouped in a well-aligned manner is the character region, and the flat pixel block is the line region. The area occupied by the black pixel block that is larger than the size and contains the square white pixel block in a well-aligned manner is the table region, the region where the irregular pixel block is scattered is the photo region, and the pixel block of any other shape is used. A drawing area, etc.

  FIG. 12 shows block information for each block obtained by the block selection process.

  Information for each block is used as information for vectorization or search described below.

[Detection of pointer information]
Next, the OCR / OMR process for extracting the file storage position shown in step 122 from the image information will be described.

  FIG. 13 is a flowchart showing a process of decoding a two-dimensional barcode (QR code symbol) added to a document image and outputting a data character string. An example of a document 310 to which a two-dimensional barcode is added is shown in FIG.

  First, an image representing the original 310 stored in the page memory in the data processing device 115 is scanned by a CPU (not shown), and a predetermined two-dimensional barcode symbol 311 is obtained from the result of the block selection process described above. Detect position. The QR code position detection pattern is composed of the same position detection element patterns arranged in three of the four corners of the symbol (step 300).

  Next, the format information adjacent to the position detection pattern is restored, and the error correction level and mask pattern applied to the symbol are obtained (step 301).

  After determining the symbol model number (step 302), the mask process is canceled by performing an XOR operation on the encoded area bit pattern using the mask pattern obtained from the format information (step 303).

  The symbol character is read in accordance with the arrangement rule corresponding to the model, and the message data and the error correction code word are restored (step 304).

  It is detected whether or not there is an error on the restored code (step 305). If an error is detected, the process branches to step 306 and is corrected.

  The data code word is divided into segments based on the mode indicator and the character number indicator from the error-corrected data (step 307).

  Finally, the data character is decoded based on the specification mode, and the result is output (step 308).

  Note that the data incorporated in the two-dimensional barcode represents the address information of the corresponding file, and is composed of path information including, for example, a file server name and a file name. Alternatively, it consists of a URL to the corresponding file.

  In the present embodiment, the document 310 to which pointer information is assigned using a two-dimensional barcode has been described. However, when pointer information is directly recorded as a character string, a block of a character string in accordance with a predetermined rule is replaced with the preceding block. It is possible to obtain the address information of the original file directly by detecting it by selection processing and recognizing each character of the character string indicating the pointer information.

  Alternatively, pointer information can be obtained by adding a modulation that is difficult to visually recognize to the character block 312 or 313 of the document 310 in FIG. Can be granted. As for the so-called watermark information, pointer information can be obtained by detecting an interval between characters when performing character recognition processing described later. It is also possible to add pointer information as a digital watermark in the natural image 314.

[File search by pointer information]
Next, the search for the electronic file from the pointer information shown in step 125 described above with reference to FIG. 3 will be described using the flowchart of FIG.

First, the file server is specified based on the address included in the pointer information. (Step 400)
Here, the file server refers to the client PC 102, the document management server 106 including the database 105, and the MFP 100 itself including the storage device 111.

  Here, the address is URL or path information including a server name and a file name.

  When the file server is identified, the address is transferred to the file server. (Step 401) Upon receiving the address, the file server searches for the corresponding file. (Step 402) If the file does not exist (Step 403-N), the MFP is notified of the fact.

If the file exists (step 403-Y), as described in FIG. 3, the file address is notified (step 133), and if the processing desired by the user is acquisition of image file data, the MFP Transfer files to. (Step 408)
[File search processing]
Next, details of the file search process shown in step 126 of FIG. 3 will be described with reference to FIGS.

  As described above, the process of step 126 is performed when there is no pointer information in the input document (input file) in step 124, or when the pointer information is present but the electronic file is not found, or when the electronic file is not found. Performed when the file is an image file.

  Here, it is assumed that each block and input file extracted as a result of step 122 includes information (block information and input file information) shown in FIG. Examples of information contents include attributes, coordinate positions, width and height sizes, and presence / absence of OCR information. Attributes are classified into characters, lines, photographs, pictures, tables and others. In addition, for the sake of simple explanation, the blocks are arranged in the ascending order of the coordinates X, that is, in the order of (for example, X1 <X2 <X3 <X4 <X5 <X6), block 1, block 2, block 3, block 4, block 5, block 6 Named. The total number of blocks is the total number of blocks in the input file. In the case of FIG. FIG. 16 is a flowchart for performing a layout search for a file similar to an input file from the database using these pieces of information. Here, it is assumed that the database file includes the same information as in FIG.

  The flow of the flowchart is to sequentially compare the input file and the file in the database. First, in step 510, initialization such as a similarity rate described later is performed. Next, in step 511, the total number of blocks is compared. If true, the information of the blocks in the file is further sequentially compared. In block information comparison, an attribute similarity rate, a size similarity rate, and an OCR similarity rate are calculated in steps 513, 515, and 518, respectively, and an overall similarity rate is calculated based on them in step 522. About the calculation method of each similarity rate, since a well-known technique is used, description is abbreviate | omitted. In step 523, if the overall similarity is higher than a preset threshold Th, the file is listed as a similarity candidate in step 524. However, N, W, and H in the figure are the total number of blocks of the input file, each block width, and each block height, and ΔN, ΔW, and ΔH consider errors based on the block information of the input file. n, w, and h are the total number of blocks in the database file, each block width, and each block height. Further, although not shown, the position information XY may be compared at the time of size comparison in step 514.

  As described above, as a result of the search, the similarity is higher than the threshold value Th, and the database file (step 524) stored as a candidate is displayed as a thumbnail or the like (step 127). If the operator needs to be selected from a plurality of files, the file is specified by the operator's input operation.

[Vectorization processing]
When the original file does not exist in the file server, the image data shown in FIG. 11 is vectorized for each block. Next, the vectorization shown in step 129 will be described in detail. First, character recognition processing is performed on each character for a character block.

"Character recognition"
The character recognition unit recognizes an image cut out in character units using a pattern matching method, and obtains a corresponding character code. This recognition process recognizes the character type with the closest distance by comparing the observed feature vector obtained by converting the feature obtained from the character image into a numerical sequence of several tens of dimensions and the dictionary feature vector obtained for each character type in advance. The resulting process. There are various known methods for extracting a feature vector. For example, there is a method characterized by dividing a character into meshes, and using a mesh number-dimensional vector obtained by counting character lines in each mesh as line elements according to directions.

  When character recognition is performed on the character area extracted in block selection (step 121), first, horizontal writing and vertical writing are determined for the corresponding area, lines are cut out in the corresponding directions, and then characters are cut out. Get a character image. Horizontal / vertical writing can be determined by taking a horizontal / vertical projection of the pixel value in the corresponding area, and determining that the horizontal projection area is large when the horizontal projection variance is large, and vertical writing area when the vertical projection variance is large. . For horizontal writing, character strings and characters are decomposed by cutting out lines using horizontal projection, and then cutting out characters from the vertical projection of the cut lines. For vertically written character areas, horizontal and vertical may be reversed. At this time, the character size can be detected.

"Font recognition"
Multiple character feature vectors used for character recognition are prepared for the character shape type, that is, font type, and the font type is output along with the character code when matching. I can do it.

"Vectorization of characters"
Using the character code and font information obtained by the character recognition and font recognition, the character portion information is converted into vector data using outline data prepared in advance. When the original document is color, the color of each character is extracted from the color image and recorded together with vector data.

  Through the above processing, the image information belonging to the character block can be converted into vector data that is almost faithful in shape, size, and color.

"Vectorization of non-character parts"
In the block selection process (step 121), the outline of the pixel block extracted therein is converted into vector data for a region that is a drawing, line, or table region. Specifically, a point sequence of pixels forming an outline is divided by points regarded as corners, and each section is approximated by a partial straight line or curve. The corner is a point where the curvature is maximized. As shown in FIG. 17, the point where the curvature is maximized is that a string is placed between k points Pi-k and Pi + k that are separated from the arbitrary point Pi. It is obtained as a point where the distance between the string and Pi becomes maximum when drawn. Furthermore, let R be the chord length / arc length between Pi−k and Pi + k, and a point where the value of R is equal to or less than a threshold value can be regarded as a corner. Each section after being divided by the corners can be vectorized by using a least square method for a straight line and a curve using a cubic spline function.

  Further, when the target has an inner contour, it is similarly approximated by a partial straight line or a curve using the point sequence of the white pixel contour extracted by block selection.

  As described above, the outline of a figure having an arbitrary shape can be vectorized by using the contour line approximation. If the original document is in color, the figure color is extracted from the color image and recorded together with vector data.

  Furthermore, as shown in FIG. 18, when the outer contour and the inner contour or another outer contour are close to each other in a certain section, the two contour lines can be combined and expressed as a line having a thickness. . Specifically, when a line is drawn from each point Pi of a certain contour to a point Qi that is the shortest distance on another contour, and each distance PQi is on average less than or equal to a certain length, the interval of interest has a midpoint PQi as a point sequence Approximate with a straight line or a curve, and the thickness is the average value of PQi. A table ruled line that is a line or a set of lines can be efficiently expressed as a set of lines having the above-described thickness.

  The vectorization using the character recognition process for the character block has been described above. As a result of the character recognition process, the character having the closest distance from the dictionary is used as the recognition result. When this distance is a predetermined value or more, However, there are many cases where characters that do not necessarily match the original characters and are similar in shape are erroneously recognized. Therefore, in the present invention, as described above, such characters are handled in the same manner as general line drawings, and the characters are outlined. That is, even a character that is erroneously recognized in the conventional character recognition processing is not vectorized into an erroneous character, and can be vectorized by outline rendering that is visually faithful to image data.

  In the present invention, since it is not possible to vectorize a block determined to be a photograph, it is left as image data.

[Figure recognition]
A process of grouping the vectorized dividing lines for each graphic object after vectorizing the outline of the figure of an arbitrary shape as described above will be described.

  FIG. 19 shows a flowchart until vector data is grouped for each graphic object. First, the start point and end point of each vector data are calculated (700). Next, a graphic element is detected using the start point and end point information of each vector (701). The detection of a graphic element is to detect a closed graphic formed by a dividing line. In detection, the detection is performed by applying the principle that each vector constituting the closed shape has vectors connected to both ends thereof. Next, other graphic elements or dividing lines existing in the graphic element are grouped to form one graphic object (702). If there is no other graphic element or dividing line in the graphic element, the graphic element is set as a graphic object.

  FIG. 20 shows a flowchart for detecting a graphic element. First, unnecessary vectors not connected to both ends are removed from the vector data, and a closed graphic component vector is extracted (710). Next, from the closed figure constituent vectors, the starting point of the vector is used as a starting point, and the vectors are sequentially followed in the clockwise direction. The process is repeated until the start point is returned, and all the passed vectors are grouped as a closed graphic constituting one graphic element (711). In addition, all closed graphic constituent vectors inside the closed graphic are also grouped. Further, the same processing is repeated with the starting point of a vector not yet grouped as a starting point. Finally, among the unnecessary vectors removed at 710, those joined to the vector grouped as a closed figure at 711 are detected and grouped as one figure element (712).

  As described above, the graphic block can be handled as an individual graphic object that can be reused individually.

[Conversion to application data]
By the way, the image data for one page is subjected to block selection processing (121), and the result of vectorization processing (129) is converted as an intermediate data format file as shown in FIG. It is called Document Analysis Output Format (DAOF).

  FIG. 21 shows the data structure of DAOF.

  In FIG. 21, reference numeral 791 denotes a header, which holds information relating to document image data to be processed. In the layout description data portion 792, attributes such as TEXT (character), TITLE (title), CAPTION (caption), LINEART (line drawing), EPICTURE (natural image), FRAME (frame), TABLE (table), etc. in the document image data. The attribute information of each block recognized every time and its rectangular address information are held. The character recognition description data portion 793 holds character recognition results obtained by character recognition of TEXT blocks such as TEXT, TITLE, and CAPTION. The table description data portion 794 stores details of the structure of the TABLE block. The image description data portion 795 cuts out image data of blocks such as PICTURE and LINEART from the document image data and holds them.

  Such a DAOF is not only used as intermediate data but may be stored as a file itself. In this file state, it is not possible to reuse individual objects in a so-called general document creation application. I can't. Therefore, the process 130 for converting DAOF into application data will be described in detail.

  FIG. 22 is an overall schematic flow.

  In 8000, DAOF data is input.

  8002 generates a document structure tree that is the source of application data.

  In step 8004, actual data in the DAOF is poured based on the document structure tree to generate actual application data.

  FIG. 23 is a detailed flow of the 8002 document structure tree generation unit, and FIG. 24 is an explanatory diagram of the document structure tree. As a basic rule of overall control, the flow of processing shifts from a micro block (single block) to a macro block (an aggregate of blocks).

  Hereinafter, the block refers to the micro block and the entire macro block.

  8100 performs regrouping based on the relevance in the vertical direction in units of blocks. Immediately after the start, judgment is made in units of micro blocks.

  Here, the relevance can be defined by the fact that the distance is close and the block width (height in the horizontal direction) is substantially the same.

  Information such as distance, width, and height is extracted with reference to DAOF.

  FIG. 24A shows an actual page configuration, and FIG. 24B shows its document structure tree. As a result of 8100, T3, T4, and T5 are generated as one group V1, and T6 and T7 are generated as one group V2 in the same hierarchy.

  8102 checks whether or not there is a separator in the vertical direction. For example, the separator is physically an object having a line attribute in the DAOF. Also, logically, it is an element that explicitly divides a block in the application. If a separator is detected here, it is subdivided at the same level.

  8104 uses the group length to determine whether there are no more divisions.

  If the group length in the vertical direction is the page height, the document structure tree generation ends.

  In the case of FIG. 24, since there is no separator and the group height is not the page height, the process proceeds to 8106.

  In step 8106, regrouping is performed based on the relevance in the horizontal direction in units of blocks. Here too, the first time immediately after the start is determined in units of microblocks.

  The definition of the relevance and the determination information is the same as in the vertical direction.

  In the case of FIG. 24, H1 at T1 and T2 and H2 at V1 and V2 are generated as a group in the same hierarchy one above V1 and V2.

  8108 checks for the presence of a horizontal separator.

  In FIG. 24, since S1 exists, this is registered in the tree, and a hierarchy of H1, S1, and H2 is generated.

  8110 uses the group length to determine whether there are no more divisions.

  If the horizontal group length is the page width, the document structure tree generation ends.

  If not, the process returns to 8102, and the relevance check in the vertical direction is repeated again at the next higher level.

  In the case of FIG. 24, since the division width is the page width, the process ends here, and finally V0 of the highest hierarchy representing the entire page is added to the document structure tree.

  After the document structure tree is completed, application data is generated in 8006 based on the information.

  In the case of FIG. 24, the details are as follows.

  That is, since there are two blocks T1 and T2 in the horizontal direction, H1 has two columns, and after T1 internal information (refer to DAOF, text of character recognition result, image, etc.) is output, the column is changed and the internal of T2 Information is output, and then S1 is output.

  Since H2 has two blocks V1 and V2 in the horizontal direction, it outputs as 2 columns, V1 outputs its internal information in the order of T3, T4, T5, then changes the column and outputs the internal information of T2, T7 of V2 To do.

  As described above, conversion processing to application data can be performed.

[Add pointer information]
Next, the pointer information addition process shown in step 133 will be described.

  When the document to be processed is specified by the search process, or when the original file can be reproduced by vectorization, when the document is recorded, pointer information is added at the time of recording on paper. Thus, the original file data can be easily obtained when various processes are performed again using this document.

  FIG. 25 is a flowchart showing a process of encoding a data character string as pointer information with a two-dimensional barcode (QR code symbol: JIS X0510) 311 and adding it to an image.

  The data to be incorporated in the two-dimensional barcode represents the address information of the corresponding file, and is composed of path information including a file server name and a file name, for example. Alternatively, the URL includes a URL to the corresponding file, a file ID managed in the database 105 in which the corresponding file is stored, or a storage device included in the MFP 100 itself.

  First, an input data string is analyzed in order to identify different characters to be encoded. Further, an error detection and error correction level is selected, and a minimum model number that can accommodate input data is selected. (Step 900).

  Next, the input data string is converted into a predetermined bit string, and an indicator indicating a data mode (numeric, alphanumeric, 8-bit byte, kanji, etc.) and a termination pattern are added as necessary. Further, it is converted into a predetermined bit code word. (Step 901).

  At this time, in order to perform error correction, the code word string is divided into a predetermined number of blocks according to the model number and the error correction level, an error correction code word is generated for each block, and is added after the data code word string ( Step 902).

  The data code words of each block obtained in step 902 are connected, and the error correction code word of each block and, if necessary, the remainder code word are followed. (Step 903).

  Next, the code word module is arranged in the matrix together with the position detection pattern, separation pattern, timing pattern, alignment pattern, and the like. (Step 904).

  Further, an optimal mask pattern is selected for the symbol coding region, and the mask processing pattern is converted into a module obtained in step 904 by an XOR operation. (Step 905).

  Finally, format information and model number information are generated in the module obtained in step 905 to complete a two-dimensional code symbol. (Step 906).

  The above-described two-dimensional barcode with address information embedded can be recorded in the data processing apparatus 115 when, for example, an electronic file is formed as print data from the client PC 102 on the recording apparatus 112 as a recorded image on paper. After being converted into raster data, it is added to a predetermined location on the raster data to form an image. The user who has distributed the image-formed paper here can read the image reading unit 110 to detect the storage location of the original electronic file from the pointer information in step 123 described above.

  In addition to the two-dimensional barcode described in this embodiment, the means for adding additional information for the same purpose includes, for example, a method of adding pointer information directly to a document as a character string, a character string in a document, A method generally referred to as a digital watermark, such as a method of embedding information by modulating a character spacing and a method of embedding in a halftone image in a document, can be applied.

[Portal sheet generator]
Hereinafter, the portal sheet 1901 will be described with reference to FIG.

  Information on six originals is recorded on the portal sheet of FIG. Each includes the following information 1902-1905.

  Reference numeral 1903 denotes a reduced image (thumbnail) of the document, which represents the first page.

  Reference numerals 1902 and 1904 denote information related to the thumbnail of 1903 such as date, title, author, ID, keyword, summary, etc., which are information attached to the document in advance, information input by the user, or the contents of the document. Or information extracted automatically from.

  Reference numeral 1905 denotes a pointer for indicating a document storage location and the like. Here, it is represented by a two-dimensional barcode, but it may be a one-dimensional barcode or a digital watermark.

[Generate index in portal sheet (as page information)]
"title"
There are the following methods for extracting a title from text data.

  A sentence having a large font that appears in the first part of text data and a frequently used sentence in the text data is suitable as a title using the layout information and font information extracted by the above-described method.

"keyword"
There are the following methods for extracting keywords from text data.

  First, the entire text data is broken down into words by morphological analysis or the like. All words are sorted according to the frequency of use, and the keywords are ordered in descending order of frequency. Furthermore, in order to extract more effective keywords, it is possible to collate with a keyword database prepared in advance.

"ID, date, author"
These pieces of information are acquired as attribute information of the file when the file exists by searching the electronic file.

"wrap up"
A method for creating a summary of text data composed of character code strings is as follows.

  First, the text data is decomposed into sentences or clauses. This can be mechanically divided using the character code of the punctuation mark as a boundary. Next, the importance is calculated and assigned to each sentence or clause. As a method of calculating the importance, the appearance frequency is obtained for the words included in the entire text data, a high score is given to the frequently appearing words, and the importance of each sentence or clause is determined for each of the included words. Using the method of summing up the number of words, layout information and font information, detect sentences corresponding to titles and headlines, increase the importance of the sentence itself, or increase the number of words included in the sentence There are ways to calculate the degree. Finally, a summary sentence is created by extracting sentences or phrases with higher importance. At this time, it is possible to generate a summary sentence of a desired size by changing the number of top ranks to be extracted.

"table of contents"
To create a table of contents for text data that spans multiple pages, you can divide the text into paragraphs from blanks on the layout, etc., and then summarize each one and assign a summary sentence of about one line. In particular, a sentence extracted as a title or a heading from layout information or font information as described above is optimal as a table of contents.

"Adding pointer information to the portal sheet"
Next, pointer information addition processing for indicating the document storage location will be described.

  FIG. 25 is a flowchart showing a process of encoding a data character string as pointer information with a two-dimensional barcode (QR code symbol: JIS X0510) 311 and adding it to an image.

  The data to be incorporated in the two-dimensional barcode represents the address information of the corresponding file, and is composed of path information including a file server name and a file name, for example. Alternatively, the URL includes a URL to the corresponding file, a file ID managed in the database 105 in which the corresponding file is stored, or a storage device included in the MFP 100 itself.

  First, an input data string is analyzed in order to identify different characters to be encoded. Further, an error detection and error correction level is selected, and a minimum model number that can accommodate input data is selected. (Step 900).

  Next, the input data string is converted into a predetermined bit string, and an indicator indicating a data mode (numeric, alphanumeric, 8-bit byte, kanji, etc.) and a termination pattern are added as necessary. Further, it is converted into a predetermined bit code word. (Step 901).

  At this time, in order to perform error correction, the code word string is divided into a predetermined number of blocks according to the model number and the error correction level, an error correction code word is generated for each block, and is added after the data code word string ( Step 902).

  The data code words of each block obtained in step 902 are connected, and the error correction code word of each block and, if necessary, the remainder code word are followed. (Step 903).

  Next, the code word module is arranged in the matrix together with the position detection pattern, separation pattern, timing pattern, alignment pattern, and the like. (Step 904).

  Further, an optimal mask pattern is selected for the symbol coding region, and the mask processing pattern is converted into a module obtained in step 904 by an XOR operation. (Step 905).

  Finally, format information and model number information are generated in the module obtained in step 905 to complete a two-dimensional code symbol. (Step 906).

  For example, when a portal sheet is generated from the client PC 102, the two-dimensional bar code in which address information is incorporated as described above is converted into raster data that can be recorded in the data processing device 115, and then the raster data is recorded on the raster data. An image is formed by being added to a predetermined portion.

  In addition to the two-dimensional barcode described in this embodiment, the means for adding additional information for the same purpose includes, for example, a method of adding pointer information directly to a document as a character string, a character string in a document, A method generally referred to as a digital watermark, such as a method of embedding information by modulating a character spacing and a method of embedding in a halftone image in a document, can be applied.

  An example of embedding code information in a halftone image (thumbnail image) posted on a portal sheet by applying a digital watermark will be described below.

  The code information is embedded as a binary data string by weighting one bit for each pixel of each color component on the image. When embedding code information in a pixel, one code information is embedded in one block composed of neighboring pixels. Code information is repeatedly embedded in a plurality of blocks that do not overlap each other. Here, a block constituted by a plurality of blocks in which the same code information is embedded is called a macro block. Note that the entire thumbnail image may be a macro block. When the embedded code information is extracted, the cross-correlation for each block is obtained from the macroblock, and the code information can be extracted from the statistics.

  Code information can be embedded in such an image by changing the pixels at a level that cannot be recognized by human vision, but the thumbnail image used in the portal sheet does not care about the image quality for that purpose. It is possible to embed with durability by embedding with a higher weight.

[Example of pointer information judgment copy]
In the previous pointer information detection embodiment, even if the original file group can be specified by the search means, there may be a problem in copying the original file. For example, this occurs when the speed is reduced, when only a part of the portal sheet is trimmed, or when the portal sheet itself is desired to be copied.

  Against this background, an example of a copy that makes an inquiry to the operator after determining the pointer information will be described with reference to FIG. Steps 120 to 134 are processing equivalent to the above-described processing outline, and description thereof is omitted.

  If pointer information is detected in step 124 and the presence of an electronic file is detected in step 125, it is determined in step 143 whether or not the previous page was also a document including pointer information. In step 143, if the previous page also has pointer information, it is copied as it is or the electronic file indicated by the pointer information is output according to the processing selected by the operator on the previous page. (Step 142). On the other hand, if the previous page does not include pointer information, the operator is inquired. A screen example at this time is shown in FIG. If the operator selects copy as it is, the scanned image is recorded in the memory in the MFP, and the process proceeds to step 134. If the last page or the operator desires output in step 134, an output image is formed in step 135 and the image is output. If the reading is continued in step 134, the process returns to step 120. Next, when the operator selects a copy of the original data in step 144, the process proceeds to step 133, the storage address is notified from the pointer information, the document or page to be output is stored in the memory in the MFP, and the process proceeds to step 134. .

  Next, when pointer information is not detected in step 124, the process proceeds to step 140.

  In step 140, it is determined whether or not the previous page was a document that does not include pointer information. If there is no pointer information on the previous page, the copied or read original is vectorized according to the processing selected by the operator on the previous page, and the resulting electronic file is output. (Step 142). On the other hand, if pointer information is included in the previous page, the process proceeds to step 141 to inquire the operator. A screen example at this time is shown in FIG. If the operator selects copy as it is, the scanned image is recorded in the memory in the MFP, and the process proceeds to step 134. Next, when the operator selects a copy after vectorizing in step 141, the process proceeds to step 126, the same data is searched, candidates are selected, vectorized, etc., and the process proceeds to step 133.

  6 and 7, when there is no input for a certain period of time, a default operation based on the user setting may be performed.

  In another embodiment, the determination processing in step 143 and step 140 is omitted, and the operator is inquired every time reading is performed.

[Example of pointer information determination transmission]
An embodiment in which a portal sheet is transmitted, which is an extension of the previous pointer information determination copy embodiment, will be described with reference to FIG.

  Steps 120 to 134 are processing equivalent to the above-described processing outline, and description thereof is omitted.

  If pointer information is detected in step 124 and the presence of an electronic file is detected in step 125, it is determined in step 143 whether or not the previous page was also a document including pointer information. In step 143, if the previous page also has pointer information, according to the processing selected by the operator on the previous page, transmission is performed as is, transmission of the electronic file indicated by the pointer information, or notification of pointer information is performed. (Step 151). On the other hand, if the previous page does not include pointer information, the operator is inquired. An example of the screen at this time is shown in FIG. When the operator selects transmission as it is, the scanned image is recorded in the memory in the MFP, and the process proceeds to step 134. If the last page or the operator desires output in step 134, an output image is formed in step 135 and image transmission is performed. If the reading is continued in step 134, the process returns to step 120. Next, when the operator selects transmission of original data in step 152, the process proceeds to step 133, the storage address is notified from the pointer information, and the document or page or data to be output is stored in the memory in the MFP. , Go to step 135. In step 135, if the transmission means can handle data such as email or file transmission, the original data is converted into an image format that can be transmitted when the transmission means handles only images such as IFAX and FAX. , Send each. On the other hand, if the operator selects the transmission of the URL of the original data in step 152, the process proceeds to step 133, the storage address is notified from the pointer information, is stored in the memory in the MFP, and the process proceeds to steps 134 and 135. In step 135, if the transmission means can handle data such as e-mail or file transmission, the storage address can be a character string. If the transmission means handles only an image such as IFAX or FAX, a character string indicating the storage address can be transmitted. Convert to a different image format and send each.

  Next, when pointer information is not detected in step 124, the process proceeds to step 140.

  In step 140, it is determined whether or not the previous page was a document that does not include pointer information. If there is no pointer information on the previous page, according to the process selected by the operator on the previous page, the document that has been transmitted or read is vectorized, and the resulting electronic file is transmitted or the storage address is notified. (Step 151). On the other hand, if pointer information is included in the previous page, the process proceeds to step 150 to inquire the operator. An example of the screen at this time is shown in FIG. When the operator selects transmission as it is, the scanned image is recorded in the memory in the MFP, and the process proceeds to step 134. Next, if the operator selects transmission after vectorizing in step 150, the process proceeds to step 126, the same data is searched, candidates are selected, vectorized, etc., and the process proceeds to step 133. In this case, in step 135, if the transmission means can handle data such as e-mail or file transmission, vectorized data can be transmitted. If the transmission means only handles images such as IFAX or FAX, the vectorized data can be transmitted. Convert to format and send each.

  In step 150, if the operator selects URL transmission after vectorization, the process proceeds to step 126, the same data is searched, candidate selection, vectorization, etc. are stored, and the process proceeds to step 133. In this case, in step 135, if the transmission means can handle data such as e-mail or file transmission, the storage address is a character string. If the transmission means handles only an image such as IFAX or FAX, a character string indicating the storage address is displayed. It converts into the image format which can be transmitted, and transmits each.

  8 and 9, when there is no input for a certain period of time, a default operation based on user settings may be performed.

  In another embodiment, the determination processing in step 143 and step 140 is omitted, and the operator is inquired every time reading is performed.

System Configuration. FIG. Outline of processing. FIG. 1 is a flowchart of a pointer information determination process overview. FIG. 2 is a flowchart of a pointer information determination process overview. Example of operator inquiry 1 screen. Example of operator inquiry 2 screen. Example of operator inquiry 3 screen. Example of operator inquiry 4 screen. Portal sheet conceptual diagram. The conceptual diagram of a block selection process. Block information. The flowchart which extracts the storage position of a file from image information. An example of a document to which a two-dimensional barcode is added. The file search flow figure by pointer information. The file search processing flowchart. Explanatory drawing of the angle determination concerning vectorization of parts other than a character. Explanatory drawing of the division line approximation of the outline concerning vectorization of parts other than a character. The flowchart of a graphical object recognition process. The flowchart of a graphic element detection process. The figure which shows the structure of the intermediate data format after vectorization. The flow figure of application data conversion processing. The flowchart of a document structure tree production | generation process. Document structure tree explanatory diagram. The flowchart which shows the process of adding the two-dimensional barcode as pointer information.

Claims (4)

  File specifying means for recognizing information indicating a storage location of one or more electronic files additionally recorded on the original from image information obtained by scanning the original, and electronic information additionally recorded on the original When file storage location information cannot be detected, vectorization means for converting image information into vector data, output image formation means for forming an output image, and processing of the read page itself when forming the output image And an output data selection unit that makes an operator inquire whether or not to perform the selection.   The image forming apparatus according to claim 1, further comprising a billing table according to the output data selection, wherein a billing method according to the output data can be set.   The output data selection means outputs the read page itself, or vectorizes the read page by the vectorization means and saves it in an internal or external server, and outputs the storage location, or the file specifying means The image forming apparatus according to claim 1, wherein an inquiry is made to an operator as to whether the storage source data is acquired and processed.   The output data selection unit is configured to change the state of the document from which the electronic file storage location information is not detected by the file specifying unit to a document from which the electronic file storage location information is detected, or the storage location of the electronic file. 4. The image forming apparatus according to claim 1, wherein an inquiry is made to an operator when information is changed from an original to be detected to an undetected original.

Images