JP4338189B2 - Image processing system and image processing method - Google Patents

Description

  The present invention relates to an image processing system and an image processing method for searching for an image stored in a database based on an image read by an image processing apparatus such as a copying machine.

  In recent years, environmental problems have been highlighted, and paperless use in offices and the like is rapidly progressing. Therefore, a document management that is conventionally stored and stored in a binder or the like is read by a scanner, converted into a portable document format (hereinafter abbreviated as “PDF”), and stored and stored in an image storage device (database). The system is built.

On the other hand, in a digital multi-function peripheral (hereinafter abbreviated as “MFP”) with an expanded function, when an image (image data) is printed and output as a paper document, the image is stored in the image storage device. Is recorded as additional information in the cover or description information of a paper document. When copying a paper document on which the image is recorded, the storage location in the image storage device in which the original image is stored is detected from the pointer information added to the cover or the like and read from the paper document. The original image stored in the image storage device can be directly used for editing and reprinting instead of using the obtained image for reprinting or the like. As a result, problems such as degradation of image quality caused by reusing an image read from a paper document many times can be prevented (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 10-143414

  However, the pointer information as described above is given the storage location and file name of the desired file, and searches are performed based on it, so if a file is moved or renamed, a user who does not know it will There may be a problem that the original file cannot be acquired.

  The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide an image processing system and an image processing method capable of accurately retrieving information described in a paper document.

In order to solve the above problems, the present invention searches an electronic file corresponding to input image data from a plurality of electronic files held in any of a plurality of electronic file servers connected to a network, and An image processing system for acquiring a searched electronic file,
  Input means for inputting image data representing a document;
  Extraction means for extracting additional information for specifying an electronic file server from the input image data;
  Recognition means for recognizing features of the input image data;
  Based on the recognized characteristics of the image data, an electronic file that matches or is similar to the input image data is searched for in the first electronic file server corresponding to the additional information extracted by the extraction means. A first search means;
  When the electronic file that matches or is similar to the input image data is not searched by the first search means, the electronic file that matches or is similar to the input image data is Second search means for searching in another electronic file server different from the electronic file server of
  An acquisition means for acquiring the searched electronic file when an electronic file that matches or is similar to the input image data is searched by the first or second search means;
  When an electronic file that matches or is similar to the input image data is not searched by the first and second search means, the input image data is converted into an electronic file of a predetermined format. Conversion means for converting;
  Holding means for holding the electronic file converted by the converting means together with additional information for specifying the electronic file server holding the electronic file in the electronic file server specified by the additional information;
  It is characterized by providing.

According to the present invention, by specifying an electronic file server from additional information extracted from input image data, an electronic file is searched in an appropriate electronic file server among a plurality of electronic file servers connected to a network. If the search is not performed, the search is performed in another electronic file server. If the search is not performed in the other electronic file server, the electronic file converted from the input image data is stored in the electronic file. Can be held together with additional information for identifying the server holding the.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing system according to the first embodiment of the present invention. The image processing system shown in FIG. 1 is realized in an environment in which an office 10 and an office 20 are connected by a network 104 such as the Internet.

  A LAN 107 constructed in the office 10 is connected to the MFP 100, a management PC 101 that controls the MFP 100, a client PC 102, a document management server 106a, a database 105a, and a proxy server 103a. A document management server 106b, its database 105b, and a proxy server 103b are connected to the LAN 108 built in the office 20. The client PC 102 includes an external storage unit, a search image input unit, and a search result output unit. The LAN 107 and the LAN 108 in the office 20 are connected to a network 104 such as the Internet via proxy servers 103a and 103b.

  In this embodiment, the MFP 100 is in charge of image reading processing for optically reading a paper document and converting it into an image signal, and part of the image processing for the read image signal. The image signal is input to the management PC 101 using the LAN 109. To do. The management PC 101 can also be realized by a normal PC, and includes an image storage unit, an image processing unit, a display unit, and an input unit. The management PC 101 may be partly or entirely integrated with the MFP 100.

  FIG. 2 is a block diagram showing the configuration of the MFP 100 according to the first embodiment of the present invention. In FIG. 2, an image reading unit 110 including an auto document feeder (hereinafter abbreviated as “ADF”) irradiates a bundle or one original image with a light source provided therein, and reflects an original reflection image with a lens. An image is formed on the element, and a raster-like image reading signal is obtained as image information having a density of 600 dpi from the solid-state imaging element. When the normal copying function is used, the image signal is processed into a recording signal by the data processing unit 115. In the case of copying every plural number, the recording data for one page is once held in the storage device 111, and then the forming apparatus. The images are sequentially output to 112 to form an image on paper.

  On the other hand, print data output from the client PC 102 is input from the LAN 107 to the MFP 100, converted into raster data that can be recorded by the data processing device 115 via the network IF 114, and then output to the forming device 112 to be recorded on paper. Formed as.

  An operator's instruction to the MFP 100 is performed from an input device 113 such as a key provided in the MFP 100 or an input device such as a keyboard and a mouse of the management PC 101. These series of operations are controlled in the data processing device 115. Controlled by the department.

  On the other hand, the status display of the operation input and the display of the image data being processed are performed on the display device 116 of the MFP 100, the monitor of the management PC 101, the client PC 102, or the like. The storage device 111 is also controlled by the management PC 101, and data exchange and control between the MFP 100 and the management PC 101 are performed using the network IF 117 and the directly connected LAN 109.

[Outline of processing]
Next, an overview of the entire image processing by the image processing system according to the embodiment of the present invention will be described. FIG. 3 is a flowchart for explaining a procedure of image processing by the image processing system according to the first embodiment of the present invention.

  First, in order to input an original image by the image reading unit 110 of the MFP 100, the image reading unit 110 is operated to scan one original in a raster shape, and an image signal of, for example, 600 dpi and 8 bits is acquired ( Image information input process: Step S120).

  The input image signal is preprocessed by the data processing unit 115 and stored in the storage device 111 as image data (image data) for one page. Then, the CPU of the management PC 101 separates the area from the image data stored in the storage device 111 into a character / line drawing portion and a halftone image portion, and the character portion is further divided into blocks in a block. Or, it is divided into a table and a figure made up of lines and segmented. On the other hand, the image portion expressed in halftone is divided into independent objects for each so-called block such as an image portion or background portion of a block separated into rectangles (block selection process: step S121).

  At this time, the object corresponding to the two-dimensional barcode or URL recorded as additional information in the document image is detected, and if it is a two-dimensional barcode, the mark is decoded. Character recognition is performed using an OCR (Optical Character Reader) or the like (step S122). Thus, pointer information relating to the address of the storage destination server or the like in which the original electronic file of the document image is stored is detected (step S123). In addition to the above, the means for adding pointer information uses a one-dimensional barcode, a method using a digital watermark that is not directly visualized, such as a method of embedding information in a space between characters and a method of embedding in a halftone image. Also good.

  Next, it is determined whether the pointer information has been detected (step S124). If the pointer information has been detected (Yes), the address of the server or the like in which the original electronic file indicated by the detected pointer is stored. Accordingly, the server or the like connected to the network 104 or LAN 107 shown in FIG. 1 is searched (step S125). In this embodiment, the electronic file (image data) is stored in the hard disk in the client PC 102 shown in FIG. 1 or the database 105a in the document management server 106a, b connected to the LAN 107, 108 in the office 10, 20. , B, or the storage device 111 included in the MFP 100 itself, and the storage device is searched according to the server address information (pointer information) obtained in step S123.

  If a specific server is not found in step S125 (No), or if pointer information cannot be detected in step S124 (that is, if the server address indicated by the pointer information does not exist), management is performed for all servers. The file search process in step S128 and subsequent steps is performed based on the image data input for the data (in this embodiment, all data stored in the databases 105a and 105b and the storage device 111 in this embodiment) (step S128). S126). On the other hand, if a server is found from the server address based on the pointer information in step S125 (Yes), the search processing from step S128 onward is performed on the server indicated by the pointer (step S127).

  Step S128 is a so-called document search processing routine. First, words are extracted from the result of the OCR performed on each character block in step S122, and a so-called layout search is performed from the full text search or the array of each object and the attribute of each object. As a result of the search, if an electronic file with a high degree of similarity is found from the search target server, the thumbnail is displayed with the electronic file as a candidate, and if the operator needs to be selected from a plurality, the operator's A file is specified by an input operation (step S129). If the candidate is one file, step S129 may be bypassed and the process may be automatically branched to step S130 and step S131 to notify the user of the storage address of the candidate file. On the other hand, if the electronic file is not found in the search process in step S128 (No in step S130), the process branches to step S133.

  In step S133, the input image data is vectorized to convert the image data into vector data. For example, when the image data is character information or the like, it is converted into vector information by encoding by OCR or by outline by contour tracing. Also, when the input image data is a figure or the like, it is converted into vector information as vector data. However, when the input image data is a natural image or the like, it is handled as image data as it is. Thereby, when the electronic file is not found in step S130, the image data is newly generated as an original electronic file. That is, for the input image data, the data after the vectorization processing in step S133 (however, in the case of a natural image or the like, the image data remains) is stored as a new electronic file.

  Next, the vectorization process in step S133 will be described. The character size, style, font, and the like are further recognized for the character block subjected to OCR in step S122, and converted into font data that is visually faithful to the character obtained by scanning the document. On the other hand, tables and graphic blocks composed of lines are outlined. The image block is processed as image data as an individual JPEG file or the like as described above.

  Further, these vectorization processes are performed for each object divided in the block selection process (step S121), and further, layout information of each object is stored, and application data that can be edited by a general application (such as document creation software). (Step S134), and then stored in the storage device 111 or the like as an electronic file (step S136). Further, as a general-purpose file format, for example, it may be converted into an RTF (Rich Text Format) format.

  In the present embodiment, after the conversion processing to application data in step S134, the address of the server or the like of the storage location of the electronic file is added as “pointer information (address information)” to the electronic file to be saved. (Step S135). By doing in this way, the server (for example, storage device 111 etc.) where the preserved electronic file is stored becomes a search object after the next time. If the pointer information is added using a two-dimensional bar code or the like when printing an electronic file, the document with the pointer information added is read and analyzed after the next time. The server that stores the original electronic file can be easily accessed.

  By the above procedure, an original electronic file or an electronic file having vector information very close to the original electronic file can be easily searched and acquired. In other words, in the present embodiment, the electronic file data is vectorized data and is stored in an editable format, and is not only acquired as a single piece of image data as in the past, but also after acquisition. The data can be directly processed and reused, or stored, transmitted, and reprinted.

  These processes are not only able to be reused and re-edited compared to the case of simply handling image data directly, but also reduce the amount of information, increase storage efficiency, reduce transmission time, When recording and displaying, it is very advantageous as high-quality data.

  Hereinafter, each processing block will be described in detail.

[Block Selection (BS) Process (Step S121)]
FIG. 4 is a diagram illustrating a state in which one image data read by the block selection process is attributed and divided into a plurality of blocks. That is, in the block selection process, the image data of one page read in step S120 indicated by reference numeral 41 is recognized as a block for each object as indicated by reference numeral 42. ), Line (line), table (table), and the like, and is divided into areas (blocks) having different attributes.

  An embodiment of the block selection process will be described below.

  First, the input image is binarized into black and white, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. Then, for the black pixel block having a large area, the white pixel block is similarly extracted by tracing the outline of the white pixel inside. Further, a black pixel block is recursively extracted from the inside of a white pixel block having a certain area or more.

  The blocks of black pixels (white pixels) thus obtained are classified according to size, shape, etc., and are classified into areas having different attributes. For example, a pixel block corresponding to a character having an aspect ratio close to 1 and having a constant size is defined as a character block, and a portion where adjacent characters can be grouped in a well-aligned manner is defined as a character region. In addition, the flat pixel block is a line region, the area occupied by the black pixel block that is more than a certain size and includes square white pixel blocks in a well-aligned manner is a table region, and an area where irregular pixel blocks are scattered A picture area and a pixel block having an arbitrary shape other than that are set as a picture area.

  FIG. 5 is a diagram illustrating an example of block information for each block obtained by the block selection process. The information for each block shown in FIG. 5 is used as information for vectorization and search processing described later.

[Detection of pointer information from image data]
First, the OCR / OMR process for extracting the storage position of the electronic file shown in step S122 from the read image data will be described. FIG. 6 is a flowchart for explaining a procedure for decoding a two-dimensional barcode (QR code symbol) added to a document image and outputting a data character string. FIG. 7 is a diagram illustrating an example of a document 310 to which a two-dimensional barcode is added.

  First, image data obtained by reading a document 310 stored in a page memory in the data processing device 115 is scanned by an internal CPU, and a predetermined two-dimensional barcode symbol 311 is scanned based on the result of the block selection process described above. Detect position. The QR code position detection pattern includes the same position detection element patterns 311a to 311c arranged at three of the four corners of the symbol (step S300).

  Next, the format information adjacent to the position detection pattern is restored, and the error correction level and mask pattern applied to the symbol are obtained (step S301). Further, after determining the symbol model number (step S302), the mask process is canceled by performing an XOR operation on the encoded area bit pattern using the mask pattern obtained from the format information (step S303).

  Then, according to the arrangement rule corresponding to the model, the symbol character is read, and the message data and the error correction code word are restored (step S304). Next, it is detected whether there is an error in the restored code (step S305). As a result, when an error is detected (Yes), the error is corrected (step S306). Then, the data code word is divided into segments based on the mode indicator and the character number indicator from the error-corrected data (step S307). Finally, the data character is decoded based on the specification mode, and the result is output (step S308). If no error is detected in step S305 (No), the process proceeds to step S307.

  Here, the data incorporated in the two-dimensional barcode represents server address information (pointer information) in which the corresponding electronic file is stored. For example, the file server name and the IP address indicating the server address correspond to the data. It consists of path information consisting of a URL or the like.

  In the present embodiment, as described above, the document 310 to which the pointer information is given using the two-dimensional barcode has been described as an example. However, when the pointer information is directly recorded as a character string, a predetermined rule is used. Server address information of the server where the original electronic file is stored directly by detecting the block of the character string according to the block selection process described above and recognizing each character of the character string indicating the pointer information. It is possible to obtain

  Further, the character string of the character block 312 or the character block 313 of the document 310 in FIG. 7 is modulated such that the space between adjacent characters is difficult to visually recognize, and watermark information using the character space is embedded. Even pointer information can be given. For such watermark information, pointer information can be obtained by detecting an interval between characters when performing character recognition processing described later. It is also possible to add pointer information as a digital watermark in the natural image block 314.

[Server search processing using pointer information]
Next, the search processing of the server storing the electronic file from the pointer information performed in the processing after step S123 described with reference to FIG. 3 will be described in detail. FIG. 8 is a flowchart for explaining a processing procedure for searching for a server storing an electronic file from detected pointer information.

  First, based on the server address included in the pointer information, the file server in which the electronic file is stored is specified (step S400). Here, the file server refers to the client PC 102, the document management servers 106a and 106b including the databases 105a and 105b, and the MFP 100 itself including the storage device 111. The address is path information including a URL and a server name.

  Then, after specifying the file server, the file server is requested to prepare for searching the server (file server) indicated by the pointer information in step S127 described with reference to FIG. 3 (step S401). The file server searches for the corresponding electronic file according to the file search process in step S128 (step S402). Then, it is determined whether or not an electronic file exists (step S403).

  As a result, if the electronic file does not exist (No), the MFP 100 is notified of the fact and the process is terminated. On the other hand, if the electronic file exists (Yes), as described above with reference to FIG. 3, candidate display is performed to perform the processing from step S129 to step S132, and then the address of the corresponding electronic file is notified. At the same time, the electronic file is transferred to the user (ie, MFP 100) (step S408).

[Server search processing based on pointer information including file access right]
Next, an embodiment that considers file access rights will be described. Among the document files that are normally handled, there are documents in which it is desirable to restrict reuse by a third party. In the search process using FIG. 8 described above, it is assumed that all electronic files stored in the file server can be freely accessed, and the entire file or a part of the objects can be reused. Explained. Hereinafter, a case will be described in which when an electronic file is searched from pointer information, the access right is restricted for the electronic file specified from the search result.

  FIG. 9 is a flowchart for explaining a processing procedure for searching for a server storing an electronic file from pointer information including a file access right. In FIG. 9, steps S400 to S403 are the same as the steps in FIG. If an electronic file is specified in step S403 (Yes), the file server checks access right information of the file (step S404). As a result, if there is an access restriction (Yes), the MFP 100 is requested to transmit a password (step S405).

  Then, MFP 100 prompts the operator to input a password, and transmits the input password to the file server (step S406). The file server collates the transmitted password (step S407). If the passwords match (Yes), the file server notifies the address of the electronic file as described with reference to FIG. Is acquired, the electronic file is transferred to the MFP 100 (step S408).

  The authentication method for controlling the access right is not limited to the password method shown in steps S405 and S406. For example, generally used biometric authentication such as fingerprint authentication or card authentication is used. Any authentication means can be used.

    On the other hand, if the file cannot be specified from the file server in step S403, the vectorization process described in step S133 of FIG. 3 can be restricted. That is, when the presence of a restriction on the access right to the original electronic file is detected from the image data obtained by scanning the paper document, the vectorization process is performed only when the authentication confirmation is obtained, Limit the use of highly confidential documents.

[File search processing]
Next, details of the file search process shown in step S128 of FIG. 3 will be described with reference to FIGS. Here, it is assumed that each block and input file extracted as a result of the OCR / OMR process in step S122 includes information (block information and input file information) shown in FIG. In the present embodiment, as information contents, as shown in FIG. 5, attributes, coordinate positions, width and height sizes, and presence / absence of OCR information are used.

  Attributes are further classified into characters, lines, photographs, pictures, tables, and the like. In FIG. 5, for simplification of description, the blocks are arranged in the order of increasing coordinates X (for example, X1 <X2 <X3 <X4 <X5 <X6) in block 1, block 2, block 3, block 4, and block 5. , Block 6. The total number of blocks is the total number of blocks in the input file, and the total number of blocks in the example shown in FIG. Hereinafter, a procedure for performing layout search of an electronic file similar to an input image file from the database using these pieces of information will be described. FIG. 10 is a flowchart for explaining a procedure for performing a layout search of an electronic file similar to an input image file from the database. It is assumed that the database file also includes information similar to the information shown in FIG. The flow of the flowchart of FIG. 10 sequentially compares the electronic file read from the input document with the electronic file in the database.

  First, initialization such as a similarity rate described later is performed to set an initial value (step S510). Next, the total number of blocks is compared (step S511). If true (Yes), the block information in the files is further compared sequentially (step S512). That is, it is checked whether the block number n of the database file is within the error ΔN range of the block number N of the input file. If it is within the error range, it is true (Yes), otherwise it is false (No). In step S512, if the block attributes of the input file and the database file are compared and matched, the process proceeds to comparison processing in step S513 and thereafter, and if not matched, the process proceeds to step S521.

  In block information comparison, attribute similarity, size similarity, and OCR similarity are calculated in steps S513, S515, and S518, respectively, and an overall similarity is calculated based on them in step S522. About the calculation method of each similarity rate, since a well-known technique can be used, description is abbreviate | omitted.

  In step S523, it is determined whether or not the total similarity is higher than a preset threshold value Th. If the total similarity is high (Yes), the electronic file is listed as a similar candidate and stored (step S524). In the figure, N, W, and H are the total number of blocks in the input file, each block width, each block height, and ΔN, ΔW, and ΔH are based on error information based on the block information of the input file. N, w, and h are the total number of blocks of the database file, each block width, and each block height. Note that position information (X, Y) may be compared during the size comparison in step S514.

  As described above, the database file stored as a candidate having a total similarity higher than the threshold Th as a result of the search is displayed as a thumbnail (step S129). Thereby, when the operator needs to be selected from a plurality of files, the file is specified by the operator's input operation.

[Vectorization processing]
Next, vectorization of the entire image shown in step S133 in FIG. 3 will be described in detail. First, character recognition processing is performed on each character for a character block.

《Character recognition processing》
In the character recognition process, an image cut out in character units from the input image data is recognized using a pattern matching method to obtain a corresponding character code. This recognition process recognizes the character type with the closest distance by comparing the observed feature vector obtained by converting the feature obtained from the character image into a numerical sequence of several tens of dimensions and the dictionary feature vector obtained for each character type in advance. The resulting process. There are various known methods for extracting feature vectors. For example, there is a method characterized by dividing a character into meshes, and using a mesh number-dimensional vector obtained by counting character lines in each mesh as line elements according to directions.

  When character recognition is performed on the character area extracted by the block selection process in step S121, first, horizontal writing or vertical writing is determined for the corresponding area. Next, a line is cut out in a corresponding direction, and then a character is cut out to obtain a character image. Horizontal / vertical writing can be determined by taking a horizontal / vertical projection of the pixel value in the corresponding area, and determining that the horizontal projection area is large when the horizontal projection variance is large, and vertical writing area when the vertical projection variance is large. . In the case of horizontal writing, the line is cut out using horizontal projection, and the characters are cut out from the projection in the vertical direction with respect to the cut out line. On the other hand, for vertical character areas, horizontal and vertical may be reversed. At this time, the character size can be detected.

《Font recognition processing》
Prepare multiple dictionary feature vectors for the number of character types used for character recognition for character shape types, that is, font types, and output the font type together with the character code at the time of matching. Can be recognized.

《Character vectorization》
Using the character code and font information obtained by the above-described character recognition processing and font recognition processing, the character portion information is converted into vector data using outline data prepared in advance. If the input document image is color, the color of each character is extracted from the color image and recorded together with vector data.

  Through the above processing, the image information belonging to the character block can be converted into vector data that is substantially faithful in shape, size, and color.

《Vectorization of parts other than characters》
In the block selection process in step S121, the contours of the extracted pixel blocks are converted into vector data for a region that is a drawing, line, or table region. Specifically, a point sequence of pixels forming an outline is divided by points regarded as corners, and each section is approximated by a partial straight line or curve. Here, the “corner” is a point where the curvature becomes a maximum.

  FIG. 11 is a diagram for explaining the point where the curvature becomes maximum. As shown in FIG. 11, when a string is drawn between k points left and right Pi-k to Pi + k with respect to an arbitrary point Pi, the distance between the string and Pi is obtained as a maximum point. . Further, let R be the chord length / arc length between Pi-k and Pi + k, and the point where the value of R is equal to or less than the threshold value can be regarded as a corner. Each section after being divided by the corners can be vectorized using a least square method for a straight line for a straight line and a cubic spline function for a curve.

  Further, when the target has an inner contour, it is similarly approximated by a partial straight line or a curve using the point sequence of the white pixel contour extracted by the block selection process.

  As described above, the outline of a figure having an arbitrary shape can be vectorized by using the contour line dividing approximation. If the input document is color, the figure color is extracted from the color image and recorded together with the vector data.

  FIG. 12 is a diagram for explaining an example of expressing as a line having a thickness when the outer contour is close to the inner contour or another outer contour. As shown in FIG. 12, when an outer contour is close to an inner contour or another outer contour in a certain section, two contour lines can be combined and expressed as a line having a thickness. Specifically, when a line is drawn from each point Pi of a certain contour to a point Qi that is the shortest distance on another contour, and each distance PQi is on average less than or equal to a certain length, the interval of interest has a PQi midpoint as a point sequence It is approximated by a straight line or a curve, and its thickness is the average value of PQi. A table ruled line, which is a line or a set of lines, can be efficiently expressed as a set of lines having the thicknesses as described above.

  For vectorization using character recognition processing for character blocks, as described above, the character having the closest distance from the dictionary as a result of the character recognition processing is used as the recognition result. Here, when this distance is greater than or equal to a predetermined value, it does not necessarily match the original character, and is often erroneously recognized as a character having a similar shape. Therefore, in the present invention, as described above, such characters are handled in the same manner as general line drawings, and the characters are outlined. In other words, even a character that has been erroneously recognized in the character recognition processing in the past can be vectorized by outline rendering that is visually faithful to the image data without being vectorized into an erroneous character. In the present invention, a block determined to be a photograph is not vectorized but is left as image data.

<< Figure Recognition >>
Here, as described above, a process for vectorizing an outline of a figure having an arbitrary shape and then grouping the vectorized dividing lines for each graphic object will be described.

  FIG. 13 is a flowchart for explaining a processing procedure until vector data is grouped for each graphic object. First, the start point and end point of each vector data are calculated (step S700). Next, a graphic element is detected using the start point and end point information of each vector (step S701). Here, the detection of a graphic element is to detect a closed graphic formed by a dividing line. In detection, the detection is performed by applying the principle that each vector constituting the closed shape has a vector connected to both ends thereof.

  Next, other graphic elements or dividing lines existing in the graphic element are grouped into one graphic object (step S702). When there is no other graphic element or dividing line in the graphic element, the graphic element is set as a graphic object.

  FIG. 14 is a flowchart for explaining a processing procedure for detecting a graphic element. First, unnecessary vectors that are not connected to both ends are removed from the vector data, and a closed graphic component vector is extracted (step S710). Next, from the closed figure constituent vectors, the starting point of the vector is set as the starting point, and the vectors are sequentially followed in the clockwise direction. Then, tracing is performed until returning to the start point, and all the passed vectors are grouped as closed figures constituting one figure element (step S711). At this time, all closed graphic constituent vectors inside the closed graphic are also grouped. Further, the same processing is repeated with the starting point of the vectors not yet grouped as the starting point. Finally, among the unnecessary vectors removed in step S710, those joined to the vector grouped as a closed graphic in step S711 are detected and grouped as one graphic element (step S712).

  With the above processing, a graphic block can be handled as an individual graphic object that can be reused individually.

[Conversion to application data]
FIG. 15 is a diagram illustrating a data structure of a file in an intermediate data format obtained as a result of converting image data for one page by block selection processing (step S121) and vectorization processing (step S133). The data format as shown in FIG. 15 is called a document analysis output format (DAOF). That is, FIG. 15 is a diagram showing a data structure of DAOF.

  In FIG. 15, reference numeral 791 denotes a header (header), which holds information relating to document image data to be processed. Reference numeral 792 denotes a layout description data section, which includes TEXT (character), TITLE (title), CAPTION (caption), LINEART (line drawing), PICTURE (natural image), FRAME (frame), TABLE (table), etc. The attribute information of each block recognized for each attribute and the rectangular address information thereof are held.

  Reference numeral 793 denotes a character recognition description data portion, which holds character recognition results obtained by character recognition of TEXT blocks such as TEXT, TITLE, and CAPTION. A table description data portion 794 stores details of the structure of the TABLE block. Reference numeral 795 denotes an image description data section, which cuts out image data of blocks such as PICTURE and LINEART from document image data and holds them.

  Such a DAOF is not only used as intermediate data, but may be stored as a file itself. However, in this file state, individual objects cannot be reused in a general document creation application. . Then, the process (step S134) for converting DAOF to application data will be described in detail.

  FIG. 16 is a flowchart for explaining a schematic procedure of the entire conversion process to application data. First, DAOF data is input (step S800). Next, a document structure tree that is the source of application data is generated (step S802). Then, based on the generated document structure tree, actual data in the DAOF is flowed to generate actual application data (step S804).

  FIG. 17 is a flowchart for explaining a detailed processing procedure of the document structure tree generation process (step S802). FIG. 18 is a diagram for explaining the outline of the document structure tree. As a basic rule of overall control, the processing flow is assumed to shift from a micro block (single block) to a macro block (an aggregate of blocks). In the following description, the block refers to the entire micro block and macro block.

  First, regrouping is performed on a block basis based on the relevance in the vertical direction (step S802a). Immediately after the start, the determination is made in units of micro blocks. Here, the relevance can be defined by the fact that the distance is close and the block width (height in the horizontal direction) is substantially the same. Information such as distance, width, and height is extracted with reference to DAOF.

  In FIG. 18, (a) shows the actual page structure, and (b) shows the document structure tree. As a result of the grouping in step S802a, T3, T4, and T5 are generated as one group V1, and T6 and T7 are generated as one group V2 as groups of the same hierarchy.

  Next, the presence / absence of a vertical separator is checked (step S802b). For example, the separator is physically an object having a line attribute in the DAOF. In addition, as a logical meaning, it is an element that explicitly divides a block in an application. If a separator is detected here, it is subdivided at the same level.

  Next, it is determined using the group length whether there are no more divisions (step S802c). For example, it is determined whether the vertical grouping length is the page height. As a result, when the group length in the vertical direction is the page height (Yes), the document structure tree generation ends. For example, in the case of the structure shown in FIG. 18, since there is no separator and the group height is not the page height, it is determined No and the process proceeds to step S802d.

  In step S802d, regrouping is performed on a block basis based on the relevance in the horizontal direction. However, even in this regrouping, the first time immediately after the start is determined in units of microblocks. The definition of the relevance and the determination information is the same as in the vertical direction. For example, in the case of the structure of FIG. 18, H1 is generated by T1 and T2, and H2 is generated by V1 and V2. H1 is one group above T1 and T2, and H2 is a group in the same hierarchy above one by V1 and V2. Generated.

  Next, the presence / absence of a horizontal separator is checked (step S802e). In FIG. 18, since there is S1, this is registered in the tree, and a hierarchy of H1, S1, and H2 is generated. Then, it is determined using the group length whether there are no more divisions (step S802f). For example, it is determined whether or not the horizontal grouping length is the page width. As a result, when the horizontal group length is the page width (Yes), the document structure tree generation ends. On the other hand, if the page width is not reached (No), the process returns to step S802b, and the relevance check in the vertical direction is repeated again at the next higher level. For example, in the case of the structure of FIG. 18, since the division width is the page width, the process ends here, and finally V0 of the highest hierarchy representing the entire page is added to the document structure tree.

  After the document structure tree is completed, application data is generated in step S804 based on the information. In the case of the structure of FIG. 18, it is specifically as follows.

  That is, since there are two blocks T1 and T2 in the horizontal direction, H1 has two columns, and after outputting the internal information of T1 (text or image of the character recognition result referring to DAOF), the column is changed, Internal information is output, and then S1 is output. Also, since H2 has two blocks V1 and V2 in the horizontal direction, it is output as two columns, V1 outputs its internal information in the order of T3, T4, and T5, and then the columns are changed, and T6 and T7 of V2 are changed. The internal information of is output. As described above, the conversion processing to application data can be performed.

[Add pointer information]
Next, the pointer information adding process shown in step S135 will be described in detail. When the document to be processed is specified by the search process, or when the original file can be reproduced by vectorization, when the document is recorded, pointer information is added at the time of recording on paper. The original file data can be easily acquired when performing various processes again using this document.

  FIG. 19 is a flowchart for explaining a procedure for encoding a data character string as pointer information using a two-dimensional bar code (QR code symbol: JIS X0510) 311 and adding it to an image.

  The data to be incorporated in the two-dimensional barcode represents server address information where the corresponding file is stored, and is constituted by path information including, for example, a file server name. Alternatively, it is configured with a URL of the corresponding server, an ID for managing the storage device 111 included in the databases 105a and 105b where the corresponding file is stored, or the MFP 100 itself.

  First, an input data string is analyzed in order to identify different characters to be encoded. Further, an error detection and error correction level is selected, and a minimum model number that can accommodate input data is selected (step S900). Next, the input data string is converted into a predetermined bit string, and an indicator indicating a data mode (numeric, alphanumeric, 8-bit byte, kanji, etc.) and a termination pattern are added as necessary. Further, the data is encoded by converting it into a predetermined bit code word (step S901).

  At this time, in order to perform error correction, the code word string is divided into a predetermined number of blocks according to the model number and the error correction level, an error correction code word is generated for each block, and is added after the data code word string ( Step S902). Further, the data code word of each block obtained in step S902 is connected, and the error correction code word of each block and, if necessary, the remainder code word are connected to construct a message (step S903).

  Next, the code word module is arranged in the matrix together with the position detection pattern, separation pattern, timing pattern, alignment pattern, and the like (step S904). Further, an optimal mask pattern is selected for the symbol coding region, and the mask processing pattern is converted into a module obtained in step S904 by an XOR operation (step S905). Finally, format information and model number information are generated in the module obtained in step S905 to complete a two-dimensional code symbol (step S906).

  The two-dimensional bar code in which the server address information is incorporated is a raster that can be recorded in the data processing apparatus 115 when an electronic file is formed as print data from the client PC 102 as a print image on a sheet of paper. After being converted to data, it is added to a predetermined location on the raster data to form an image. Here, the user who has distributed the image-formed paper reads the image reading unit 110 to appropriately detect the location of the server where the original electronic file is stored from the pointer information in step S123 described above. Can do.

  In addition to the two-dimensional barcode described in this embodiment, the means for adding additional information for the same purpose includes, for example, a method of adding pointer information directly to a document as a character string, a character string in a document, A method generally referred to as a digital watermark such as a method of embedding information by modulating the space between characters and a method of embedding in a halftone image in a document may be applied.

  As described above, according to the present embodiment, since the additional information amount to be added to the document as pointer information is only the address indicating the location of the server, the watermark, barcode, character, The size of a figure or the like is reduced, and image degradation that occurs when a watermark is added to a document can be reduced. Further, by performing a search based on the similarity of the originals, it is possible to expect an effect such that other documents related to the originals can be searched. Further, even when the corresponding document is not found, by performing vectorization and saving, it is possible to greatly enhance the reusability and editability of the input document.

  In the first embodiment described above, as described with reference to FIG. 3, the server is searched from the pointer information in step S125, and the file in the server corresponding to the server address obtained by the pointer information. In step S128 and below, a file search is performed, and if the corresponding file is not found as a result of the file search, vectorization is performed in step S133 and the subsequent steps to save the file.

  On the other hand, if an electronic file is not found during the process of step S130, the process proceeds to step S126, and a search is performed for all servers on the network. After that, if the search after step S128 is performed and no server is found after searching the server other than the server indicated by the pointer, the process branches to step S133 and is vectorized. Good.

  Although the embodiments have been described in detail above, the present invention can take an embodiment as, for example, a system, an apparatus, a method, a program, or a storage medium, and specifically includes a plurality of devices. The present invention may be applied to a system that is configured, or may be applied to an apparatus that includes a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the figure) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

1 is a block diagram illustrating a configuration of an image processing system according to a first embodiment of the present invention. 1 is a block diagram showing a configuration of an MFP 100 according to a first embodiment of the present invention. 3 is a flowchart for explaining a procedure of image processing by the image processing system according to the first embodiment of the present invention; It is a figure which shows a mode that the attribute of one sheet of image data read by the block selection process is determined and divided into a plurality of blocks. It is a figure shown about an example of the block information with respect to each block obtained by the block selection process. 6 is a flowchart for explaining a procedure of decoding a two-dimensional barcode (QR code symbol) added to a document image and outputting a data character string. 3 is a diagram illustrating an example of a document 310 to which a two-dimensional barcode is added. FIG. It is a flowchart for demonstrating the process sequence which searches the server in which the electronic file is stored from the detected pointer information. It is a flowchart for demonstrating the process sequence which searches the server in which the electronic file is stored from the pointer information containing a file access right. It is a flowchart for demonstrating the procedure of performing the layout search of the electronic file similar to an input image file from the database. It is a figure for demonstrating the point from which a curvature becomes the maximum. It is a figure for demonstrating the example expressed as a line with thickness when an outer contour is adjoining with an inner contour or another outer contour. It is a flowchart for demonstrating the process sequence until grouping vector data for every graphic object. It is a flowchart for demonstrating the process sequence which detects a graphic element. It is a figure which shows the data structure of the file of the intermediate data format obtained as a result of having converted the image data for one page by the block selection process and the vectorization process. It is a flowchart for demonstrating the schematic procedure of the whole conversion process to application data. It is a flowchart for demonstrating the detailed process sequence of a document structure tree production | generation process. It is a figure for demonstrating the outline | summary of a document structure tree. It is a flowchart for demonstrating the procedure which encodes the data character string as pointer information using a two-dimensional barcode, and adds it to an image.

Explanation of symbols

100 Digital MFP (MFP)
101 Management PC
102 Client PC
103a, 103b Proxy server 104 Network 105a, 105b Database 106a, 106b Document management server 107, 108, 109 LAN
110 Image Reading Unit 111 Storage Device 112 Forming Device 113 Input Device 114, 117 Network I / F
115 Data Processing Device 116 Display Device

Claims (8)

An image processing system for retrieving an electronic file corresponding to input image data from a plurality of electronic files held in any of a plurality of electronic file servers connected to a network, and acquiring the retrieved electronic file There,
Input means for inputting image data representing a document;
Extraction means for extracting additional information for identifying the electronic file server from the entered image data;
Recognition means for recognizing the entered characteristic of the image data,
Based on the recognized feature of the image data, a match or electronic file similar to the image data that has been entered, the first electronic file server corresponding to the additional information extracted by the extraction means A first search means for searching;
When the electronic file that matches or is similar to the input image data is not searched by the first search means, the electronic file that matches or is similar to the input image data is Second search means for searching in another electronic file server different from the electronic file server of
An acquisition means for acquiring the searched electronic file when an electronic file that matches or is similar to the input image data is searched by the first or second search means ;
When an electronic file that matches or is similar to the input image data is not searched by the first and second search means, the input image data is converted into an electronic file of a predetermined format. Conversion means for converting;
And holding means for holding the electronic file converted by the converting means together with additional information for specifying the electronic file server holding the electronic file in the electronic file server specified by the additional information. Image processing system. Said extracting means, the image processing according to claim 1, wherein the extracting watermark information, bar code information, the additional information added to the image data using one of a character information or graphic information system. It said input means, the image processing system according to claim 1 or 2, characterized in that inputs the image data representing the document by converting optically reads a document to a digital image signal. And password request means for requesting a password input,
Password determining means for determining whether or not the password input in response to the request from the password request means is correct ,
If the correct password is determined to have been inputted by the password determination means, the acquisition means, to claim 1, characterized in that to obtain the electronic file retrieved by the first or second search means 4. The image processing system according to any one of items 3 . The converting means includes
Character processing means for character encoding input said image data;
A vectorization means for converting the input is the image data into vector data,
Further comprising an image converting means for converting the input is the image data into a predetermined image format,
The image data that has been entered, in accordance with an attribute of the image data, the character processing means, the vectorization means, to claim 1, wherein the converting using at least one of said image converting means 5. The image processing system according to any one of 4 above. The recognition means is
The input is the image data, dividing means for dividing into a plurality of blocks based on the attribute of the image data,
Anda block number recognizing means for recognizing a number of blocks divided by the dividing means,
The first and second search means are:
And the number of blocks of the image data that has been entered, a first comparison means for comparing the number of blocks of the image data of the electronic file stored in the electronic file server,
As a result of comparison by the first comparison means, when the difference in the number of blocks is within a predetermined range, second comparison means for comparing the similarity of the attribute of each block for each compared block When,
Using the result of comparison by the second comparison means, a calculating means for calculating said image data input, the kind similarity score between the image data of the electronic file stored in the electronic file server,
If the previous SL such similarity score is equal to or larger than the reference threshold, be provided with a determination unit and the image data of the electronic file stored in the image data and the electronic file servers that are entered are identical or similar The image processing system according to claim 1, wherein: In an image processing system for searching for an electronic file corresponding to input image data from a plurality of electronic files held in any of a plurality of electronic file servers connected to a network, and acquiring the searched electronic file An image processing method to be executed ,
An input step in which the input means of the image processing system inputs image data representing a document;
Extraction means of the image processing system, an extraction step of extracting an additional information for the input has been the image data for specifying the electronic file server,
Recognition means of the image processing system, and recognizing step a characteristic of the image data that has been entered,
First search means of the image processing system, based on the feature of the image data that has been recognized, the identical or similar to that electronic file to the image data that has been entered, extracted in the extraction step the A first search step for searching in a first electronic file server corresponding to the additional information;
The second search means of the image processing system matches the input image data when an electronic file that matches or is similar to the input image data is not searched in the first search step. Or a second search step of searching for a similar electronic file in another electronic file server different from the first electronic file server;
The acquisition means of the image processing system acquires the searched electronic file when an electronic file that matches or is similar to the input image data is searched in the first or second search step. Acquisition process;
When the conversion means of the image processing system finds that the electronic file that matches or is similar to the input image data is not searched in the first and second search steps, the input image data is A conversion process to convert the electronic file into a predetermined format;
The holding means of the image processing system holds the electronic file converted in the conversion step in the electronic file server specified by the additional information together with additional information for specifying the electronic file server holding the electronic file. An image processing method comprising: a holding step . An image processing system that retrieves an electronic file corresponding to input image data from a plurality of electronic files held in any of a plurality of electronic file servers connected to a network, and acquires the retrieved electronic file On the included computer,
An input process for inputting image data representing a document;
An extraction step of extracting additional information for specifying an electronic file server from the input image data;
A recognition step for recognizing features of the input image data;
Based on the recognized characteristics of the image data, an electronic file that matches or is similar to the input image data is searched for in the first electronic file server corresponding to the additional information extracted in the extraction step. A first search step;
If an electronic file that matches or is similar to the input image data is not searched in the first search step, an electronic file that matches or is similar to the input image data is A second search step for searching in another electronic file server different from the electronic file server of
An acquisition step of acquiring the searched electronic file when an electronic file that matches or is similar to the input image data is searched in the first or second search step;
When the electronic file that matches or is similar to the input image data is not searched in the first and second search steps, the input image data is converted into an electronic file of a predetermined format. A conversion process to convert;
A holding step of holding the electronic file converted in the conversion step together with additional information for specifying the electronic file server holding the electronic file in the electronic file server specified by the additional information . program.

Images