JP2006031181A - Image retrieval system, method, storage medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieval system which precisely retrieves an original electronic document, when the original electronic document is retrieved from printed matter, even if the printed matter is a document comprising a plurality of pages or having addition, removal, alteration or replacement of pages, and also provide a method, a recording medium and a program therefor. <P>SOLUTION: Image data groups including a plurality of document images each comprising a plurality of pages are stored in a client PC 102 or the like in association with their feature quantities. An MFP 100 electronically reads printed matter having a plurality of pages in units of pages, converts them to electronic data, and extracts the feature quantities. Then, the feature quantities of the electronic data are compared, for each page, with the feature quantity of each document image included in the image data groups to compute similarity, and a document image corresponding to the electronic data is retrieved, as the candidate image data, from the image data groups based on the similarity. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image search apparatus, method, storage medium, and program, and in particular, original electronic data that can be used for printing, distribution, storage, editing, and the like from a paper document read by an image input device such as a copying machine. The present invention relates to an image search device, a method, a storage medium, and a program.

  In recent years, the use of electronic documents has become popular as paperless offices have been advocated. On the other hand, paper documents have advantages not found in electronic documents, such as less eye fatigue, easy overview, and writing. For this reason, many offices are currently using both electronic documents and paper documents by taking advantage of the merits of electronic documents and paper documents. Therefore, in order to reduce the difference in handling between paper documents and electronic documents in such a situation, an environment that facilitates mutual transition from electronic documents to paper documents or from paper documents to electronic documents is required. It has been.

  Transition from an electronic document to a paper document can generally be easily achieved by printing the electronic document on paper using a printer. On the other hand, a method of shifting from a paper document to an electronic document is generally performed by reading the paper document using a scanner and digitizing it as image data.

  However, if the original electronic document corresponding to the paper document already exists, the original electronic document corresponding to the paper document is used to convert the paper document to the electronic document, rather than transferring the paper document to the electronic document again. It is better to move to.

  As a method for realizing this, for example, an electronic document is converted into a raster image, stored in association with the electronic document, a raster image obtained by scanning a paper document, and a raster image obtained by converting the electronic document, A method is known that makes it possible to search an original electronic document from a raster image of a paper document by comparing the similarity of the feature amounts (see, for example, Patent Document 1).

  For example, materials used for presentations at meetings are generally distributed to attendees as paper documents, but the original electronic document can be extracted from the distributed paper documents using the method described above. become. Therefore, meeting attendees can use the distributed paper document to edit the corresponding original electronic document or distribute the paper document obtained from the electronic document to those who are not attending the meeting. It becomes possible.

  Further, when the original document is a paper document, raster image data obtained by scanning the paper document is stored as original raster image data, and the raster document obtained by scanning the distributed paper document is stored. A method for searching an original document by comparing image data with original raster image data is known (see, for example, Patent Document 2).

  At present, original documents are often obtained as paper documents from outside the company, such as product brochures. In such a case, if the original paper document is copied for generations, the image quality deteriorates. Thus, the raster image data of the original document is retained in this way, and the original raster image data is stored. It makes sense to search for.

Further, when the original document is searched by comparing the raster image data obtained by scanning the paper document with the original raster image data, the raster image data obtained by scanning each page and the original image data are searched. A method is known in which an original document is searched by performing comparison without scanning all paper documents (see, for example, Patent Document 3).
JP 2001-256256 A JP-A-5-37748 Japanese Patent Application No. 2003-143493

  However, general paper documents include more documents composed of a plurality of pages than documents composed of only one page. On the other hand, according to the technique described in Patent Document 1, in order to use the feature values of raster images for all pages as a comparison condition, there is a memory that temporarily stores and holds raster images of scanned paper documents. There is a problem that the memory capacity increases because all pages are required.

  In the technique described in Patent Document 1, in order to specify an original electronic document without using the feature values of raster images for all pages as a comparison condition, it is necessary to compare all the pages of each original document as a comparison destination image. Raster images need to be compared, which causes a problem that processing time increases.

  Further, according to the technique described in Patent Document 2, a method of reducing the memory to be used by using only a specific page such as the first page as a comparison condition is shown. However, there are many similar documents used in the office, and there are cases where the original document cannot be specified properly only by using only a specific page such as the first page as a comparison condition. There was a problem that it was not possible.

  Further, according to the technique described in Patent Document 3, a method is shown in which the memory capacity required for processing is not increased by narrowing down candidates when sequentially comparing from the first page. However, paper documents tend to get while being handled and fall apart. For example, some pages in a paper document may be lost, the order may be changed, or pages from other documents may be mixed. Therefore, if such a page exists in a scanned paper document, the technique described in Patent Document 3 causes the original document that should originally be a candidate to be removed from the candidate during the search, and the original document is appropriately searched. There was a problem that it could not be done.

  Furthermore, since the scanning of the paper document is completed when the original document is specified, the addition, omission, change, or replacement of the paper document existing after that page cannot be detected, and the difference from the original document can be detected. There was a problem that it could not be properly recognized.

  The present invention has been made in consideration of such circumstances, and when searching for an original electronic document from a printed matter, the case where the printed matter is a document composed of a plurality of pages, or a page of the printed matter. An object of the present invention is to provide an image search apparatus, method, recording medium, and program capable of accurately searching an original electronic document even when there are additions, omissions, changes, and replacements.

In order to solve the above problems, an image search device according to the present invention provides:
Storage means for storing a plurality of document images composed of a plurality of pages in association with respective feature amounts;
Reading means for electronically reading a plurality of pages of printed matter on a page basis and converting it into electronic data;
Extraction means for extracting feature quantities of the electronic data;
The feature amount of the electronic data and the feature amount of each document image stored in the storage unit are compared for each page to calculate a similarity, and based on the similarity, the plurality of document images And a search means for searching for a document image corresponding to the electronic data as candidate data.

In addition, in order to solve the above problem, an image search method according to the present invention includes:
A storage step of storing a plurality of document images of a plurality of pages in a storage device in association with respective feature amounts;
A reading step of electronically reading a plurality of pages of printed matter in units of pages and converting it into electronic data;
An extraction step of extracting a feature amount of the electronic data;
The feature amount of the electronic data and the feature amount of each document image stored in the storage unit are compared for each page to calculate a similarity, and based on the similarity, the plurality of document images And a search step of searching for a document image corresponding to electronic data as candidate data.

Furthermore, in order to solve the above-described problem, a program according to the present invention provides:
On the computer,
A storage procedure for storing a plurality of document images composed of a plurality of pages in a storage device in association with respective feature amounts;
A reading procedure for electronically reading a multi-page printed matter page by page and converting it into electronic data;
An extraction procedure for extracting feature quantities of the electronic data;
The feature amount of the electronic data and the feature amount of each document image stored in the storage unit are compared for each page to calculate a similarity, and based on the similarity, the plurality of document images And a search procedure for searching for a document image corresponding to electronic data as candidate data.

  Furthermore, the present invention is a computer-readable storage medium storing the program.

  According to the present invention, when searching for an electronic document from a printed material, even if the printed material is a document composed of a plurality of pages, or there are additions, omissions, changes, and replacements of pages in the printed material. The electronic document can be searched with high accuracy.

  In addition, according to the present invention, when an electronic document is searched from a printed material, if a page is added, missing, changed, or replaced on the printed material, the state can be suitably detected.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of an image processing system according to the first embodiment of the present invention. As shown in FIG. 1, the image processing system according to the first embodiment is realized in an environment in which an office 10 and an office 20 are connected by a network 104 such as the Internet.

  The LAN 107 constructed in the office 10 includes an MFP (Multi-Function Peripheral) 100 that is a multi-function machine that realizes a plurality of functions, a management PC 101 that controls the MFP 100, a client PC 102, a document management server 106a, and a database 105a thereof. And a proxy server 103a. A document management server 106b and its database 105b are connected to the LAN 108 built in the office 20.

  The LAN 107 in the office 10 and the LAN 108 in the office 20 are connected to the network 104 via proxy servers 103a and 103b in both offices.

  In particular, MFP 100 includes an image reading unit that electronically reads a paper document and an image processing unit that performs image processing on an image signal obtained from the image reading unit. Then, the MFP 100 can transmit this image signal to the management PC 101 via the LAN 109.

  The management PC 101 can be realized by a normal PC and includes various components such as an image storage unit, an image processing unit, a display unit, and an input unit. Note that some of the components may be integrated with the MFP 100.

  The network 104 is typically the Internet, a LAN, a WAN, a telephone line, a dedicated digital line, ATM, a frame relay line, a communication satellite line, a cable TV line, a data broadcasting wireless line, or the like. It is a so-called communication network realized by a combination, and it is sufficient if data can be transmitted and received.

  In addition, various terminals such as the management PC 101, client PC 102, and document management server are standard components (for example, CPU, RAM, ROM, hard disk, external storage device, network interface, display, keyboard) mounted on a general-purpose computer. , Mouse etc.).

  Next, a detailed configuration of the MFP 100 will be described with reference to FIG. FIG. 2 is a block diagram showing a detailed configuration of the MFP 100 according to the first embodiment of the present invention.

  In FIG. 2, an image reading unit 110 including a document table and an auto document feeder (ADF) irradiates a bundle or one document image with a light source (not shown), and reflects a document reflection image with a lens as a solid-state image sensor. An image is formed above, and a raster-like image reading signal is obtained from the solid-state imaging device as a raster image having a predetermined density (for example, 600 DPI).

  In the first embodiment, a paper document is described as an example of a printed matter read by the image reading unit 110. However, a recording medium other than paper (for example, an OHP sheet, a transparent original such as a film, a cloth, or the like). The printed material may be a reading target of the image reading unit 110.

  The MFP 100 also has a copying function for printing an image corresponding to the image reading signal on a recording medium by the printing unit 112. When copying one original image, the MFP 100 performs image processing on the image reading signal by the data processing unit 115 to generate a recording signal, and causes the printing unit 112 to print it on a recording medium. On the other hand, when copying a plurality of document images, the MFP 100 once stores and holds one recording signal in the storage unit 111 and then sequentially outputs the recording signal to the printing unit 112 to print it on a recording medium.

  On the other hand, the recording signal output from the client PC 102 is received by the data processing unit 115 via the LAN 107 and the network IF 114, and the data processing unit 115 converts the recording signal into raster data that can be recorded by the printing unit 112. The printing unit 112 prints on the recording medium.

  An operator's instruction to the MFP 100 is performed from a key operation unit equipped in the MFP 100 and an input unit 113 including a keyboard and a mouse connected to the management PC 101. These series of operations are performed by a control unit (in the data processing unit 115). (Not shown). Further, the display of the operation input status and the image data being processed is performed on the display unit 116.

  The storage unit 111 is also controlled by the management PC 101, and data transmission / reception and control between the MFP 100 and the management PC 101 are performed via the network IF 117 and the LAN 109.

  The database 118 is a document image of a paper document read by the image reading unit 110, information about the document image, and databases 105a and b of the document management servers 106a and 106b in the hard disk in the client PC 102 or in the offices 10 and 20. Registers and manages information related to image images in the electronic document.

  In MFP 100, display unit 116 and input unit 113 realize a user interface that provides a user with various operations and displays for executing various processes described below.

  The processing executed by the image processing system according to the present embodiment can be broadly divided into a registration processing for registering image data and a search processing for searching for desired image data. In this embodiment, in order to improve the processing efficiency of the entire image processing system, various processes described below are executed by being distributed to various terminals constituting the image processing system. It may be executed on a terminal (for example, MFP 100).

  First, details of registration processing in the image processing system according to the first embodiment will be described.

[Overview of registration process]
As registration methods of image data to be registered, an electronic file created by a document creation application or the like is registered as an original document, and an electronic file obtained by scanning a paper document is registered as an original document.

(A) Outline of Electronic Document Registration Process First, an outline of a registration process for registering an electronic document as an original document in the database 118 in the MFP 100 under the control of the management PC 101 will be described with reference to FIG. 3A. FIG. 3A is a flowchart for explaining the electronic document registration process in the image processing system according to the first embodiment of the present invention. The electronic document to be registered is stored in either the hard disk in the client PC 102 or the databases 105a and 105b of the document management servers 106a and 106b in the offices 10 and 20.

  Therefore, first, the management PC 101 acquires electronic documents one by one according to the order determined from these storage sources, and stores them in the storage unit 111 via the network I / F 114 (step S3101). Next, it is determined whether or not all electronic documents have been acquired (step S3102). As a result, when the acquisition of the electronic document ends and no new electronic document is acquired (Yes), the registration process ends. On the other hand, if the acquisition of the electronic document has not been completed (No), the process proceeds to step S3103.

  In step S3103, a unique document ID is issued to the acquired electronic document, and is registered as address information in the database 118 in association with an address indicating the storage location of the electronic document (electronic file). Here, the document ID is an ID for identifying an electronic document stored in the database 118, and an unused ID is issued each time one electronic document is registered. The address is full path information indicating a storage destination of an electronic document including a URL, a server name, a host name, a shared name, a directory, and a file name. FIG. 4 is a diagram illustrating an example of address information including a document ID and an address registered in the first embodiment.

  Next, the acquired electronic document is converted into a raster image page by page by the data processing unit 115 to generate an image, and is stored in the storage unit 111 (step S3104). Thereafter, it is determined whether or not the conversion of each page of the electronic document into a raster image has been completed (step S3105). As a result, if it is determined that the conversion has been completed (Yes), the process returns to step S3101 to acquire the next electronic document. On the other hand, if it is determined that the conversion has not ended (No), the process proceeds to step S3106.

  In step S3106, block selection (BS) processing is performed on the raster image for one page stored in the storage unit 111. This process is executed under the control of the management PC 101.

  More specifically, the CPU of the management PC 101 divides the raster image to be processed stored in the storage unit 111 into a character / line image portion and a halftone image portion. Next, the character / line drawing portion is further divided for each block grouped as a lump in the paragraph, or for each table or figure composed of lines. On the other hand, the halftone image part is divided into blocks such as an image part of a block separated into a rectangle and a background part.

  Next, the page number of the page image to be processed in the block selection process in step S3106, the block ID for specifying each block in the page, is issued, the attribute (image, character, etc.), size, and original document in each block. Is stored (registered) in the database 118 as block information in association with each block (step S3107). FIG. 5 is a diagram illustrating an example of block information registered in the database 118 in the first embodiment.

  Next, the data processing unit 115 performs feature amount information extraction processing for extracting feature amount information of each block according to the type of each block (step S3108). In particular, for character blocks, a character code is extracted by performing OCR processing, and this is used as a character feature amount. For image blocks, image feature values relating to colors are extracted.

  Next, the feature amounts corresponding to the respective blocks are collected in units of pages, and stored as feature amount information in the database 118 in association with the document ID, page number, and block ID (step S3109). 6 and 7 are diagrams illustrating examples of color feature amount information and character feature amounts stored in the database 118 in the first embodiment, respectively.

  Thereafter, the process returns to step S3104 to process the next page.

(B) Outline of Paper Document Registration Process Next, an outline of a registration process for registering a paper document as an original document in the database 118 in the MFP 100 under the control of the management PC 101 will be described with reference to FIG. 3B. FIG. 3B is a flowchart for explaining a paper document registration process according to the first embodiment of this invention. This process starts when a paper document to be registered is set in the ADF of the image reading unit 110 and the registration button of the input unit 113 is operated.

  First, when the operator instructs the registration process of the original document from the paper document from the input unit 113, the management PC 101 issues a new document ID and determines the storage location of the electronic data of the paper document in the database 118. The document ID is registered as address information in the database 118 in association with the address indicating the storage location of the electronic data of the paper document (step S3201). Since the storage location of the electronic data of the paper document corresponds to a plurality of pages, a new directory is created, and an address indicating this directory is stored in the address information.

  Next, in accordance with an instruction from the management PC 101, the image reading unit 110 of the MFP 100 is operated, and a raster image obtained by raster scanning a paper document page by page is read into the storage unit 111 (step S3202). At this time, raster images for one page are sequentially stored as electronic files in the directory created in step S3201. The file name of the electronic file includes a page number, and the page of the file can be identified only by referring to the file name.

  The file format of the electronic data when the original document is a paper document is, for example, the BMP format, but is not limited to this, and is a file format that can store color information (for example, GIF, JPEG, etc.) may be used. On the other hand, when the original document is electronic data, the file format of the electronic data is a file created by an application that created the electronic data (for example, MS-Word of Microsoft Corporation, Acrobat of Adobe Systems Inc., etc.). Format (* .doc, * .pdf, etc.).

  Next, it is determined whether all pages have been read from the image reading unit 110 (step S3203). As a result, when the reading is finished (Yes), this processing is finished. On the other hand, if the reading has not been completed (No), the process proceeds to step S3204.

  In step S 3204, block selection (BS) processing is performed on the raster image for one page stored in the storage unit 111. Since this process is the same as the process of step S3106, detailed description thereof is omitted.

  Next, block information is generated based on each block in the page image to be processed in step S3204 and stored in the database 118 (step S3205). Since this process is the same as the process of step S3107, detailed description thereof is omitted.

  Next, the data processing unit 115 performs feature amount information extraction processing for extracting feature amount information of each block according to the type of each block (step S3206). Note that this process is the same as the process of step S3108, and thus detailed description thereof is omitted.

  Next, the feature amounts corresponding to the respective blocks are collected in units of pages, and stored as feature amount information in the database 118 in association with the document ID, page number, and block ID (step S3207). Since this process is the same as that in step S3109, detailed description thereof is omitted.

  Thereafter, the process returns to step S3202, and the next page is processed.

(C) Outline of Search Process Next, an outline of a search process for searching for a corresponding original electronic file or image data based on a paper document and performing printing or the like will be described with reference to FIG. 3C. FIG. 3C is a flowchart for explaining search processing according to the first embodiment of the present invention.

  First, the image reading unit 110 of the MFP 100 is operated to read a raster image obtained by raster scanning one page of paper document into the storage unit 111 (step S3301). Next, it is determined whether all pages have been read from the image reading unit 110 (step S3302). As a result, when the reading is completed (Yes), the process proceeds to step S3307. On the other hand, if the reading has not been completed (No), the process proceeds to step S3303.

  In step S3303, block selection (BS) processing is performed on the raster image for one page stored in the storage unit 111. Since this process is the same as that in step S3106, detailed description thereof is omitted. However, the attribute, size, and position of each block generated by this block selection process are only temporarily stored, and block information for the paper document need not be stored in the storage unit 111.

  Next, feature amount information extraction processing for extracting feature amount information of each block is performed (step S3304). Since this process is the same as the process of step S3108, detailed description thereof is omitted. However, the feature amount of each block generated by this process is only temporarily stored, and it is not necessary to store the feature amount information for the paper document in the storage unit 111.

  Next, feature amount information for the page image (comparison source image) of the input paper document and feature amounts for all pages of the original document (comparison destination image) corresponding to the document ID of the electronic document stored in the database 118 The information is compared (step S3305).

  Next, the similarity is calculated from the comparison result of step S3305, and the document ID and page of the electronic document whose similarity is higher than the first threshold are added to the search result list (step S3306). Here, the search result list is a list for managing search results. The page number of the input paper document, the document ID of the candidate original document, the page number of the page of the original document, the similarity, etc. And is stored in the storage unit 111. FIG. 8 is a diagram illustrating a configuration example of a search result list according to the first embodiment.

  Then, the process returns to step S3301 to read the paper document of the next page.

  On the other hand, in step S3302, when the reading of the paper documents of all pages from the image reading unit 110 is completed (Yes), the process proceeds to step S3307. Here, the search result list is evaluated to create a document candidate list. Note that the document candidate list includes the document ID of the original document as a candidate and the total similarity, and is stored in the storage unit 111. When the similarity total is below the second threshold, the similarity is deleted from the document candidate list. FIG. 9 is a diagram illustrating a configuration example of a document candidate list in the first embodiment.

  Next, it is determined whether or not the number of candidates in the document candidate list is one (step S3308). As a result, when the number of candidates is one (Yes), the process proceeds to step S3310, and when the number of candidates is two or more (No), the process proceeds to step S3309.

  In step S3309, a user interface including thumbnail images of documents registered in the document candidate list is realized by the display unit 116 and the input unit 113, and original document candidates are displayed / selected by this user interface. In particular, this selection is realized by displaying thumbnail images of original document candidates on the display unit 116 and allowing the user to select thumbnail images of desired original document candidates from among a plurality of original document candidates. When the original document is selected by this user interface, the document ID of the original document is specified. Details of the configuration of this user interface will be described later.

  Finally, in step S3310, the original document (electronic file or image) is stored in the storage unit 111 with reference to the address corresponding to the document ID of the address information in the database 118 with respect to the document ID of the specified original document. Data) is read and the original document is printed, distributed, stored, or edited based on the operator's instructions.

(D) Document Information Extraction Next, an outline of processing for extracting information on increase, deletion, change, and replacement of pages of a paper document from an original electronic file or image data search result corresponding to the paper document will be described. .

  As described above, a document candidate list as shown in FIG. 9 is obtained as a result of the search process shown in FIG. 3C. In the following, it is assumed that the document indicated by the document ID 0001 is detected as an example. Therefore, in the following processing, the part of the image ID 0001 is extracted from the search result list shown in FIG. 8, and the page correspondence table shown in FIGS. 21A and 21B is created.

  FIG. 21A is a diagram showing a table arranged in the order of page numbers of processed documents, and includes a page number of processed documents, a page number of detected documents, and a similarity. FIG. 21B is a diagram showing a table arranged in the order of page numbers of detected documents, and includes a page number of detected documents and a page number of processed documents. In addition, “X” is described in a portion where there is no corresponding page.

  The portion “X” in FIG. 21A is a page that exists only in the processed document, indicating that the third page of the processed document is mixed with pages of other documents. Further, “X” in FIG. 21B is a page that exists only in the detected document, indicating that the fourth page of the detected document is missing.

  The portion other than X in FIG. 21A indicates that the second page of the processed document and the fourth page of the processed document can be sorted, and the page order is switched.

  In addition, a page (fourth page of the processed document) that is below the third threshold (80 in the present embodiment) with the similarity shown in FIG. 21A indicates that the page has been reorganized.

[Details of each process]
Details of each process will be described below.

  First, details of the block selection process in steps S3106, S3204, and S3303 will be described. Here, FIG. 10 is a diagram showing an example of image block extraction in the first embodiment of the present invention.

  In block selection processing, for example, the raster image 10A in FIG. 10 is recognized as a block for each meaningful block as shown in 10B, and the attribute (character (TEXT) / drawing (PICTURE) / photograph ( (PHOTO) / line (LINE) / table (TABLE), etc.), and is divided into blocks having different attributes.

  Here, an embodiment of the block selection process will be described in detail.

  First, an input image is binarized into a black and white image, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. For a black pixel block having a large area, the white pixel block is extracted by tracing the outline of the white pixel inside, and recursively from the inside of the white pixel block having a certain area or more. A black pixel block is extracted.

  The blocks of black pixels obtained in this way are classified by size and shape, and are classified into blocks having different attributes. For example, a block whose aspect ratio is close to 1 and whose size is constant is a pixel block corresponding to a character, and a portion where adjacent characters can be grouped with good alignment is a character block. In addition, the flat pixel block is a line block, the area occupied by the black pixel block that is larger than a certain size and contains rectangular white pixel blocks in a well-aligned manner is a table block, and the area where irregular pixel blocks are scattered is a photo block The pixel block having any other shape is used as a graphic block.

  Next, details of the feature amount information extraction processing in steps S3108, S3206, and S3304 will be described. In the feature amount information extraction process, the processing method is different for each of the image block and the character block, and will be described separately. For example, in the case of the example shown in 10B in FIG. 10, the image block includes a photographic block and a graphic block, but the image block is set to at least one of the photographic block and the graphic block according to the use and purpose. Is also possible.

  First, feature amount information extraction processing for an image block will be described. When a plurality of image blocks exist in one document, the following processing is repeated for the total number of the image blocks. In the present embodiment, as an example, color feature amount information extraction processing for extracting a color feature amount related to the color of an image is performed.

  Details of the color feature amount information extraction processing will be described with reference to FIG. FIG. 11 is a flowchart for explaining details of the color feature amount information extraction processing according to the first embodiment of the present invention. In the process described below, the color having the most frequent color in the color histogram of each mesh block obtained by dividing the processing target image into a plurality of mesh blocks and the information in which the position information of each mesh block is associated with each other are colored. Extracted as feature information.

  First, the image is divided into a plurality of mesh blocks (step S1101). FIG. 12 is a diagram illustrating an example of dividing an image into a plurality of mesh blocks in the first embodiment. As shown in FIG. 12, in this embodiment, the image is divided into 9 mesh blocks in the vertical and horizontal directions. In particular, in the present embodiment, an example in which it is divided into 9 × 9 = 81 mesh blocks is shown for convenience of description, but actually, it is preferable that the number is about 15 × 15 = 225 mesh blocks.

  Next, the target mesh block to be processed is set as the upper left block (step S1102). The mesh block of interest is set with reference to an order determination table in which the processing order is determined in advance as shown in FIG. 13, for example. That is, FIG. 13 is a diagram illustrating an example of an order determination table used in the first embodiment. The example shown in FIG. 13 shows a scanning example in which scanning is performed from the upper left end to the right, and when the line is completed, scanning is performed from the left end of the next line to the right.

  Next, the presence / absence of an unprocessed target mesh block is determined (step S1103). As a result, if there is no unprocessed target mesh block (No), the process is terminated. On the other hand, when there is an unprocessed target mesh block (Yes), the process proceeds to step S1104.

  In step S1104, the density values of all the pixels of the target mesh block are projected onto a color bin, which is a partial space created by dividing the color space shown in FIG. 14, and a color histogram for the color bin is generated. That is, FIG. 14 is a diagram illustrating an example of the configuration of the color bins on the color space according to the first embodiment of this invention.

  In the present embodiment, as shown in FIG. 14, the density values of all the pixels of the target mesh block are projected onto the color bin obtained by dividing the RGB color space into 3 × 3 × 3 = 27. Actually, it is preferable to project the density values of all pixels of the target mesh block into color bins obtained by dividing the RGB color space into about 6 × 6 × 6 = 216.

  Thereafter, the color bin ID of the most frequent color bin of the color histogram is determined as the representative color of the target mesh block, and stored in the storage unit 111 in association with the target mesh block and its position (step S1105).

  Then, the target mesh block to be processed next is set with reference to the order determination table shown in FIG. 13 (step S1106). Thereafter, the process returns to step S1103, and the above-described processing of steps S1103 to S1106 is repeated until there is no unprocessed target mesh block.

  Through the above processing, information in which the representative color for each mesh block of the processing target image (image block) and the position information of each mesh block are associated can be extracted as color feature amount information.

  Next, feature amount information extraction processing for character blocks will be described. If there are a plurality of character blocks in one document, the process described below is repeated for the total number of character blocks. The character feature amount information for the character block is a character code obtained by performing OCR (character recognition) processing on the character block.

  Here, in the OCR (character recognition) process, character recognition is performed on a character image cut out from a character block in character units using a pattern matching technique, and a corresponding character code is acquired. This character recognition process compares an observed feature vector obtained by converting a feature obtained from a character image into a numerical sequence of several tens of dimensions with a dictionary feature vector obtained in advance for each character type. Is the recognition result. There are various known methods for extracting a feature vector. For example, there is a method characterized by dividing a character into meshes and using a mesh number-dimensional vector obtained by counting character lines in each mesh block as line elements according to directions.

  When character recognition is performed on the character block extracted in the block selection process (step S3106, step S3204, or step S3303), horizontal / vertical writing is first determined for the corresponding character block, and the corresponding direction is determined. A character string is cut out, and then a character image is obtained by cutting out the character from the character string.

  Here, horizontal / vertical writing is determined by taking a horizontal / vertical projection of the pixel value in the corresponding character block, and determining horizontal writing when the horizontal projection variance is large and vertical writing when the vertical projection variance is large. . In the case of a horizontally written character block, the character string and character are decomposed by cutting out the line using the horizontal projection and cutting out the character from the vertical projection of the cut out line. To do. On the other hand, for vertically written character blocks, horizontal and vertical may be reversed.

  Next, details of the comparison processing in step S3305 will be described with reference to FIG. FIG. 15 is a flowchart for explaining details of the comparison processing in the first embodiment of the present invention.

  First, document IDs are acquired in order from the top of a document candidate list for managing comparison target documents (step S1501). Next, it is determined whether all document IDs have been acquired (step S1502). As a result, when all the document IDs have been acquired (Yes), this process ends. On the other hand, if all document IDs have not been acquired (No), the process proceeds to step S1503.

  In step S1503, layouts are compared. Here, the layout refers to the attribute, size, and position of the block in the block information. Specifically, the attribute, size, and position of each block are compared with the attribute, size, and position of each block in the block information corresponding to the document ID acquired in step S1501 and the currently processed page number. Determine if they are the same.

  As a result, if the comparison source image (paper document) and the comparison destination image (original document) have the same layout (Yes), the process proceeds to step S1504. On the other hand, when the layouts of the comparison source image and the comparison destination image are not the same (No), the process returns to step S1501.

  In step S1504, a page comparison process for comparing pages of the comparison source image (paper document) and the comparison destination image (original document) is executed. This comparison is performed in a composite manner using feature amounts corresponding to characters and images according to the block attributes, and the similarity is calculated. Details of this process will be described later.

  Next, it is determined whether or not the calculated similarity is greater than or equal to a first threshold (step S1505). As a result, when the similarity is less than the first threshold (No), the process returns to step S1501. On the other hand, if the similarity is greater than or equal to the first threshold (Yes), the process proceeds to step S1506.

  In step S1506, the currently processed document ID and similarity are added to the search result list, and the process returns to step S1501.

  Next, details of the page comparison process in step S1504 will be described with reference to FIG. FIG. 16 is a flowchart for explaining details of the comparison processing in the first embodiment of the present invention.

  First, block information is referenced to determine whether there is an uncompared block in the electronic data corresponding to the document ID and page number to be processed (step S1601). As a result, if there is no uncompared block (No), the process proceeds to step S1605. On the other hand, if there is an uncompared block (Yes), the process proceeds to step S1602.

  In step S1602, the attribute of the comparison target block is determined. As a result, if the attribute is an image block, the process advances to step S1603. On the other hand, if the attribute is a character block, the process advances to step S1604.

  If the attribute is an image block, color feature amount information comparison processing that is a similarity comparison with the comparison target block is performed with the feature amount information about the color (step S1603). Although details of this processing will be described later, the similarity obtained by this processing is temporarily stored in the storage unit 111 in association with the document ID, page number, and block ID of the comparison destination.

  On the other hand, if the attribute is a character block, a character feature information comparison process that is a similarity comparison between the comparison source block and the comparison target block in the character feature information is performed (step S1604). Although details of this processing will be described later, the similarity obtained thereby is temporarily stored in the storage unit 111 in association with the document ID, page number, and block ID of the comparison destination.

  In step S1601, when comparison with all the blocks is completed (No), the process proceeds to step S1605. In step S1605, the process is performed in steps S1603 and S1604. An integration process is performed to integrate the similarities of all blocks included in the pages of the comparison target document (original document) and calculate the similarity (integrated similarity) between the paper document as a search condition and the pages in the original document. .

  Next, details of the color feature amount information comparison processing in step S1603 will be described with reference to FIG. FIG. 17 is a flowchart for explaining details of the color feature amount information comparison processing according to the first embodiment of the present invention.

  First, the color feature amounts of the comparison source image and the comparison destination image are read from the color feature amount information (step S1701). Next, the target mesh block in the image to be processed is set at the head (step S1702). Further, the similarity distance indicating the similarity between the color feature amount of the comparison source image and the color feature amount of the comparison target is reset to 0 (step S1703).

  Thereafter, the presence / absence of an uncompared target mesh block is determined (step S1704). As a result, when there is no uncompared target mesh block (No), the process proceeds to step S1708. On the other hand, if there is an uncompared target mesh block (Yes), the process proceeds to step S1705.

  In step S1705, the color bin ID of each target mesh block is acquired from the color feature amounts of the comparison source image and the comparison destination image. Next, the local similarity distance of the target mesh block corresponding to the acquired color bin ID is acquired with reference to the color bin penalty matrix shown in FIG. 18 (step S1706). Then, the acquired local similarity distance is cumulatively added to the similarity distance acquired in the immediately preceding process, and the similarity distance is stored in the storage unit 111 (step S1707).

  Here, the color bin penalty matrix will be described with reference to FIG. FIG. 18 is a diagram illustrating a configuration example of the color bin penalty matrix used in the first embodiment of the present invention. The color bin penalty matrix is a matrix that manages the local similarity distance between the color bin IDs. According to FIG. 18, the similarity distance of the color bin penalty matrix is 0 for the same color bin ID, and the similarity distance increases as the difference between the color bin IDs increases, that is, the similarity decreases. It is configured. Further, the color bin penalty matrix has a similarity distance of 0 for all diagonal positions of the same color bin ID, and has a target property with that as a boundary.

  As described above, in the image processing system according to the first embodiment, the similarity distance between the color bin IDs can be acquired only by referring to the color bin penalty matrix as shown in FIG. Can be achieved.

  Then, the target mesh block to be processed next is set with reference to the order determination table shown in FIG. 13 (step S1707). Thereafter, the process returns to step S1704.

  On the other hand, if it is determined in step S1704 that there is no uncompared target mesh block (No), the similarity distance stored in the storage unit 111 is converted into a similarity and output as a pair with the block ID (step S1708). ). The conversion to similarity is, for example, 100% similarity when the similarity distance is the minimum value, 0% similarity when the similarity distance is the maximum value, and the similarity to the similarity distance within the range is the minimum What is necessary is just to calculate based on the difference with respect to a value or a maximum value.

  Next, details of the character feature amount information comparison processing in step S1604 will be described. In this process, the character codes in the character blocks in the comparison source image and the comparison target image are compared with each other, and the similarity is calculated from the matching degree. It should be noted that when comparing a paper document as a search condition with an original document, the similarity is ideally 100%. However, in actuality, OCR processing is performed on a character block in a paper document as a search condition. In some cases, misrecognition may occur. Therefore, even when compared with the original document, the degree of similarity may not be 100%, but if the values are close to 100%, they can be regarded as the same document.

  Next, details of the integration processing in step S1605 will be described. In this integration processing, the similarity of the calculated block-by-block similarity is such that the similarity of the block that accounts for a large proportion in the original document that is the comparison target image is more largely reflected as the similarity of the entire original document. Perform integration.

  For example, it is assumed that the similarity ratio for each block is calculated as n1 to n6 for the blocks B1 to B6 in the original document. At this time, the overall similarity N of the entire original document is expressed by the following equation.

N = w1 * n1 + w2 * n2 + w3 * n3 + ... + w6 * n6 (1)
Here, w1 to W6 are weighting factors for evaluating the similarity of each block.

The weighting factors w1 to w6 are calculated based on the occupation ratio of the block in the original document. For example, if the sizes of the blocks 1 to 6 are S1 to S6, the occupation ratio w1 of the block 1 is
w1 = S1 / (S1 + S2 + ... + S6) (2)
Can be calculated as

  By the weighting process using such an occupancy rate, the similarity of blocks that occupy a large area in the original document can be reflected in the similarity of the entire original document.

  Next, an example of a user interface at the time of candidate display / selection processing in step S3309 will be described with reference to FIG. FIG. 19 is a diagram illustrating an example of a user interface according to the first embodiment of the present invention.

  In FIG. 19, reference numeral 1901 denotes a display area, which is composed of, for example, a touch panel, and a user can make a selection instruction by directly touching the screen. Reference numerals 1902 to 1911 denote thumbnail images of predetermined pages of original document (electronic file or image data) candidates to be output as search results. The thumbnail images are displayed in descending order of similarity starting from 1902 in numerical order.

  In this example, a maximum of 10 thumbnail images are displayed, and when the number of original document candidates is 10 or more, the top 10 thumbnail images are displayed. A desired original document can be selected by selecting a desired thumbnail image from the thumbnail image groups 1902 to 1911 by touching or the like. Then, various processes can be executed on the selected original document.

  In FIG. 19, reference numeral 1912 denotes the number of pages in the document on which candidates are displayed. Further, by selecting this, other page candidates can be switched and displayed.

  As described above, according to the first embodiment, when searching for a corresponding original document from a paper document composed of a plurality of pages, comparison is made with all data in the database. In other words, this embodiment is not a simple process of comparing the second page of the search source document with only the second page of the document in the database. Therefore, even if the document to be processed is in a state where pages are added, missing, changed, replaced, etc., the original document can be detected appropriately.

  Even after a document is detected, by comparing the pages for all the pages of the processing target document, even if there are pages added, missing, changed, replaced, etc. Can be suitably detected.

<Second Embodiment>
In the first embodiment described above, all pages of the document to be processed are compared with all pages in the database. However, in the second embodiment, as the page to be processed advances, Processing for narrowing down the data to be compared will be described. Hereinafter, search processing according to the second embodiment will be described in detail with reference to the drawings.

  Note that the configuration of the image processing system, the detailed configuration of the MFP, the electronic document registration process, and the paper document registration process according to the second embodiment are the same as those in the first embodiment described above. The detailed description is omitted.

  Search processing in the second embodiment will be described with reference to FIG. 3D. FIG. 3D is a flowchart for explaining search processing according to the second embodiment of the present invention.

  First, when a search for an original document (electronic file or image data) is instructed from the input unit 113 by the operator, the comparison destination list is initialized under the control of the management PC 101 (step S3401).

  The comparison destination list is a list for narrowing down the original documents to be compared in the comparison process. The document ID of the original document to be compared, the paper document read from the image reading unit 110, And the average of the similarities of the retrieved pages, which are stored in the storage unit 111. FIG. 20 is a diagram illustrating a configuration example of a comparison destination list used in the second embodiment. In the initialization of the comparison destination list, all the document IDs of the electronic documents stored in the database 118 are stored in the comparison destination list, and 0 is set as the value of the total similarity.

  Since the processing from the next step S3402 to step S3407 is the same as the processing from step S3301 to step S3306 in the first embodiment, detailed description thereof is omitted here.

  Then, after the search result list is added in step S3407, the comparison destination list is updated (step S3408). In the update of the comparison destination list, candidates whose total similarity is below the second threshold and candidates whose average similarity of the searched pages is below the first threshold are deleted from the candidates. For the other candidates, the sum of the similarities and the average of the similarities are calculated again, and the list is updated. FIG. 20 shows an example of the result of sequentially updating the comparison destination list from (A) to (D).

  Then, the process returns to step S3402 to read the paper document of the next page.

  If reading of paper documents for all pages from the image reading unit 110 is completed in step S3403 (Yes), the process advances to step S3409. Note that the processing from step S3409 to step S3412 is the same as the processing from step S3307 to step S3310 in the first embodiment described above, and thus detailed description thereof is omitted here.

  As described above, according to the second embodiment, when searching for a corresponding original document from a paper document composed of a plurality of pages, the comparison target is narrowed down based on the comparison process, and all of the data with the comparison target narrowed down is searched. Compare with page. In other words, this embodiment is not a simple process of comparing the second page of the search source document with only the second page of the document in the database. Therefore, even when a document to be processed includes pages added, missing, changed, or replaced, the original document can be detected appropriately. Further, by narrowing down the comparison targets, the load of comparison processing is reduced, and the processing speed is greatly improved. However, if a page is added to the document to be processed, the document to be searched may be excluded from the comparison target during the search.

  Even after a document is detected, the page is compared against all pages of the document to be processed, so that even if pages are added, missing, changed, or replaced, the status is appropriately Can be detected.

<Third Embodiment>
Next, in the third embodiment, a description will be given of a process in which the result of the comparison process for the first page is used as a comparison target for the subsequent pages. The configuration of the image processing system, the detailed configuration of the MFP, the electronic document registration process, and the paper document registration process according to the third embodiment are the same as those in the first embodiment described above, and are described in detail here. The explanation is omitted.

  Hereinafter, search processing according to the third embodiment will be described with reference to FIG. 3E. FIG. 3E is a flowchart for explaining search processing according to the third embodiment of the present invention.

  First, the comparison destination list is initialized (step S3501). Since this processing is the same as step S3401 of the second embodiment described above, detailed description thereof is omitted. Further, the subsequent processing from step S3502 to step S3507 is the same as the processing from step S3301 to step S3306 in the first embodiment, and thus detailed description thereof is omitted.

  In step S3507, after the search result list is added, it is determined whether the page being processed is the first page (step S3508). As a result, if it is determined that the page being processed is the first page (Yes), the process proceeds to step S3509. On the other hand, if it is determined that the page being processed is the second page or later (No), the process returns to step S3502. Note that the comparison destination list update process in step S3509 is the same process as step S3408 in the second embodiment described above, and thus detailed description thereof is omitted. Then, returning to step S3502, the paper document of the next page is read.

  On the other hand, if it is determined in step S3503 that the reading of the paper documents for all pages from the image reading unit 110 has been completed (No), the process proceeds to step S3510. Since the subsequent processing from step S3510 to step S3513 is the same as the processing from step S3307 to step S3310 in the first embodiment, detailed description thereof will be omitted.

  As described above, according to the third embodiment, when a corresponding original document is searched from a paper document composed of a plurality of pages, the comparison target is narrowed down based on the comparison process of the first page, and the subsequent pages. Compares with all pages of the data narrowed down. In other words, this embodiment is not a simple process of comparing the second page of the search source document with only the second page of the document in the database. Therefore, even if the document to be processed is in a state where pages are added, missing, changed, replaced, etc., the original document can be detected appropriately.

  Further, when a document to be searched is set in the image reading unit 110, at least the first page may be one of the documents to be searched even if there is a possibility that other documents are mixed in the middle page. High nature. Therefore, by narrowing down the comparison target on the first page, the load of the comparison process is reduced and the processing speed is improved as compared with the process for comparing all pages of all documents without lowering the original document detection accuracy. Expected.

  In addition, by comparing the pages for all pages of the document to be processed even after the document is detected, if the page has been added, missing, changed, or replaced, it is preferable to detect the state. Can do.

<Fourth Embodiment>
In the fourth embodiment, until the number of comparison target documents falls below a predetermined number (4 in the present embodiment), the comparison target is narrowed down, and the comparison processing result of the page is used as the comparison target for the next page and subsequent pages. It is. Note that the configuration of the image processing system, the detailed configuration of the MFP, the electronic document registration processing, and the paper document registration processing according to the fourth embodiment are the same as those in the first embodiment, and thus are described in detail here. Is omitted. Hereinafter, search processing according to the fourth embodiment will be described with reference to FIG. 3F.

  FIG. 3F is a flowchart for explaining search processing according to the fourth embodiment of the present invention.

  First, the comparison destination list is initialized (step S3601). Since this processing is the same as step S3401 of the second embodiment, detailed description thereof is omitted here. Further, the subsequent processing from step S3602 to step S3607 is the same as the processing from step S3301 to step S3306 in the first embodiment, and thus detailed description thereof is omitted.

  Then, after the search result list is added in step S3607, it is determined whether or not the number of candidates included in the comparison destination list is equal to or greater than a predetermined number (4 in the present embodiment) (step S3608). As a result, when the number of candidates included in the comparison destination list is equal to or greater than a predetermined number (4 in the present embodiment) (Yes), the process proceeds to step S3609. On the other hand, when the number of candidates included in the comparison destination list is less than a predetermined number (4 in the present embodiment) (No), the process returns to step S3602.

  In step S3609, a comparison destination list update process is performed. Since this process is the same process as step S3408 in the second embodiment, a detailed description thereof will be omitted. Then, the process returns to step S3602, and the paper document of the next page is read.

  On the other hand, in step S3603, when reading of all pages of the paper document from the image reading unit 110 is completed (Yes), the process proceeds to step S3610. The subsequent processing from step S3610 to step S3613 is the same as the processing from step S3307 to step S3310 in the first embodiment, and thus detailed description thereof is omitted.

  As described above, according to the fourth embodiment, when a corresponding original document is searched from a paper document consisting of a plurality of pages, the number of candidates for comparison is a predetermined number (4 in this embodiment). The comparison target is narrowed down until it falls below, and after the next page, comparison is made with all pages of the data with the comparison target narrowed down. In other words, this embodiment is not a simple process of comparing the second page of the search source document with only the second page of the document in the database. Therefore, even if the document to be processed is in a state where pages are added, missing, changed, replaced, etc., the original document can be detected appropriately.

  Further, when setting a document to be searched in the image reading unit 110, for example, even if there is a possibility that other documents are mixed in the middle page, the previous page is one page of the documents to be searched. Is likely. In addition, if narrowing is performed only on the first page, if the first page is a cover, sufficient narrowing may not be performed and the processing speed may not increase as expected. Therefore, according to the present embodiment, the load of comparison processing is reduced and the processing speed is expected to be improved without lowering the original document detection accuracy.

  Even after a document is detected, by comparing pages for all pages of the document to be processed, even if there are pages added, missing, changed, or replaced on the printed matter, the state is suitable. Can be detected.

<Other embodiments>
In the above-described embodiment, the example of extracting the most frequent color of the processing target image as the color feature information has been described in the color feature amount information extraction process of FIG. 11. However, the present invention is not limited to this. May be extracted as color feature information.

  In addition, although the color feature amount is used as the image feature amount, the present invention is not limited to this. For example, the feature amount is represented by luminance feature amounts such as mode luminance and average luminance, co-occurrence matrix, contrast, entropy, Gabor conversion, and the like. A plurality of types of image feature amounts such as texture feature amounts, shape features such as edges and Fourier descriptors, or any combination of image feature amounts may be used.

  Further, in the above-described embodiment, the block selection process is performed, the document to be processed is divided into the character block and the image block, and the search is performed using the feature amount of each block in combination. The original document may be searched by regarding it as one image. Furthermore, if the accuracy is within the allowable range, the original document may be searched using only the image block in the document.

  Furthermore, in the above-described embodiment, processing is performed by converting an electronic document into a raster image. However, character codes and images may be directly extracted from the electronic document and compared. Furthermore, in the embodiment described above, a character code is used as the character feature amount. However, for example, the word part of speech is extracted by matching with a word dictionary in advance, and a word that is a noun is used as the character feature amount. Also good.

  Furthermore, in the embodiment described above, in the comparison processing in step S3305 in FIG. 3C, step S3406 in FIG. 3D, step S3506 in FIG. 3E, and step S3606 in FIG. However, a configuration for directly comparing the image data may be used.

  Furthermore, in the above-described embodiment, the MFP 100 in which the image reading unit 110 (scanner) and the printing unit 112 (printer) are integrally configured has been described as an example. However, the image reading unit 110 and the printing unit 112 are respectively configured. As an individual device, it may be configured to be connected to the LAN 107 or the client PC 102 or the like. In this case, the other components of the MFP 100 are included in the management PC 101, and the connection between the image reading unit 110 (scanner) and the printing unit 112 (printer) is via the LAN 107 via the network I / F 114. Will be connected.

  The first to third thresholds described above do not need to be fixed, and may be arbitrarily set by a user, an administrator, or the like depending on the type of document to be used.

  Although the embodiment has been described in detail above, the present invention can take an embodiment as, for example, a system, apparatus, method, program, or storage medium (recording medium). The present invention may be applied to a system composed of a single device or an apparatus composed of a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the figure) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

1 is a block diagram illustrating a configuration of an image processing system according to a first embodiment of the present invention. FIG. 2 is a block diagram illustrating a detailed configuration of the MFP 100 according to the first embodiment of the present invention. 5 is a flowchart for explaining electronic document registration processing in the image processing system according to the first embodiment of the present invention; It is a flowchart for demonstrating the registration process of the paper document of the 1st Embodiment of this invention. It is a flowchart for demonstrating the search process in the 1st Embodiment of this invention. It is a flowchart for demonstrating the search process in the 2nd Embodiment of this invention. It is a flowchart for demonstrating the search process in the 3rd Embodiment of this invention. It is a flowchart for demonstrating the search process in the 4th Embodiment of this invention. It is a figure which shows an example of the address information which consists of document ID and address registered in 1st Embodiment. It is a figure which shows an example of the block information registered into the database 118 in 1st Embodiment. It is a figure which shows an example of the color feature-value information memorize | stored in the database 118 in 1st Embodiment. It is a figure which shows an example of the character feature-value memorize | stored in the database 118 in 1st Embodiment. It is a figure which shows the structural example of the search result list | wrist in 1st Embodiment. It is a figure which shows the structural example of the document candidate list | wrist in 1st Embodiment. It is a figure which shows the example of image block extraction in the 1st Embodiment of this invention. It is a flowchart for demonstrating the detail of the color feature-value information extraction process in the 1st Embodiment of this invention. It is a figure which shows the example of a division | segmentation into the some mesh block in 1st Embodiment. It is a figure which shows an example of the order determination table used in 1st Embodiment. It is a figure which shows an example of a structure of the color bin on the color space of the 1st Embodiment of this invention. It is a flowchart for demonstrating the detail of the comparison process in the 1st Embodiment of this invention. It is a flowchart for demonstrating the detail of the comparison process in the 1st Embodiment of this invention. It is a flowchart for demonstrating the detail of the color feature-value information comparison process in the 1st Embodiment of this invention. It is a figure which shows one structural example of the color bin penalty matrix used in the 1st Embodiment of this invention. It is a figure which shows an example of the user interface in the 1st Embodiment of this invention. It is a figure which shows the structural example of the comparison destination list used by 2nd Embodiment. It is a figure which shows the table | surface arranged in order of the page number of the process document. It is a figure which shows the table | surface arranged in order of the page number of the detected document.

Explanation of symbols

100 MFP
101 Management PC
102 Client PC
103a, 103b Proxy server 104 Network 105a, 105b Database 106a, 106b Document management server 107, 108, 109 LAN
110 Image Reading Unit 111 Storage Unit 112 Printing Unit 113 Input Unit 114, 117 Network I / F
115 Data Processing Unit 116 Display Unit 118 Database

Claims (14)

Storage means for storing a plurality of document images composed of a plurality of pages in association with respective feature amounts;
Reading means for electronically reading a plurality of pages of printed matter on a page basis and converting it into electronic data;
Extraction means for extracting feature quantities of the electronic data;
The feature amount of the electronic data and the feature amount of each document image stored in the storage unit are compared for each page to calculate a similarity, and based on the similarity, the plurality of document images An image search apparatus comprising: search means for searching for a document image corresponding to electronic data as candidate data.   The search means compares all pages included in the electronic data with all pages of the respective document images stored in the storage means to calculate a similarity for each page, and for each page The image search apparatus according to claim 1, wherein an integrated similarity obtained by integrating similarities is calculated, and a document image corresponding to the electronic data is searched as candidate data based on the integrated similarity.   The search means calculates a similarity by comparing with each page of the plurality of document images in order from the first page of the electronic data, and deletes a document image having the similarity less than a predetermined value from the candidate data. 2. The image search apparatus according to claim 1, wherein the candidate data is searched up to the last page.   The search means calculates the similarity by comparing the first page of the electronic data and the pages of the plurality of document images, and corresponds to the electronic data from the plurality of document images based on the similarity The document image to be searched is searched as candidate data, and the degree of similarity is calculated in comparison with the page of each document image included in the candidate data in order from the second page included in the electronic data. The image search apparatus according to claim 1, wherein the candidate data is searched to the last page while deleting less than less document images from the candidate data.   The search means calculates the similarity by comparing with the respective pages of the plurality of document images in order from the first page of the electronic data, and deletes the document image having the similarity less than a predetermined value from the candidate data. The image search apparatus according to claim 1, wherein candidate data is searched up to a predetermined page.   The image search apparatus according to claim 5, wherein the search means searches to a page when the number of candidates for the candidate data falls below a predetermined number.   7. The apparatus according to claim 1, further comprising detection means for detecting addition, omission, replacement, and replacement of pages of the printed material based on the similarity for each page calculated by the search means. The image search device according to claim 1. The extraction means comprises:
Block selection means for dividing the electronic data into a plurality of blocks;
A specifying means for specifying the type of the block;
First extraction means for extracting the feature amount of each block based on the identified block type;
8. The apparatus according to claim 1, further comprising: a second extracting unit configured to extract the feature values of the electronic data by collecting the feature values of the extracted blocks in units of pages. Image search device.   The feature amount is color feature information in which a representative color in a color histogram of each mesh block obtained by dividing an image into a plurality of mesh blocks and position information of each mesh block are associated with each other. 9. The image search device according to any one of up to 8.   The image search apparatus according to claim 9, wherein the similarity is a similarity distance between color bin IDs acquired from color feature information on each mesh block of the electronic data and the document image. . Determination means for determining whether or not a plurality of candidate data are searched by the search means;
When a plurality of candidate data are searched, display means for displaying a list of thumbnail images of the plurality of candidate data;
11. The image search apparatus according to claim 1, further comprising selection means for selecting one candidate data from thumbnail images of a plurality of candidate data displayed in a list. A storage step of storing a plurality of document images of a plurality of pages in a storage device in association with respective feature amounts;
A reading step of electronically reading a plurality of pages of printed matter in units of pages and converting it into electronic data;
An extraction step of extracting a feature amount of the electronic data;
The feature amount of the electronic data and the feature amount of each document image stored in the storage unit are compared for each page to calculate a similarity, and based on the similarity, the plurality of document images And a search step for searching for a document image corresponding to electronic data as candidate data. On the computer,
A storage procedure for storing a plurality of document images composed of a plurality of pages in a storage device in association with respective feature amounts;
A reading procedure for electronically reading a multi-page printed matter page by page and converting it into electronic data;
An extraction procedure for extracting feature quantities of the electronic data;
The feature amount of the electronic data and the feature amount of each document image stored in the storage unit are compared for each page to calculate a similarity, and based on the similarity, the plurality of document images And a search procedure for searching a document image corresponding to electronic data as candidate data.   A computer-readable storage medium storing the program according to claim 13.

Images