JP2004348467A - Image retrieval apparatus and its control method, program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieval apparatus and its control method, and a program which enable original electronic data to be accurately retrieved from a paper document. <P>SOLUTION: A printed matter is read out as electronic data and inputted as a retrieval key image. The retrieval key image is divided into a character area and a non-character area. Then similarity between the retrieval key image and registered images is calculated on the basis of the feature quantities of the character in the character area, the feature quantities of the image in the non-character area and positional relation between the character area and the non-character area. Then a registered image corresponding to the retrieval key image is retrieved on the basis of the calculation result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image search apparatus, a method, and a program for inputting a printed material as a search key and searching for an image corresponding to the printed material.
[0002]
[Prior art]
In recent years, paperless offices have rapidly become paperless, and not only documents created on terminals such as personal computers (PCs), but also past paper documents that have been stored in binders and the like from the past as well as image data (electronic Document) and store it in a database.
[0003]
On the other hand, paper documents are still preferred for handouts and the like at conferences, and there are many occasions where electronic documents stored in a database are printed out on paper and handed over to users.
[0004]
Here, when a user who receives a paper document wants to store and transmit it electronically or extract the contents and reuse it, rather than re-electronicizing the paper document to obtain an electronic document, a database is obtained. It is more desirable to acquire and use the original electronic (image) data of the paper document than to re-electronicize and use the paper document.
[0005]
In order to respond to such a request, for example, Patent Document 1 proposes a system capable of reading a paper document with a scanner and searching for image data similar to the content from a database.
[0006]
Further, assuming that a search target is a document generally used in an office or the like, there are roughly two types of information in a document: textual information and non-textual information such as a photograph or a figure. At the time of the search, the processing accuracy can be further improved by using the similarity calculation processing according to each characteristic.
[0007]
Further, if the area identification processing disclosed in Patent Document 2 is used, a character area and a non-character area in a document image obtained by scanning a paper document can be separated and extracted. By using this technology, each area is specified even for a document in which characters and non-characters are mixed. From the character area, keywords are extracted from the character code string obtained by the character recognition processing, and the character code string of the registered data is extracted. And the similarity using the degree of matching at the time of collation is obtained. On the other hand, from the non-character area, a similarity based on image feature amounts such as colors and edges is obtained. By adding this similarity with an appropriate distribution, a search using both character and non-character similarity information was possible.
[0008]
[Patent Document 1]
Patent No. 3017851 [Patent Document 2]
US Pat. No. 5,680,478 [0009]
[Problems to be solved by the invention]
In such an environment as described above, in particular, document image data registered in an office or the like may include many similar document image data. For example, by using drawing parts and texts, many documents containing the same photograph but different texts, or many documents with the same text but different figures are created and registered.
[0010]
On the other hand, in the method of comparing the similarity between a paper document image and a registered image, as in the conventional example, the similarity between a character and a non-character is obtained independently and summed up in an appropriate distribution. However, it is difficult to always find an appropriate distribution for search when the similarity between the two is highly biased.
[0011]
In addition, in order to specify a document image of an original document from a group of document images having similar contents, there has been a method of performing a search including the position of a character or non-character area in the feature amount.
[0012]
However, when the original document is generated from the electronic document, the layout may not exactly match the paper document created from the electronic document. For example, in the case of an electronic document in the HTML format, the layout at the time of printout greatly differs depending on the environment. In such a case, if an attempt is made to search for the original electronic document using the position information of the characters and non-character areas in the paper document, the search accuracy may be degraded.
[0013]
SUMMARY An advantage of some aspects of the invention is to provide an image retrieval apparatus capable of accurately retrieving original electronic data from a paper document, a control method thereof, and a program.
[0014]
[Means for Solving the Problems]
An image search device according to the present invention for achieving the above object has the following configuration. That is,
An image search device for inputting a printed material as a search key image and searching for a registered image corresponding to the search key image,
Region dividing means for dividing the search key image into a character region and a non-character region,
Calculating means for calculating a similarity between the search key image and the registered image based on a character feature amount of the character region, an image feature amount of the non-character region, and a positional relationship between the character region and the non-character region. When,
A search unit for searching for a registered image corresponding to the search key image based on a calculation result by the calculation unit.
[0015]
Preferably, the character feature amount is a character code obtained by character recognition of the character region.
[0016]
Also preferably, the calculating means, when the similarity based on the image feature amount of the non-character area is equal to or more than a threshold, based on the positional relationship between the predetermined character string in the character area and the non-character area, A similarity between the search key image and the registered image is calculated.
[0017]
Further, preferably, the calculating unit sets a correlation between predetermined character strings in the character area located within a predetermined range of the non-character area to be processed for both the search key image and the registered image as the similarity. calculate.
[0018]
Preferably, the calculation means compares distances between the non-character area to be processed in both the search key image and the registered image and a predetermined character string in the character area located within the predetermined range. Then, a similarity is calculated based on the comparison result.
[0019]
Preferably, the calculation means compares angles from the non-character area to be processed of both the search key image and the registered image to a predetermined character string in the character area located within the predetermined range. Then, a similarity is calculated based on the comparison result.
[0020]
An image search method according to the present invention for achieving the above object has the following configuration. That is,
An image search method for inputting a printed material as a search key image and searching for a registered image corresponding to the search key image,
An area dividing step of dividing the search key image into a character area and a non-character area;
Based on the character feature amount of the character region, the image feature amount of the non-character region, and the positional relationship between the character region and the non-character region, the search key image and a plurality of registered images stored in the storage medium are used. Calculating a similarity of
A search step of searching a registered image corresponding to the search key image from a plurality of registered images stored in the storage medium, based on a result of the calculation in the calculation step.
[0021]
A program according to the present invention for achieving the above object has the following configuration. That is,
A program for inputting a printed material as a search key image and implementing an image search for searching for a registered image corresponding to the search key image,
Program code of an area dividing step of dividing the search key image into a character area and a non-character area,
Based on the character feature amount of the character region, the image feature amount of the non-character region, and the positional relationship between the character region and the non-character region, the search key image and a plurality of registered images stored in the storage medium are used. Program code for a calculation step of calculating the similarity of
A program code for a search step of searching for a registered image corresponding to the search key image from a plurality of registered images stored in the storage medium based on a calculation result in the calculation step.
[0022]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0023]
FIG. 1 is a block diagram illustrating a configuration of an image processing system according to an embodiment of the present invention.
[0024]
This image processing system is realized in an environment where the office 10 and the office 20 are connected by a network 104 such as the Internet.
[0025]
A LAN 107 built in the office 10 includes an MFP (Multi Function Peripheral) 100, which is a multifunction peripheral that implements a plurality of types of functions, a management PC 101 that controls the MFP 100, a document management server 106 and its database 105, and a proxy server 103. Is connected.
[0026]
A user PC 102 functioning as a user of the MFP 100 or the document management server 106 is connected to the LAN 108 built in the office 20, for example.
[0027]
The LAN 107 in the office 10 and the LAN 108 in the office 20 are connected to the network 104 via the proxy server 103 in both offices.
[0028]
The MFP 100 particularly includes an image reading unit that electronically reads a paper document and an image processing unit that performs image processing on an image signal obtained from the image reading unit. The image signal is transmitted to the management PC 101 via the LAN 109. can do.
[0029]
The management PC 101 is a normal PC, and includes various components such as an image storage unit, an image processing unit, a display unit, and an input unit. Some of the components are integrated with the MFP 100. .
[0030]
The network 104 is typically one of the Internet, a LAN, a WAN, a telephone line, a dedicated digital line, an ATM, a frame relay line, a communication satellite line, a cable TV line, a data broadcasting wireless line, or the like. It is a so-called communication network realized by the combination, and it is sufficient that data can be transmitted and received.
[0031]
Various terminals such as a management PC 101, a user PC 102, and a document management server are respectively standard components (eg, CPU, RAM, ROM, hard disk, external storage device, network interface, display, keyboard) mounted on a general-purpose computer. , Mouse, etc.).
[0032]
Further, the configuration of the image processing system in FIG. 1 is an example, and for example, the office 10 and the office 20 may be configured on the same LAN.
[0033]
Next, a detailed configuration of MFP 100 will be described with reference to FIG.
[0034]
FIG. 2 is a block diagram showing a detailed configuration of the MFP according to the embodiment of the present invention.
[0035]
In FIG. 2, an image reading unit 110 including a document table and an automatic document feeder (ADF) irradiates a bundle or one document image with a light source (not shown), and reflects a document reflection image on a solid-state image sensor with a lens. And a raster image reading signal is obtained from the solid-state imaging device as a raster image having a predetermined density (for example, 600 DPI).
[0036]
In the present embodiment, a paper document is described as an example of a printed material read by the image reading unit 110. However, a printed material formed of a recording medium other than paper (for example, a transparent original such as an OHP sheet or a film, or a cloth) is used. May be set as a reading target of the image reading unit 110.
[0037]
Further, MFP 100 has a copy function of printing an image corresponding to the image reading signal on a recording medium by printing unit 112, and when copying one document image, the image reading signal is transmitted to data processing unit 115 by image processing unit 115. Processing is performed to generate a recording signal, and this is printed by a printing unit 112 on a recording medium. On the other hand, when a plurality of original images are copied, a recording signal for one copy is once stored and held in the storage unit 111, and is sequentially output to the printing unit 112 to be printed on a recording medium.
[0038]
As a printing function of the MFP 100, a recording signal output from the user PC 102 is received by the data processing unit 115 via the LAN 107 and the network IF 114, and the data processing unit 115 can record the recording signal by the printing unit 112. After converting the data into raster data, the printing unit 112 can print the data on a recording medium.
[0039]
As a transmission function of the MFP 100, the MFP 100 converts a raster image into electronic data in a compressed image file format such as TIFF or JPEG or a network file format such as PDF and outputs the same from the network IF 114. The output electronic data can be transmitted to the document management server 106 via the LAN 107 or further transferred to the user PC 102 via the network 104.
[0040]
An operator's instruction to the MFP 100 is performed from a key operation unit provided in the MFP 100 and an input unit 113 including a keyboard and a mouse connected to the management PC 101. A series of these operations are performed by a control unit ( (Not shown). The display of the status of the operation input and the display of the image data being processed are performed on the display unit 116.
[0041]
The storage unit 111 is also controlled by the management PC 101, and data transmission and reception and control between the MFP 100 and the management PC 101 are performed via the network IF 117 and the LAN 109.
[0042]
In the MFP 100, the display unit 116 and the input unit 113 implement a user interface for providing a user with various operations and displays for executing various processes described below.
[0043]
The processing executed by the image processing system according to the present invention is roughly classified into a registration processing for registering image data and a search processing for searching for desired image data.
[0044]
In the present embodiment, the processing described below is performed by, for example, the MFP 100 alone. However, in order to improve the processing efficiency of the entire image processing system, various processing described below is performed by the image processing system. May be executed in a distributed manner among the various terminals constituting the program.
[0045]
First, the registration process will be described.
[0046]
As a method for registering image data to be registered by the registration process, there are a case where electronic data created by a document creation application or the like is registered as an original document, and a case where a paper document is registered as an original document. In this registration process, the registered original document data is subjected to a feature amount extraction process of a feature amount necessary for a search, and the original document data is associated with the registered original document data and registered in a database.
[0047]
FIG. 3 is a flowchart showing the registration processing according to the embodiment of the present invention.
[0048]
First, in step S301, it is determined whether the registration target is a paper document or electronic data.
[0049]
This determination is made, for example, based on the operation of the paper document registration button and the electronic data registration button configured on the input unit 101.
[0050]
In step S301, if the paper document registration button is operated (YES in step S301), the process proceeds to step S302, and the image reading unit 110 of the MFP 100 scans the paper document in a raster form to obtain a raster image (page image). .
[0051]
On the other hand, if the electronic data registration button has been operated (NO in step S301), the process proceeds to step S303, in the hard disk in the client PC 102, in the database 105 in the document management server 106 in the office 10, or in the storage unit 111 of the MFP 100. The electronic data of the original document to be registered, which is stored in any of the above, is read and input to the data processing unit 115 via the network IF 114, and the data processing unit 115 converts the electronic data into a raster image (page image). I do.
[0052]
The conversion into the raster image may be realized by using the rasterization function of the application itself or the additional software that generated the electronic data to be registered.
[0053]
Next, in step S304, a region division process is performed on the page image to extract a character region and a non-character region (for example, a photograph, a drawing, and the like) from the page image.
[0054]
Here, in the area division processing, for example, as shown in FIG. 4B, the page image of FIG. 4A is recognized as a meaningful block for each block, and the attribute (character (TEXT)) of each block is recognized. / Picture (PICTURE) / photograph (PHOTO) / line (LINE) / table (TABLE)), and is divided into blocks having different attributes.
[0055]
An embodiment of the area dividing process will be described below.
[0056]
First, the input image is binarized into black and white, and contour tracing is performed to extract a block of pixels surrounded by black pixel contours. For a block of black pixels having a large area, contour tracing is also performed on the white pixels inside to extract a block of white pixels. To extract the lump.
[0057]
The block of black pixels obtained in this manner is classified according to size and shape, and classified into blocks having different attributes. For example, a block whose aspect ratio is close to 1 and whose size is fixed is a pixel block equivalent to a character, a portion where adjacent characters can be grouped in a well-aligned manner is a character block, a flat pixel block is a line block, A table block shows the area occupied by black pixel blocks that are larger than the size and contain rectangular white pixel blocks in a well-aligned manner, a photo block shows areas where irregular pixel blocks are scattered, and other arbitrary pixel blocks Block.
[0058]
In the present embodiment, a character block is a character region, and a drawing block and a photo block are non-character regions. Further, a table block or a line block may be included as a non-character area.
[0059]
Next, in step S305, each non-character area in the page image is set as one piece of non-character image information, and the image feature amount is extracted. Then, the image feature amount of each non-character area and the position in the page image are registered as search data in the database 105 in association with the page image.
[0060]
It should be noted that the image feature amount extraction may be performed by using a known method, and the details will not be described here. As an example, a method of dividing a processing target image into meshes and vectorizing the average color of each mesh region as an element is used. is there.
[0061]
In addition to the example of extracting the average color of the image to be processed, there is also an example of extracting the most frequent color as the image feature amount. Furthermore, for example, a plurality of types of luminance feature amounts such as mode luminance, average luminance, co-occurrence matrix, contrast, entropy, texture feature amounts represented by Gabor transform, and shape feature amounts such as edges, Fourier descriptors, etc. One image feature amount or an image feature amount obtained by arbitrarily combining the image feature amounts may be used.
[0062]
Next, in step 306, character recognition is performed on all character regions in the page image to obtain a character code string, and this is registered in the database 105 as a character feature amount in association with the page image.
[0063]
The character recognition process may use a known method.For example, an edge component or the like is acquired from a character image and converted into a feature vector, and a feature vector in a recognition dictionary registered in advance for each character type and a similarity are calculated. There is a method in which the character code of the character having the highest similarity is used as the recognition result.
[0064]
Although a character code is used as the character feature amount, for example, the word part of the word may be extracted by performing matching with a word dictionary in advance, and the word that is a noun may be used as the character feature amount.
[0065]
In step S307, a word serving as a keyword is extracted from the character code string acquired in step S306, added to the character feature amount along with its position, and registered in the database 105 in association with the page image.
[0066]
There are various methods for keyword extraction. Here, as a result of analyzing character codes with a high frequency of appearance and part of speech analysis, character codes recognized as proper nouns are given priority, and up to five types of keywords are extracted. To However, this is only an example, and it goes without saying that other keyword extraction methods and an arbitrary number of keywords can be set.
[0067]
Next, the search processing will be described.
[0068]
In the search processing, a feature amount is extracted from a page image (search key image) obtained by scanning a paper document as a search key, and the extracted feature amount is compared with a feature amount group of a registered image (page image) registered in the database 105. A registered image having a high degree of similarity is output as a search result. Further, the user can execute various processes such as printing, distribution, accumulation, and editing on the output registered image.
[0069]
FIG. 5 is a flowchart showing a search process according to the embodiment of the present invention.
[0070]
First, in step S401, a paper document serving as a search key is input via the image reading unit 110 of the MFP 100. This process is the same as the process in step S302. However, the raster image (page image) generated by this process is only temporarily stored.
[0071]
Next, in step S402, a region division process is performed on the page image (search key image) to extract a character region and a non-character region in the search key image. This processing is the same as the processing in step S304. However, the position of each area generated by this processing is only temporarily stored.
[0072]
Next, in step S403, each non-character area in the search key image is set as one piece of non-character image information, and the image feature amount is extracted. Then, the image feature amount of each non-character area and the position in the page image are stored in the storage unit 111 as the image feature amount of the search key image in association with the search key image. This process is the same as the process in step S305.
[0073]
Next, in step S404, character recognition is performed on all character regions in the search key image to obtain a character code string, and this is stored as the character feature amount of the search key image in association with the search key image. The information is stored in the unit 111. This process is the same as the process in step S306.
[0074]
Next, in step S405, a word serving as a keyword is extracted from the character code string acquired in step S404, added to the character feature amount of the search key image along with its position, and associated with the search key image in the storage unit 111. To memorize. This process is the same as the process in step S307.
[0075]
Next, in step S406, between the search key image and each registered image (page image) registered in the database 105 in the registration processing of FIG. A similarity comparison is performed for keywords in a character area determined as a processing target based on the comparison, and a similarity is calculated as a result of the comparison.
[0076]
This similarity comparison performs the following three types of similarity comparisons.
[0077]
The first is the degree of similarity between the image features in the non-character area, and specifically, is calculated based on the closeness of the distance between the image features of both the search key image and the registered image in the feature vector space. I do. Here, when there are a plurality of non-character areas in the comparison target image (the registered image in the database 105), the comparison is performed with respect to the image feature amount of one non-character area in the search key image (comparison source image). The degree of similarity with the image feature amount of each of the plurality of non-character area images in the previous image is calculated, and by associating with the non-character area having the highest similarity, the cumulative similarity for the plurality of non-character areas is calculated.
[0078]
However, when the number of non-character areas in the comparison destination image is large and the number of combinations increases, the combination of non-character areas to be compared may be limited depending on the position. The similarity based on the image feature amount calculated as described above is defined as S1.
[0079]
The second is the similarity between the character feature amounts of the character areas, which is calculated based on the closeness between the character code strings of both the search key image and the registered image. However, the character codes obtained by character recognition include erroneous recognition due to various factors, and therefore do not originally completely match. Therefore, in the present embodiment, the probability that a plurality of keywords extracted from the character code string corresponding to the character area of the search key image are present in the character code string corresponding to the character area of the registered image is determined by the character feature. The similarity based on the quantity is defined as the similarity S2.
[0080]
Third, a similarity is calculated between the search key image and the registered image based on the positional relationship between the non-character area and the character area. Details of the similarity calculation processing will be described with reference to FIG.
[0081]
FIG. 6 is a flowchart showing the similarity calculation processing according to the embodiment of the present invention.
[0082]
First, in step S601, among the non-character region group in the search key image, it sets a non-character region to be processed in A _1. Incidentally, during the search key image, when the non-character area there is only one, and sets the non-character area A _1.
[0083]
Next, in step S602, among the non-character region in the reference image, to obtain the highest degree of similarity and the non-character areas A ₁ non-character region A _2. This processing may be acquired in the process of calculating the similarity based on the above-described image feature amount.
[0084]
Next, it is determined in step S603, the similarity is above a threshold, i.e., whether the non-character area A ₁ and the non-character area A ₂ are the same (or similar) images. If it is less than the threshold (NO in step S603), it is determined that the images are not the same (or similar), and the process proceeds to step S607. On the other hand, if it is equal to or larger than the threshold (YES in step S603), it is determined that the images are the same (or similar), and the process proceeds to step S604.
[0085]
Next, in step S604, referring to a character area group within a predetermined range of the non-character image area A1, a keyword group of a search key image included in the character area group is selected, and these are referred to as {N ₁₁ , N ₁₂ ,…｝
[0086]
Here, a specific example of step S604 will be described with reference to FIG.
[0087]
FIG. 7 is a diagram for explaining an example of selecting a keyword in a character area according to the embodiment of the present invention.
[0088]
In the example of FIG. 7, a case is shown in which a keyword whose center is located at the center of gravity of the non-character area 700 and whose side is within a 2 × L square range is selected from the character area. In FIG. 7, when the keywords of the search key image are “yyyy” and “xxxx”, the non-character area 700 is within a predetermined range (in this embodiment, a 2 × L square range). An example is shown in which, of the character areas 701 to 703, a keyword “yyy” is selected from the character area 701 and a keyword “xxxx” is selected from the character area 702.
[0089]
Note that the value of L may be fixed or may be changed according to the size of the non-character area to be processed or the amount of characters in a nearby character area.
[0090]
Next, in step S605, referring to a character area group within a predetermined range of the non-character area A2 in the registered image, a keyword group of the registered image included in the character area group is selected, and these are set to ｛N ₂₁ , N ₂₂ ,...}.
[0091]
Next, in step _S606, the based on the correlation of _{{N 11, N 12, ...} } × {N 21, N 22, ...}, calculates a score indicating a correlation.
[0092]
Specifically, if there is a matching keyword in the keyword group {N ₂₁ , N ₂₂ ,...} For the first keyword N ₁₁ , the score is set to +1 and the final number of matching keywords is It is calculated as a score indicating the correlation. Alternatively, with respect to the keyword N _2n matching the keyword N _11, the search key image and the center of gravity of the non-character region both between the registered image, close to the ratio of the distance between keywords in center of gravity 1 keyword or non-character, A keyword with a high degree of coincidence of the angle from the center of gravity of the region to the center of gravity of the keyword to be processed may be greatly added.
[0093]
Hereinafter, similarly, the keywords to be processed are changed, the score with each keyword is calculated, and the total value of the scores of each keyword is set as the similarity S3.
[0094]
Then, for the calculated three types of similarities S1, S2, and S3, using the coefficients (a, b, and c),
Total similarity S = a × S1 + b × S2 + c × S3
Is calculated.
[0095]
Note that the coefficients (a, b, c) may be fixed values, may be changed by the distribution of character areas and non-character areas in the processing target image, or may be arbitrarily set by the user when executing the search processing. May be. Here, a is a coefficient defining the weight of the similarity comparison based on the image feature amount, b is a coefficient defining the weight of the similarity comparison based on the character feature amount, and c is a coefficient defining the weighting of the similarity comparison based on the keyword. can do.
[0096]
Then, in step S607, it is determined whether there is an unprocessed non-character area in the search key image. If there is a non-character area unprocessed (YES in step S607), the non-character area of the untreated in order to set the A _1, returns to step S601, for all the non-character region in the key image, Until the processing in FIG. 6 is completed, the non-character area to be processed is switched and the processing is repeated recursively. On the other hand, if there is no unprocessed non-character area (YES in step S607), the processing ends, and the flow advances to step S407 in FIG.
[0097]
In step S407, a registered image serving as a search result is displayed on the display unit 116 based on the calculated overall similarity 407. For this display, for example, only registered images having the highest overall similarity may be displayed, or registered images having overall similarities equal to or greater than a threshold are displayed as a list of search result candidates, and the final selection is presented to the user. It may be entrusted.
[0098]
In step S408, any one of printing, distribution, storage, and editing of the registered image serving as a search result is performed based on an operation from the user via the user interface realized by the display unit 106 and the input unit 114. The processing of is performed.
[0099]
As described above, according to the present embodiment, a printed matter printed using a registered image is input as a search key image to a database for managing registered images registered by a paper document or an electronic document. Then, a registered image corresponding to the search key image is searched based on the character feature amount of the search key image, the image feature amount, and the positional relationship between the keyword in the character region of the search key image and the non-character region.
[0100]
As a result, even if there are a plurality of registered images having similar contents, the registered image corresponding to the search key image can be more appropriately searched.
[0101]
More specifically, in the present embodiment, even when a character area and a non-character area are mixed and a large number of registered images having similar configurations are registered, the non-character area and the character area By using the positional relationship of the keyword, the registered image corresponding to the search key image can be more accurately searched.
[0102]
Moreover, even when the layout of the contents between the search key image and the registered image does not exactly match, a flexible search can be performed.
[0103]
As described above, the embodiment has been described in detail. However, the present invention can take an embodiment as, for example, a system, an apparatus, a method, a program, a storage medium, or the like. The system may be applied to a system including a single device or an apparatus including one device.
[0104]
According to the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the drawings) for realizing the functions of the above-described embodiment is directly or remotely supplied to a system or an apparatus, and a computer of the system or the apparatus is supplied. Is also achieved by reading and executing the supplied program code.
[0105]
Therefore, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. That is, the present invention includes the computer program itself for realizing the functional processing of the present invention.
[0106]
In that case, as long as it has the function of the program, it may be in the form of object code, a program executed by the interpreter, script data supplied to the OS, or the like.
[0107]
As a recording medium for supplying the program, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, non-volatile memory card , ROM, DVD (DVD-ROM, DVD-R) and the like.
[0108]
In addition, as a method of supplying the program, a client computer connects to an Internet homepage using a browser, and downloads the computer program itself of the present invention or a compressed file including an automatic installation function to a recording medium such as a hard disk from the homepage. Can also be supplied. Further, the present invention can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. In other words, the present invention also includes a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.
[0109]
In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and downloaded to a user who satisfies predetermined conditions from a homepage via the Internet to download key information for decryption. It is also possible to execute the encrypted program by using the key information and install the program on a computer to realize the program.
[0110]
The functions of the above-described embodiments are implemented when the computer executes the read program, and an OS or the like running on the computer executes a part of the actual processing based on the instructions of the program. Alternatively, all the operations are performed, and the functions of the above-described embodiments can be realized by the processing.
[0111]
Further, after the program read from the recording medium is written into the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board or the A CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the processing also realizes the functions of the above-described embodiments.
[0112]
【The invention's effect】
As described above, according to the present invention, it is possible to provide an image search apparatus capable of accurately searching original electronic data from a paper document, a control method thereof, and a program.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an image processing system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a detailed configuration of the MFP according to the embodiment of the present invention.
FIG. 3 is a flowchart illustrating registration processing of an electronic document according to the embodiment of the present invention.
FIG. 4 is a diagram showing an example of image block extraction according to the embodiment of the present invention.
FIG. 5 is a flowchart illustrating a search process according to the embodiment of the present invention.
FIG. 6 is a flowchart illustrating a similarity calculation process according to the embodiment of the present invention.
FIG. 7 is a diagram illustrating an example of selecting a keyword in a character area according to the embodiment of the present invention.
[Explanation of symbols]
100 MFP
101 Management PC
102 User PC
103 Proxy server 104 Network 105 Database 106 Document management server 107 LAN
110 Image reading unit 111 Storage unit 112 Printing unit 113 Input unit 114, 117 Network I / F
115 Data processing unit 116 Display unit

Claims (8)

An image search device for inputting a printed material as a search key image and searching for a registered image corresponding to the search key image,
Region dividing means for dividing the search key image into a character region and a non-character region,
Calculating means for calculating a similarity between the search key image and the registered image based on a character feature amount of the character region, an image feature amount of the non-character region, and a positional relationship between the character region and the non-character region. When,
An image search device comprising: a search unit that searches for a registered image corresponding to the search key image based on a calculation result by the calculation unit. The apparatus according to claim 1, wherein the character feature amount is a character code obtained by character recognition of the character area. The calculating means, when the similarity based on the image feature amount of the non-character area is equal to or more than a threshold, based on the positional relationship between the predetermined character string in the character area and the non-character area, the search key image and The apparatus according to claim 1, wherein a similarity between the registered images is calculated. The calculation means calculates, as the similarity, a correlation between predetermined character strings in the character region located within a predetermined range of the non-character region to be processed for both the search key image and the registered image. The image search device according to claim 1. The calculation means compares the distance between the non-character area to be processed for both the search key image and the registered image and a predetermined character string in the character area located within the predetermined range, and the comparison result The image search device according to claim 1, wherein the similarity is calculated based on: The calculation means compares angles from the non-character area to be processed for both the search key image and the registered image to a predetermined character string in the character area located within the predetermined range, and compares the comparison result. The image search device according to claim 1, wherein the similarity is calculated based on: An image search method for inputting a printed material as a search key image and searching for a registered image corresponding to the search key image,
An area dividing step of dividing the search key image into a character area and a non-character area;
Based on the character feature amount of the character region, the image feature amount of the non-character region, and the positional relationship between the character region and the non-character region, the search key image and a plurality of registered images stored in the storage medium are used. Calculating a similarity of
A search step of searching a registered image corresponding to the search key image from a plurality of registered images stored in the storage medium based on a result of the calculation in the calculation step. A program for inputting a printed material as a search key image and implementing an image search for searching for a registered image corresponding to the search key image,
Program code of an area dividing step of dividing the search key image into a character area and a non-character area,
Based on the character feature amount of the character region, the image feature amount of the non-character region, and the positional relationship between the character region and the non-character region, the search key image and a plurality of registered images stored in the storage medium are used. Program code for a calculation step of calculating the similarity of
A program code for a search step of searching a registered image corresponding to the search key image from a plurality of registered images stored in the storage medium, based on a result of the calculation in the calculation step.

Images