JP5660574B2 - Document image database registration and search method - Google Patents

Description

  The present invention relates to a document image database registration method and search method. More specifically, the present invention relates to a registration method and a search method for extracting feature points from a document image and registering them together with the document image in a document image database. The present invention relates to application to a document image database.

  In recent years, digital cameras are commonly attached in the field of mobile phones. In addition, the quality is remarkably improved, which is comparable to that of a normal digital camera. As a result, a situation occurs in which a general user always carries a high-quality digital camera. Therefore, image retrieval using an image taken with a digital camera has attracted attention. In the present invention, document image search is considered among image searches.

In this specification, a document image refers to a document or an image including a document.
The document image search is a process of finding a document image corresponding to a given search question (query) from a database. Among them, document image search using a digital camera uses a document image taken with a digital camera as a search question. By using a document image search of such a format, it is possible to apply to various services by photographing a printed document and retrieving the photographed printed document using a retrieval question. Specifically, services such as obtaining references by taking academic papers and accessing related websites can be considered.

  Locally Likely Arrangement Hashing (LLAH) has been proposed as a technique for retrieving a document image corresponding to a photographed image from a database (see, for example, Patent Documents 1 and 2 and Non-Patent Document 1). LLAH is a method of searching for a feature amount from the feature point arrangement using, for example, the centroid of a word in a document image as a feature point. A characteristic of LLAH is that it is robust against disturbances such as changes in the shooting direction, hiding, and curvature of the paper that occur in practical use. In addition, it is fast enough to realize real-time search by repeating simple search. This is because the calculation amount of the feature amount calculation is O (N) with respect to the number N of feature points to be searched (for example, see Non-Patent Document 2). Here, O (N) is a notation method of the approximate amount of calculation required to solve the problem, and O (N) is the amount of calculation when N is determined as a × N + b (a and b are constants) ) Indicates that it fits below. Due to these properties, applications such as augmented reality (for example, see Non-Patent Documents 3 and 4) and camera pen systems (for example, see Non-Patent Document 5) are applied to documents.

JP 2009-32109 A JP 2009-70066 A

Tomohiro Nakai, Koichi Kise, Masakazu Iwamura, “Use of Affine and Similar Invariants in High-speed Document Image Retrieval Using Digital Cameras”, IEICE Technical Report, vol.105, no.PRMU-614, pp.25-30, Feb. 2006. Tomohiro Nakai, Koichi Kise, Masakazu Iwamura, “High-speed document image retrieval using a digital camera based on local arrangement of feature points”, IEICE Transactions D, vol.J89-D, no.9, pp. 2045-2054, Sept. 2006. R.T.Azuma, “A survey of augmented reality,” Presence, vol.6, no.4, pp.355-385, 1997. Tomohiro Nakai, Koichi Kise, Masakazu Iwamura, “Real-time document image retrieval based on local arrangement of feature points and its application to augmented reality”, IEICE technical report, vol.106, no.PRMU-229, pp.41-48, Sept. 2006. Megumi Konno, Kazumasa Iwata, Koichi Kise, Masakazu Iwamura, Seiichi Uchida, Shinichiro Omachi, “A Method for Improving Document Image Retrieval Considering Projection Distortion in Camera Pen Systems”, Image Recognition and Understanding Symposium (MIRU2010) , Pp.239-246, July 2010.

  On the other hand, LLAH has an aspect of using a large amount of memory in order to achieve high accuracy and robustness. As a specific example, if 10,000 pages of document images are registered in the database, about 200 MB of memory is required, and about 150 GB of memory is required to register 10 million pages of document images. Such poor memory efficiency limits the scalability of LLAH. Further, as the number of registered pages in the database increases, the possibility that documents having similar feature point arrangements are registered increases. This is thought to cause a decrease in search accuracy. Therefore, in order to increase the scale, a feature quantity having higher discriminability is required.

  The present invention has been made in consideration of the above-described circumstances, and solves the problem of memory efficiency of LLAH and the problem of distinguishability of feature quantities that become apparent as the document image database becomes larger. Provide improvement methods to solve.

This invention
(I) A feature point extracting step in which a computer extracts a feature point representing a local feature of a document image from a document image to be registered in the document image database, and each feature point using an invariant for geometric transformation The characteristic values of each feature point are determined by a predetermined calculation for a plurality of geometric elements determined by each feature point and its neighboring n (n is a natural number) feature points. (1) for a feature quantity calculating step in which a plurality of invariants are combined, and a feature quantity is a vector whose dimension is the calculated invariant, and each feature point extracted from the document image A reference set used for referring to the document image, (2) an identifier for distinguishing the feature point from other feature points, and (3) a data set in which the features (1) to (3) of the feature point are associated with each other. Generate the data A registration step of registering a data set together with the document image in the document image database, wherein the feature point extraction step determines a connected component of lines constituting the document, and features a centroid of the connected component smaller than a predetermined area. When the number of extracted feature points is less than the threshold value, the centroid of the connected components in the vicinity of each feature point is further extracted as a feature point so that feature points above the threshold value can be obtained. In the document image database, the feature amount calculating step calculates each invariant using a geometric element that does not overlap with a geometric element used for one invariant as another invariant. Provide a method.

Furthermore, this invention
(II) A feature point extraction step in which a computer extracts a plurality of query feature points representing local features of a document image from a document image taken as a search question, and calculates a query feature amount related to each query feature point A feature quantity calculation step, and a feature quantity that is the same as or similar to each query feature point from a data set registered in the document image database with reference to the document image database in which the document image is registered by the registration method. Search for a data set that has an individual search step to obtain a reference for the found data set, search for each query feature point, statistically process each obtained document identifier, and recommend it as a search result A voting step for specifying a document image to be determined, and the feature point extracting step determines a connected component of lines constituting the document and specifies a center of gravity of each connected component. Extracted as points, the feature amount calculation step

Is obtained by calculating each characteristic value by a predetermined operation for a plurality of geometric elements determined by each feature point and its neighboring n (n is a natural number) feature points, and combining those characteristic values Calculate multiple invariants and calculate each invariant using a geometric element that does not overlap with the geometric element used for one invariant for the other invariant. Provided is a document image search method characterized in that a dimension vector is used as the feature amount.

  In the present invention, the basic idea for reducing the memory usage of the conventional LLAH is to sample feature points stored in the database. In addition, the number of dimensions is increased in order to improve the distinguishability of the feature amount. However, if the number of dimensions is simply increased, the stability of the feature amount is impaired (see Non-Patent Document 1 described above). Thus, the number of dimensions of the feature quantity is increased and at the same time redundant dimensions are deleted. As a result, it is possible to improve the stability of the feature quantity and realize high search accuracy while preventing loss of information.

In addition, this invention
(III) A feature point extracting step in which a computer extracts a feature point representing a local feature of a document image from a document image to be registered in the document image database, and each feature point using an invariant for geometric transformation The characteristic values of each feature point are determined by a predetermined calculation for a plurality of geometric elements determined by each feature point and its neighboring n (n is a natural number) feature points. (1) for a feature quantity calculating step in which a plurality of invariants are combined, and a feature quantity is a vector whose dimension is the calculated invariant, and each feature point extracted from the document image A reference set used for referring to the document image, (2) an identifier for distinguishing the feature point from other feature points, and (3) a data set in which the features (1) to (3) of the feature point are associated with each other. Generate the data A registration step of registering a data set together with the document image in the document image database, wherein the feature point extraction step determines a connected component of lines constituting the document, and features a centroid of the connected component smaller than a predetermined area. When the number of extracted feature points is less than the threshold value, the centroid of the connected component in the vicinity of each feature point is further extracted as the feature point so that a feature point equal to or higher than the threshold value can be obtained. A document image database registration method is provided.

Furthermore, this invention
(IV) A feature point extraction step in which a computer extracts a plurality of query feature points representing local features of a document image from a document image taken as a search question, and calculates a query feature amount related to each query feature point A feature quantity calculation step, and a feature quantity that is the same as or similar to each query feature point from a data set registered in the document image database with reference to the document image database in which the document image is registered by the registration method. Search for a data set that has an individual search step to obtain a reference for the found data set, search for each query feature point, statistically process each obtained document identifier, and recommend it as a search result and executes the Tohyo Suteppu identifying the Subeki Bunsho Gazo, the feature point extraction Suteppu is to Kettei the Renketsu Seibun of lines that constituting the document, the center of gravity of Kaku Renketsu Seibun The feature amount calculating step extracts each characteristic value by performing a predetermined operation on a plurality of geometric elements determined by each feature point and n feature points in the vicinity (n is a natural number). There is provided a document image search method characterized in that a plurality of invariants obtained by combining these characteristic values are calculated, and a vector having each dimension of the calculated invariants is used as the feature amount.

  When it is necessary to prioritize the reduction of the memory amount over the search accuracy, there may be a mode in which the redundant dimension is not reduced instead of increasing the number of feature quantity dimensions.

  In the registration methods of (I) and (III), the feature point extraction step extracts the centroids of a plurality of connected components smaller than a predetermined area as feature points, and when the number of extracted feature points is less than a threshold value, The centroid of the connected components in the vicinity of each feature point is further extracted as a feature point so that feature points above the threshold can be obtained, so it is extracted compared to the case where the centroid of all connected components is used as the feature point. The number of feature points to be reduced is reduced, and the required amount of memory can be saved.

Further, in the registration method of (I), the feature amount calculating step calculates each invariant using a geometric element that does not overlap with a geometric element used for one invariant as another invariant. Therefore, the feature quantity formed by combining these invariants does not have the relationship between the invariants that the same characteristic value is used for calculating different feature quantities, that is, there is no kind of redundancy. Therefore, each feature point is represented by a feature amount having high discrimination and eliminating the redundancy. In other words, each feature point is represented by a stable feature amount in which the influence of errors due to projection distortion, noise, and / or a difference in resolution is suppressed.
According to the present invention, it is possible to apply to a large-scale document image database in which the number of registered pages ranges from 1 million to 10 million.

  The search methods (II) and (IV) provide a search method used for the document image database created by the registration method described above, so that the number of registered pages ranges from 1 million to 10 million. A document image database can be searched.

  A local feature of a document image refers to a geometric feature of a character, word, or image described in a partial area in the document image. In the embodiment, the present invention refers to a geometric feature indicated by a region corresponding to a word, but is not necessarily limited thereto.

  A geometric transformation is a transformation that rotates, translates, scales, shears, and / or fans out a document image. There is Euclidean transformation including rotation and translation, similarity transformation with scaling, affine transformation with shear deformation, and projection transformation with polygons added to the back (see, for example, “Computer Vision” Visual geometry-", 1st edition, Corona, Inc., July 30, 2003, p.42-65).

  An invariant is a value that does not change through geometric transformation, and there is an invariant according to the type of geometric transformation. For example, the area of a figure is an example of an invariant for Euclidean transformation. The ratio of side lengths is an example of an invariant for similarity transformation. As will be described later, the area ratio of a figure is an example of an invariant (affine invariant) for affine transformation. The cross ratio is an example of an invariant (projection invariant) for the projective transformation. If the area of the figure is small, the above-mentioned area ratio can be treated approximately as a projection invariant.

  In the present invention, a characteristic value obtained from a geometric element by a predetermined calculation refers to an element constituting an invariant. Specifically, the area of the figure and the length of the side relating to the invariant correspond to the characteristic values.

  The connected component is a component in which lines constituting the characters of the word are connected, or a component connected when the line is blurred by a predetermined amount. A connected component is a collection of pixels in an image that are connected to each other. In the embodiment described later, a connected component when the resolution is lowered by performing the blurring process is employed. The amount of blurring processing may be set to such an extent that one connected component corresponds to a word in a document such as English to be written separately. In a document such as Japanese that is not divided, one connected component may be set to an extent corresponding to, for example, “hen” or “making” that constitutes a kanji.

  The statistical processing of the document identifier is processing for selecting an appropriate document identifier from among a large number of document identifiers listed as options. Specifically, a voting process can be mentioned.

  The distinguishability of a feature point is a characteristic of the feature point that makes it possible to distinguish one feature point from another. The stability of feature points refers to the property that feature quantities related to feature points are less susceptible to errors due to distortion, noise, and / or resolution differences.

  As will be described later, as a result of experiments using a database in which 10,000 pages of document images were registered, it was confirmed that the memory usage could be reduced by about 70% by sampling feature points. In an experiment using a 10 million page database, it was confirmed that search was possible with a search accuracy of 99.4% and a processing time of 38 ms.

It is explanatory drawing which shows the process of the conventional LLAH used as this invention foundation. It is explanatory drawing which shows the structure of the hash table used by conventional LLAH used as the foundation of this invention. It is explanatory drawing which shows the sampling of the feature point used by improved LLAH of this invention. It is explanatory drawing explaining deletion of the redundant dimension of the feature-value used by improved LLAH of this invention. It is explanatory drawing which shows an example of the document image registered into the database in the effect verification experiment of the improved LLAH of this invention. FIG. 6 is an explanatory diagram showing an example of a search question in the LLAH effect demonstration experiment of the present invention. It is a graph which shows the relationship between the number of registration pages, and a required memory amount as a result of the verification experiment of the improved LLAH of this invention. It is a graph which shows the relationship between the number of registration pages and search precision as a result of the verification experiment of the improved LLAH of this invention. It is explanatory drawing which shows the example of the document image which failed in the search in the verification experiment of the effect of improved LLAH of this invention. It is a graph which shows the relationship between the number of registration pages, and processing time as a result of the effect verification experiment of the improved LLAH of this invention. It is a graph which shows the relationship between the number of registration pages, and average list length as a result of the effect verification experiment of the improved LLAH of this invention. It is explanatory drawing which shows the 1st example of the search question used in order to investigate the tolerance of partial imaging | photography in the effect verification experiment of the improved LLAH of this invention. It is explanatory drawing which shows the 1st example of the search question used in order to investigate the tolerance of partial imaging | photography in the effect verification experiment of the improved LLAH of this invention. It is explanatory drawing which shows the 1st example of the search question used in order to investigate the tolerance of partial imaging | photography in the effect verification experiment of the improved LLAH of this invention. It is a graph which shows the difference in the search precision by the imaging | photography range as a result of the verification experiment of the improved LLAH of this invention. It is explanatory drawing which shows the example of the document image which failed in the search by partial imaging | photography in the effect verification experiment of the improved LLAH of this invention. It is explanatory drawing which shows the example of the feature point which failed in the search in the effect verification experiment of the improved LLAH of this invention. (A) is a failure example of “” ”and (b) is a failure example of“. ”. 6 is a graph showing a difference in average list length depending on whether or not a feature amount is selected in the LLAH effect demonstration experiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described.
In the registration method of the present invention, the registration step includes (1) the reference, (2) the identifier, and (3) the feature quantity or a simplified feature quantity obtained by simplifying the feature quantity (1) to (3 ) Is generated, and a predefined calculation is performed to classify each feature point according to the feature amount to obtain an index of a class to which the feature point belongs. Data sets may be classified and registered in the document image database together with the document images. In this way, since the data sets are classified and registered, the search target is narrowed down to the feature points registered in any one of the classes, and the processing time required for the search can be shortened.

  In the feature quantity calculating step, a triangle having three feature points as vertices is used as the geometric element, the area of the triangle is used as the characteristic value, and any three of four or more feature points are used as vertices. An area ratio of two triangles sharing one side in a combination of triangles may be set as one invariant. In this way, the figure area ratio as an affine invariant can be obtained from the smallest feature point. Further, since the area of a triangle having a feature point as a vertex is smaller than a quadrilateral or more figure having those feature points as vertices, it has approximate characteristics more excellent as a projection invariant.

In the search method of the present invention, for the search method of the document image database in which the data set is classified and registered in a class according to the index, the feature point extraction step determines a connected component of lines constituting the document , The center of gravity of each connected component is extracted as a feature point, and the feature amount calculation step performs a predetermined operation on a plurality of geometric elements determined by each feature point and n feature points in the vicinity (n is a natural number). Each characteristic value is obtained by calculating a plurality of invariants by combining these characteristic values, and a geometric element that does not overlap with the geometric element used for one invariant is changed to another invariant. Each invariant is used to calculate a vector having the calculated invariant as each dimension as the feature quantity, and the individual search step obtains an index of each query feature point by calculation corresponding to the registration method. , Referring to a data set registered as a class related to the index, and if the data set has a simple feature quantity, the simple feature quantity related to the query feature point is obtained and then the same or most similar simple feature quantity is obtained. It is also possible to obtain a reference related to a data set having the same or the most similar feature quantity when the registered data set has a feature quantity. In this way, a search method is also provided for a document image database in which data sets are classified and registered according to the index. Since the data set is classified, the search can be performed in a short time.

In the search method of the present invention, the feature amount calculating step includes a triangle having three feature points as vertices as the geometric element, an area of the triangle as the characteristic value, and any of four or more feature points. Alternatively, the area ratio of two triangles sharing one side in a combination of triangles having three vertices as vertices may be set as one invariant.
Preferred embodiments of the present invention include combinations of any of the plurality of embodiments shown here.

Hereinafter, the present invention will be described in more detail with reference to the drawings. In addition, the following description is an illustration in all the points, Comprising: It should not be interpreted as limiting this invention.
In order to facilitate understanding of the present invention, a conventional LLAH that is the basis of the present invention will be briefly described here.

≪Document image search by conventional LLAH≫
1. Outline of Processing FIG. 1 is an explanatory diagram showing an outline of processing of document image retrieval by conventional LLAH which is the basis of the present invention. As shown in FIG. 1, the conventional LLAH is a stage for building a document image search environment. When a document image to be registered is given, feature points are extracted from the document image and stored in a document image database. Applies to the registration stage. Furthermore, when a document image as a search question is given, the conventional LLAH is applied at the stage of extracting feature points from the search question and searching the document image corresponding to the search question from the document image database. . Both registration and search are executed by a computer. In this specification, a computer is used as a general term for devices that perform processing according to a program, such as a CPU or a microcomputer. The computer that executes the process is not limited to a single CPU configuration, but includes a configuration of a plurality of CPUs such as so-called cloud computing and grid computing. Furthermore, the computer that executes the process related to the search may not be the same as the computer that executes the process related to the registration.

  In the registration and the search, the computer executes a common feature point extraction process. In the feature point extraction process, the computer extracts a plurality of feature points from the document image and represents the document image as a set of feature points. Furthermore, the computer executes registration processing or search processing using the extracted feature points. That is, registration processing is executed at the registration stage, and search processing is executed at the search stage. The registration process and the search process include a common feature amount calculation process.

  In the registration process, the computer calculates a feature amount for each feature point extracted from the document image to be registered. Then, the computer registers the feature amount calculated from each feature point and the reference code for the document image from which the feature point is extracted in the document image database in association with the feature point. That is, the registration process registers the document image in the document image database so that the document image is indexed by the feature amount.

In the search process, the computer calculates a feature amount for each feature point extracted from the given search query image, as in the registration process. The computer accesses the document image database using the feature amount calculated from each feature point. That is, if a document image including a feature point similar to each feature point of the search query image is registered, a reference code for the document image is obtained. A large number of feature points are usually extracted from the search query. Therefore, a voting process using these feature points as a vote unit is performed to obtain a reference code for the document image corresponding to the search question.
The voting process generally refers to a process of assigning a score to any of the options based on the obtained evidence and selecting the option based on the result of totaling all the evidence. Normally, the option with the highest number of votes is selected, but not only from the number of votes, but also from the statistical point of view, such as whether the number of votes exceeds the specified number of votes, or whether there is a difference in the number of votes from lower options By evaluating, a plurality of options may be selected or nothing may be selected. Details of the processing will be described below.

2. Feature Point Extraction Processing In conventional LLAH, a computer calculates feature amounts based on the arrangement of feature points, and matches document images using the calculated feature amounts. Therefore, in the feature point extraction processing, it is necessary to extract the same feature point even if a given document image has projection distortion or noise or has a low resolution. One method focuses on an area occupied by words constituting the document of the document image (hereinafter, word area) and uses the centroid of the word area as a feature point. In addition, for documents that are not separated, such as Japanese, instead of the word area, focus on the connected components that make up the kanji characters "hen" and "make". And a technique for calculating a feature amount related to the center of gravity (feature point) using the area ranking has been proposed (see, for example, paragraphs 0006, 0043, and 0046 of Patent Document 1). It can be said that the word region is a connected component obtained by blurring the lines constituting the characters. Hereinafter, word regions as examples of connected components will be described.

  The procedure for extracting the centroid of the word region as a feature point is shown below. First, a given document image is adaptively binarized to obtain a binary image. Next, when the binary image is blurred with a Gaussian filter and adaptive binarization is performed again, an image concatenated for each word is obtained. Finally, the centroid of the connected component is calculated as a feature point.

3. Feature Quantity Calculation A feature quantity is a value that represents a feature point of a document image, and matching of feature points is performed based on the feature quantity. A robust feature is required for accurate search. When a given document image is taken by a camera, projection distortion is usually included due to the positional relationship between the document image and the camera. Therefore, it is necessary to use a geometric invariant that is invariant to the projection distortion as the feature quantity. Therefore, as a feature amount corresponding to a certain feature point S0, a value obtained by the following expression is calculated from four neighboring points ABCD including the point S0 and used as the feature amount.

  Here, P (A, B, C) is the area of a triangle composed of vertices ABC, and these two triangles always share one side. Equation (1), which is the area ratio of two triangles, is an affine invariant. Furthermore, the area ratio of the triangle is a value that can be approximated to a projection invariant in a local area having a small area. LLAH uses affine invariants to achieve robustness against projection distortion.

It is assumed that a document image is registered in the document image database in a state where there is no projection distortion, and when a search is performed using a search question corresponding to the document image, the search question includes projection distortion. In that case, the four neighboring points of the feature point S0 extracted from the search question may differ from the four neighboring points of the feature point extracted from the registered document image. This is the case when the distance relationship between the point S0 and the neighboring feature points differs from the original document image due to the influence of the projection distortion. Even so, since the distance from the point m of the neighboring n points (m and n are 4 ≦ m a natural number <n) is considered to be likely be common to obtain, all of the combinations _n C _Let's examine _m streets. That is, _n C _m feature values are calculated for each feature point. The combination is ways _m C ₄ to select the four points necessary for the calculation of the affine invariant from the point m. An affine invariant is obtained from all combinations, and a sequence of _m C ₄ numerical values (r ₍₀₎ , r ₍₁₎ , ..., r _(mC4) ) is used as a feature quantity.

  In addition, when four points are selected from the neighboring m points, a method has been proposed in which a rule for selecting the four points from which point and in which order to calculate the affine invariant is determined, and the amount of invariant calculation is reduced. (For example, refer to paragraph 0049 of the specification of Patent Document 2). This is to shorten the processing time.

In addition, the feature obtained from the sequence (r ₍₀₎ , r ₍₁₎ , ..., r _(mC4) ) obtained as described above is added to the feature obtained from the area of the connected component obtained in the feature point extraction process. Add more quantity. First, the area of the connected components at the neighborhood m points used in the calculation of the affine invariant is obtained. Next, a number is assigned to a set of adjacent connected components, and the area ratio of the connected components is calculated. That is, a total of m connected component sets are created. Then, the numbers of sets of connected components are arranged in the order of the area ratio. The m numeric columns are added to the already obtained _m C ₄ columns (r ₍₀₎ , r ₍₁₎ ,..., R _(mC4) ). Therefore, _n C _m feature quantities are obtained for each feature point, and each feature quantity is represented by a sequence of ( _m C ₄ + m) numeric values. That is, each feature amount is represented as a ( _m C ₄ + m) -dimensional vector (feature vector) having the above-described numerical sequence as an element.

4). Registration Process In the registration process, the computer registers each feature point in the hash according to the feature amount. All document images are registered in the same hash. The index H _{index of the} hash table is calculated by the following hash function.

Here, r _(i) is the value of each dimension of the feature quantity, d is the number of discretization levels, and H _size is the _size of the hash table (number of bins). The quotient Q is uniquely determined for all feature quantities. That is, the same H _index and quotient Q can be obtained from the same feature vector. In the search process, a feature point corresponding to each feature point extracted from the search question is searched for among the feature points registered in the document image database. The search is performed by checking the index H _index and the quotient Q. And can be divided into That is, the search target can be narrowed down first to a class of bins having an index H _index equal to each feature point extracted from the search question. Next, the feature points registered in the bin are collated with the feature points extracted from the search question. The matching with the feature points registered in the same bin can be performed by comparing the quotients Q instead of comparing the ( _m C ₄ + m) dimensions of the feature vectors. The quotient Q can be said to be a characteristic value obtained by simplifying the feature value for collation.

Therefore, as shown in FIG. 2, the data set of the document ID, the point ID, and the quotient Q is registered in the hash table. The hash table has a bin structure partitioned for each index (here, the value of H _index ), and is a data structure that classifies and registers data to be registered into bins according to the value of the index. The data registered in the hash table by LLAH is a data set of document ID, point ID, and quotient Q as described above. Here, the document ID is an identification number of the document image and corresponds to the above-mentioned reference code. The point ID is a unique number assigned to a feature point to distinguish it from other feature points of the document image. Data is registered in the bin according to the index value. When there are a plurality of data having the same index value among the data to be registered, these data must be registered in the same bin. This state is called hash collision. If a collision occurs, those data are registered in the bin in the form of a list. A limit is set for the length of the list. When the limit value is exceeded, all data linked in the list is deleted together with the index. Thereafter, the deleted index is not used. That is, the data set corresponding to the index value is not registered in the hash table. This is because many of the feature points related to these data sets are similar and are considered not useful for searching. In order to reduce the memory usage, a method for deleting all lists in which hash collisions occur and simplifying the hash table has already been proposed (see, for example, paragraphs 0044 and 0045 of the specification of Patent Document 2). .

5. Search Process At the search stage, feature points are extracted from the search question, feature quantities of each feature point are calculated, and an index (value of H _index ) and a quotient Q are obtained. Using the obtained index, if there is a data set registered in the corresponding bin, it is referred to. Check if the quotient Q matches for the referenced data set. It is an individual feature point search. An individual feature point search is performed for each feature point extracted from the search question. If the quotient Q of the referenced data set matches, vote for the document ID of that data set. This is the voting process. A voting table with a counter with an initial value of zero is prepared for each document ID. The computer increments the counter of the document ID to be voted out of the document IDs in the voting table. Refer to the hash table for each feature point of the search question, and vote according to the contents of the bin. The document image with the document ID that obtained the maximum number of votes is output as the correct image.

≪Improvements of LLAH according to this invention≫
1. Reduction of memory usage In conventional LLAH, all feature points obtained in the feature point extraction process are stored in a hash table in principle. The only exception is when the number of hash collisions (list length) exceeds the limit. As a result, the amount of memory used increases. However, it is possible to search without saving all feature points. This is because voting is used for the search. Therefore, feature points are sampled to reduce the amount of data stored in the hash. As a result, the memory usage is reduced.

It should be noted when sampling feature points that consideration should be given so as not to cause positional bias in the feature points registered in the hash.
If the distribution of the feature points in the document image can be made sparse / dense, it is considered that the correct image cannot obtain a sufficient number of votes in the sparse part, resulting in a decrease in accuracy.
That is, in order to be robust in the shooting range, the feature points to be sampled must be distributed evenly to some extent.

Further, it is important that the feature points to be sampled are effective for the search.
Therefore, in the present invention, sampling is performed by paying attention to the area of the connected component. An example of sampling is shown in FIG. FIG. 3 shows a part of the document image. S0 to S6 are symbols indicating respective feature points obtained as centroids of the six connected components. Now, focusing on the word “in” in the center, it can be seen that the connected component has a smaller area than the surrounding connected components. In sampling, the feature point S0 extracted from such a connected component is adopted as an effective feature point. A connected component having an area smaller than that of the surrounding is a connected component having a small number of characters, and frequently appears in a document as a preposition or article. Therefore, it can be considered that sampling without positional deviation can be performed.

  Moreover, the fact that the area of the connected component is small means that the distance from nearby feature points is small. It is considered that the smaller the distance from the nearby feature points S1 to S6, the less affected by the projective distortion, and the more stable feature value can be obtained. However, there are cases where the number of feature points registered in the hash is too small. Therefore, k points near the sampled feature points are also registered in the hash. In this embodiment, the value of k is set so that the number of feature points to be registered is about 200 per document. Therefore, for a document image having less than 200 feature points, the feature points are not sampled and all feature points are registered in the hash. In addition, it is difficult to determine the presence or absence of sampling during a search. Therefore, at the time of retrieval, the hash is referred to for all feature points obtained from the retrieval question.

2. Improvement of distinguishability and stability of feature points As the database becomes larger, the amount of features registered in the hash table becomes enormous. Therefore, it is generally considered that the number of hash collisions increases. In other words, it is considered that similar feature points increase. For this reason, the number of false votes increases, which is thought to affect the search accuracy. However, if the limit value is set low in order to suppress the number of hash collisions, it is considered that a large number of data sets to be deleted appear. Therefore, it is considered that data necessary for the search is deleted, and the search accuracy is lowered. From the above, it is necessary to improve the distinguishability of feature points and suppress the number of collisions. This means that it can be easily distinguished from different feature points.

The basic idea of improving discrimination is to increase the number of dimensions of feature quantities. In the conventional LLAH, the number of feature points near the point S0 used for the feature amount calculation of the feature point S0 of interest is n = 7 and m = 6, and 21-dimensional feature amounts are used. In conventional LLAH, since the number of registered pages is assumed to be about 10,000, the feature points are sufficiently discriminable even with this number of dimensions. It is also considered that stability is enhanced by using low-dimensional feature values.
In the present invention, the number of dimensions is increased by setting the number of feature points in the vicinity of the feature point S0 to n = 8 and m = 7. Specifically, the affine invariant vector is ₇ C ₄ = 35 dimensions and the area ratio feature quantity is 7 dimensions, and a total of 42 dimensions can be obtained. The number of dimensions is twice that of conventional LLAH.

However, simply increasing the dimensionality of the feature value is disadvantageous in terms of stability. The larger the value of m, the more invariants will be calculated. Therefore, the possibility that a feature point having the same feature amount as the feature point related to the search question appears accidentally in a document image unrelated to the search question is low. However, in order for the feature points of the search question and the feature points of the correct image to match, the feature amounts must match in all dimensions. However, when the dimensionality of the feature quantity increases, there is a high possibility that a different feature quantity is calculated from the feature points to be handled due to the influence of errors due to projection distortion, noise, and / or resolution differences. Therefore, if the number of dimensions of the feature quantity is increased, the distinguishability of the feature points is improved, but in exchange, the dilemma that the stability of the feature points is lowered occurs.
Therefore, in the present invention, in order to solve the above-mentioned dilemma, dimensions that are related to each other are deleted from the feature amount.

The dimensions related to each other are considered as follows. In the conventional LLAH, as shown in FIG. 4, the feature amount is calculated by overlapping the same triangle for calculating different affine invariants. Here, the different affine invariants are _m C ₄ numerical sequences (r ₍₀₎ , r ₍₁₎ ,…, r _(mC4) ). Invariants that use overlapping triangles are related to each other. This relationship can be considered as a kind of redundancy.

  In FIG. 4, feature points S1, S2, S3, S4, S5, and S6 exist in the vicinity of the feature point S0 so as to surround the feature point S0. The affine invariant relating to the feature point S0 includes invariant 1, invariant 2 and invariant 3 shown on the right side of FIG. The invariant 1 is calculated as the area ratio of the triangle 1 whose vertices are made up of feature points S0, S1, S2 and the triangle 2 made up of feature points S0, S2, S3. The invariant 2 is calculated as the area ratio of the triangle 2 and the triangle 3 whose vertices are feature points S0, S3, S4. The invariant 3 is calculated as the area ratio of the triangle 3 and the triangle 4 whose vertices are feature points S0, S4, S5. The invariant 2 is highly related to the invariant 1 in that the triangle 2 is redundantly used, and is highly related to the invariant 3 in that the triangle 3 is redundantly used. Therefore, it is considered that the invariant 2 in which both the triangle 2 and the triangle 3 are used redundantly has the above-described redundancy.

The dimension like this invariant 2 is deleted. As a result, any triangle used for feature quantity calculation is not used redundantly for different invariant calculations. By deleting redundant dimensions, the stability of feature quantities is improved while suppressing information loss. The original 35-dimensional affine invariant vector is reduced to about half of 17 dimensions by deleting redundant dimensions, and the total of area ratio feature quantity 7 dimensions is 24 dimensions.
The improved LLAH described so far is an aspect of a processing program for causing a computer to perform a predetermined process, an aspect of a non-volatile storage medium that stores the processing program in a computer-readable manner, and the processing program. Can be regarded as an aspect of a system that realizes an improved LLAH function.

≪Experimental example≫
1. Experimental Example 1: Effect of improved LLAH In order to clarify the effects of reducing the amount of memory used and improving the distinguishability and stability of feature points, the following three versions of LLAH were created.
i. Traditional LLAH method
ii. Memory usage reduction version
iii. Comprehensive version i with distinction and stability improvement processing added to the memory usage reduced version is conventional LLAH as a reference for comparison, and ii and iii are related to improved LLAH.

The number of registered pages in the document image database used in the experiment is 10,000.
Registered document images are mainly single-column and double-column English articles collected from the CD-ROM of the proceedings. These English papers are PDF files converted to images with a resolution of 200 dpi. An example of the registered document image is shown in FIG. As a search question, we prepared 1003 images of the entire printed document taken from an oblique direction (approximately 60 °) to the paper. A 12-megapixel digital camera was used for shooting. An example of the search question image is shown in FIG. The shooting angle of the registered image and search query is different. Search queries with different shooting angles than those registered can clearly show the robustness to projection distortion of each version. The size of the hash table is 2 ³⁰ −1. The computer used in the experiment has an Opteron (registered trademark) CPU manufactured by AMD, a CPU clock frequency of 2.8 GHz, and a memory capacity of 128 GB.
The results are shown in Table 1 below. In Table 1, “used memory” represents the amount of data stored in the hash table. The memory capacity used for the hash table itself as a data structure is not included.

The “correct vote rate” represents the proportion of the corresponding points found in the search corresponding to the correct image. The higher the percentage of correct votes, the higher the discrimination and stability.
The memory reduction version of ii realized about 70% memory reduction compared to the conventional method. This is synonymous with deleting about 70% of the number of feature points registered in the hash by the conventional method. In addition, the comprehensive version of iii realized a memory reduction of about 65% compared to the conventional method. Despite deleting these feature points, the accuracy was not affected at all. One reason for this is that the search question used in this experiment is a photograph of the entire document. Therefore, it is necessary to verify the effect on accuracy due to the difference in the shooting range, which will be described later in the item “resistance to partial shooting”.

Next, comparing the memory-reduced version of ii with the comprehensive version of iii, the memory usage has increased by 10MB. This is because the feature amount obtained from one feature point is increased by changing the value of the number of neighboring feature points n. In addition, the mis-voting is reduced by about 20%, so it is considered that the distinguishability of the feature is improved. Furthermore, since the accuracy is not affected at all, it is considered that the stability of the feature amount can be secured. The processing time is increased by 3 ms because the number of affine invariants calculated at one feature point is increased.
From the above, the improved LLAH has achieved a reduction in memory usage without degrading accuracy, and has also achieved a reduction in memory usage and improved distinction and stability of feature points. I can say that.

2. Experimental example 2: Correspondence to enlargement The improved scalability of LLAH for the enlargement of document image database was verified. Four different databases with 10,000, 100,000, 1 million, and 10 million registered pages were created, and the relationship between the number of registered pages, required memory, search accuracy, and processing time was examined. The improved LLAH in Experimental Example 2 corresponds to the comprehensive version of iii in Experimental Example 1.
For comparison, the results of the conventional LLAH method are also shown. The same search question as in Experimental Example 1 was used. The size of the hash table is 2 ³⁰ -1, list length limit is 100.

2-1. Required Memory Figure 7 shows the relationship between the number of pages registered in the document image database and the required memory. Here, the necessary memory amount represents all the memory usage necessary for searching. Also, 8GB of memory is used to secure the hash table. Since the conventional method cannot create a database of 10 million pages, 1 million to 10 million represents an estimated value. Both methods require more memory as the number of registered pages increases. However, it can be seen that the required memory capacity of the improved LLAH is about 50% compared to the conventional LLAH.

2-2. Search accuracy The relationship between the number of registered pages and search accuracy is shown in FIG. The improved LLAH showed higher search accuracy than the conventional LLAH. In addition, it was confirmed that the improved LLAH can be searched with high accuracy of 99.4% in a document image database of 10 million pages. From this, it is considered that the improved feature amount has high discrimination and stability that can withstand a large scale. However, as the number of registered pages increased, the search accuracy decreased. FIG. 9 shows an example of a search failure due to an increase in scale. Like this image, charts occupy most of the page, and search failed with a search question with a small amount of text. This is probably because the correct document cannot obtain a sufficient number of votes because the number of obtained feature points is small.

2-3. Processing Time The relationship between the number of registered pages and processing time is shown in FIG. Compared with an increase in the number of registered pages, an increase in processing time is suppressed. This is considered to be the effect of searching using a hash. In addition, it was confirmed that a 10 million page document image database can be searched in 38 ms. Thus, the improved LLAH can be searched in real time in a 10 million page document image database.

With the enlargement of the document image database, the processing time was reversed between the conventional LLAH and the improved LLAH. When the number of registered pages is small, the processing time of conventional LLAH is shorter. This is due to the difference between the number of features per feature point and the number of affine invariants per feature. In conventional LLAH, ₇ C ₆ = 7 feature quantities and ₆ C ₄ = 15 invariants per feature quantity are calculated. The improved LLAH calculates ₈ C ₇ = 8 features and 24 invariants. Thus, since the amount of calculation of the conventional LLAH is less than that of the improved LLAH, it is considered that the processing time is shortened.

  However, as the number of registered pages increased, the improved LLAH performed better than the conventional LLAH. This is considered to be caused by the difference in the list length in the hash table. Here, FIG. 11 shows an average of the list lengths of the hash table whose list length is not zero. It can be seen that both the conventional LLAH and the improved LLAH increase the average list length as the number of registered pages increases.

  In particular, in a document image database of 1 million pages, the list length is about twice that of LLAH, which is an improvement of conventional LLAH. As the list length increases, the processing time required to follow the list increases. Conventional LLAH requires more processing time in that part. Therefore, it is considered that the processing time has been reversed. From the above, it is considered that as the number of registered pages increases, the improved LLAH is more advantageous than the conventional LLAH in terms of processing time.

3. Experiment 3: Tolerance to partial shooting As described in Experiment 1, one of the reasons why the improved LLAH has high search accuracy is that the search query shows the entire document image. . This is because by capturing the entire document image, the number of feature points to be referenced increases, and the correct image can obtain a large number of votes. In a search query in which a part of a document image is taken, the number of feature points decreases, so that the accuracy is considered to decrease. Therefore, we created a search query that captured a part of the document image and verified its resistance to partial shooting.

  In the document image database used in Experimental Example 3, 10 million page document images are registered. First, 989 items were selected from the search questions used in Experimental Examples 1 and 2, excluding those that occupy most of the charts. Next, the part where the text was reflected was extracted from those search questions, and a pseudo partial shooting query was created. Three patterns of 1/2, 1/4, and 1/8 of the entire photographing range were prepared. Examples of shooting ranges are shown in FIGS. 12, 13, and 14, respectively.

  The results are shown in FIG. It can be seen that the search accuracy decreases as the shooting range becomes narrower. This is considered to be because the number of feature points obtained from the search question is small, and the correct image cannot obtain a sufficient number of votes. An example of the search failure is shown in FIG. As described above, when a large number of “.” And “” ”are distributed in the sentence, the search tends to fail. This is because they become noise and it is difficult to extract stable feature points. FIG. 17 shows an example of feature points when the search fails. In each of FIGS. 17A and 17B, the left side represents a search question and the right side represents a registered document image. As shown in (a), “” ”is separated from the connected component and becomes a new feature point, or“. ”Is not extracted as a feature point as shown in (b). This is considered to be an effect of adaptive binarization in the feature point extraction process. For this, it is necessary to improve the feature point extraction by adding a new process.

4). Experimental example 4: Effect of thinning out feature points and selection of feature amount The experimental examples described so far show how to reduce the memory usage by sampling feature points (hereinafter referred to as thinning) and distinguishability of feature amounts. This is a mode in which a technique for eliminating redundancy of feature quantities (hereinafter referred to as feature quantity selection) is combined in order to ensure stability while improving. Next, an experiment was conducted to confirm the single effect of these methods.
First, the required memory amount, search accuracy, and processing time were confirmed by combining the presence / absence of thinning and the feature selection / non-existence. However, if there is no thinning-out, the required amount of memory becomes enormous and the current experimental apparatus cannot cope. Therefore, the experiment was conducted with 1 million registered pages. The results are shown in Table 2.

Looking at Table 2, the amount of used memory without thinning out under the condition without feature selection is 16.2 GB, and with thinning out is 6.4 GB. In addition, the amount of used memory without thinning out under the condition with feature amount selection is 18.7 GB, while that with thinning out is 7.3 GB. Regarding the reduction of memory usage, there is not much effect of feature quantity selection, and the effect of thinning out is conspicuous. It seems that the difference in search accuracy depending on whether or not a feature is selected is not so large. However, when the number of registered pages reaches 10 million pages, the difference in search accuracy without performing feature amount selection becomes larger.
Next, we compared the difference between the number of registered pages and the selection of feature quantity for 10 million pages. The results are shown in Table 3.

In Table 3, “necessary memory amount” is all the memory usage necessary for the search. The “processing time” does not include the time required for the feature point extraction process. “Average list length” is an average value of non-zero list lengths in the hash. “Hash table usage rate (hereinafter referred to as usage rate)” is a ratio of a list whose length is not zero.
The difference in the required memory amount depending on whether or not the feature amount is selected is mainly due to the difference in the number of feature amounts (the number of dimensions of the feature vector) calculated per feature point. When there is no feature selection, it is 21 dimensions, and when there is feature selection, it is 24 dimensions.

  The search accuracy without feature amount is reduced to 98.6% compared to the search accuracy with feature amount selection of 99.4%. The ratio of search errors when considering the difference from the accuracy of 100% has a difference of about twice. The decrease in search accuracy without feature quantity selection is considered to be due to an increase in false votes. However, since the average list length is almost the same depending on whether or not the feature amount is selected, it is not known whether the increase in false voting is due to the increase in the number of collisions. Therefore, the frequency of the list length of the hash table was examined. The result is shown in FIG.

From FIG. 18, it can be seen that when there is no feature selection, there are many items with a longer list length than when feature amount selection is performed. This is thought to have led to an increase in false votes.
From the above results, it can be understood that the thinning out of the feature points greatly contributes to the reduction of the memory usage amount, while the selection of the feature amount contributes to the improvement of the search accuracy.
Both methods are very effective in dealing with a large-scale document image database with 10 million registered pages.

  In the improved LLAH, in order to support large-scale document image retrieval, improvements were made in terms of reducing the amount of memory required and improving the distinguishability and stability of feature points. As a result, it was confirmed by experiments that a search can be performed with an accuracy of 99.4% and a processing time of 38 ms in a 10 million page database. From the above, it can be said that the improved LLAH has improved scalability.

  In addition to the embodiments described above, there can be various modifications of the present invention. These modifications should not be construed as not belonging to the scope of the present invention. The present invention should include the meaning equivalent to the scope of the claims and all modifications within the scope.

S0, S1, S2, S3, S4, S5, S6: Features

Claims (8)

Computer
A feature point extracting step for extracting a feature point representing a local feature of the document image from the document image to be registered in the document image database;
A feature value of each feature point using an invariant for geometric transformation, and a predetermined amount of geometric elements determined by each feature point and n feature points in the vicinity (n is a natural number) Each characteristic value is obtained by calculation, a plurality of invariants obtained by combining these characteristic values is calculated, and a feature quantity calculating step using a vector having each dimension as the calculated invariant as the feature quantity,
For each feature point extracted from the document image, (1) a reference used for referring to the document image, (2) an identifier for distinguishing the feature point from other feature points, and (3) a feature amount of the feature point Generating a data set formed by associating (1) to (3), and registering the data set together with the document image in the document image database,
The feature point extracting step determines a connected component of lines constituting the document, extracts a centroid of a connected component smaller than a predetermined area as a feature point, and each feature is extracted when the number of extracted feature points is less than a threshold value. Extract the centroid of the connected components in the vicinity of the point as a feature point and extract it so that a feature point above the threshold is obtained,
The document image database registration method characterized in that the feature quantity calculating step calculates each invariant using a geometric element that does not overlap with a geometric element used for one invariant as another invariant. .   The registration step includes (1) the reference, (2) the identifier, and (3) the feature quantity or a simple feature quantity obtained by simplifying the feature quantity (1) to (3) in association with the data set. And performing a predefined calculation to classify each feature point according to the feature amount to obtain an index of a class to which the feature point belongs, classify the data set into a class according to the index, and The registration method according to claim 1, wherein the registration is performed together with the document image in the document image database.   In the feature amount calculating step, a triangle having three feature points as vertices is used as the geometric element, and the area of the triangle is used as the characteristic value. The registration method according to claim 1 or 2, wherein an area ratio of two triangles sharing one side of the combination is one invariant. Computer
A feature point extraction step of extracting a plurality of query feature points representing local features of the document image from the document image captured as a search question;
A feature amount calculating step for calculating a query feature amount related to each query feature point;
The document image database in which the document image is registered by the registration method according to claim 1 is referred to, and the feature amount is the same as or similar to each query feature point from the data set registered in the document image database. A separate search step for searching the data set to obtain a reference to the found data set;
A search is performed for each query feature point, each document identifier obtained is statistically processed, and a voting step for identifying a document image to be recommended as a search result is performed.
The feature point extraction step determines a connected component of lines constituting the document, extracts a centroid of each connected component as a feature point,
The feature amount calculating step obtains respective characteristic values by a predetermined calculation for a plurality of geometric elements determined by each feature point and n feature points (n is a natural number) in the vicinity thereof, and the characteristic values Calculates multiple invariants that are a combination of, and calculates each invariant using a geometric element that does not overlap with the geometric element used for one invariant for the other invariant. A document image search method, wherein a vector having invariants as dimensions is used as the feature amount. Computer
A feature point extraction step of extracting a plurality of query feature points representing local features of the document image from the document image captured as a search question;
A feature amount calculating step for calculating a query feature amount related to each query feature point;
A feature point in which a local feature is similar to each query feature point from among feature points registered in the document image database with reference to the document image database in which the document image is registered by the registration method according to claim 2. And a separate search step to obtain a reference associated with the found similar feature point;
A search is performed for each query feature point, each document identifier obtained is statistically processed, and a voting step for identifying a document image to be recommended as a search result is performed.
The feature point extraction step determines a connected component of lines constituting the document, extracts a centroid of each connected component as a feature point,
The feature amount calculating step obtains respective characteristic values by a predetermined calculation for a plurality of geometric elements determined by each feature point and n feature points (n is a natural number) in the vicinity thereof, and the characteristic values Calculates multiple invariants that are a combination of, and calculates each invariant using a geometric element that does not overlap with the geometric element used for one invariant for the other invariant. A vector having an invariant as each dimension as the feature amount,
The individual search step obtains an index of each query feature point by calculation corresponding to the registration method according to claim 2, refers to a data set registered as a class related to the index, and the data set is a simple feature amount. Is obtained after obtaining a simple feature amount related to the query feature point, and a reference is obtained for the data set having the same or the most similar simple feature amount, and the same when the registered data set has the feature amount. Alternatively, a document image search method comprising obtaining a reference related to a data set having the most similar feature amount.   In the feature amount calculating step, a triangle having three feature points as vertices is used as the geometric element, and the area of the triangle is used as the characteristic value. The search method according to claim 4 or 5, wherein an area ratio of two triangles sharing one side of the combination is one invariant. Computer
A feature point extracting step for extracting a feature point representing a local feature of the document image from the document image to be registered in the document image database;
A feature value of each feature point using an invariant for geometric transformation, and a predetermined amount of geometric elements determined by each feature point and n feature points in the vicinity (n is a natural number) Each characteristic value is obtained by calculation, a plurality of invariants obtained by combining these characteristic values is calculated, and a feature quantity calculating step using a vector having each dimension as the calculated invariant as the feature quantity,
For each feature point extracted from the document image, (1) a reference used for referring to the document image, (2) an identifier for distinguishing the feature point from other feature points, and (3) a feature amount of the feature point Generating a data set formed by associating (1) to (3), and registering the data set together with the document image in the document image database,
The feature point extracting step determines a connected component of lines constituting the document, extracts a centroid of a connected component smaller than a predetermined area as a feature point, and each feature is extracted when the number of extracted feature points is less than a threshold value. A document image database registration method, wherein a centroid of connected components in the vicinity of a point is further extracted as a feature point so as to obtain a feature point equal to or greater than a threshold value. Computer
A feature point extraction step of extracting a plurality of query feature points representing local features of the document image from the document image captured as a search question;
A feature amount calculating step for calculating a query feature amount related to each query feature point;
The document image database in which the document image is registered by the registration method according to claim 7 is referred to, and the feature amount is the same or similar to each query feature point from the data set registered in the document image database. A separate search step for searching the data set to obtain a reference to the found data set;
A search is performed for each query feature point, each document identifier obtained is statistically processed, and a voting step for identifying a document image to be recommended as a search result is performed.
The feature point extraction step determines a connected component of lines constituting the document, extracts a centroid of each connected component as a feature point,
The feature amount calculating step obtains respective characteristic values by a predetermined calculation for a plurality of geometric elements determined by each feature point and n feature points (n is a natural number) in the vicinity thereof, and the characteristic values Calculates multiple invariants that are a combination of, and calculates each invariant using a geometric element that does not overlap with the geometric element used for one invariant for the other invariant. A document image search method, wherein a vector having invariants as dimensions is used as the feature amount.