JP2015203978A - Image processor, image retrieval device, and method for controlling image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make those images retrievable without registering a local feature amount from the same image.SOLUTION: An image processor includes an input part for inputting an image of a registration object, a division part for dividing the inputted image into a plurality of areas in accordance with the image, a calculation part for calculating a feature amount of the image in each divided image area, a retrieval part for retrieving similar images from images which have already been registered by using the feature amount in each divided area, and registration part for registering the feature amount of the image of the corresponding divided area in the case that the retrieval part does not retrieve a similar image.

Description

  The present invention relates to an image search technique.

  There has been proposed a method of searching for a similar image using a local feature amount (local feature amount) of an image. In this method, first, characteristic points (local feature points) are extracted from an image (Non-Patent Document 1). Next, a feature amount (local feature amount) corresponding to the local feature point is calculated based on the local feature point and surrounding image information (Non-Patent Document 2).

  In the method using the local feature amount, the local feature amount is defined as information including a plurality of elements that are rotation invariant and enlargement / reduction invariant. Thereby, even when the image is rotated, enlarged or reduced, the search can be performed. The local feature amount is generally expressed as a vector.

  In order to extract the rotation-invariant local feature value, for example, in Non-Patent Document 2, the main direction is calculated from the pixel pattern of the local region around the local feature point, and the local region is rotated based on the main direction when calculating the local feature value. To normalize the direction. Further, in order to calculate the local feature quantity that does not change in size, the image of different scales is generated internally, and local feature points are extracted from the images of the respective scales and the local feature quantities are calculated. Here, a series of image sets of different scales generated internally is generally called a scale space.

  With the above-described method, a plurality of local feature points are extracted from one image. In image retrieval using local feature amounts, matching is performed by comparing local feature amounts calculated from respective local feature points. A widely used voting method (Patent Document 1) finds a feature point that is similar to a local feature amount of each feature point extracted from a search source image by a nearest neighbor process. The more votes, the more similar the votes are.

  On the other hand, there has been proposed a technology related to an information leakage suppression system that accumulates the contents of a job output by a user from a printer or the like and can track when and who executed what job (Patent Document 2). In order to suppress the leakage of the image, the feature amount extracted from the image is registered, and a similar image is searched to identify the job that has output the image.

JP 2009-284084 Japanese Patent No.4921202

C. Harris and M.J. Stephens, “A combined corner and edge detector,” In Alvey Vision Conference, pages 147-152, 1988. David G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, 60, 2 (2004), pp.91-110.

  In the office, there are many opportunities for multiple people to print the same document. In the above-described information leakage suppression system, when the same document is printed, the feature amount is registered in the system every time it is printed. Therefore, the same feature amount is redundantly registered in the system a plurality of times. Then, comparison is made with all the registered feature quantities, and search results are generated.

  The non-local (global) feature used in the past was a fixed size, small feature size, and high-speed search was possible, so it was a redundant database, but it was not a big problem. .

  However, the size of the local feature amount is hundreds to thousands times larger than the conventional feature amount size. Therefore, when the number of registrations increases, the enlargement of the database becomes a problem. Furthermore, the enlargement of the database affects the search speed. In the conventional method, since redundant registration is performed, there has been a problem of accelerating the enlargement of the database.

  In addition, when printing a document in an office, in order to save paper, toner, and the like, there are many opportunities to reduce a plurality of pages of documents and assign them to one page for printing. Since, in principle, almost the same local feature amount is extracted from such a reduced image, there is a problem that redundant registration is similarly performed if these are registered as they are.

  The present invention intends to provide a method capable of searching for images without redundant registration of local feature values extracted from the same image.

In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is,
An image processing apparatus for registering an image in an image search apparatus,
An input means for inputting an image to be registered;
Dividing means for dividing the input image into a plurality of regions according to the image;
Calculating means for calculating the feature amount of the image for each divided area;
Search means for searching for similar images from already registered images using the feature amount for each divided area;
And a registration unit that registers the feature amount of the image of the divided corresponding area when a similar image does not exist in the search result of the search unit.

  According to the present invention, it is possible to search for images without redundant registration of local feature amounts extracted from the same image.

The block diagram of the computer apparatus to which 1st Embodiment is applied. FIG. 2 is a block diagram illustrating a functional configuration example of the image processing apparatus according to the first embodiment. FIG. 5 is a diagram illustrating an example of an input image according to the first embodiment. 6 is a flowchart illustrating an example of a registration process according to the first embodiment. 5 is a flowchart illustrating an example of image feature amount extraction processing according to the first embodiment. FIG. 6 is a diagram illustrating an example of a reduced image generation process according to the first embodiment. The figure which shows the example of a division | segmentation into the lattice shape of the two-dimensional feature-value space in 1st Embodiment. 6 is a flowchart illustrating an example of an image registration processing procedure according to the first embodiment. The figure which shows an example of the schema when registering the image feature-value in 1st Embodiment in a database. The figure which shows an example of the coincidence area | region specific process result in 1st Embodiment. The figure which shows an example of the database table after registering the link information in 1st Embodiment. The flowchart which shows the allocation image determination processing procedure in 1st Embodiment. The figure which shows an example of the information regarding the layout image in 1st Embodiment. The figure explaining the area | region of the search result image corresponding to the area | region when the matching area | region in the process target image in 1st Embodiment is extended to the whole process target image. 6 is a flowchart illustrating an image division re-registration process procedure according to the first embodiment. The figure which shows an example of the database table after the division | segmentation re-registration of the image in 1st Embodiment. 6 is a flowchart illustrating a similarity reference image search processing procedure according to the first embodiment. 5 is a flowchart illustrating a matching area specifying process procedure according to the first embodiment. The figure which shows an example of the schema of the shortest distance corresponding | compatible point list | wrist in 1st Embodiment. The flowchart which shows the corresponding point list creation process sequence in 1st Embodiment. 5 is a flowchart illustrating an example of a search processing procedure according to the first embodiment. 10 is a flowchart illustrating a procedure for an allocated image determination process according to the second embodiment. FIG. 10 is a diagram illustrating an example of an image area separation result according to a second embodiment. The figure which shows the example of the division position when assigning and dividing the image of the image area separation result in the second embodiment. The flowchart which shows the registration processing procedure in 3rd Embodiment. 10 is a flowchart illustrating an image registration processing procedure according to the third embodiment.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[First Embodiment]
First, the configuration of a computer device constituting the server device and the client device in the first embodiment will be described with reference to the block diagram of FIG. Each of the server device and the client device may be realized by a single computer device, or may be realized by distributing each function to a plurality of computer devices as necessary. When configured by a plurality of computer devices, they are connected by a local area network (LAN) or the like so that they can communicate with each other. The computer device can be realized by an information processing device such as a personal computer (PC) or a workstation (WS).

  In FIG. 1, a CPU 101 is a central processing unit that controls the entire computer apparatus 100. The ROM 102 is a Read Only Memory that stores programs and parameters that do not need to be changed. A RAM 103 is a Random Access Memory that temporarily stores programs and data supplied from an external device. The external storage device 104 is a storage device such as a hard disk or a memory card that is fixedly installed in the computer device 100. The external storage device 104 may include an optical disk such as a flexible disk (FD) and a Compact Disk (CD) that can be detached from the computer apparatus 100, a magnetic or optical card, an IC card, a memory card, and the like. The input device interface 105 is an interface with an input device 109 such as a pointing device or a keyboard that receives data from a user and inputs data. The output device interface 106 is an interface with the monitor 110 for displaying data held by the computer apparatus 100 and supplied data. The communication interface 107 is a communication interface for connecting to a network line 111 such as the Internet, a digital camera 112, a digital video camera 113, a smartphone 114, and the like. The system bus 108 is a transmission path that connects the units 101 to 107 so that they can communicate with each other.

  Each operation described later is executed by the CPU 101 executing a program stored in a computer-readable storage medium such as the ROM 102.

  FIG. 2 is a block diagram illustrating a functional configuration example of the image processing apparatus according to the first embodiment. Details of the operation in each configuration shown in FIG. 2 will be described later.

  The first embodiment relates to a process of calculating and registering an image feature amount from an input image when searching for a similar image using the image feature amount. Further, the present invention relates to processing for searching for similar images by calculating image feature amounts from input images and comparing them with registered image feature amounts. When an image generated by allocating a plurality of images (hereinafter referred to as an allocated image) is registered, the image is divided into a plurality of original images, and an image feature amount is calculated and registered for each of the images. Here, each of the divided plurality of original images is referred to as a reference image.

  In FIG. 2, an image input unit 201 inputs an image to be registered (registration target image) and a search source image (query image). In the present embodiment, as shown in FIG. 3A, an image obtained by rasterizing a document as an image will be described. This input image can also be generated by a printer driver when printing a document. In particular, when generated by a printer driver, the print data can include information indicating how many pages are printed on one sheet of recording paper, which is advantageous in the embodiment.

  The allocated image determination unit 202 determines whether the input image input from the image input unit 201 is an allocated image, and determines the number of allocated pages if the input image is an allocated image. Here, the layout image is an image in which a plurality of pages of a document are grouped into one (one), as shown in examples in FIGS. This can be done when printing a document. When performing layout printing, a plurality of pages are reduced by the same size and allocated to one recording sheet (in one image). Therefore, the number of pages that can be assigned and printed is 2, 4, 6, 8, 8, 9, 16, 32, etc. In general, allocating and printing N pages on one recording sheet is called Nin1. It is theoretically possible to evenly assign the number of pages exceeding 32 pages to one (1 page), and the number of pages is not limited to these. When performing allocation, for example, when a 4-page layout image is generated from a 3-page document, a blank page is placed at the position of the 4th page. The layout image determination unit 202 further determines how many pages the layout image is when it is determined as a layout image. Here, a description will be given assuming that the image is determined to be an N-page layout image.

  The area dividing unit 203 divides the input image into N areas when the allocated image determining unit 202 determines that the page is an N page allocated image. The allocation method is determined in advance and is divided according to the method. The image feature amount calculation unit 204 calculates an image feature amount for an input image when the input image is not an assigned image, and for each image divided by the region dividing unit 203 when the input image is an assigned image.

  The image search unit 205 uses the image feature amount calculated by the image feature amount calculation unit 204 as a search source, compares the image feature amount of an already registered image, and searches for similar images. When a similar image is found by the image search unit 205, the link information registration unit 206 registers link information that the image feature amount image used as the search source is the same image as the similar image. When a similar image is not found by the image search unit 205, the image feature amount registration unit 207 registers an image feature amount as a search source. The storage unit 208 is a memory / HDD that stores data being processed, various indexes, image files, and the like. Each of these components is centrally controlled by a CPU (not shown).

[registration process]
FIG. 4 is a flowchart illustrating an example of a registration processing procedure in the image processing apparatus according to the first embodiment.

  First, in step S401, an image to be registered is input via the image input unit 201. An image ID is assigned to the input image to be registered. In the first embodiment, as an example, a case where an image rasterized by a printer driver is input when a document is printed will be described. The input image is stored in the storage unit 208.

  In step S402, the allocated image determination unit 202 determines whether the input registration target image is an allocated image. As described above, it is possible to instruct layout printing by setting the printer driver when printing a document. In the present embodiment, the information accompanying the input image is used. If it is determined in step S403 that the input image is not an allocated image, the process proceeds to step S404, where the input image is set as a processing target image. If the input image is an allocated image, in step S405, an allocation number N indicating how many pages of the input registration target image are acquired. The number N assigned also uses information set by the printer driver. However, the present invention is not limited to the case where the printer driver setting information is used, and operation information of a device such as a printer or a multifunction peripheral may be used. For example, when copying with a multifunction machine, it is possible to copy two sheets allocated to one sheet, and such information can also be used.

  In step S406, the area dividing unit 203 divides the input image into N areas. This division is a predetermined division method and is divided into regions of equal size. In step S407, an image of the first area divided into N is set as a processing target image. In step S408, the image feature amount calculation unit 204 extracts the image feature amount of the processing target image determined in step S404 or step S407.

  FIG. 5 is a flowchart illustrating an example of an image feature amount extraction processing procedure in step S408. In the first embodiment, a local feature amount (local feature amount) of an image is used as the image feature amount. First, in step S501, a luminance component is extracted from an image to be processed, and a luminance component image is generated based on the extracted luminance component. In step S502, the luminance component image is successively reduced in accordance with the magnification (reduction ratio) p, and n reduced images including the original image, which are reduced stepwise from the original size image, are generated. . Here, it is assumed that the magnification p and the number n of reduced images are determined in advance.

  FIG. 6 is a diagram illustrating an example of a reduced image generation process. The example shown in FIG. 6 is a case where the magnification p is “2 to the power of − (1/4)” and the number n of reduced images is “9”. Of course, the magnification p is not necessarily "2 to the power of-(1/4)". In FIG. 6, reference numeral 601 denotes the luminance component image generated in step S501. Reference numeral 602 denotes a reduced image obtained by recursively reducing the luminance component image 601 four times according to the magnification p. Reference numeral 603 denotes a reduced image obtained by reducing the luminance component image 601 eight times according to the magnification p.

  In this example, the reduced image 602 is an image obtained by reducing the luminance component image 601 to ½, and the reduced image 603 is an image obtained by reducing the luminance component image 601 to ¼. Any method may be used to reduce the image. In the first embodiment, a reduced image is generated by a reduction method using linear interpolation.

  Next, in step S503, local feature points (local feature points) that are robustly extracted even if there is image rotation in each of the n reduced images are extracted. As a method for extracting local feature points, Harris operator is used in the first embodiment (Non-patent Document 1: C. Harris and MJ Stephens, “A combined corner and edge detector,” In Alvey Vision Conference, pages 147-152. , 1988.).

  Specifically, for the pixels on the output image H obtained by applying the Harris operator, the pixel values of the pixels and the pixels in the vicinity of the pixels (eight pixels in total) (total nine pixels) are examined. Then, a point at which the pixel becomes a local maximum (a pixel value of the pixel becomes the maximum among the nine pixels) is extracted as a local feature point. Here, even when the pixel reaches a local maximum, it is not extracted as a local feature point if the value of the pixel is less than or equal to the threshold value.

  It should be noted that any feature point extraction method is applicable as long as it is a method capable of extracting local feature points, not limited to the feature point extraction method using the Harris operator described above.

  Next, in step S504, for each of the local feature points extracted in step S503, a feature quantity (local feature quantity) defined so as to remain unchanged even when the image is rotated is calculated. As a method for calculating the local feature, the first embodiment uses a combination of Local Jet and derivatives thereof (JJ Koenderink and AJvan Doorn, “Representation of local geometry in the visual system,” Riological Cybernetics, vol. 55, pp.367-375, 1987.).

ただし、式（１）の右辺で用いている記号は、以下に示す式（２）から式（７）で定義される。ここで、式（２）右辺のＧ（ｘ，ｙ）はガウス関数、Ｉ（ｘ，ｙ）は画像の座標（ｘ，ｙ）における画素値であり、“＊”は畳み込み演算を表す記号である。また、式（３）は式（２）で定義された変数Ｌのｘに関する偏導関数、式（４）は当該変数Ｌのｙに関する偏導関数である。式（５）は式（３）で定義された変数Ｌｘのｙに関する偏導関数、式（６）は式（３）で定義された変数Ｌｘのｘに関する偏導関数、式（７）は式（４）で定義されたＬｙのｙに関する偏導関数である。

尚、局所特徴量を算出可能な方法であれば、上述したような特徴量算出方法に限らず、どのような特徴量算出方法でも適用可能である。 Specifically, the local feature amount V is calculated by the following equation (1).

However, the symbols used on the right side of the equation (1) are defined by the following equations (2) to (7). Here, G (x, y) on the right side of Expression (2) is a Gaussian function, I (x, y) is a pixel value at image coordinates (x, y), and “*” is a symbol representing a convolution operation. is there. Equation (3) is a partial derivative of variable L defined by equation (2) with respect to x, and equation (4) is a partial derivative of variable L with respect to y. Equation (5) is the partial derivative of variable Lx defined in equation (3) with respect to y, equation (6) is the partial derivative of variable Lx defined in equation (3) with respect to x, and equation (7) is the equation. It is a partial derivative with respect to y of Ly defined in (4).

It should be noted that any feature amount calculation method is applicable as long as it is a method capable of calculating a local feature amount, not limited to the above-described feature amount calculation method.

  A group of image feature values can be obtained by grouping the local feature values for all the local feature points of the target image.

  Next, in step S505, the image feature amount extracted in step S504 is quantized to generate a quantized image feature amount of the processing target image.

  The image feature quantity used in the first embodiment, that is, the local feature quantity calculated from each local feature point by a combination of Local Jet and their derivatives is an N-dimensional vector. Here, the nth dimension of the N dimensions is quantized to the Kn gradation. Here, it is assumed that N and Kn are determined in advance.

Specifically, quantization is performed by the following equation (8).
Q _n = (V _n * K _n ) / (V _{n — max} −V _{n — min} +1) (8)
Here, Q _n is a value obtained by quantizing the feature quantity V _n of the n-th dimension among the N dimensions. V _{n_max} and V _{n_min} are the maximum and minimum values that can be taken by the feature quantity of the nth dimension.

  Quantized image feature values can be obtained by grouping the quantized values of each dimension, and quantized image feature groups can be obtained by grouping all quantized image feature values of the target image. .

  In the above quantization, the number of quantization gradations is determined for each dimension. However, the number of gradations common to all dimensions or several dimensions may be used. This quantization method is a method of dividing the feature amount space into a lattice shape as shown in FIG. 7A, but may be divided into a lattice shape as shown in FIG. 7B. In this figure, reference numeral 701 denotes a quantization area in the feature amount space, and reference numeral 702 denotes each feature. Both FIGS. 7A and 7B are examples in which the two-dimensional feature amount space is quantized and divided, and the division is performed by expanding this to multi-dimensions corresponding to the number of dimensions of the local feature amount.

  Further, as long as it is a method that can divide the feature amount space, any division method can be applied without being limited to the method of quantizing based on the rules as described above. For example, a clustering rule may be created by machine learning of a plurality of images, and the feature amount space may be divided and quantized according to the rule.

尚、ラベル化可能な算出方法であれば、上述したような算出方法に限らずに、どのようなラベル化方法でも適用可能である。第１の実施形態では、ラベル化値ＩＤＸを特徴量ＩＤという。 In addition, after performing quantization for each dimension, the quantization value group is labeled according to the following equation (9), so that it can be handled substantially equivalent to a one-dimensional feature amount. .
IDX = Q ₁ + Q ₂ * K ₁ + Q ₃ * K ₁ * K ₂ + ... + Q _n * K ₁ * K ₂ * ... * K _n-1 (9)
Further, when the number of gradations of all dimensions is common, the quantization value group can be labeled by the following equation (10). Here, K is the number of gradations.

As long as the calculation method can be labeled, any labeling method can be applied without being limited to the calculation method as described above. In the first embodiment, the labeled value IDX is referred to as a feature amount ID.

  The processing content of step S408 in FIG. 4 has been described above. In the next step S409, the image search unit 205 searches for a similar image by comparing the image feature amount already registered with the image feature amount calculated by the image feature amount calculation unit 204 as a search source. Based on the result, the processing target image is registered by the link information registration unit 206 or the image feature amount registration unit 207.

  Here, an example of the image registration processing procedure in step S409 will be described with reference to the flowchart of FIG.

  First, in step S801, the image search unit 205 uses the image feature amount calculated in step S408 as a search source, compares the image feature amount of a reference image already registered, and searches for a similar reference image. . In the similar reference image search process, a comparison with the image feature amount of the registered reference image is performed, and the similarity is calculated. Then, a result sorted in descending order of similarity is obtained. Details of the similar reference image search processing will be described later.

  If no search result equal to or higher than the predetermined similarity is found in step S802, the process advances to step S803 to register the image feature amount of the processing target image. That is, the processing target image is registered as one of new reference images.

  Here, FIG. 9 is an example of a schema for registering image feature amounts in a database. These are examples, and different types of schemas may be used.

  FIG. 9A shows an example of a schema that associates an image ID with a reference image ID. Here, the image ID is an ID given to the registration target image. The reference image ID is an ID given to each reference image divided for each page when the registration target image is an assigned image. When the registration target image is not an assigned image, a reference image ID is assigned to the registration target image. In this schema, which region of the registered image is the reference image is further stored as region information. As the area information, for example, pixel values of upper left coordinates and lower right coordinates are stored. The reference image ID and the image ID are stored using an 8-byte type, although it depends on the number of registered images. The area information is stored using four values of a 2-byte type. Therefore, a 24-byte storage area is required to register one reference image. However, this storage size is an example and is not limited to this value.

  FIG. 9B is an example of a schema that associates the feature amount ID with the reference image ID. Here, the feature amount ID is a label value when the image feature amount is labeled. Since image feature amounts having the same feature amount ID may be calculated from a plurality of reference images, a plurality of reference image IDs may be associated with one feature amount ID. In this schema, pixel values of feature point coordinates are further stored. Image feature amounts having the same feature amount ID may be calculated from a single reference image. In this case, the feature point coordinates are used as a list, and the list itself is held as a list. The image feature value is also stored. Each of the feature amount ID and the reference image ID is stored using an 8-byte type. The feature point coordinates are stored using two values of a 2-byte type. Here, an image feature amount represented by a 128-dimensional double-precision decimal type is assumed, and stored using 128 values of an 8-byte type. Therefore, in order to register one feature point, a storage area of about 1 kilobyte is required. Generally, several thousand feature points are extracted from one reference image. Here, assuming that the number of feature points extracted from one reference image is 3000, a storage area of about 3 megabytes is required to register one reference image. Of course, it is possible to reduce the storage area by reducing the number of dimensions of the image feature amount by principal component analysis or by using a fixed decimal type instead of using a double precision decimal type.

  Returning to the flow of FIG. 8, if there is a search result equal to or higher than the predetermined similarity in step S802, the process proceeds to step S804. In step S804, a matching area between the processing target image and the search result image is specified. In the matching area specifying process, as shown in the example of FIG. 10, the inclusion ratio RQ and location of the matching area in the processing target image, and the inclusion ratio RS and location of the matching area in the search result image are obtained.

  10A to 10D show the relationship between the processing target image and the search result image. In FIGS. 10A to 10D, reference numerals 100 to 103 on the left side indicate examples of processing target images, and reference numerals 110 to 113 on the right side indicate examples of search result images. In each figure, a range surrounded by a bold rectangle indicates the matching area and its location. The position of the coincidence area is shown in association with a dotted line. In addition, a numerical value (%) starting with RQ described beside each image indicates the ratio of the matching area in the processing target image, and a numerical value starting with RS indicates the ratio of the matching area in the search result image. 100% indicates that the matching area is the entire corresponding image.

  As the relationship between the matching areas of the processing target image and the search result image, when the entire area matches as in the relationship of FIG. 10A, the processing target image is the search result image as in the relationship of FIG. May be included. In addition, the processing target image may be a part of the search result image as in the relationship of FIG. 10C or FIG. Here, the search result image 112 in FIG. 10C includes two pages, and is an image that should be registered in units of pages. The main image is an image that has been registered without being determined as an assigned image due to some influence in step S402 at the time of registration. Details of the matching area specifying process will be described later.

  In step S805, the inclusion ratio RQ of the matching area in the processing target image is compared with the inclusion ratio RS of the matching area in the search result image. If the ratio is approximately the same, the process proceeds to step S806. Here, it is assumed that the RS is within the range having a margin of, for example, about ± 10% with respect to RQ. Here, the value of the margin of 10% is an example, and this value may be obtained by experiments, or may be set by the system administrator. When RQ and RS have substantially the same ratio, it can be considered that the processing target image and the search result image are the same image. Accordingly, in step S806, link information from the processing target image to the search result image is registered. Here, instead of registering a new image feature amount, only information that is the same as the already registered image feature amount is registered.

  As an example, it is assumed that the image ID of the processing target image is 5 and the search result reference image ID is 1. In this case, the table for managing the image ID and the reference image ID shown in FIG. 9A is updated as shown in FIG. That is, as shown in FIG. 11, “5” is inserted (added) into the “image ID” field in the record with the reference image ID “1”, and the coordinate information of the matching region of the processing target image is inserted into the region information. To do. The storage capacity necessary for registering one reference image is only a total of 16 bytes including 8 bytes of image ID and 8 bytes of area information. In the case of registering the image feature amount, a storage area of about 3 megabytes is required, so that it can be seen that a significant reduction is possible.

  If the values of RQ and RS are not substantially the same in step S805, the process proceeds to step S807. If RQ is smaller than RS in step S807, the process proceeds to step S803, and the image feature amount of the processing target image is registered. The case where RQ is smaller than RS is a case where the processing target image includes the search result image as shown in FIG.

  If RQ is larger than RS in step S807, the process proceeds to step S808. When RQ is larger than RS, as shown in FIGS. 10C and 10D, the search result image includes the processing target image. Here, FIG. 10C shows one page of the allocated image, and FIG. 10D is not a single page of the allocated image, but merely a part of the search result image. In step S808, it is determined which of these is the processing target image, that is, whether the search result image is an allocated image. Furthermore, when the search result image is an assigned image, the number of assignments and the assigned position are acquired.

  FIG. 12 is a flowchart showing an example of the assigned image determination processing procedure for the search result image in step S808.

  First, in step S1201, allocation number candidates are acquired based on the RS / RQ value. Here, description will be made by taking, as an example, the allocated images of pages 2, 4, 6, 8, 8, 9, and 16 shown in FIGS. FIG. 13 is a table summarizing information about the allocated images. The ratio in the table of FIG. 13 is the reduction ratio of each page by the allocation. Rotation is whether to rotate 90 degrees when allocating, Yes indicates that the rotation is performed, and No indicates that the rotation is not performed. When rotating, the orientation of the image to be assigned and the orientation of the assigned image are rotated 90 degrees. The center x-coordinate and the center y-coordinate are the ratios when the width or height is set to 1 for the center x-coordinate and y-coordinate of the assigned page. Coordinates obtained by combining any one of the center x coordinates and any one of the center y coordinates are the center coordinates of the allocated pages.

  In step S1201, an allocation number having an RS / RQ value substantially the same as the ratio in the table of FIG. 13 is determined as an allocation number candidate. Here, substantially the same value means that the RS / RQ value falls within a range having a margin of about ± 10% with respect to the ratio in the table of FIG. Here, the margin value of 10% is an example, and this value may be obtained by experiment. Further, the margin value may be changed for each allocation number. At this point, there may be a plurality of candidates.

  Next, in step S1202, when there is no allocation number candidate, it progresses to step S1203 and determines with a search result image not being an allocation image. If there is an allocation number candidate in step S1202, the process proceeds to step S1204.

  In step S1204, allocation number candidates are narrowed down depending on the orientation of the processing target image and the search result image, that is, whether the image is a vertically long image or a horizontally long image. When the image orientations are the same, only the number of assignments for which the rotation of the table of FIG. 13 is No is left in the assignment number candidates, and the others are excluded from the assignment number candidates. When the orientations of the images are different, only the allocation number whose rotation in the table of FIG. 13 is Yes is left as the allocation number candidate, and the other is excluded from the allocation number candidates.

  Next, in step S1205, if there is no allocation number candidate, the process proceeds to step S1203, and it is determined that the search result image is not an allocation image. If there is an allocation number candidate in step S1205, the process proceeds to step S1206.

  In step S1206, a region in the search result image corresponding to the region when the matching region in the processing target image is expanded to the entire processing target image is obtained, and the center coordinates of the region are acquired. A region surrounded by a thin line 1401 in the processing target image 1400 in FIG. 14 is a matching region, and it is considered that this region is expanded to the entire image region surrounded by a thick line. At this time, the region of the search result image corresponding to the entire region of the processing target image can be obtained by expanding the matching region surrounded by the thin line 1411 in the search result image 1410 by the same ratio in the same direction. . And the center coordinate of the area | region is acquired.

  In step S1207, the allocation number candidates are narrowed down by the center coordinates of the area of the search result image. That is, only the allocation number in which either the center x coordinate or the center y coordinate in the table of FIG. 13 is substantially the same value as the center coordinate of the area of the search result image is left as the allocation number candidate. Remove from the candidate.

  If there is no allocation number candidate in step S1208, the process advances to step S1203 to determine that the search result image is not an allocation image. If it is determined in step S1208 that there is an allocation number candidate, the process advances to step S1209 to determine that the search result image is an allocation image. In step S1207, an inclusion position, that is, an allocation position is acquired depending on which center x coordinate and center y coordinate are substantially the same.

  This completes the assigned image determination processing for the search result image in step S808.

  Returning to the flow of FIG. 8, in step S809, if the search result image is not an assigned image, the process proceeds to step S803 to register the image feature amount of the processing target image. If the search result image is an allocated image, the process proceeds to step S810, and the search result image is re-registered in a state of being divided according to the allocation method.

  FIG. 15 is a flowchart illustrating an example of the procedure for dividing and re-registering the search result image in step S810. The details of step S810 will be described below with reference to FIG. First, in step S1501, the assigned number of search result images is substituted for K, and the maximum number of registered reference image IDs is substituted for N. In step S1502, an association table of reference image IDs and image IDs is referred to, and an image ID and area information are acquired from the reference image ID of the search result image. In step S1503, the obtained area information is divided into K pieces according to an assignment method predetermined by the number of assignments.

  In step S1504, N + 1 to N + K reference image IDs are added to the reference table of reference image IDs and image IDs. Furthermore, the image ID obtained in step S1502 and the K-divided area information obtained in step S1503 are registered in association with those reference image IDs. In step S1505, the reference image ID of the search result image is deleted from the reference table of the reference image ID and the image ID.

  In step S1506, the feature point coordinates of the reference image ID of the search result image are acquired by referring to the association table between the feature amount ID and the reference image ID. In step S1507, it is determined which of the K-divided regions the feature point coordinates obtained in step S1506 are included, and a corresponding reference image ID is acquired. Then, the reference image ID of the search result image is rewritten to the reference image ID after K division.

  Returning to the flow of FIG. 8, in step S811, link information from the processing target image to the corresponding divided image is registered among the plurality of images obtained by assigning and dividing the search result image. The process in step S811 is equivalent to the process in step S806. Even if the allocated image is once registered as a reference image for some reason by the processing in steps S810 and S811, if one page image included in the allocated image is input as a processing target, it is included in the allocated image. Each page that has been registered is re-registered as a reference image.

  Here, the processing of steps S810 and S811 will be described using a specific example. A case of additional registration from the state of FIG. 9 will be described. Here, it is assumed that the relationship between the processing target image and the search result image is as shown in FIG. The image ID of the processing target image is 5 and the image ID of the search result image is 4. As shown in FIG. 10C, the search result image is a two-page layout image, and the left half of the search result image matches the processing target image. Since one reference image (reference image ID is 5) is registered for the search result image, it is re-registered as two reference images. If the registered maximum reference image ID is 5, the left page is registered as 6 for the reference image ID, and the right page is registered as 7 for the reference image ID. FIGS. 16A and 16B show the state of each table after division re-registration.

  For the association between the reference image ID and the image ID in FIG. 16A, the reference image IDs 6 and 7 are added, and the image ID 4 registered in the reference image ID 5 is inserted. Further, for the row with the reference image ID 6, the region information on the left side of the region information with the image ID 4 registered in the reference image ID 5 is displayed. The area information is inserted as each area information. The row with the reference image ID 5 is deleted. Further, link information of the processing target image is added. That is, 5 is inserted into the image ID of the row whose reference image ID is 6, and the coordinate information of the processing target image is inserted into the region information.

  For the association between the feature amount ID and the reference image ID in FIG. 16B, the feature amount ID in which the reference image ID 5 is registered is searched. Feature amount IDs 1 and 4 are acquired from FIG. And the coordinate information of those feature points is acquired. Since the coordinates of the feature points registered in the feature amount ID 1 are found to be points on the left page of the search result image, the reference image ID is changed from 5 to 6. Since the coordinates of the feature point registered in the feature amount ID 4 are found to be points on the right page of the search result image, the reference image ID is changed from 5 to 7.

  This completes the image registration process in step S409 in FIG. Next, returning to the flow of FIG. 4, if the input image is not an assigned image in step S410, the registration process of the first embodiment is terminated. If the input image is an assigned image, the process proceeds to step S411.

  If all the images have been processed in step S411, the image registration processing of the first embodiment is ended. If all the images have not been processed, the image in the next area is set as the processing target image in step S412, and the processing in step S408 and subsequent steps is repeated.

[Similar standard image search processing]
Details of the similarity reference image search process in step S801 in FIG. 8 will be described with reference to the flowchart in FIG. The search processing method described here is a method called a voting method.

  First, in step S1701, ballot boxes for the number of registered reference images are prepared. Technically, it is a process of securing a variable for storing the number of votes for each reference image on the RAM.

  In step S1702, the number of votes in all ballot boxes is reset to zero. In step S1703, the counter k is reset to 1. In step S1704, the k-th feature amount ID of the processing target image is acquired, and in step S1705, the reference image ID list registered in the feature amount ID k is acquired. In step S1706, votes are cast in the ballot boxes for all the reference image IDs in the reference image ID list. The number of votes at the time of voting may be always 1 or may be based on the similarity of feature quantities. Alternatively, the number of votes may be based on the feature point coordinates.

  In step S1707, 1 is added to the counter k (increment). If it is determined in step S1708 that the counter k is equal to or smaller than the number of feature points of the processing target image, the process returns to step S1704, and if the number exceeds the number of feature points, the process proceeds to step S1709. Finally, in step S1709, sorting is performed in descending order of votes.

  When the feature amount is quantized, the quantized value may be different even if the feature amount is similar. In that case, the feature amount ID is different. As a countermeasure, in step S1705, not only the reference image ID list registered in the feature quantity ID k is acquired, but also the reference image ID registered in the feature quantity ID having a high similarity to the feature quantity ID k. Also get. In step S1706, a method of voting for all the reference image IDs may be employed.

[Matched area identification process]
Details of the matching area specifying process in step S804 in FIG. 8 will be described with reference to the flowchart in FIG.

  First, in step S1801, the feature points of the comparison destination image having the shortest distance between feature amounts with respect to the feature points of the comparison source image are set as corresponding points, and those lists are created as the shortest distance corresponding point list. Specifically, the distance between the feature amounts of the feature amount of a feature point in the comparison source image and all the feature amounts of the comparison destination image is calculated. Among them, if the shortest distance is within the threshold Tv, the feature point ID of the comparison destination image that is closest to the feature point ID of the comparison destination image is set as the shortest distance correspondence point, and the shortest distance correspondence point list is displayed. sign up. At that time, the coordinates of the feature points and the distance between the feature amounts are also registered. By performing this process on all feature points of the comparison source image, a shortest distance corresponding point list can be created. The schema of the shortest distance corresponding point list includes, for example, as shown in FIG. 19, a feature point ID and its coordinates of the comparison source image, and a feature point ID and its coordinates of the comparison destination image. However, it is not always necessary to have these items in one schema, and it is only necessary to obtain these information by combining a plurality of schemas.

  In the following description, it is assumed that m sets of shortest distance corresponding points are registered in the shortest distance corresponding point list. For the k-th corresponding point registered in the shortest distance corresponding point list, the feature point IDs of the comparison source image and the comparison destination image are represented as Qk and Sk, respectively, and their feature point coordinates are represented as (x′k, y). 'k) and (xk, yk).

  Next, in step S1802, only the corresponding points that are estimated to be correct corresponding points on the actual image are selected from the shortest distance corresponding point list generated in step S1801, and a correct corresponding point list is created. There are similar feature amounts in one image. Accordingly, the corresponding points that are associated only by the shortest distance between the feature amounts, that is, the corresponding points created in step S1801 include points that are not correctly corresponding points on the actual image. In this process, by using the positional relationship between a plurality of feature points, the corresponding points that are estimated to be correct are selected.

  FIG. 20 is a flowchart illustrating an example of a processing procedure for creating a corresponding point list in step S1802. The system described here is a system called RANSAC (RANdom Sample Consensus).

  First, in step S2001, a variable VoteMax representing the final number of votes is initialized to zero. Next, in step S2002, a variable Count indicating the number of repetition counts of the similarity calculation process is initialized to zero.

  In step S2003, if the iteration count number Count does not exceed the maximum number of iterations Rn, the process proceeds to step S2004, and a variable Vote indicating the number of votes is initialized to zero. Next, in step S2005, two pairs of corresponding points are extracted at random from the shortest distance corresponding point list. Here, a case where the i-th corresponding point and the j-th corresponding point in the shortest distance corresponding point list are extracted will be described. The feature point IDs of the i-th corresponding point are Qi and Si, and their coordinates are (x ′ i, y ′ i) and (xi, yi), respectively. The feature point IDs of the j-th corresponding point are Qj and Sj, and their coordinates are (x'j, y'j) and (xj, yj), respectively.

  Next, in step S2006, the extracted Qi (x′i, y′i), Si (xi, yi), Qj (x′j, y′j), and Sj (xj, yj) are expressed in the equation (11). Assuming that the conversion shown is satisfied, variables a to f in Equation (11) are obtained.

ここで、第１の実施形態では、簡略化のため、相似変換だけを考える。このとき、上記式（１１）は以下の式（１２）のように書き換えられる。

このとき、変数ａ、ｂ、ｅ、ｆはｘ’ｉ、ｙ’ｉ、ｘｉ、ｙｉ、ｘ’ｊ、ｙ’ｊ、ｘｊ、ｙｊを使って式（１３）から式（１６）で表される。

However, in step S2006 shown in FIG. 20, a matrix composed of variables a to d is denoted by M, and a matrix composed of variables ef is denoted by T.

Here, in the first embodiment, only the similarity transformation is considered for simplification. At this time, the above equation (11) is rewritten as the following equation (12).

At this time, the variables a, b, e, and f are expressed by equations (13) to (16) using x′i, y′i, xi, yi, x′j, y′j, xj, and yj. The

  Next, in step S2007, the corresponding point selection variable k is initialized with 1. In step S2008, it is determined whether k is neither i nor j in order to select points other than the two sets of points randomly extracted from the shortest distance corresponding point list in step S2005 described above. When k is i or j, the process proceeds to step S2015, and after incrementing k, the process returns to step S2008. If k is neither i nor j, the process advances to step S2009 to determine whether the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list. Here, if it exceeds, the process moves to step S2016, which will be described later. If it is determined in step S2009 that the corresponding point selection variable k does not exceed the number m of corresponding points registered in the shortest distance corresponding point list, the process proceeds to step S2010. In step S2010, points other than the two sets of points Si and Sj randomly extracted from the shortest distance corresponding point list in step S2006 are extracted from the shortest distance corresponding point list. In the first embodiment, the extracted point is Sk, and the coordinates of Sk are (xk, yk).

  Next, in step S2011, the coordinates (uk, vk) of the point S'k to which the coordinates (xk, yk) of Sk are moved using the equation (12) are obtained.

  After that, in step S2012, the geometric distance between the coordinates (uk, vk) of S′k and the coordinates (x′k, y′k) of Qk is calculated as the Euclidean distance, and the Euclidean distance is calculated as the threshold value Td. It is determined whether or not: If the Euclidean distance is equal to or less than the threshold value Td, the process proceeds to step S2013, the corresponding point information of the two images is stored in the provisional corresponding point list, the vote number Vote is incremented in step S2014, and the process proceeds to step S2015. If the Euclidean distance is greater than the threshold value Td, the process proceeds to step S2015 without doing anything. In step S2015, the corresponding point selection variable k is incremented, and the process returns to step S2008, and the above processing is performed until the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list. repeat.

  Next, step S2016, which is processing when the corresponding point selection variable k exceeds the number m of corresponding points registered in the shortest distance corresponding point list in step S2009, will be described. In step S2016, the value of the vote number Vote and the value of the final vote number VoteMax are compared. If the value of the vote number Vote is larger than the value of the final vote number VoteMax, the process proceeds to step S2017. In this step S2017, the value of the final vote number VoteMax is replaced with the value of the vote number Vote, and in the next step S2018, the correct corresponding point list is replaced with the provisional correct corresponding point list. Thereafter, the iteration count number Count is incremented in step S2019, and the process returns to step S2003 described above.

  If the value of the vote number Vote is equal to or less than the value of the final vote number VoteMax in step S2016, the process proceeds to step S2019, the repeat count number Count is incremented, and the process returns to step S2003 described above.

  Next, in step S2003, if the iteration count number Count exceeds the predetermined maximum number of iterations Rn, the process proceeds to step S2020, the final vote number VoteMax, the correct corresponding point list is output, and this process ends. .

  In the description of the similarity calculation method according to the first embodiment, only the similarity transformation is described. However, for other geometric transformations such as affine transformation, a transformation matrix corresponding to each is obtained in step S2006. Can be supported. For example, in the case of affine transformation, first, in step S2005, the number of coordinates of a set of corresponding points to be selected at random is set to 3. Next, in step S2006, equation (11) is used instead of equation (12), and the variables a to f may be obtained using the three sets of corresponding points (total of 6 points) selected in step S2005.

  This is the end of the corresponding point list creation process in step S1802 of FIG. Next, returning to the flow of FIG. 18, in step S1803, the outermost bounding rectangle of the processing target image and the search result image is obtained based on the coordinates of the feature points in the correct corresponding point list. The minimum value and maximum value of each of the x-coordinate and y-coordinate of each feature point in the positive corresponding point list are obtained. A rectangle surrounded by two points (minimum value of x coordinate, minimum value of y coordinate) and (maximum value of x coordinate, maximum value of y coordinate) is defined as a circumscribed rectangle.

  In step S1804, the area of the outermost rectangle is divided by the area of the entire image to obtain the inclusion ratio RQ of the matching area in the processing target image and the inclusion ratio RS of the matching area in the search result image.

[Search processing]
FIG. 21 is a flowchart illustrating an example of a search processing procedure in the image processing apparatus according to the first embodiment.

  First, in step S2101, a search source image (query image) is input via the image input unit 201. In step S2102, the image feature amount calculation unit 204 extracts the image feature amount of the search source image. The image feature amount extraction process in step S2102 is the same as the process in step 308 in FIG.

  Next, in step S2103, the image search unit 205 uses the image feature amount calculated in step S2102 as a search source, compares it with the image feature amount of the already registered reference image, and searches for a similar reference image. To do. The similarity reference image search process in step S2103 is the same as the process in step S801 in FIG. In the similar reference image search process, a comparison with the image feature amount of the registered reference image is performed, and the similarity is calculated. Then, a result sorted in descending order of similarity is obtained.

  In step S2104, the image search unit 205 acquires an image ID corresponding to the reference image ID searched in step S2103. As shown in FIG. 9A, FIG. 11, and FIG. 16A, a plurality of image IDs are registered in association with the reference image ID, and all image IDs corresponding to the reference image ID are acquired. To do.

  Finally, in step S2105, the image acquired in step S2104 is displayed together with the similarity.

  In the above example, the similarity of the image corresponding to the reference image is displayed with the same value, but the similarity may be adjusted based on the region information. For example, when acquiring an image ID registered for the reference image ID searched in step S2103, the inclusion ratio of the reference image in the image with the image ID is obtained based on the region information. Then, the similarity may be adjusted based on the inclusion ratio, and the result sorted again by the similarity may be output as the search result. Alternatively, the number of feature points may be stored for each reference image, and the degree of similarity may be adjusted based on the number of feature points.

  In the above example, the similarity with the search result image is displayed as the search result. However, based on the region information, which region is the same region as the search source image may be displayed.

  As described above, in the first embodiment, it is determined whether or not an image to be registered is an allocated image, and in the case of an allocated image, registration is performed for each allocated original image (reference image). At this time, if the reference image to be registered is similar to the already registered reference image, link information to the already registered reference image is registered. Thus, when the same image is registered a plurality of times, it is possible to register with a small storage area without affecting the search result. Also, even images with different numbers of layouts can be registered with a small storage area by registering with link information in units of pages.

[Second Embodiment]
Hereinafter, a second embodiment will be described. In the first embodiment described above, in step S402 in FIG. 4, which is a process at the time of image registration, an input image assigned image is set based on printer driver setting information and operation information of a device such as a printer or a multifunction peripheral. Judgment was made. In the second embodiment, an example will be described in which an assigned image is determined by analyzing an input image (image region separation).

  FIG. 22 is a flowchart illustrating an example of an assigned image determination processing procedure for an input image in the image processing apparatus according to the second embodiment.

  First, in step S2201, the input image is analyzed, and image area separation processing (or rectangular area determination processing) is performed. With image area separation processing, for example, an image 2301 in FIG. 23 is recognized as a cluster for each meaningful area as indicated by 2302. Then, the attribute (text (TEXT) / drawing (PICTURE) / photograph (PHOTO) / line (LINE) / table (TABLE)), etc.) of each area is determined and divided into areas having different attributes.

  Here, an embodiment of the image area separation process will be described in detail. First, the input image is binarized into a black and white image, and contour tracing is performed in the binary image obtained thereby to extract a block in which pixels surrounded by a black pixel contour are continuous. For a black pixel block having a large area, the white pixel block is extracted by tracing the outline of the white pixel inside, and recursively from the inside of the white pixel block having a certain area or more. A black pixel block is extracted.

  The rectangular regions circumscribing the black pixel block thus obtained are classified according to their sizes and shapes, and are classified into regions having different attributes. For example, an area having an aspect ratio close to 1 and having a constant size is a pixel block corresponding to a character, and when adjacent characters are aligned, they are connected and grouped, and the grouped rectangular region Is the text area. For pixel blocks other than the character area, flat pixel blocks are line areas, black pixel blocks that are larger than a certain size and that contain rectangular white pixel blocks in a well-aligned manner are table areas, and irregular pixel blocks are scattered. An area of the image is a photographic area, and a pixel block of any other shape is a drawing area.

  Next, in step S2202, allocation number candidates are obtained based on the image area separated regions. First, it is considered whether the input image can be divided by a predetermined division method. An example is shown in FIG. FIG. 24A is a diagram showing a state where image areas are separated. FIGS. 24B to 24G are diagrams in which the division positions obtained by dividing the image by 2 to 16 are shown by broken lines (2501 etc.) in order, and these pieces of information are stored in advance in a storage device such as a hard disk. It shall be. Here, this line is referred to as a dividing line (also a page boundary position). At this time, it is checked whether or not the region where the image area is separated straddles the dividing line (ie, whether it straddles the page). From this figure, it can be seen that the regions in FIGS. 7B and 7C do not cross the dividing line. Also, FIGS. 4D to 4G show that the region straddles the dividing line. Therefore, in this example, {2, 4} is a candidate for the number of allocation.

  In step S2203, if there is no allocation number candidate, the process proceeds to step S2204, where it is determined that the image is not an allocation image, and the process ends. If there is an allocation number candidate in step S2203, the process advances to step S2205 to determine that the image is an allocation image. In step S2206, the maximum number among the allocation number candidates is set as the allocation number, and the process ends. In the case of the above example, since the allocation number candidates are {2, 4}, it is determined that the larger “4” is the final allocation number.

  As described above, in the second embodiment, the number of pages included in the processing target image can be determined by analyzing the input image (image region separation). As a result, even when there is no printer driver setting information or device operation information, the assigned image of the input image can be determined, and registration for each reference image is possible.

[Third Embodiment]
Hereinafter, a third embodiment will be described. In the first and second embodiments, a document image layout image is assumed as the input image. The third embodiment is an example when an image in which a plurality of images are pasted on a one-page document is input instead of an allocated image. Here, the image may be a natural image only, or a draw or a line drawing. Text images may also be targeted. In this case, registration is performed for each pasted image.

  FIG. 25 is a flowchart illustrating an example of a registration processing procedure in the image processing apparatus according to the third embodiment. This flow is similar to the flow of FIG. 4, and the same step numbers are assigned to portions that perform the same processing as the flow of FIG. 4. Here, only a different part of the process will be described.

  In step S2502, an image area separation process is performed on the input image. The image area separation process is the same as step S2201 in FIG. Steps S2503 and S2510 branch depending on whether or not it is determined that there are a plurality of regions. In step S2505, the number of areas N is acquired, and in step S2506, area division is performed. In this area division, processing for extracting the area obtained in step S2502 is performed. In step S2509, processing target image registration processing is performed.

  FIG. 26 is a flowchart illustrating an example of an image registration processing procedure in step S2509. This flow is similar to the flow of FIG. 8, and the same step numbers are assigned to the portions that perform the same processing as the flow of FIG. 8. Here, only a different part of the process will be described.

  In step S2605, the inclusion ratio RQ of the matching region in the processing target image is compared with the inclusion ratio RS of the matching region in the search result image. If it is determined that they are not substantially the same, the process proceeds to step S803, and the processing target image Register feature values. The processing corresponding to steps S807 to S811 in FIG. 8 is not performed.

  As described above, in the third embodiment, when a plurality of images are pasted on a one-page document, registration is performed for each pasted image (reference image). At this time, if the reference image to be registered is similar to the already registered reference image, link information to the already registered reference image is registered. As a result, even when the same image is pasted on different documents, it is possible to register with a small storage area by registering with link information in units of images.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  DESCRIPTION OF SYMBOLS 201 ... Image input part, 202 ... Assigned image determination part, 203 ... Area division part, 204 ... Image feature-value calculation part, 205 ... Image search part, 206 ... Link information registration part, 207 ... Image feature-value registration part, 208 ... Memory

Claims (11)

An image processing apparatus for registering an image in an image search apparatus,
An input means for inputting an image to be registered;
Dividing means for dividing the input image into a plurality of regions according to the image;
Calculating means for calculating the feature amount of the image for each divided area;
Search means for searching for similar images from already registered images using the feature amount for each divided area;
A registration unit that registers the feature amount of the image of the divided corresponding area when a similar image does not exist in the search result of the search unit;
An image processing apparatus comprising: The dividing means includes
Whether or not the input image by the input means is an image obtained by allocating a plurality of pages to one image, and if it is an allocated image, includes a page number determining means for determining the number of allocated pages;
The image processing apparatus according to claim 1, wherein the input image is divided according to the number of pages determined by the page number determination unit. The input means further inputs setting information indicating how many pages are included in one image to be registered,
The image processing apparatus according to claim 2, wherein the page number determination unit determines the number of pages with reference to the setting information. The page number determination means includes
Binarization means for binarizing an input image by the input means;
A rectangular area determining means for obtaining an area in which black pixels are continuous from the binary image obtained by the binarizing means and obtaining a plurality of rectangular areas based on the size and shape of each area;
With reference to the information on the page boundary position determined by the number of pages included in one image, the rectangular area determined by the rectangular area determining means divides the maximum number of pages that do not cross the page boundary. The image processing apparatus according to claim 2, further comprising means for determining the number of pages. The dividing means includes
Binarization means for binarizing an input image by the input means;
A means for determining an attribute of a region where black pixels are continuous in the binary image obtained by the binarization means, and determining a region obtained by connecting regions having the same attribute as a region to be divided. The image processing apparatus according to claim 1, wherein: The registration means includes
When the search means searches for a similar image similar to the image of the divided area and the image of the divided area includes the similar image, the similar image is divided and each divided First registration means for re-registering the feature quantity of the image in the region as the feature quantity of the reference image searched by the search means;
When the search means searches for a similar image similar to the image of the divided area and the image of the divided area is included in the similar image, the feature amount of the image of the divided area is calculated. The image processing apparatus according to claim 1, further comprising: a second registration unit configured to register as a feature amount of an image linked to the similar image. The first registration means includes:
Means for determining whether or not the similar image is an image to which a plurality of pages are allocated based on a ratio of a ratio of the regions in which the image of the divided region and the similar image obtained by the search match each other. Have
The image processing apparatus according to claim 6, wherein when the image is determined to be an image to which a plurality of pages are assigned, the similar image is divided and registered. A means for inputting a query image as a search source;
Means for calculating a feature amount of the query image;
Means for retrieving a similar image from the image processing apparatus according to any one of claims 1 to 7, using the feature amount calculated by the means;
An image search apparatus comprising: means for searching for an image having link information with the similar image. An image processing apparatus control method for registering an image in an image search apparatus,
An input step in which an input means inputs an image to be registered;
A dividing step in which the dividing unit divides the input image into a plurality of regions according to the image;
A calculation step in which the calculation means calculates a feature amount of the image for each divided area;
A search step for searching for a similar image from an already registered image using the feature amount for each divided area;
A registration step in which a registration unit registers a feature amount of an image of the divided corresponding area when a similar image does not exist in the search result of the search step;
A control method for an image processing apparatus, comprising:   The program for functioning each means which the apparatus of any one of Claim 1 thru | or 8 has by making the computer read and execute.   A computer-readable storage medium storing the program according to claim 10.

Images