JP4468736B2 - Similar image retrieval device, similar image retrieval method, and similar image retrieval program - Google Patents

Description

  The present invention relates to a similar image retrieval technique, and in particular, a similar image retrieval device, a similar image retrieval method, and a similar image that determine and retrieve similar images by applying matching by dynamic programming (DP) in multiple stages. Relating to the search program.

Conventionally, for example, as in the technique described in Patent Document 1 below, a feature vector of an area in an image is extracted and image retrieval is performed.
JP 2000-082075 A

  However, in the conventional method, even when the subject is the same, when the shooting position is different and the image of the subject is distorted, it is determined that the extracted feature vectors are different and the image similarity is different.

  The present invention solves the above-described problems of the prior art, and even when the shooting position is different and the image of the subject is distorted, it is possible to appropriately determine the similarity of images and search for similar images. An object is to provide a new similar image retrieval technique.

  In the present invention, DP matching is used in multiple stages, and image similarity is determined and searched in consideration of vertical and horizontal shifts and image size or distortion.

That is, in order to solve the above-described problem, the present invention provides a similar image search device that searches for a similar image of a target image, the target image, and a search target image selected from the search target image group. Is divided into image elements each having a × b pixels (a ≧ 1, b ≧ 1), and latticed, and vertical or horizontal lattice rows in the latticed target image and search target image A cost calculating means for calculating a minimum cost corresponding to a DP path between two image element rows arranged in the grid row direction by performing DP matching between the image element rows arranged in the direction; and the minimum cost, and DP cost lattice space generating means for generating a DP cost lattice space by assigning a matrix position determined by the calculating source and turned positions of the two image-element sequence, the DP By performing DP matching on strike lattice space, obtains the DP paths in the DP cost lattice space, the DP path minimum cost determination means for calculating a minimum cost for that DP path as a final minimum cost, the final minimum The image processing apparatus includes a similarity determination unit that determines a similarity between the target image and the search target image based on a cost.

Further, the present invention is a similar image search method for searching for similar images of a target image by a computer, wherein the target image and a search target image selected from the search target image group are respectively a × A gridding step that divides and grids image elements of b pixels (a ≧ 1, b ≧ 1), and image elements arranged in the vertical or horizontal grid row direction in the grid target image and search target image By performing DP matching between columns, a cost calculating step for calculating a minimum cost corresponding to a DP path between two image element columns arranged in the grid column direction, and the minimum cost as a calculation source. and DP cost lattice space generating step of generating a DP cost lattice space by assigning a matrix position determined by the position of the two image element sequence has the By performing DP matching for P costs lattice space, obtains the DP paths in the DP cost lattice space, the DP path minimum cost determination step of calculating the minimum cost for that DP path as a final minimum cost, the final And a similarity determination step for determining a similarity between the target image and the search target image based on a minimum cost.

  The similar image search program according to the present invention is a program for causing a computer to function as each unit of the similar image search device. This program can be provided by being recorded on a computer-readable recording medium or can be provided through a network.

  In the present invention, DP matching is used in multiple stages, and image similarity is determined and searched in consideration of vertical and horizontal shifts and image size or distortion. Therefore, according to the present invention, even when the shooting position is different and the image of the subject is distorted, it is possible to appropriately determine the similarity of images and search for similar images.

  Hereinafter, the present invention will be described in detail according to embodiments. FIG. 1 is a diagram showing an example of the configuration of a similar image retrieval apparatus of the present invention. In FIG. 1, reference numeral 1 denotes a similar image retrieval apparatus including a CPU, a memory, and a software program for causing the CPU to execute similar image retrieval processing. Reference numeral 2 denotes an image input device for inputting a target image (an image serving as a search key), and reference numeral 3 denotes a search target image storage device for storing a search target image group.

  The similar image search device 1 includes a lattice unit 11 that lattices a target image and a search target image selected from the search target image group stored in the search target image storage device 3, and a target image. A first processed image memory 12 for storing the result of gridding, a second processed image memory 13 for storing the result of gridting the search target image, and an image element of the grid target and the search target image Column image selecting means 14 for selecting a column, each column cost calculating means 15 for calculating the minimum cost between the columns by performing DP matching on the two selected image element columns, and the calculated minimum cost for each column DP cost lattice space generating means 16 for generating a DP cost lattice space by assigning to a matrix position corresponding to the position, a DP path in the DP cost lattice space, and DP path / minimum cost determination means 17 for determining the minimum cost (final minimum cost), final minimum cost storage means 18 for storing the final minimum cost, and final minimum cost stored in the final minimum cost storage means 18 The similarity determination means 19 for determining the similarity between the target image and the search target image.

  The lattice unit 11 divides the target image input from the image input device 2 into image elements of a × b pixels (a ≧ 1, b ≧ 1) and generates a first processed image. , The first processed image memory 12 is written. Note that the target image is not input from the image input device 2 but may be stored in the storage device in advance. Also, if the target image is an analog image, it is digitized before being gridded.

  In addition, the lattice unit 11 converts the search target images selected in order from the search target image group stored in the search target image storage device 3 into image elements of a × b pixels (a ≧ 1, b ≧ 1). A second processed image is generated by dividing and gridding, and is written in the second processed image memory 13. The search target image may be an image read from another device via a network.

  As a result of the processing of the grid forming means 11, the first processed image memory 12 stores the first processed image divided into (n + 1) × (m + 1) grids, for example, as shown in FIG. The second processed image memory 13 stores a second processed image divided into (q + 1) × (p + 1) grids as shown in FIG. 3, for example.

  The column image selection means 14 selects the column image (group of image elements arranged in the i-th column) in the i-th column (i = 0 to m) of the first processed image stored in the first processed image memory 12. At the same time, the second processed image memory 13 selects and reads out the j-th column (j = 0 to p) column images (image element group arranged in the j-th column) of the second processed image.

  In each column cost calculating unit 15, as shown in FIG. 4, the DP matching unit 151 includes the column image of the i-th column included in the first processed image output from the column image selecting unit 14, and the column image selecting unit 14. By performing DP matching with the j-th column image of the second processed image to be output, the DP path of the shortest path is obtained, and the minimum cost calculation means 152 has a minimum cost corresponding to the DP path. d [i, j] is calculated.

  FIG. 5 shows the DP matching for obtaining the similarity between the image element constituting the image in the i-th column of the first processed image and the image element constituting the image in the j-th column of the second processed image. It is a figure explaining calculation of minimum cost d [i, j].

  As shown in FIG. 5, the image elements [0, 1],..., [N, i] constituting the i-th column image of the first processed image, and the j-th column of the second processed image. , [Q, j] are matched with the image elements [0, j],. For the calculation of the distance in DP matching, for example, the corresponding Euclidean distance in the image element of a × b pixels, the sum of squares of the Euclidean distance (the sum of squares of the difference values), or the like is used. The shortest path that minimizes the distance from the start state to the end state is obtained, and the DP path indicated by the bold line corresponding to the shortest path is obtained. Then, the minimum cost d [i, j] corresponding to the obtained DP path is calculated.

  This process is repeated from i to 0 to m and j from 0 to p. Instead of performing all combinations of i and j, the combination of i and j is limited, and only the portion of the diagonal lattice point that may be the shortest path is selectively calculated. The calculation process may be speeded up by omitting the calculation of other parts.

  The DP cost lattice space generating unit 16 assigns the minimum cost d [i, j] calculated by each column cost calculating unit 15 to the matrix position (i, j), thereby generating a DP cost lattice space.

  FIG. 6 shows the DP cost lattice space generated by the DP cost lattice space generating means 16 in this way. In this DP cost lattice space, the minimum cost d [i, j] calculated by each column cost calculation means 15 is assigned to the matrix position (i, j). The DP cost lattice space has a size of (m + 1) × (p + 1) as shown in FIG.

  The DP path / minimum cost determination means 17 performs DP matching on the DP cost lattice space. In other words, this is the DP path / minimum cost using the minimum cost d [i, j] assigned to the matrix position (i, j) of the DP cost lattice space in the DP recurrence equation in DP matching. Means seeking.

  In this way, the DP path / minimum cost determining means 17 determines the DP path in the DP cost lattice space as shown in FIG. 7, for example. The DP path / minimum cost determining means 17 determines a cost (final minimum cost) corresponding to the determined DP path and stores it in the final minimum cost storage means 18.

  The similarity determination unit 19 determines the similarity between the target image and the search target image based on the final minimum cost stored in the final minimum cost storage unit 18.

  An example of the processing flow of the similar image search method of the present invention will be described with reference to FIG. First, the lattice unit 11 of the similar image search device 1 inputs a target image from the image input device 2, digitizes it as necessary, and divides it into blocks of a × b pixels to form a lattice ( Step S1). By this lattice processing, a first processed image divided into (n + 1) × (m + 1) lattices as shown in FIG. 2, for example, is obtained (step S1).

  Further, the gridding means 11 selects a search target image from the search target image group stored in the search target image storage device 3, and grids the selected search target image (step S2). By this lattice processing, a second processed image divided into (q + 1) × (p + 1) lattices as shown in FIG. 3 is obtained.

  DP matching is performed on each of the [0] to [m] columns of the first processed image and the [0] to [p] columns of the second processed image to obtain a DP path (shortest path). A minimum cost corresponding to the path is calculated (step S3).

  A DP cost lattice space as shown in FIG. 6 is generated by substituting the minimum cost corresponding to the calculated DP path into the corresponding lattice points in the DP cost lattice space. (Step S4). Using each of the substituted minimum costs, a DP path in the DP cost lattice space is obtained, and a cost (final minimum cost) corresponding to the DP path is determined (step S5).

  The determined final minimum cost is stored in the final minimum cost storage means 18, and the similarity between the target image and the search target image is determined based on the stored final minimum cost (step S6). The smaller the final minimum cost, the greater the similarity. The above process is repeated for all search target images, and if there are no unprocessed search target images (step S7), the process is terminated.

  As a result of determining the similarity between the target image and each search target image, the search target image with the highest similarity, the search image with the highest similarity, or a search with a similarity greater than or equal to a predetermined threshold The target image is output as a search result.

  In step S2, DP matching is performed on each of the [0] to [m] columns of the first processed image and the [0] to [p] columns of the second processed image. Then, the image for DP matching may be a row image in which the image elements are arranged in the horizontal direction. For example, the [0] to [n] rows of the first processed image and the [0] row of the second processed image. DP matching may be performed on each of the [q] rows.

  As described above, according to the present invention, after DP matching is performed on a column image or a row image, DP matching is performed using a DP cost lattice space generated by the minimum cost, and vertical and horizontal shifts and images are detected. The image similarity is determined and searched in consideration of the size or distortion of the image.

It is a figure which shows an example of a structure of the similar image search device of this invention. It is a figure which shows a 1st process image. It is a figure which shows a 2nd process image. It is a figure explaining calculation of minimum cost d [i, j] through DP matching. It is a figure explaining calculation of minimum cost d [i, j] by DP matching between sequence images. It is a figure which shows DP cost lattice space. It is a figure which shows DP path in DP cost lattice space. It is a figure which shows an example of the processing flow of the similar image search method of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Similar image search device 2 Image input device 3 Search object image storage device 11 Gridding means 12 1st process image memory 13 2nd process image memory 14 Column image selection means 15 Each row cost calculation means 16 DP cost grid space generation Means 17 DP path / minimum cost determining means 18 Final minimum cost storing means 19 Similarity determining means 151 DP matching means 152 Minimum cost calculating means

Claims (5)

A similar image retrieval device for retrieving a similar image of a target image,
Gridding means for dividing and gridting the target image and the search target image selected from the search target image group into image elements each having a × b pixels (a ≧ 1, b ≧ 1);
By performing DP matching between the image element row arranged in the vertical or horizontal grid row direction in the grid target image and the search target image, between the two image element rows arranged in the grid row direction A cost calculating means for calculating a minimum cost corresponding to the DP path;
DP cost lattice space generation means for generating a DP cost lattice space by assigning the minimum cost to a matrix position determined by the position of the two image element sequences from which the calculation is made;
By performing DP matching for the DP cost lattice space, obtains the DP paths in the DP cost lattice space, the DP path minimum cost determination means for calculating a minimum cost for that DP path as a final minimum cost,
A similar image search apparatus comprising: a similarity determination unit that determines a similarity between the target image and the search target image based on the final minimum cost. The DP path / minimum cost determining means, when performing DP matching on the DP cost lattice space, substitutes the minimum matrix value in the DP cost lattice space into the term of the inter-vector distance of the DP recurrence formula in DP matching. The DP path / minimum cost is calculated using the cost and the final minimum cost is determined.
The similar image retrieval apparatus according to claim 1. A similar image retrieval method for retrieving similar images of a target image by a computer,
A gridding step for splitting the target image and the search target image selected from the search target image group into image elements each having a × b pixels (a ≧ 1, b ≧ 1);
By performing DP matching between the image element row arranged in the vertical or horizontal grid row direction in the grid target image and the search target image, between the two image element rows arranged in the grid row direction A cost calculating step for calculating a minimum cost corresponding to the DP path;
A DP cost lattice space generation step of generating a DP cost lattice space by assigning the minimum cost to a matrix position determined by the position of the two image element sequences from which the calculation is made;
By performing DP matching for the DP cost lattice space, obtains the DP paths in the DP cost lattice space, the DP path minimum cost determination step of calculating the minimum cost for that DP path as a final minimum cost,
A similarity image search method comprising: a similarity determination step for determining a similarity between the target image and the search target image based on the final minimum cost. In the DP path / minimum cost determination step, when performing DP matching for the DP cost lattice space, the minimum value substituted into the matrix position of the DP cost lattice space in the term of the inter-vector distance of the DP recurrence formula in DP matching Using the cost, find the DP pass / minimum cost and determine the final minimum cost
The similar image retrieval method according to claim 3, wherein: A similar image search program for causing a computer to execute the similar image search method according to claim 3.

Images