JP2011253442A - Image retrieval device, data processing method for image retrieval device and image retrieval system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image retrieval device capable efficiently and accurately retrieving an image, and to provide a data processing method for an image retrieval device and an image retrieval system.SOLUTION: An image retrieval device 100 includes: a configuration straight line information storage section 102 for storing straight line information configuring the outline in a retrieval target image; a configuration straight line candidate extraction section 104 for extracting combination candidate information of configuration straight lines from the configuration straight line information storage section 102 on the basis of the similarity of angle information from a query including angles formed by polygons of a retrieval query; and a distance calculation section between shapes 106 for calculating a distance between shapes from the combination candidate information of the configuration straight lines extracted in the configuration straight line candidate extraction section 104 and the angle information from the query.

Description

  The present invention relates to an image search device, a data processing method for the image search device, and an image search system, and more particularly to an image search device using the shape information as a query, a data processing method for the image search device, and an image search system.

In order to realize an image search using a query shape specified by the user, it is necessary to calculate a similarity between the query shape specified by the user and the shape in the search target image.
In order to perform high-speed collation at the time of retrieval, it is necessary to extract feature amounts from the retrieval target image group in advance. When the entire image is the search target, the feature amount may be extracted from the entire image. However, when the object that is a part of the image is the search target, it is necessary to extract the feature amount of the corresponding region from the search target image. .

FIG. 17 is a diagram illustrating an example of an image search method using a shape as a query. When a user inputs a shape to be searched as a query shape, an image including a similar shape is output as a search result.
Examples of image retrieval devices are described in the following patent documents.
The image search apparatus described in Patent Literature 1 acquires the shape and color feature of a query to be searched for selected by the user and the position to be searched, and scans the search window from the images of the search target image group. Then, a similar region is searched and a similar image is extracted.

  The image search device described in Patent Document 2 determines the type of main shape (for example, a circle, a triangle, or a quadrangle) included in a search target image in advance and registers it, thereby specifying a user. Search using object shape type as a query.

The image search apparatus described in Patent Document 3 automatically extracts a region of interest from a search target image in advance and determines the similarity with a query region.
The straight line image search device described in Patent Document 4 registers straight line features (start point, end point, center of gravity, inclination, or length) included in a search target image, and provides straight line information and straight lines given by the user. Search for similarities such as surrounding colors. By considering a straight line that may be continuous as a straight line, it is possible to search for a straight line that is partially missing.

  The image feature extraction device described in Patent Document 5 registers the inclination of a straight line included in an image, calculates the similarity with the straight line information in the image given by the user, and searches. Further, the image forming system described in Patent Document 6 calculates the similarity of shapes using the angle formed by the line segment.

JP 2000-76302 A JP 2009-193398 A JP 2007-199749 A JP 2000-82075 A JP-A-5-135174 JP 2006-252007 A

However, the above-described conventional system has the following problems.
The problem with the example described in Patent Document 1 is that the search window is set and therefore it is easily affected by scale and rotation, and it takes a very long time to search for a search target image and extract a feature amount during the search. is there.
The problem with the example described in Patent Document 2 is that there are various query shapes that the user uses for searching, and it is difficult to prescribe the shapes in advance. Also, it is difficult to automate the shape registration.

  The problem of the example described in Patent Document 3 is that a search result desired by the user cannot be obtained. The reason is that the region of interest in the search target image is extracted as a dominant single color and is fixed regardless of the user's interest, but the region that becomes the query can be freely selected by the user. This is because the selected item is different from the user's intention. The attention area in which the user is interested is not constant, and changes depending on the purpose of the search. Therefore, it is very difficult to determine the attention area from the search target image in advance.

The problem with the example described in Patent Document 4 is that the degree of similarity such as a straight line position or a peripheral color in an image is determined, so that a search with a specific shape cannot be performed, and it is robust to rotation because it depends on position information. It is not.
The problem with the example described in Patent Document 5 is that it is not robust to large angle rotations. The reason for this is that the rotational deviation that can be absorbed by the introduction of the membership function is limited to a small angle, and it cannot cope with rotation of 45 degrees or 90 degrees. Further, since a set of straight line inclinations of the entire image is used as a feature amount, it is considered that the search accuracy is lowered unless a shape similar to the query occupies most of the image.

  Further, since the problem in the example described in Patent Document 6 is the similarity based only on the angle formed by the line segment, for example, the angle formed by any of a rectangle and a square is 90 degrees, 90 degrees, 90 degrees, and the like. In the case of 90 degrees, it is indistinguishable.

  In this way, in the image search by shape, the query shape used by the user for the search is unknown, so it is difficult to determine in advance from its diversity, and the shape determined at the time of search is searched. Then there is a problem that takes too much time. In the method of automatically extracting a search target area in an image based on a special selection criterion, the selection criterion is fixed, and thus it cannot cope with various shapes. Although there is a method of determining the degree of coincidence by extracting a straight line component from the search target image group, there is a problem that it is vulnerable to image rotation because the straight line inclination is used.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image search device capable of efficiently and accurately searching an image, a data processing method for the image search device, and an image search system. There is.

The first image retrieval apparatus of the present invention is:
Constituent line information storage means for storing information on straight lines constituting the contour in the search target image;
Configuration line candidate extraction means for extracting combination line combination candidate information from the configuration line information storage means based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query;
The inter-shape distance calculating means for calculating the inter-shape distance from the constituent straight line combination candidate information extracted by the constituent straight line candidate extracting means and the angle information made by the query;
Is provided.

The second image retrieval apparatus of the present invention is
Query polygon information acquisition means for acquiring polygon information including a vertex position or a line segment representing the shape of the search query;
Angle information calculating means for calculating angle information made by a query including an angle formed by a polygon from polygon information of the search query;
Constituent line information storage means for storing information on straight lines constituting the contour in the search target image;
Configuration line candidate extraction means for extracting configuration line combination candidate information from the configuration line information storage means based on the similarity of the angle information made by the query acquired by the angle information calculation means made;
From the combination line combination candidate information extracted by the component line candidate extraction means and the polygon information of the search query obtained by the query polygon information acquisition means, at least of polygon movement, rotation, and scale An inter-shape distance calculating means for calculating a distance based on a positional deviation amount of a vertex with a shape after calculating one geometric deformation and correcting the geometry;
Is provided.

The data processing method of the first image retrieval apparatus of the present invention is as follows:
A data processing method of an image search device having a constituent straight line information storage device for storing information on straight lines constituting a contour in a search target image,
The image search device
Based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query, the combination line combination candidate information is extracted from the component line information storage device,
A distance between shapes is calculated from the extracted combination straight line combination candidate information and the angle information made by the query.

The data processing method of the second image retrieval apparatus of the present invention is as follows:
A data processing method of an image search device having a constituent straight line information storage device for storing information on straight lines constituting a contour in a search target image,
The image search device
Obtain polygon information that includes either the vertex position or line segment that represents the shape of the search query,
Calculating the angle information made by the query including the angle formed by the polygon from the polygon information of the search query;
Based on the similarity of the obtained angle information made by the query, the component line combination candidate information is extracted from the component line information storage device,
After calculating the geometric correction amount by calculating at least one geometric deformation amount among the movement, rotation, and scale of the polygon from the extracted combination candidate information of the constituent straight lines and the polygon information of the obtained search query The distance is calculated based on the amount of positional deviation of the apex from the shape.

The image search system of the present invention includes:
Constituent line information storage means for storing information on straight lines constituting the contour in the search target image;
Configuration line candidate extraction means for extracting combination line combination candidate information from the configuration line information storage means based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query;
The inter-shape distance calculating means for calculating the inter-shape distance from the constituent straight line combination candidate information extracted by the constituent straight line candidate extracting means and the angle information made by the query;
Presenting means for presenting the calculated distance between the shapes to the user;
Is provided.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  Further, although a plurality of procedures are described in order in the data processing method or computer program of the present invention, the described order does not limit the order in which the plurality of procedures are executed. For this reason, when implementing the data processing method and computer program of this invention, the order of the several procedure can be changed in the range which does not interfere in content.

  Furthermore, the plurality of procedures of the data processing method or computer program of the present invention are not limited to being executed at different timings. For this reason, another procedure may occur during the execution of a certain procedure, or some or all of the execution timing of a certain procedure and the execution timing of another procedure may overlap.

  ADVANTAGE OF THE INVENTION According to this invention, the image search apparatus which can perform an efficient and highly accurate image search, the data processing method of an image search apparatus, and an image search system are provided.

It is a functional block diagram which shows the structure of the image search device which concerns on embodiment of this invention. It is a figure for demonstrating extraction of the straight line information from the search object image in the image search device which concerns on embodiment of this invention. It is a figure which shows the example of the straight line information recorded on the structure straight line information storage part of the image search device which concerns on embodiment of this invention. It is a figure which shows the example of the straight line information recorded on the structure straight line information storage part of the image search device which concerns on embodiment of this invention. It is a figure for demonstrating the search of the similar figure in the search object image corresponding to the search query in the image search device which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the image search device which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the image search device which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the image search device which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the image search device which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the image search device which concerns on embodiment of this invention. It is a figure for demonstrating the structure straight line detection process of the image search device which concerns on embodiment of this invention. It is a figure which shows an example of the search result display screen of the image search device which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the image search device which concerns on embodiment of this invention. It is a flowchart which shows an example of operation | movement of the image search device which concerns on embodiment of this invention. It is a figure for demonstrating the calculation precision of the intersection coordinate of 2 straight lines of the image search device which concerns on embodiment of this invention. It is a figure for demonstrating the angle parameter accuracy of the straight line in the straight line extraction of the search object image of the image search device which concerns on embodiment of this invention. It is a figure for demonstrating the image search using the search query shape from a search object image group.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

(First embodiment)
FIG. 1 is a functional block diagram showing a configuration of an image search apparatus 100 according to the embodiment of the present invention.
The image search apparatus 100 according to the present embodiment is similar to a constituent straight line information storage unit 102 that stores information on straight lines that form a contour in a search target image, and angle information made by a query including an angle formed by a polygon of a search query. Based on the configuration line candidate extraction unit 104 for extracting the configuration line combination candidate information from the configuration line information storage unit 102, the configuration line combination candidate information extracted by the configuration line candidate extraction unit 104, and the angle information to be queried. An inter-shape distance calculation unit 106 that calculates the inter-shape distance.

  Specifically, referring to FIG. 1, image search apparatus 100 according to the embodiment of the present invention includes a configuration line information storage unit 102, a configuration line candidate extraction unit 104, and an inter-shape distance calculation unit 106.

  Each component of the image search apparatus 100 includes a CPU (Central Processing Unit) of an arbitrary computer, a memory, a program that realizes the components of this figure loaded in the memory, a storage unit such as a hard disk that stores the program, a network It is realized by any combination of hardware and software with a focus on the connection interface. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. Each figure described below shows functional unit blocks, not hardware unit configurations.

Specifically, the image search device 100 includes, for example, a CPU, a memory, a hard disk, and a communication device (not shown) and is connected to an input device such as a keyboard and a mouse and an output device such as a display and a printer. It can be realized by a computer or a device corresponding to them. Each function of each unit can be realized by the CPU reading the program stored in the hard disk into the memory and executing it.
In the following drawings, the configuration of parts not related to the essence of the present invention is omitted and is not shown.

The constituent straight line information storage unit 102 stores information regarding straight lines extracted from the image group to be searched.
In the present invention, the image to be searched can include a still image and a moving image.

As a method for extracting a straight line, Hough transform described in Patent Document 4 may be used. Or Harris Corner Detector (C. Harris, 1 other, “A combined corner and edge detector”, Proc. 4th Alvey Vision Conf.), (UK), The Plessey Company plc., August 1988, p.147-151), etc. Alternatively, a line having a large number of pixels having edges on a straight line connecting the points may be extracted as a straight line.
A pair of the distance ρ from the origin in the straight line extracted in this way and the inclination angle θ of the perpendicular drawn to the straight line is recorded in the constituent straight line information storage unit 102 as straight line information.

FIG. 2 shows an example of extracting straight line information from the search target image.
The straight line information may use coefficients a, b, and c of a straight line equation ax + by + c = 0 in addition to the distance and the slope. Can be obtained as a line segment, the coordinates (x1, y1) and (x2, y2) of the two end points of the line segment may be stored as information on the straight line. However, in the present invention, since the angle formed is frequently used, it is desirable to record the tilt angle θ.

  The extracted straight line information is recorded as values of ρ and θ for each straight line as shown in FIG. At this time, in order to facilitate subsequent processing, as shown in FIG. 4, it is preferable to record a value representing a difference in angle with respect to a combination of straight lines as a table. At this time, if only polygons whose vertices are all located in the image are to be searched, the processing amount can be reduced by recording only a set of straight lines that intersect within the image range.

  These data are stored in a recording medium such as a CD-ROM, a DVD-R, a hard disk, or a memory in a batch or divided into a plurality. The stored data can be read from a single or a plurality of media by a reading device such as a CD-ROM drive, or can be acquired via a network such as the Internet or a LAN (Local Area Network) through a network interface.

  The constituent straight line candidate extraction unit 104 inputs information 10 of an angle θqi (where i = 1 to n is a natural number) formed by a polygon of an n-square search query, and an angle formed to match the angle formed by the input query. Are extracted from the configuration line information storage unit 102, and information on the configuration line candidates (corresponding configuration line candidate information 12) is output.

  Here, the angle information 10 made by the query input to the constituent straight line candidate extraction unit 104 may be, for example, information specified by the user operating the operation unit, or information prepared in advance via a recording medium or a network. It may be information read from another device and input, and is not particularly limited. The angle information 10 made by the query can be received by a receiving unit (not shown) and input to the constituent straight line candidate extracting unit 104 and the inter-shape distance calculating unit 106.

The number of angles formed by the constituent lines in the image j among all the N images stored in the constituent line information storage unit 102 is mj, and the set of angles formed in the image j is represented by the following expression (1). .
Gj = {θj1, θj2,..., Θjmj} (1)
At this time, n combinations of the following formula (2) are searched for the angle q {θq1, θq2,..., Θqn} formed by the search query.
θqi = θjk (2)
However, considering the angle extraction error and the like, the following equation (3) may be used by using the threshold th as a condition for regarding θa and θb to be the same.
| Θa−θb | <th (3)

  From the detected permutation of angles, a permutation of straight lines corresponding to the sides of the query shape is obtained, and information on candidate constituent straight lines such as θ and ρ values or vertex positions of each straight line is output.

  In the example of FIG. 5, since θq1 = θj1, θq2 = θj2, and θq3 = θj3, it can be seen that L1 having θq1 and θq2 at both ends and L1 ′ having θj1 and θj2 at both ends are corresponding straight lines. Similarly, it can be seen that L2 and L2 ′ and L3 and L3 ′ are respectively corresponding straight lines.

Information about straight lines such as permutations of corresponding straight line pairs and the inclination angle θ and distance ρ of each straight line is output.
The inter-shape distance calculation unit 106 calculates and outputs the inter-shape distance 14 for each image from which the sides of the query shape detected by the constituent straight line candidate extraction unit 104 and the corresponding straight line candidates are extracted.
The distance between shapes is a scale representing the degree of difference between two shapes. The smaller the distance between shapes, the more similar the two shapes, and the larger the distance, the less similar.

  Note that a determination unit (not shown) that determines similarity to the search query based on the calculated distance between shapes can be further provided.

As a calculation method of the distance between shapes, a permutation q {θq1, θq2,..., Θqn} formed by a query shape and a permutation Gj ′ = {θj1, θj2, formed by a straight line group in the search target image j. ,..., Θjn}, the sum of absolute values of the corresponding angle differences is calculated as the distance dj by the following equation (4).
dj = Σ | θqi−θji | (4)
However, i is a natural number of 1 to n.
In addition to this, any one that satisfies the distance axiom, such as a sum of squares or a more generalized Lp norm, may be used.

In the present embodiment, this distance between shapes is used as a measure of similarity. However, in the following description, it should be noted that the degree of similarity increases as the distance between shapes decreases. Of course, instead of the distance between shapes, an index representing similarity (which increases as the degree of similarity increases) may be used as a measure of similarity. For example, an index such as 1 / (1 + d) (d is a distance) may be used.
This process is performed for all search target images (j = 1 to N), and the calculated inter-shape distance dj is output.

  In the image search apparatus 100 of the present embodiment, the various units as described above are realized as various functions by the CPU executing various processing operations corresponding to the computer program.

  The computer program of the present embodiment causes the computer for realizing the image search apparatus 100 to store the configuration line information from the configuration line information storage unit 102 based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query. It is described that the procedure for extracting the combination candidate information, and the procedure for calculating the distance between shapes from the extracted combination candidate information of the constituent lines and the angle information to be queried are described.

  The computer program of this embodiment may be recorded on a computer-readable storage medium. The recording medium is not particularly limited, and various forms can be considered. The program may be loaded from a recording medium into a computer memory, or downloaded to a computer through a network and loaded into the memory.

  In the configuration as described above, a data processing method by the image search apparatus 100 according to the present embodiment will be described below. FIG. 6 is a flowchart showing an example of the operation of the image search apparatus 100 of the present embodiment.

  In the data processing method according to the present embodiment, the image search apparatus 100 extracts component line combination candidate information from the component line information storage unit 102 based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query. Then, the distance between shapes is calculated from the extracted combination line combination candidate information and the angle information made by the query (step S107).

The operation of the image search apparatus 100 of the present embodiment configured as described above will be described below with reference to FIG.
FIG. 6 is a flowchart showing an example of the operation of the image search apparatus 100 of the present embodiment.
The angle information 10 made by the query of the n-side search query is input to the constituent line candidate extraction unit 104 (step S101).

  Then, the constituent line candidate extraction unit 104 refers to the constituent line information storage unit 102 and extracts constituent lines in one image j from all the N search target images stored in the constituent line information storage unit 102. Then, candidates for constituent straight lines that may have an angle formed that matches the angle formed by the input query are extracted (step S105). Here, permutations of corresponding straight line pairs and information on straight lines such as the inclination angle θ and distance ρ of each straight line are output.

  Then, the inter-shape distance calculation unit 106 calculates and outputs the inter-shape distance 14 for each image from which the query shape side detected by the constituent straight line candidate extraction unit 104 and the corresponding straight line candidate are extracted (step S107). . These processes are repeated for all the search target images stored in the component straight line information storage unit 102 (NO in step S109), and when the process is completed for all search target images (YES in step S109), this process is performed. Exit. In this way, the inter-shape distance from the query polygon is output for the search target image.

  As described above, according to the image search device 100 of the present embodiment, it is possible to efficiently reconstruct a shape similar to the query shape at the time of search. For this reason, there is an effect that it is not necessary to extract a target shape from a search target image for each search. The reason is that the straight line information, which is the minimum unit of the shape, is extracted in advance for the search target image to be searched. In addition, since the shape component is described as a straight line, there is an effect that it is robust against partial chipping or shielding.

(Second Embodiment)
Next, the image search system according to the embodiment of the present invention will be described in detail with reference to FIG. FIG. 7 is a functional block diagram showing the configuration of the image search apparatus according to the embodiment of the present invention. In the image search system according to the present embodiment, the image search device 200 is different from the image search device 100 of the above-described embodiment in that it stores information on search target images or presents search results to the user. Is different. Referring to FIG. 7, the image search apparatus 200 of the present embodiment includes a search target image information storage unit 202 and a search result display unit 204 in addition to the configuration of the image search apparatus 100 of the above embodiment of FIG. In addition.

  In addition to the configuration of the image search device 100 of the above embodiment, the image search system of the present embodiment further includes a presentation unit (search result display unit 204) that presents the calculated distance between shapes to the user. In the image search system of the present embodiment, the presentation unit may be another terminal different from the image search apparatus 200, and can be configured to be connectable to the image search apparatus 200 via a network or the like.

  Specifically, the search target image information storage unit 202 stores information related to all images as search targets stored in the component line information storage unit 102. As information to be stored, an image file name, size, resolution, creation date, URL in the case of a downloaded image, an original image, a thumbnail image, and the like can be included. These pieces of information may be recorded in one-to-one correspondence with the information in the constituent straight line information storage unit 102, but a plurality of “information stored in the constituent straight line information storage unit 102” is used to reduce the storage capacity. One piece of information may be assigned to the “image extracted from”. In this case, for example, several frames before and after in the moving image represent almost the same content, so the same information (for example, the central frame of the frame group is set as a representative frame for such a highly similar frame group, Information on this may be assigned.

  The search result display unit 204 sorts the inter-shape distances to be searched output by the inter-shape distance calculation unit in ascending order, and presents the search results to the user. The presentation method may be a method of displaying the file name of the image or an image. The order of images (ranking) and the distance between shapes may be displayed simultaneously in ascending order of distance between shapes (or in descending order). In addition, it is good to make the region of the shape similar to the query shape stand out by enclosing it with a circle or changing the color of the sides of the polygon. The search result may be output as a result information file or printed out as well as presented on the screen. In addition, based on the calculated distance between shapes, a result of determining similarity to a search query may be presented.

Further, as the data to be output, the distance between shapes in all images may be output, but it is preferable to output only an image having a distance between shapes that is equal to or less than a preset threshold.
As shown in FIG. 12, the search result can be displayed in an easy-to-understand manner by displaying the searched shape by making the line segment of the corresponding shape thick or changing it to a conspicuous color such as red. it can.

  As described above, according to the image search system of the present embodiment, the same effect as that of the above-described embodiment can be achieved, and the search result can be visually presented to the user.

(Third embodiment)
Next, the image search apparatus 300 according to the embodiment of the present invention will be described in detail with reference to FIG. FIG. 8 is a functional block diagram showing the configuration of the image search apparatus according to the embodiment of the present invention.
The image search apparatus 300 of this embodiment is different from the above-described embodiment of FIG. 1 in that the user can specify the shape of the query polygon using the user interface. Referring to FIG. 8, in addition to the configuration of the image search device 100 of the above-described embodiment of FIG. 1, the image search device 300 of this embodiment includes a query polygon information acquisition unit 312, an angle information calculation unit 314 to be formed, Further included. Note that the image search apparatus 300 of the present embodiment can also include the configuration of the image search apparatus 200 of the above-described embodiment of FIG.

  The image search apparatus 300 according to the present embodiment includes a query polygon information acquisition unit 312 that acquires polygon information including a vertex position or a line segment that represents the shape of a search query, and a query from the polygon information of the search query. An angle information calculation unit 314 for calculating the angle information to be formed, and the constituent line candidate extraction unit 104 and the inter-shape distance calculation unit 106 use the angle information made by the query calculated by the angle information calculation unit 314 to be formed.

  In the present embodiment, the user of the image search device 300 can arbitrarily select a pointing device (not shown) such as a pen type or a mouse type on an operation screen displayed on a display unit (not shown) such as a display. The polygon shape can be specified.

  The shape can be specified by specifying the vertices of the polygon, by combining the sides, or by combining the positions, angles, and sizes of the polygons prepared in advance. May be specified. In either method, the query polygon information acquisition unit 312 sequentially receives the coordinate positions of the vertices that are connected by following the sides of the polygon specified by the user, and acquires them as the query vertex coordinate information 30.

The query polygon information acquisition unit 312 sequentially sets the acquired vertices as V1, V2,.
When the user selects a vertex of the polygon, the user can select the vertex position of the object in the displayed image by using the pointing device by displaying a reference image on the display unit. The query polygon information acquisition unit 312 may accept the vertex position of the object. According to this configuration, the user can easily select a specific object shape as a query.

だたし、・は内積、｜｜はノルムを表す。 The formed angle information calculation unit 314 calculates an angle θi formed at the acquired vertex Vi and outputs the calculated angle information 10 as a query. Here, the angle θi formed by the acquired vertex Vi can be calculated by the following equation (5) using the vectors Li−1 = Vi−1−Vi and Li = Vi−Vi + 1.

However, · represents an inner product, and || represents a norm.

By performing this for 1 to n, the angle information 10 made by the query is calculated and output.
However, in general, two angles of α or 180−α are considered as angles formed by two vectors, where α is a positive number. Therefore, the ambiguity can be eliminated by setting the small angle to be the angle between the two vectors. For this reason, when θi exceeds 90 degrees, 180−θi may be changed to θi again. That is, the following equation (6) may be used.
θi = min (θi, 180−θi) (6)
Here, min is a function for selecting the smaller value.

  As described above, according to the image search apparatus 300 of the present embodiment, the same effects as those of the above-described embodiment can be obtained, and the user can easily specify the query shape using a pointing device or the like.

(Fourth embodiment)
Next, the image search apparatus 400 according to the embodiment of the present invention will be described in detail with reference to FIG. FIG. 9 is a functional block diagram showing the configuration of the image search apparatus according to the embodiment of the present invention.
The image search apparatus 400 according to the present embodiment is different from the above-described embodiment of FIG. 8 in that the angle formed by each side, which is a feature robust to changes in rotation, enlargement, or movement from a polygonal query shape specified by the user. Based on the above, a line group constituting the candidate shape is extracted from the set of straight line components of the search target image, and the similarity is determined. Note that the image search apparatus 400 of the present embodiment can also include the configuration of the image search apparatus 200 of the above-described embodiment of FIG.

  The image search apparatus 400 according to the present embodiment includes a query polygon information acquisition unit 412 that acquires polygon information including either a vertex position or a line segment that represents the shape of a search query, and a polygon from the polygon information of the search query. An angle information calculation unit 414 that calculates angle information made by a query including an angle formed by a square, a configuration line information storage unit 402 that stores information on a straight line that forms a contour in the search target image, and an angle information calculation unit 414 The component line candidate extraction unit 404 that extracts the component line combination candidate information from the component line information storage unit 402 based on the similarity of the angle information made by the query acquired in step 1, and the combination of the component lines extracted by the component line candidate extraction unit 404 From the candidate information and the polygon information of the search query obtained by the query polygon information acquisition unit 412, among polygon movement, rotation, and scale, Without even including the shape distance calculation unit 406 for calculating the distance based on the position deviation amount of the vertex of a single geometric deformation amount (geometric parameters) shape after geometric correction is calculated, and the.

  Specifically, referring to FIG. 9, the image search apparatus 400 according to the present embodiment includes a query polygon information acquisition unit 412, an angle information calculation unit 414, a constituent straight line candidate extraction unit 404, and an inter-shape distance calculation. Unit 406.

The query polygon information acquisition unit 412 acquires polygon information including either a vertex position or a line segment representing the shape of the search query.
The formed angle information calculation unit 414 calculates the angle information 10 made by the query including the angle formed by the polygon from the polygon information of the search query.

The constituent straight line information storage unit 402 stores information on straight lines constituting the contour in the search target image.
The configuration line candidate extraction unit 404 extracts configuration line combination candidate information from the configuration line information storage unit 402 based on the similarity of the angle information made by the query acquired by the angle information calculation unit 414.

  The inter-shape distance calculation unit 406 moves and rotates polygons from the combination line combination candidate information extracted by the component line candidate extraction unit 404 and the polygon information of the search query obtained by the query polygon information acquisition unit 412. , And at least one geometric deformation amount (geometric parameter) of the scale, and the distance is calculated based on the amount of positional deviation of the vertex from the shape after geometric correction.

  Similar to the first embodiment, the method for calculating the distance between shapes is a permutation q {θq1, θq2,..., Θqn} of angles formed by the query shape and a straight line group in the search target image j. The permutation Gj ′ = {θj1, θj2,..., Θjn} of angles formed is acquired. Then, at least two intersections of three corresponding sides are extracted from the corresponding straight line candidates, geometric transformation parameters are calculated, geometric correction is performed, and the distance between shapes is calculated.

  As a selection criterion for the two intersection points, for example, as a method of extracting two points from n points, the absolute value | vr × vl | of the outer product when both sides of the vertex Xqi of the query shape are vectors vr and vl Two points may be selected in descending order. The reason for using the outer product is to use the orthogonality of the angle formed with the length of the side length.

  First, the calculation accuracy of the intersection coordinates of the two straight lines L1 and L2 will be described with reference to FIG. When the width (thickness) of the straight line L1 is Δd, the blur width selected by the user as the intersection of L1 and L2 is in a range indicated by a broken line in the figure. This blur width is Δd / sin θ, and becomes smaller as the angle θ formed is closer to a right angle.

  Further, with reference to FIG. 16, the accuracy of the angle parameter of the straight line in the straight line extraction of the search target image will be described. The angle error Δθ of the straight line L1 passing through the two points P1 and P2 can be approximated to Δd / L using the position blur width Δd of the point P2 and the distance L between the points. For this reason, the longer the side length which is the distance L between the points, the more the angle parameter of the extracted line can be extracted stably.

The geometric correction method will be described in detail below.
The geometric transformation of the rotation Δθ, the enlargement ratio α, and the translations ΔX and ΔY from the point P (x, y) to P ′ (x ′, y ′) can be expressed by the following equation (7). From equation (7), two points P1 (x1, y1), P2 (x2, y2) and their corresponding geometrically corrected points P1 ′ (x1 ′, y1 ′), P2 ′ (x2 ′, y2 ′) ), The parameters Δθ, α, ΔX, ΔY can be solved.

  For example, in FIG. 5, since θq1 and θj1 are equal, and θq2 and θj2 are equal, it can be seen that the straight lines L1 and L1 ′ are corresponding straight lines. Similarly, L2, and L2 ′, and L3 and L3 ′ are It can be seen that each is a corresponding straight line. If the three straight lines L1, L2, L3 and L1 ′, L2 ′, L3 ′ associated with each other are used, the intersections (Xq1, Yq1) of L1 and L3 correspond to the intersections (Xj1, Yj1) of L1 ′ and L3 ′. It turns out that they are vertices.

Similarly, (Xq2, Yq2) and (Xj2, Yj2) are the corresponding vertices. By substituting these two corresponding vertices into Equation (7) and solving the simultaneous equations, the parameters Δθ, α, ΔX, ΔY can be obtained. Then, the shape of the search target image is geometrically corrected, the vertices are projected on the same plane as the query shape, and the displacement of the position of each vertex is calculated. The positional deviation can be obtained as the sum of the absolute values of the differences between the X and Y coordinate values of the two points as in the following equation (8).
dj = Σ (| Xqi−Xji | + | Yqi−Yji |) (8)
Here, i = 1 to n is a natural number.

  Also, anything that satisfies the distance axiom, such as the sum of squares of differences, may be used, and is the same as in the above embodiment, and is not limited thereto.

  As a modification of the present embodiment, in addition to the configuration of the image search device 300 of the embodiment of FIG. 8, the combination line combination candidate information extracted by the configuration line candidate extraction unit 104 and the query polygon information acquisition unit 312 are obtained. And a correction unit (not shown) for calculating at least one geometric deformation amount of polygon movement, rotation, and scale from the polygon information of the retrieved query, and a geometric distance calculation unit 106. However, the inter-shape distance may be calculated based on the positional deviation amount of the apex from the shape after geometric correction by the correction unit.

  A data processing method performed by the image search apparatus 400 according to the present embodiment having the above-described configuration will be described below. FIG. 10 is a flowchart illustrating an example of the operation of the image search apparatus 400 according to the present embodiment.

  In the data processing method of this embodiment, the image search apparatus 400 acquires polygon information including either a vertex position or a line segment representing the shape of the search query (step S301), and from the polygon information of the search query. The angle information made by the query including the angle formed by the polygon is calculated (step S303), and the combination line combination candidate information is extracted from the component line information storage unit 402 based on the similarity of the obtained angle information made by the query (step S103). Step S105), calculating at least one geometric deformation amount among polygon movement, rotation, and scale from the extracted combination line combination candidate information and polygon information of the obtained search query, and geometric correction A distance is calculated based on the amount of positional deviation of the apex from the shape after the processing (step S107).

Hereinafter, the operation of the image search apparatus 400 in the present embodiment will be described using the flowchart shown in FIG.
10 includes steps similar to steps S103 to S109 in the flowchart of FIG. 6 of the above embodiment, and further includes steps S301, S303, and S311.
Polygon vertex positions (coordinates) designated by the user using a pen-type pointing device or the like are sequentially obtained and stored in the memory (step S301).

For example, when the user designates an n-gon, the positions of n vertices are recorded in the memory as follows.
Q [1]. x = 10
Q [1]. y = 30
Q [2]. x = 40
Q [2]. y = 70
:
Q [n]. x = 100
Q [n]. y = 20
Q [0]. x = 100
Q [0]. y = 20
At this time, since the n-gon is a closed shape, the same value as Q [n] is substituted for Q [0].
Here, Q [n]. x, Q [n]. y represents the x-coordinate and y-coordinate components of Q [n].

  When an n-square query shape has a short side as shown in FIG. 11A and the sum of angles A and B formed by both ends of this side exceeds 180 degrees, n as shown in FIG. Assuming a -1 polygon, it is better to use it as a query shape for the search. This is because the search can be performed even when a short side is not detected in the straight line detection in the search target image.

  That is, the formed angle information calculation unit 314 may calculate the angle information made by the query by using the polygon formed by extending the line segments at both ends of the side having the obtuse angle in the query polygon as the polygon of the search query.

The formed angle information calculation unit 314 calculates an angle formed by each vertex of the query polygon shape Q acquired by the query polygon information acquisition unit 312 (step S303).
The array of angles θi formed (i = 1, 2,..., N) can be calculated as in the following formulas (9) and (10).
θ [i] = arccos ((β [i−1] .x * β [i] .x + β [i−1] .y * β [i] .y) / √ ((β [i−1] .x ^ 2 + β [i−1] .y ^ 2) (β [i] .x ^ 2 + β [i] .y ^ 2))) Equation (9)

if (θ [i]> π) {θ [i] = π−θ [i]} (10)

Here, (beta) can be calculated | required by the following formula | equation (11)-Formula (14).
β [i]. x = Q [i + 1]. xQ [i]. x ... Formula (11)
β [i]. y = Q [i + 1]. yQ [i]. y ... Formula (12)

β [i-1]. x = Q [i-1]. xQ [i]. x ... Formula (13)
β [i-1]. y = Q [i-1]. yQ [i]. y ... Formula (14)

The straight line information of one search target image stored in the constituent straight line information storage unit 102 is extracted (step S103).
The straight line information is stored in the array L [M], and the angle information formed is stored in the two-dimensional array A [M] [M].
If the angle information to be made is not stored, the angle information to be made using straight line information is created.

The example in FIG. 3 is as follows.
L [1]. ρ = 50, L [1]. θ = 60,
L [2]. ρ = 60, L [2]. θ = 15,
L [3]. ρ = 40, L [3]. θ = 105,
A [1] [1] = 0, A [1] [2] = 45, A [1] [3] = 45,
A [2] [1] = 45, A [2] [2] = 0, A [2] [3] = 90,
A [3] [1] = 45, A [3] [2] = 90, A [3] [3] = 0

The constituent line candidate extraction unit 404 extracts from the constituent line information storage unit 402 candidates for search target images that may match the query polygon calculated by the angle information calculation unit 314 to be formed.
A combination of angles formed with the array θ [i] representing the permutation of angles formed by the query polygon is searched from the angle information two-dimensional array A [M] [M] stored in the component line information storage unit 402. To do.

  That is, the formed angle information calculation unit 414 includes the angle permutation information formed by extracting the angles formed by the polygons of the search query in the adjacent order, and the constituent line candidate extraction unit 404 further includes the angle permutation information formed. The combination of constituent lines having similar angles is extracted from the constituent line information storage unit 402 according to the permutation.

As simple filtering, in all i (1 to n), the following processing is performed only when p and q (1 = <p, q <= M) satisfying the following formula (15) exist, If not, the process returns to step S103, and the straight line information of another image stored in the constituent straight line information storage unit 102 is extracted.
| Θ [i] −A [p] [q] | <th (15)

Next, a permutation similar to the permutation of θ [i] (i = 1 to n) is searched from A [M] [M].
As a search method, the following recursive method may be used.
Step S201: Find p0, q0 where A [p0] [q0] == θ [1].
Step S202: Search for q other than p0 where A [q0] [q] == θ [2] and set it to p1.
Step S203: Search for q other than p0 and p1 that satisfies A [p1] [q] == θ [3] and set it to p2.
Step S204: Search for q other than pj (j = 0,..., I-1) where A [pi-1] [q] == [theta] [i], and set it as pi.
Step S205: If i <n, set i = i + 1 and repeat step S204.
Step S206: If i = n and pi = p0, the same combination straight line combination pj (1, 2,..., N) as the angle permutation formed by the extracted vertices is output. There are no other combinations.
In this way, the combination pj (1, 2,..., N) of the same constituent straight line as the angle permutation formed by the vertices is acquired (step S105).

The inter-shape distance calculation unit 406 calculates the inter-shape distance for each image obtained by extracting the sides of the query polygon detected by the constituent straight line candidate extraction unit 404 and the corresponding straight line candidates (step S107).
When a pair of pj is obtained, one vertex coordinate can be calculated by obtaining an intersection (Xb, Yb) of two straight lines of pa and pb. By performing this operation for n pairs, n-corner vertex coordinates (Xi, Yi) are calculated. From the correspondence between the vertex coordinates (Xi, Yi) (i = 1 to n) extracted in this way and the query polygon (Q [i] .x, Q [i] .y), Geometric deformation parameters α, Δθ, ΔX, ΔY are calculated. It is obtained by substituting into the equation (7) using two of the n vertices and solving the simultaneous equations.

  Based on the obtained parameters, the query polygon (Q [i] .x, Q [i] .y) is substituted into X and Y in the above equation (7), so that the query polygon after geometric deformation ( Q [i] ′. X, Q [i] ′. Y) can be calculated.

The inter-shape distance is the distance between the query polygon (Q [i] ′. X, Q [i] ′. Y) after geometric deformation and the extracted n-corner vertex coordinates (Xi, Yi). It is obtained by calculating according to (16). Here, i = 1 to n.
d = Σ (| Q [i] .x−Xj | + | Q [i] .y−Yj |) Expression (16)
In addition, the calculation of the distance is not limited to this, and any calculation that satisfies the distance axiom may be used.

It is confirmed whether or not the processing has been performed on all the search target images stored in the component straight line information storage unit 402. If not (NO in step S109), the process returns to step S103, and a new search target is obtained. Extract images.
When processing has been performed on all search target images stored in the component straight line information storage unit 402 (YES in step S109), the calculated inter-shape distances are sorted in ascending order of distance. Image information corresponding to the sorted shapes is displayed on the screen as a search result in the order of short distance (step S311).

  As described above, according to the image search apparatus 400 of the present embodiment, the same effects as those of the above-described embodiment can be obtained, and an effect that a search robust to rotation, movement, or scale fluctuation can be performed. . The reason for this is to extract line candidates having the same permutation angle from the search target image group by using the permutation of the angle between adjacent sides of the polygon that is invariant to rotation, movement, or scale variation. This is for determining.

(Fifth embodiment)
Next, the image search apparatus 500 according to the embodiment of the present invention will be described in detail with reference to FIG. FIG. 13 is a functional block diagram showing the configuration of the image search apparatus according to the embodiment of the present invention.
The image search apparatus 500 of the present embodiment is different from the above-described embodiment of FIG. 9 in that the distance is calculated in consideration of the visual feature amount corresponding to the internal region of the query polygon.

  The image search apparatus 500 according to the present embodiment uses a query polygon information based on information including at least one vertex position and a line segment representing the polygon shape of the search query acquired by the query polygon information acquisition unit 412. Query visual feature quantity extraction unit 522 that extracts visual information corresponding to the internal region, search target image information storage unit 202 that records information including at least one of the color and edge of the search target image, and constituent straight line candidates A visual feature amount extraction unit 524 that extracts the visual feature amount of the search target image in the region surrounded by the constituent straight lines extracted by the extraction unit 404 based on information stored in the search target image information storage unit 202; A visual feature amount distance calculating unit 526 that calculates a distance between visual feature amounts extracted by the query visual feature amount extracting unit 522 and the visual feature amount extracting unit 524; Further comprising a total distance calculating portion 528 for calculating a total distance based on the calculated distance between Jo distance calculation unit 406 and the visual feature quantity distance calculation unit 526, a.

  Referring to FIG. 13, in addition to the configuration of the image search device 400 of the above-described embodiment of FIG. 9, the image search device 500 of the present embodiment includes a query visual feature amount extraction unit 522 and a visual feature amount extraction unit 524. And a visual feature amount distance calculation unit 526 and an overall distance calculation unit 528. Note that the image search apparatus 500 of the present embodiment can also include the configuration of the image search apparatus of the above-described embodiment of FIG.

  The query visual feature amount extraction unit 522 obtains visual information corresponding to the internal region of the query polygon based on the vertex position and line segment information representing the query polygon acquired by the query polygon information acquisition unit 412. Extract. As a method of extracting visual features, a dominant color (Dominant Color) (a color that controls the impression of the entire color scheme) and a color layout (feature related to the arrangement of colors in the target region). Color Layout) or features such as edge histograms that are features related to patterns may be extracted. These feature amounts may be created in advance for a plurality of sample colors and sample patterns and may be selected by the user, or an image from which the feature amounts should be extracted when the user selects a polygon for the search query. And a region in the image may be specified, and the feature amount may be calculated from the region of the image.

  The visual feature amount extraction unit 524 extracts the visual feature amount of the search target image in the region surrounded by the configuration line extracted by the configuration line candidate extraction unit 404. From the image information stored in the search target image information storage unit 202, the color and edge of the pixel corresponding to the region surrounded by the constituent straight lines are extracted, and feature quantities such as Dominant Color, Color Layout, and edge histogram are calculated. The feature amount calculated here is the same as that of the query visual feature amount extraction unit 522 described above. In addition, the search target image is divided into fine regions, and feature amounts are extracted and stored in advance for each divided region, so that the feature amounts for the region surrounded by the constituent straight lines are By substituting the feature amount, the feature amount calculation process can be omitted.

  The visual feature amount distance calculation unit 526 calculates a visual feature amount distance that is a distance between the visual feature amounts extracted by the query visual feature amount extraction unit 522 and the visual feature amount extraction unit 524, respectively. As a method for calculating the visual feature amount distance, the Euclidean distance between the visual feature amounts may be used. Specifically, the distance can be calculated by calculating the square of the difference between the values of the visual feature values in each dimension and taking the square root of the sum. In addition to the square of the difference, anything that satisfies the distance axiom, such as the absolute value of the difference, may be used, and the present invention is not limited to this.

  The total distance calculation unit 528 calculates an integrated distance using the two distances calculated by the inter-shape distance calculation unit 406 and the visual feature amount distance calculation unit 526. The distance calculation can be obtained by adding two distances or adding weights weighted according to the importance of shape and visual similarity.

  The search target image information storage unit 202 and the search result display unit 204 are the same as those described in the above embodiment of FIG. 7, but in this embodiment, the search target image information storage unit 202 stores the visual feature amount. It is better to store an original image or a thumbnail image for use in calculation.

  The operation of the present embodiment configured as described above will be described below with reference to FIG. FIG. 14 is a flowchart showing an example of the operation of the image search apparatus according to the embodiment of the present invention. In the present embodiment, in the flowchart of the image search apparatus 400 of the above embodiment of FIG. 10, the distance between shapes of each image is obtained and the following processing is performed.

  First, the query visual feature amount extraction unit 522 uses the information on the positions and line segments of the vertices representing the query polygon acquired by the query polygon information acquisition unit 412 to display the visual corresponding to the internal region of the query polygon. Information is extracted (step S501).

  Then, the visual feature quantity extraction unit 524 extracts the visual feature quantity of the search target image in the region surrounded by the configuration line extracted by the configuration line candidate extraction unit 404 (step S503). Further, the visual feature amount distance calculation unit 526 calculates a visual feature amount distance that is a distance between the visual feature amount extracted by the query visual feature amount extraction unit 522 and the visual feature amount extraction unit 524 ( Step S505). Then, the total distance calculation unit 528 calculates and outputs the integrated distance using the two distances calculated by the inter-shape distance calculation unit 406 and the visual feature amount distance calculation unit 526 (step S507).

  The output total distance can be presented to the user in the same manner as the distance between shapes in the second embodiment.

  As described above, according to the image search device 500 of the present embodiment, the same effect as the above embodiment can be obtained, and the distance can be calculated in consideration of the visual feature amount of the inner region of the query polygon. In addition to the contour, an image more similar to the query can be detected, and the search processing load can be reduced.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.

  Further, in the image search device 500 of the above embodiment of FIG. 13, the image search device 400 may be configured as the image search device of the above embodiment of FIG. 1, FIG. 2, or FIG. In this case, the query visual feature amount extraction unit 522 extracts visual information for the sample color or pattern selected by the user as visual information corresponding to the internal region of the query polygon. The visual feature amount extraction unit 524 extracts the visual feature amount of the search target image in the region surrounded by the configuration line extracted by the configuration line candidate extraction unit 104. The total distance calculation unit 528 calculates the integrated distance using the two distances calculated by the inter-shape distance calculation unit 106 and the inter-shape distance calculation unit 406 and the visual feature amount distance calculation unit 526.

  In the above-described embodiment of FIG. 9, when displaying the search result in step S311 of FIG. 10, the image information corresponding to the sorted shape is stored as the search target image information storage unit 202 of the image search apparatus 200 of FIG. And the original image and the thumbnail image may be displayed in order from the shortest distance. According to this configuration, the search result can be presented to the user in an easily understandable manner.

  While the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

1 Query angle information 12 Corresponding constituent straight line candidate information 14 Inter-shape distance 30 Query vertex coordinate information 100 Image search device 102 constituent straight line information storage unit 104 constituent straight line candidate extractor 106 Inter-shape distance calculator 200 Image search device 202 Search Target image information storage unit 204 Search result display unit 300 Image search device 312 Query polygon information acquisition unit 314 Angle information calculation unit 400 Image search device 402 Configuration line information storage unit 404 Configuration line candidate extraction unit 406 Inter-shape distance calculation unit 412 Query Polygon information acquisition unit 414 Angle information calculation unit 500 Image search device 522 Query visual feature amount extraction unit 524 Visual feature amount extraction unit 526 Visual feature amount distance calculation unit 528 Total distance calculation unit

Claims (10)

Constituent line information storage means for storing information on straight lines constituting the contour in the search target image;
Configuration line candidate extraction means for extracting combination line combination candidate information from the configuration line information storage means based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query;
The inter-shape distance calculating means for calculating the inter-shape distance from the constituent straight line combination candidate information extracted by the constituent straight line candidate extracting means and the angle information made by the query;
An image search apparatus comprising: The image search device according to claim 1,
Query polygon information acquisition means for acquiring polygon information including a vertex position or a line segment representing the shape of the search query;
Angle information calculating means for calculating angle information made from the polygon information of the search query,
Further comprising
The constituent line candidate extraction unit and the inter-shape distance calculation unit use the angle information made by the query calculated by the angle information calculation unit made. The image search apparatus according to claim 2,
From the combination line combination candidate information extracted by the component line candidate extraction means and the polygon information of the search query obtained by the query polygon information acquisition means, at least of polygon movement, rotation, and scale A correction means for calculating one geometric deformation amount and correcting the geometry;
The inter-shape distance calculating means calculates the inter-shape distance based on a positional deviation amount of a vertex from the shape after geometric correction by the correcting means. Query polygon information acquisition means for acquiring polygon information including a vertex position or a line segment representing the shape of the search query;
Angle information calculating means for calculating angle information made by a query including an angle formed by a polygon from polygon information of the search query;
Constituent line information storage means for storing information on straight lines constituting the contour in the search target image;
Configuration line candidate extraction means for extracting configuration line combination candidate information from the configuration line information storage means based on the similarity of the angle information made by the query acquired by the angle information calculation means made;
From the combination line combination candidate information extracted by the component line candidate extraction means and the polygon information of the search query obtained by the query polygon information acquisition means, at least of polygon movement, rotation, and scale An inter-shape distance calculating means for calculating a distance based on a positional deviation amount of a vertex with a shape after calculating one geometric deformation and correcting the geometry;
An image search apparatus comprising: The image search device according to any one of claims 2 to 4,
The angle information calculation means is configured to calculate a polygon formed by extending a line segment of both sides of an edge having an obtuse angle in the query polygon as the polygon of the search query and calculating the angle information made by the query. A featured image retrieval device. In the image search device according to claim 1 or 4,
The angle information calculating means includes the angle permutation information formed by extracting the angles formed by the polygons of the search query in the adjacent order in the angle information to be queried.
The constituent line candidate extraction unit uses the angle permutation information formed, and extracts a combination of constituent lines having similar formed angles according to the permutation from the constituent line information storage unit. The image search device according to claim 4,
Visual information corresponding to the internal region of the query polygon is extracted based on the information including at least one vertex position and line segment representing the polygon shape of the search query acquired by the query polygon information acquisition means. Query visual feature amount extraction means,
Search target image information storage means for recording information including at least one of the color and edge of the search target image;
Visual feature amount extraction means for extracting the visual feature amount of the search target image in the region surrounded by the constituent straight line extracted by the constituent line candidate extraction means based on the information stored in the search target image information storage means; ,
Visual feature amount distance calculating means for calculating a distance between the query visual feature amount extracting means and the visual feature amount extracted by the visual feature amount extracting means;
Total distance calculation means for calculating a total distance based on the distance calculated by the inter-shape distance calculation means and the visual feature amount distance calculation means;
An image search device further comprising: A data processing method of an image search device having a constituent straight line information storage device for storing information on straight lines constituting a contour in a search target image,
The image search device
Based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query, the combination line combination candidate information is extracted from the component line information storage device,
A data processing method of an image search apparatus for calculating a distance between shapes from the extracted combination line combination candidate information and the angle information made by the query. A data processing method of an image search device having a constituent straight line information storage device for storing information on straight lines constituting a contour in a search target image,
The image search device
Obtain polygon information that includes either the vertex position or line segment that represents the shape of the search query,
Calculating the angle information made by the query including the angle formed by the polygon from the polygon information of the search query;
Based on the similarity of the obtained angle information made by the query, the component line combination candidate information is extracted from the component line information storage device,
After calculating the geometric correction amount by calculating at least one geometric deformation amount among the movement, rotation, and scale of the polygon from the extracted combination candidate information of the constituent straight lines and the polygon information of the obtained search query A data processing method of an image search device for calculating a distance based on a positional deviation amount of a vertex from a shape. Constituent line information storage means for storing information on straight lines constituting the contour in the search target image;
Configuration line candidate extraction means for extracting combination line combination candidate information from the configuration line information storage means based on the similarity of the angle information made by the query including the angle formed by the polygon of the search query;
The inter-shape distance calculating means for calculating the inter-shape distance from the constituent straight line combination candidate information extracted by the constituent straight line candidate extracting means and the angle information made by the query;
Presenting means for presenting the calculated distance between the shapes to the user;
An image search system comprising:

Images