JP3884468B2 - Fast image search method - Google Patents

Description

The present invention relates to an image search method by a computer, and more specifically, has a similar or identical relationship with an original figure pattern (hereinafter also referred to as an original pattern) having an arbitrary shape including a character pattern to be searched using a computer. In addition, the present invention relates to a method for quickly searching for a pattern on a searched image that may be partially hidden in an arbitrary posture.

Conventionally, using a computer to find a pattern that is similar or identical to the original pattern from the image to be searched,
There is a template matching method using the normalized correlation method that is often used when the original pattern is a specific pattern and the patterns on the searched image are separated without overlapping, and the original pattern is an arbitrary figure When the pattern on the image to be searched includes a partially hidden figure, a continuous figure is interrupted, or a lot of noise components are included, a method using a general theory of generalized Hough transform , A method using the general theory of Geometric Hashing, a method disclosed in JP 2000-502824A (P2000-502824A), and the like.

The template matching method using the normalized correlation method cannot be applied to a searched figure whose pattern on the searched image has an arbitrary posture, a similar size, or a partially hidden pattern.
The method using the generalized Hough transform and the general theory of Geometric Hashing is used when the original pattern has an arbitrary shape, the pattern on the searched image is partially hidden, or has an arbitrary posture, similarity, or the same size with respect to the original pattern. It can be applied to all, but when the original pattern becomes complicated or large, or when the image to be searched becomes large, the number of feature points increases, resulting in a large number of calculations. This is a method that takes a long processing time. In particular, in the method using the general theory of Geometric Hashing, when the original pattern is normalized and registered in the reference coordinate system, or in order to compare the pattern in the searched image with the registered original pattern, The combination of selection of feature point base pairs, which is the standard for normalization when expanding to the reference coordinate system, is basically in the order of the square of the number of feature points, and the original pattern becomes more complex and larger Obviously, the number of calculations increases and the processing time becomes longer.
Also in the method disclosed in JP 2000-502824 (P2000-502824A), when the original pattern has an arbitrary shape, the pattern on the searched image is partially hidden or has an arbitrary posture, similarity, or the same size with respect to the original pattern However, when the original pattern becomes complicated or has a large size, or when the image to be searched has a large size, it takes a long processing time.
Special table 2000-502824 (P2000-502824A) Computer Vision: Technical Review and Future Prospects Page 149-161 New Technology Communications Inc.

In the conventional technique, when the original pattern becomes complicated or large, or when the size of the image to be searched becomes large, there is a problem that the number of feature points increases and processing time is long. .
On the other hand, as a necessary performance in many image search operations, when the original pattern is an arbitrary shape, the pattern on the searched image is partially hidden or has the ability to handle the case of an arbitrary posture, similarity or the same size with respect to the original pattern I want to search at high speed from the image to be searched even when the original pattern has a complicated external shape, or when various figures or characters are combined or combined to become a large size, or There are many requests to search at high speed even when the image to be searched has a very large size.
In the present invention, when the original pattern is an arbitrary shape, the pattern on the searched image is partially hidden, and can be applied to a searched figure corresponding to all cases of an arbitrary posture, similarity, or the same size with respect to the original pattern. The object is to provide a method for quickly searching for a pattern to be searched for in a search image in processing time.

A plurality of feature points are set on the contour of the original pattern by a predetermined method 1, and a feature point pair (hereinafter, referred to as a first feature point and a second feature point) is formed by a predetermined method 2 using the plurality of feature points. A source pattern feature point / pair setting step for setting a plurality of sets of original pattern base pairs) and a vector having the first feature point of the original pattern base pair as the start point and the second feature point as the end point (hereinafter referred to as the original pattern base vector) And a point group representing the shape of the original pattern on the original pattern local coordinate system arranged in a predetermined relationship with the normalized original pattern basis vector, relative to the original pattern basis vector. The original pattern local coordinate point group is normalized and expanded while maintaining the positional relationship, and the original pattern local coordinate point group is transformed into the original coordinate reference coordinate point group by converting the original pattern local coordinate point group to a predetermined reference coordinate system. An area is divided into small areas, and an original pattern base is added to an information cell corresponding to a small area where any of the original pattern reference coordinate points exists on a hash table including information cells corresponding to the divided small areas. An original pattern registration process for registering an original pattern from an original pattern feature registration step for executing a procedure for writing identification information of a pair on a plurality of necessary original pattern base pairs;
A plurality of feature points are set on the contour of the pattern on the image to be searched by a predetermined method 1, and a feature point comprising the first feature point and the second feature point by a predetermined method 2 using the plurality of feature points A pattern feature point / pair setting step for setting a pair (hereinafter also referred to as a pattern basis pair) and a vector having a first feature point as a start point and an end point as a second feature point (hereinafter also referred to as a pattern basis vector) And the normalized pattern basis vector on the pattern local coordinate system arranged to maintain the same predetermined relationship as the relationship between the original pattern basis vector and the original pattern local coordinate system in the registration process. The point group representing the pattern shape is normalized and expanded as a pattern local coordinate point group while maintaining the relative positional relationship with the pattern basis vector, The point group is coordinate-converted on the reference coordinate system to form a pattern reference coordinate point group. If any of the pattern reference coordinate point groups exists in the small area, the small area is described in the information cell of the corresponding hash table. Voting in a ballot box provided corresponding to each of the original pattern base pair identification information (hereinafter also referred to as a base pair ballot box) for each of the small regions, Calculates the target pose, which is a specification representing the similarity, posture, and positional relationship between the original pattern base pair corresponding to the ballot box that has obtained a high number of votes and a vote number higher than a predetermined threshold, and the pattern base pair at that time. A first search step for obtaining a plurality of target poses by executing a series of procedures on the required plurality of pattern base pairs and the most accurate distribution from the obtained distributions of the plurality of target poses. One or more target poses are selected by a predetermined method, and the pattern developed on the reference coordinate system is the same as the original pattern using the pattern base pair that is the target of calculation of the selected target pose as a base vector, or A searched image by going through a pattern search process for selecting a pattern on one or more searched images that is identical to or highly likely to be similar to the original pattern from the second search step that is selected as a pattern having a high possibility of similarity Take a way to search for the pattern above.
As the predetermined relationship between the original pattern basis vector and the original pattern local coordinate system, the origin of the original pattern local coordinate system is placed at the midpoint of the original pattern basis vector, and the X axis or Y axis of the original pattern basis vector and the original pattern local coordinate system Taking the relationship of superimposing and is one of the preferable methods in consideration of processing convenience. Of course, any other relationship that is uniquely determined may be used.

The predetermined method 1 is a method of setting a plurality of feature points at predetermined intervals on the original pattern or the contour of the pattern, and the predetermined method 2 uses the feature points as the first feature points, One or more second points passing through one feature point and having a predetermined angle with respect to the direction of the normal of the contour at the first feature point (hereinafter also referred to as a feature point setting line) intersect the contour. A procedure for setting as a feature point pair by setting two feature points and setting each of the first feature point and one or more second feature points on the same feature point setting line as the first feature point as the first feature point It is desirable to have a method of setting a plurality of feature point pairs by executing each point.
The predetermined interval for setting the feature points may be an interval between two feature points determined by a predetermined method, and is not necessarily a constant interval, but is preferably a constant interval in principle.

  In the pattern feature point / pair setting step, it is desirable to set the pattern base pair so that the distance between the first feature point and the second feature point of the pattern base pair satisfies the condition within the first predetermined distance. .

A distance corresponding to the maximum diameter of the original pattern can be adopted as the first predetermined distance.

  Furthermore, in the first search step, as a method of expanding the pattern local coordinate point group, the distance between the pattern base pair and the point on the searched image corresponding to each of the pattern local coordinate point groups is a second predetermined distance. It is desirable to expand the pattern local coordinate point group by performing coordinate calculation on the pattern local coordinate system for the feature points satisfying the conditions within the range.

In the first search step, as a method of expanding the pattern local coordinate point group, a distance between a predetermined position representing the pattern base pair and a point on the searched image corresponding to each of the pattern local coordinate point groups is a second value. A pattern local coordinate point group can also be developed by performing coordinate calculation on the pattern local coordinate system for feature points satisfying a condition within a predetermined distance.
Here, the predetermined position representing the pattern base pair is preferably the middle point of the pattern base pair for convenience of processing.

  As the second predetermined distance, a distance in consideration of the length of the pattern base pair can be adopted as a distance corresponding to the maximum diameter of the original pattern.

In the second search step, a predetermined method is
Each of the obtained target poses is set as a reference target pose, and the distance between the reference target pose and the other target pose is obtained for each reference target pose, and then, for each reference target pose, The value of a given function that is dependent on the distance between poses and decreases with increasing distance between poses, with respect to the distance between each other pose within which the difference is within a given range And the addition value of each calculation result is used as the score of the reference target pose, and the reference target pose whose score is equal to or higher than a predetermined value is recognized as the core of a small set of target poses (hereinafter also referred to as a cluster). A procedure for performing all the reference target poses, and selecting a reference target pose with the highest score among the certified reference target poses as the most accurate target pose; Rukoto is desirable.

Within a predetermined range,
Reference target pose (txo, tyo, θo, so)
Other target poses (tx, ty, θ, s)
When
The absolute value of (tx-txo) is less than (predetermined maximum allowable range txmax) (ty-tyo) is less than (predetermined maximum allowable range tymax) (θ-θo) is (predetermined maximum allowable range θmax) The absolute value of less than (s-so) can be a range of (tx, ty, θ, s) that satisfies less than (predetermined maximum allowable range somax).

A given function,
Reference target pose (txo, tyo, θo, so)
Other target poses (tx, ty, θ, s)
The distance between the reference target pose and another target pose is d ((tx, ty, θ, s), (txo, tyo, θo, so)).
When
1 / {0.5 + d ((tx, ty, θ, s), (txo, tyo, θo, so))}
It can be.

In addition, the original pattern local coordinate point group and the pattern local coordinate point group may be a plurality of feature points constituting a feature point pair selected on the contour of each of the original pattern and the pattern, or a plurality of feature points selected on the contour. Any of the combinations of the feature points constituting the pair of feature points and other points selected by a predetermined method on the contour can be used.
With respect to the predetermined method for selecting the other points, the distance between the points, either the straight line distance or the contour line distance, is measured on the contour between the first feature points adjacent to the first feature point. There is a method of selecting as a point.

According to the present invention, when the original pattern is an arbitrary shape, when various figures or characters are complicated by being combined or combined, or when the size is large, the pattern on the searched image is compared to the original pattern. Even in the case of arbitrary posture, similarity or the same size, partially hidden or large size, it is now possible to quickly search for the pattern to be searched for in the search image in a short processing time. .
Normally, in an actual image search, it is sufficient to search for a pattern on the searched image that matches the original pattern under a predetermined search condition in the first search operation, and a searched image that seems to roughly match the original pattern. First search for the above pattern as the first work (this search work is often referred to as an ancestor search), and the second work is to check whether or not it exactly matches the found pattern Depending on the situation, there are both cases of performing an operation for obtaining an accurate posture of the searched pattern (this operation is often referred to as a fine search), which corresponds to the first ancestor search in the former case or the latter case By performing the work to be performed by the means according to the present invention, the search time can be greatly shortened.

  The best mode for carrying out the present invention will be described together with examples.

  The high-speed image search method according to the first embodiment of the present invention basically executes a registration process in which an original graphic pattern (original pattern) to be searched is registered in advance using a geometric hashing technique prior to search. By executing a search process for searching for a graphic pattern (target pattern) to be searched in the search image using a geometric hashing technique, the search pattern has the same or similar relationship as the original pattern to be searched for. This is a method for finding one or more graphic patterns at high speed, and is a method according to first to fourth means for solving a problem.

The registration process will be described below based on a flowchart showing an example of the registration process shown in FIG.
<Step S101>
A feature point consisting of a set of two feature points (the first feature point is Pi1 and the second feature point is Pi2) on the contour of the original pattern which is the original figure pattern of the figure pattern to be searched in the searched image A plurality of pairs (i = 1, 2,..., N) are extracted and set.
The feature point extraction / setting method is determined in advance over the entire area of the original pattern outline obtained by appropriate processing such as preprocessing, edge extraction, thinning, and tracking to clarify the outline of the original pattern. First feature points Pi1 are set at intervals. Next, a normal line of the outline at each first feature point Pi1 (hereinafter also referred to as a feature point setting line).
And a point other than Pi1 at which the normal intersects the contour is extracted as the second feature point Pi2. As a matter of course, a plurality of second feature points Pi2 may be extracted for the same first feature point Pi1.
Here, the “predetermined interval” in the setting of the first feature point Pi1 adopts a method of determining the linear distance between two adjacent first feature points to be a predetermined number of pixels. . As another method, “the number of pixels along the contour (the contour passes)” can be adopted instead of the “linear distance” in the above method. As the number of pixels for determining the “predetermined interval”, an optimal number of pixels is selected depending on the complexity and size of the shape of the original pattern, the accuracy of the image search, and the time limit required for the search. In general, the interval need not be a fixed interval, and may be an interval (predetermined interval) determined according to a predetermined function including a constant.
<Step S102>
An arbitrary new feature point pair (Pi1, Pi2) is selected. The feature point pair (Pi1, Pi2) is also called an original pattern base pair.
<Step S103>
A vector consisting of the selected feature point pair (original pattern basis vector starting from the first feature point Pi1 and ending at the second feature point Pi2) is normalized, and a local coordinate system (on the normalized unit vector ( The local coordinate system is determined so that the xi axis of the xiii coordinate system is present and the origin of the local coordinate system coincides with the midpoint of the start and end points of the unit vector. Also, normalization corresponding to the normalization of feature point pairs is performed, these local coordinates are calculated, and the original pattern is developed on the local coordinate system. Whether the calculation for the expansion is performed only on the point group including only the first feature point Pi1 or whether the calculation is performed on the point group including both the first feature point Pi1 and the second feature point Pi2. The pattern may be selected based on a comparison balance between the accuracy when the pattern is developed on the local coordinate system and the calculation time reduction for the high-speed search which is the object of the present invention.
<Step S104>
In order to convert the normalized original pattern developed on the local coordinate system onto the reference coordinate system having the same coordinate scale as the local coordinates, the feature points developed on the local coordinate system (generally, The coordinates (local coordinates) of the above point group) are converted into coordinates on the reference coordinate system.
<Step S105>
A predetermined area of the reference coordinate system is subdivided according to a predetermined reference, and each calculated feature point (or the above-described point group) on a hash table having information cells corresponding to the subdivided small areas exists. The identification information of the feature point pair selected at that time is registered in the information cell corresponding to the small area. As identification information, it is convenient to use the feature point number i. The reference for subdividing the reference coordinate system is desirably finer than the set pitch of the feature point Pi1.
In step 6, it is determined whether or not all the n feature point pairs set for registration have been selected. If all the selections have not been made, the process jumps to step S102 and the subsequent steps are executed again. If all are completed, the registration process ends.

Next, the search process will be described based on a flowchart showing an example of the search process shown in FIG.
<Step S201>
A plurality of feature point pairs (j = 1, 2) consisting of a set of two feature points (the first feature point is Oj1 and the second feature point is Oj2) on the contour of the pattern on the searched image. , ..., m) Extraction and setting. The method is the same as the method of setting a plurality of feature point pairs including a set of two feature points (first feature point is Pi1 and second feature point is Pi2) in step S101 of FIG. .
Here, in general, the image to be searched is a complex figure that has a different aspect from the original pattern, such as various figure patterns mixed and partly hidden and overlapped. Typically, the number of feature point pairs made is greater than in the original pattern.
On the other hand, as means for speeding up the image search by further reducing the number of feature point pairs, a means for limiting the distance between feature point pairs to be selected to be equal to or less than a first predetermined distance as will be described later.
<Step S202>
An arbitrary new feature point pair (Oj1, Oj2) on the contour of the pattern on the searched image is selected, and all data on the reference coordinate system is deleted for the operation of step S203 for the new feature point pair ( Reset. The feature point pair (Oj1, Oj2) is also called a pattern base pair on the searched image.
<Step S203>
A vector consisting of the selected feature point pair (a pattern base vector on the image to be searched having the feature point Oj1 as the start point and the feature point Oj2 as the end point) is normalized, and the local coordinate system (xjyj coordinates is set on the normalized unit vector. System) xj axis exists, the local coordinate system is defined so that the origin of the local coordinate system coincides with the midpoint of the start and end points of the unit vector, and the pattern on the searched image is defined on the local coordinate system based on this Therefore, the normalization corresponding to the normalization of the feature point pair is performed on the feature points other than the feature point pair, and the local coordinates are calculated. Then, in order to convert the normalized target pattern developed on the local coordinate system onto the reference coordinate system, the coordinates of the feature points (local coordinates) developed on the local coordinate system are converted into the reference coordinate system. Convert to the above coordinates.
The reference coordinate system is generally not limited with respect to the coordinate system, but is preferably the coordinate system of the same system as the local coordinate system from a practical aspect. It is further preferable that the coordinate axis scale system is the same as the local coordinate system.
When performing the above “normalization corresponding to normalization of the feature point pair for feature points other than the feature point pair and calculating their local coordinates”, the number of feature points to be calculated is reduced. As a means for speeding up the image search, the range of feature points to be subjected to local coordinate calculation is limited to a range not more than a second predetermined distance having a predetermined relationship from a predetermined position in the local coordinate system, as will be described later. Take measures.
<Step S204>
If the identification information of the feature point pair of the original pattern exists in the information cell of the hash table corresponding to the coordinate of each converted feature point, the vote corresponding to the feature point pair of the original pattern indicated by each of all existing identification information Vote in a box (hereinafter also referred to as a feature point pair ballot box).
<Step S205>
The feature point pair ballot box having the highest number of votes is selected, and when the number of votes is higher than a predetermined threshold, the feature point pair of the original pattern corresponding to this feature point pair ballot box is selected. A position / angle (posture) / scale (similarity) relationship (hereinafter also referred to as a target pose) between feature point pairs of the pattern on the search image is calculated. An example of the definition of the target pose will be described later.
<Step S206>
It is determined whether or not all the executions after step S202 have been completed for each selection of the set m feature point pairs, and if not all, the process jumps to step S202 and executes the subsequent steps again. If completed, the process proceeds to step S207.
<Step S207>
By selecting the target pose with the highest accuracy in each of the one or more similar group distributions from the target pose group having one or more similar group distributions obtained in the above steps by a clustering method to be described later, One or more most similar patterns are selected by selecting the pattern on the searched image that is the target of calculation of the target pose as the pattern most similar to the original pattern.

  The method of selecting the target pose with the highest accuracy by the definition of the target pose and the clustering method will be described below.

An example of the definition of the target pose in the first embodiment is shown below.
As described in step S205, the feature point pair ballot box having the highest number of votes is selected, and when the number of votes is higher than a predetermined threshold, the feature point pair of the original pattern corresponding to this feature point pair ballot box And “the original feature pattern corresponding to the feature point pair ballot box” as the pattern shown in FIG. 3A, and the feature point pair at that time is ( P1, P2). The pattern on the image to be searched that is selected at that time is the pattern shown in FIG. 3B, and the feature point pair at that time is (O1, O2).
Assume an xy coordinate system having parallel coordinate axes x and y on two orthogonal sides of the screen in common with the original image and the searched image on the rectangular quadrilateral screen.
The scale of the searched image with respect to the original image in which the original pattern exists is s, the relative angle of the pattern on the searched image with respect to the original pattern is θ, and the reference point of the original pattern (in FIG. 3, as an example, Pr is the reference point) If the relative position of the reference point (for example, Or is the reference point) of the pattern on the image to be searched corresponding to a certain) is tx in the x direction and ty in the y direction,
The target pose is defined as equations (1) to (4) as a set (s, θ, tx, ty) of s, θ, tx, and ty.

Next, how to determine the target pose with the highest accuracy from the target pose group will be described.

Here, the means for limiting the distance between the feature point pairs to be selected to be equal to or less than the first predetermined distance described in step S201 will be described. Hereinafter, this restriction is also referred to as a first restriction.
When searching for a pattern having the same or similar relationship to the original pattern, the feature points of the pattern that may have the same or similar relationship in the search target image when selecting the feature point pair of the pattern on the searched image The distance between the pairs is at most within the distance on the searched image corresponding to the distance between the two points that are farthest apart on the contour of the original pattern (for convenience, this is also referred to as the maximum diameter of the original pattern). Therefore, as the distance between the first feature point Oj1 and the second feature point Oj2, the distance on the searched image corresponding to the maximum diameter of the original pattern is adopted as an example of the first predetermined distance. Choose to be within distance.
The calculation of the distance on the searched image corresponding to the maximum diameter of the original pattern is obtained as follows.
When the scale relationship between “search target image” and “original image” is clear in advance, the maximum diameter of the original pattern is multiplied by this scale relationship. The maximum diameter of the original pattern is actually determined by visually deciding two points on the contour of the original pattern that are close to the maximum diameter, measuring the distance, and applying a safety factor that does not fall below the true maximum diameter. It is also possible to take a method of using the maximum diameter by considering the original pattern. Further, regarding the determination of two points on the original pattern contour close to the maximum diameter, it is also possible to determine the maximum distance between the feature point pairs of the original pattern. In that case, the safety factor is also changed according to the current situation. Of course, other simple calculation methods can be used.
The first limitation described above has an effect that the calculation time increases by selecting an unnecessary feature point pair, and the pattern search time on the searched image can be prevented from becoming unnecessarily long.

Here, the means for limiting the range of the feature points to be subjected to local coordinate calculation described in step S203 to a range equal to or less than a distance having a second predetermined relationship from a predetermined position in the local coordinate system will be described. Hereinafter, this restriction is also referred to as a second restriction.
Normalize each feature point pair (each pattern base pair) of the pattern on the searched image and place it in the local coordinate system, perform normalization corresponding to normalization of the pattern base pair for feature points other than the pattern base pair, When a pattern normalized by calculating these local coordinates is developed on the local coordinate system, a restriction is added so that unnecessary feature points are not calculated unnecessarily.
The means uses the fact that all the feature points to be calculated are within the maximum diameter from the pattern basis vector that represents the pattern basis pair as a vector, no matter how far away, and the pattern basis vector is within the maximum diameter. The distance from the feature point to the feature point is calculated, and local coordinates of the feature point are calculated only when the distance is equal to or smaller than the distance on the searched image corresponding to the maximum diameter of the original pattern.
In this case, in reality, strictly calculating the distances from all the points on the pattern basis vector to the feature points is time consuming, and may hinder the speeding up of the object of the present invention. Absent. Therefore, as one of the simplified methods, the first restriction restricts the maximum distance between feature point pairs of the pattern on the searched image within the distance on the searched image corresponding to the maximum diameter of the original pattern. In view of this, the distance from the midpoint of the pattern basis vector (corresponding to the origin of the local coordinates) to each feature point is the distance on the searched image corresponding to the maximum diameter of the original pattern. As an example of the distance having the second predetermined relationship, the distance obtained by subtracting one half of the length of the pattern basis vector from the distance multiplied by only the feature points within the distance having the second predetermined relationship Is to perform local coordinate calculation.
In the above calculation of the distance, the distance from the midpoint of the pattern basis vector to each feature point is calculated, but it is not necessarily the distance from the center of the pattern basis vector, and is selected by a predetermined method. It may be a distance from another point (predetermined position) representing the position of. It is natural that the distance having the second predetermined relationship depending on the predetermined position can be appropriately changed according to the concept when the predetermined position is the midpoint of the pattern basis vector. Naturally, other simple calculation methods are easily conceivable, and any method can be used.

  In S101, a method has been described in which a normal of the contour at each first feature point Pi1 is obtained, and a point other than Pi1 where the normal intersects the contour is extracted as the second feature point Pi2. There is no limitation, and a method may be used in which a straight line having a predetermined angle with respect to the normal is obtained as a feature point setting line, and the second intersection point of this and the contour is used as the second feature point Pi2. In general, the direction of the normal is uniquely determined from the concentration distribution when the contour is obtained, and the direction of a straight line having a predetermined angle with respect to the direction of the normal thus determined is also unique. This is because the feature point pair (Pi1, Pi2) can be set by a predetermined method common to both the original pattern and the pattern on the searched image.

In step S103 or step S203, the local coordinates of feature points constituting a plurality of feature point pairs on the contour of the pattern are calculated when the pattern is developed in the local coordinate system. However, the present invention is not limited to this. A combination of feature points constituting a point pair and other points selected on the contour by a predetermined method, or a plurality of points other than the feature points constituting the feature point pair selected on the contour Local coordinates can also be calculated for the group.
As one method for calculating local coordinates for the point group of the above “combination of feature points constituting a feature point pair and points selected on other contours”, feature points are set in advance and There is a method in which a predetermined integer number of feature points to be used for the point pair are selected and the pattern expansion is calculated for all the feature point groups. There is also a method of selecting a point on the contour at a predetermined distance from one feature point as another point between two feature points.

In step S103 or step S203, the origin of the local coordinate system is set at the midpoint of the start point and end point of the feature point pair vector, but the present invention is not limited to this. Theoretically, the origin only needs to have a predetermined relationship with the feature point pair vector, the start point of the normalized unit vector coincides with the origin of the local coordinate system (xy coordinates), and the end point is on the x axis. The local coordinate system can be determined so as to coincide with one unit coordinate. Also, the x-axis of the local coordinate system has been matched with the feature point pair vector, but theoretically it is not necessary to match and there is no problem if they are in a predetermined relationship with each other. However, the relationship of the present embodiment has an advantage that various calculations are relatively easy, such as simplification of the distance calculation related to the second restriction.

In the first embodiment, a normal line of the contour at each first feature point Pi1 is obtained, and a point other than Pi1 at which the normal line intersects the contour is extracted as the second feature point Pi2, or the first feature point Although it has been described that a straight line that passes through Pi1 and forms a predetermined angle with respect to the normal line is obtained as a feature point setting line, and a second intersection point of this and the second contour point Pi2 is used as a second feature point Pi2. It is not necessary to limit to these methods, and a pair of feature points Pi1 and Pi2 may be extracted on the contour by any method different from these methods. However, in that case, the shorter the process of extracting the two feature points, the better in terms of processing time and accuracy.

  In the first embodiment, it has been described that the clustering method is selected as the method for selecting the target pose with the highest accuracy in step S207, but it is not necessarily limited to the clustering method. Other methods may be used.

In this embodiment, an image including an original pattern with a large number of feature points for expressing a shape (hereinafter also referred to as an original image) is divided into a plurality of small images (hereinafter also referred to as divided original images). Arrangement information of each divided original image boundary frame (position, inclination, scale information, hereinafter also referred to as boundary frame arrangement information) with respect to a reference in the defined original image and an original pattern divided for each divided original image boundary frame ( Hereinafter, arrangement information (position, inclination, scale information, hereinafter also referred to as divided original pattern arrangement information) is registered and is the same as or similar to each divided original pattern in the searched image. A pattern that has a high probability of occurrence (hereinafter, also referred to as a divided pattern) is searched, and the search is performed using the boundary frame arrangement information and the divided original pattern arrangement information so that the divided original pattern is superimposed on the searched divided pattern. On the image Verify whether or not a divided pattern other than the searched divided pattern exists in the searched image as if it overlapped with the original pattern when the original pattern (or original image) is overlaid. The present invention relates to a method for quickly searching for a pattern that is the same as or similar to the original pattern, and relates to a fifth means for solving the problem. The contents are shown below.

When the original pattern is complex or large and the number of feature points for expressing the shape is large, searching for a pattern that is the same or similar to the original pattern in the searched image is time-consuming. It takes too much and often becomes a problem. Incidentally, the calculation amount is on the order of the cube of the number of feature points.
Embodiment 2 is an embodiment that realizes a method for avoiding this problem, and relates to a method of performing image search at high speed by dividing an original image and reducing the number of feature points provided for calculation as a result. Is. FIG. 5A is an example of an original image composed of five character strings, and is an example of an original pattern suitable for applying the method of this embodiment. FIG. 5B is an example in which the original image in FIG. 5A is divided into five corresponding to five characters.
Hereinafter, a method of registering an original pattern and searching for a target pattern will be described with reference to a flowchart illustrating the method of the present embodiment shown in FIG.
<Step S301>
An original image including the original pattern is divided into a plurality of divided original images.
Then, boundary frame arrangement information that is the relationship between the mutual position, angle, and scale of each divided original image boundary frame, and the divided original pattern that is the relationship between the mutual arrangement position, angle, and scale of each divided original image boundary frame and the divided original pattern Arrangement information is stored. Although there are various ways of border frame arrangement information, it is one of preferable methods to digitize the relationship between an arbitrarily determined reference of the original image and each divided image border frame.
<Step S302>
For each divided original pattern, replace the “divided original pattern” with “original pattern”, execute the above-described original pattern registration method based on FIG. 1 to register each divided original pattern, and FIG. The pattern search method on the searched image described above is executed based on the image, and one or more patterns in the searched image that are the same or similar to each divided original pattern ("one or more" is "detected" Search for all possible patterns.
<Step S303>
The scale of the original image is matched with the scale of the searched image, and the divided original pattern corresponding to the pattern is superimposed on the newly selected pattern in the searched pattern so as to overlap the original pattern or the original image. Overlapping search images.
The method of superimposing the divided original pattern corresponding to the pattern on the newly selected pattern among the searched patterns is a statistical method such as superimposing the two normalized correlations. However, in general, the method is not necessarily limited to a statistical method, and other generally known superposition methods can be adopted.
<Step S304>
Next, a comparison is made between the original pattern part other than the divided original pattern superimposed on the search target image and the search target image, and a vote is given for the matching pixels, and a vote is given for the non-matching pixels. This method is performed for each pixel, and the comparison result is digitized according to a predetermined standard, and whether or not a pattern that matches the original pattern exists in the searched image in the superimposed state is determined by the numeric value. If it exists, the pattern is stored in the match list. If it does not exist, the process proceeds to step S305.
The above comparison method is performed in order to investigate whether or not a pattern corresponding to the original pattern exists in a predetermined width around the outline of the original pattern portion other than the divided original patterns superimposed in step S303. A software window for selectively inspecting the state of pixels in a predetermined pixel range with respect to the center is prepared, and the pattern on the searched image is moved while moving the center of the window along the outline of the divided original pattern. If there is a ratio of “the number of votes” to “the number of points on the examined contour” is equal to or greater than a predetermined value, it is determined that there is a pattern that matches the original pattern. By adjusting the size of the window, it is possible to absorb errors in searching for the target pattern, errors in overlaying, and the like.
The comparison standard / system is not necessarily limited to this standard / system, and other systems may be used.
<Step S305>
For each divided original pattern, check whether step S303 and step S304 are executed for all patterns searched in step S302, and if not, select a new searched pattern The process proceeds to step S303. If it has been executed, it is determined that there is a high possibility that all patterns stored in the match list are the same as or similar to the original pattern before division, and this high-speed image search is terminated.

  Each of the above two embodiments alone can provide an image search performance that is more advanced than that of the prior art. However, by combining the two embodiments, a further advanced search performance can be obtained.

Flowchart showing an example of a registration process in the first embodiment A flowchart showing an example of a search process in the first embodiment. It is an example which shows arrangement | positioning on the screen of the original pattern of object pose calculation, and the pattern on a searched image. (A) is for the original pattern corresponding to the feature point pair ballot box, and (b) is for the pattern on the selected image to be searched. Flowchart showing an example of the method of the second embodiment It is a figure which shows an example of the original image suitable for the division | segmentation of an original image, and the original image after a division | segmentation. (A) is an original image composed of five character strings. (B) is an example in which the original image (a) is divided into five corresponding to five characters.

Explanation of symbols

P1 The starting point of the original pattern basis vector corresponding to the feature point pair voting box P2 The end point of the original pattern basis vector corresponding to the feature point pair voting box Pr The reference point of the original pattern O1 The pattern basis vector on the selected image to be searched Start point O2 End point of the pattern basis vector on the selected searched image Or Reference point of the pattern on the searched image s Scale of the searched image with respect to the original image θ Relative angle of the pattern on the searched image with respect to the original pattern tx Relative position (x direction) of the reference point of the pattern on the searched image corresponding to the reference point of the original pattern
ty Relative position (y direction) of the reference point of the pattern on the searched image corresponding to the reference point of the original pattern

Claims (10)

A plurality of first feature points are set on the contour of the original pattern at a predetermined fixed interval or at a predetermined interval determined so as to vary according to the form of the contour according to a predetermined function, and thereafter One or more straight lines that pass through one first feature point and have a predetermined angle with respect to the direction of the normal of the outline at the first feature point (hereinafter also referred to as feature point setting line) intersect the outline The second feature point is set as a second feature point, and the procedure for setting the first feature point and each of the second feature points as a feature point pair is executed for each of the first feature points. the first feature points and feature point pair consisting of the second feature point of the first feature points (hereinafter, the original pattern base pairs referred to as) a plurality of sets setting to the original pattern feature point pair setting step and the original pattern base pairs Vector with start point and second feature point as end point (Hereinafter, also referred to as an original pattern basis vector), and a point group representing the shape of the original pattern on an original pattern local coordinate system arranged in a predetermined relationship with the normalized original pattern basis vector. The original pattern local coordinate point group is normalized and expanded as the original pattern local coordinate point group while maintaining the relative positional relationship with the original pattern base vector, and the original pattern local coordinate point group is subjected to coordinate conversion on the reference coordinate system. A predetermined area of the reference coordinate system is divided into small areas, and any of the original pattern reference coordinate points on the hash table including information cells corresponding to the divided small areas exists. Original pattern feature for executing a procedure for writing the identification information of the original pattern base pair in the information cell corresponding to a small region for a plurality of necessary sets of the original pattern base pairs And the original pattern registration process of registering the original pattern from the recording step,
A plurality of first feature points are set on the contour of the pattern on the image to be searched at a predetermined fixed interval or at a predetermined interval determined so as to vary according to the form of the contour according to a predetermined function. Thereafter, a straight line (hereinafter also referred to as a feature point setting line) passing through one first feature point and having a predetermined angle with respect to the direction of the normal line of the contour at the first feature point intersects the contour. One or more second points to be set as second feature points, and a procedure for setting each of the first feature points and each of the second feature points as a feature point pair is set for each of the first feature points. And a pattern feature point / pair setting step for setting a feature point pair (hereinafter also referred to as a pattern base pair) composed of the first feature point and the second feature point, and the first feature point of the pattern base pair as a starting point. And a vector with the second feature point as the end point (Hereinafter also referred to as a pattern basis vector), and the predetermined relationship that is the same as the relationship between the original pattern basis vector and the original pattern local coordinate system in the registration process with respect to the normalized pattern basis vector And a point group representing the shape of the pattern on the pattern local coordinate system arranged while maintaining the pattern is normalized and expanded as a pattern local coordinate point group while maintaining the relative positional relationship with the pattern base vector, and the pattern The local coordinate point group is coordinate-converted on the reference coordinate system to be a pattern reference coordinate point group, and when any of the pattern reference coordinate point groups exists in the small area, the small area corresponds to the hash table corresponding to the small area. A ballot box (hereinafter referred to as a base pair ballot box) provided corresponding to each identification information of the original pattern base pair described in the information cell. The original pattern base pair corresponding to the voting box that has obtained the highest number of votes and the number of votes higher than a predetermined threshold and the pattern at that time. First, a plurality of target poses are obtained by executing a series of procedures for calculating target poses, which are specifications representing similarity, posture, and positional relationship with the base pairs, on a plurality of necessary pattern base pairs. One or more most likely target poses are selected by a predetermined method from the distribution of the plurality of target poses obtained in one search step, and the pattern basis pair that is the target of calculation of the selected target poses is used as a basis vector. A second search step of selecting a pattern developed on a reference coordinate system as a pattern having the same or high possibility of similarity to the original pattern. A high-speed image search method, wherein a pattern on a search image is searched by going through a pattern search process for selecting a pattern on one or more search images that are likely to be the same as or similar to a pattern. The high-speed image search method according to claim 1, wherein the interval is a linear distance between adjacent first feature points or a distance along the contour.   In the pattern feature point / pair setting step, the pattern base pair is set so that a distance between the first feature point and the second feature point of the pattern base pair satisfies a condition within a first predetermined distance. The high-speed image search method according to claim 1 or 2.   The high-speed image search method according to claim 3, wherein the first predetermined distance is a distance corresponding to a maximum diameter of the original pattern.   In the first search step, the method of expanding the pattern local coordinate point group is such that a distance between the pattern base pair and a point on the searched image corresponding to each of the pattern local coordinate point groups is a second predetermined value. 5. The pattern local coordinate point group is developed by performing coordinate calculation on the pattern local coordinate system for a feature point satisfying a condition within a distance of 5. The high-speed image search method described.   In the first search step, a method of developing the pattern local coordinate point group is a distance between a predetermined position representing the pattern base pair and a point on the searched image corresponding to each of the pattern local coordinate point groups. 2. The pattern local coordinate point group is developed by performing coordinate calculation on the pattern local coordinate system for a feature point satisfying a condition that is within a second predetermined distance. 6. The high-speed image search method according to any one of 5 above.   7. The high-speed image search method according to claim 5, wherein the second predetermined distance is a distance in consideration of a length of the pattern base pair with a distance corresponding to a maximum diameter of the original pattern. .   In the second search step, the predetermined method uses each of the obtained target poses as a reference target pose, obtains a pose distance between the reference target pose and another target pose for each reference target pose, For each reference target pose, the distance between poses depends on the distance between poses and the distance between poses with respect to each other target pose whose difference from the reference target pose is within a predetermined range. The value of a predetermined function having a decreasing relationship with respect to the increase in the number is calculated, and the added value of each calculation result is used as the score of the reference target pose. A procedure for recognizing as a core of a small set (hereinafter also referred to as a cluster) of all the reference target poses is performed, and the base having the highest score among the determined reference target poses is executed. High-speed image search method according to claim 1 to 7, characterized in that a method of selecting a subject poses as the most accurate likely target pose. Within the predetermined range,
The reference target pose is (txo, tyo, θo, so)
The other target pose is (tx, ty, θ, s)
When
The absolute value of (tx-txo) is less than (predetermined maximum allowable range txmax) (ty-tyo) is less than (predetermined maximum allowable range tymax) (θ-θo) is (predetermined maximum allowable range θmax) 9. The high-speed image search method according to claim 8, wherein an absolute value of less than (s-so) is in a range of (tx, ty, θ, s) satisfying less than (predetermined maximum allowable range somax). The predetermined function is
The reference target pose is (txo, tyo, θo, so)
The other target pose is (tx, ty, θ, s)
The distance between the reference target pose and the other target pose is defined as d ((tx, ty, θ, s), (txo, tyo, θo, so)).
When
1 / {0.5 + d ((tx, ty, θ, s), (txo, tyo, θo, so))}
The high-speed image search method according to claim 8, wherein:

Images