JP5970512B2 - Information processing method, information processing apparatus, and program - Google Patents

Description

  The present invention relates to an information processing method, an information processing apparatus, and a program.

  A video processing technique for searching for a similar image from a plurality of candidate images prepared in advance for an image obtained by camera photography (hereinafter referred to as a target image) has been researched and developed.

  In general, image feature amounts are described in advance for candidate images, image feature amounts are calculated for target images at the time of retrieval, and similarity is measured by measuring the similarity between these image feature amounts. A candidate image is determined. In the calculation of the image feature amount, a characteristic portion in the image is detected as a point of interest, and a change amount of the image signal in the vicinity of the point of interest is obtained as a feature amount descriptor. At this time, the feature descriptor is composed of a plurality of numerical values, and can be handled as a vector quantity in the feature quantity space.

  As a conventional method for obtaining a feature descriptor, a digital filter in an image is applied to detect a point satisfying a predetermined condition as a point of interest, and another digital filter is applied in the vicinity of the point of interest to determine the direction of the point of interest. There is a method in which a group of digital filter responses in a coordinate space with the point of interest as the origin and the direction of the point of interest as an axis is calculated as a feature descriptor.

  In addition, as a conventional method related to interest point detection, an x-axis and a y-axis in an image are based on characteristics required for the interest point (equivalent points are selected equally in each of the target image and the candidate image). In addition, there is a method for detecting a point of interest from a three-dimensional space composed of scale dimensions, for example, by a minimum check of Hessian determinant. Hereinafter, such a method for detecting a point of interest will be described as a point of interest detection step.

  As a conventional method for determining the direction of the point of interest, there is a method for determining the direction of the point of interest based on a plurality of tiles provided in a region surrounding the point of interest. A plurality of tiles include a plurality of pixels. Specifically, in this method, digital filters in the x and y directions are applied to a plurality of tiles to generate a contrast-related response (that is, a vector having an x direction component and a y direction component is calculated from each tile). To do. Then, the vector calculated from each tile is synthesized for each orientation determination window. The direction of the point of interest is determined by selecting the maximum vector from the synthesized vectors. In addition, the tile mentioned here is a concept corresponding to an image (partial region) within a direction calculation range including a sample point described below.

  As an application of the digital filter, there is a method in which box filters in the x direction and the y direction are applied to corresponding tiles on the integral image (see Patent Document 1).

Japanese Patent No. 498559

  In the conventional method, after the direction of the interest point is determined, the interest point may be calculated around the periphery, and the direction of the interest point may be similar. At this time, there are cases where a plurality of points of interest having similar feature descriptors are selected together (that is, a plurality of points of interest that cannot be distinguished from each other are selected). In such a case, for example, a plurality of points of interest in the target image are similar to a single point of interest in the candidate image, and an image corresponding to the target image cannot be accurately searched from the plurality of candidate images. Problems can occur.

  Therefore, the present invention has been made in view of the above-described problems of the prior art, and provides an information processing method, an information processing apparatus, and a program that can accurately detect an image corresponding to a target image.

(1) One aspect of the present invention is a sample point direction calculation that calculates a direction that represents a feature related to each of a plurality of sample points within a predetermined range including a target point of interest selected from one or more points of interest detected from an image. And a cluster detection step of detecting a cluster of the sample points for each of the one or more points of interest, based on a direction representing a characteristic regarding each of the plurality of sample points calculated by the sample point direction calculation step; , Determining the points of interest to be excluded from the one or more points of interest based on the detection result of the cluster of the sample points for each of the points of interest by the cluster detection step, and the exclusion according to the determination result A point of interest exclusion step of excluding the target point of interest from the one or more points of interest.

( 2 ) According to another aspect of the present invention, in the point of interest exclusion step, the point of interest in which one or more of the clusters are detected by the cluster detection step is the point of interest as the exclusion target. The information processing method according to ( 1 ), characterized in that a determination is made.

( 3 ) According to another aspect of the present invention, in the point-of-interest exclusion step, the two or more clusters satisfy a predetermined condition with respect to the point of interest in which two or more clusters are detected by the cluster detection step. The information processing method according to ( 1 ) or ( 2 ), wherein if satisfied, the point of interest is determined as the exclusion target.

( 4 ) Further, according to another aspect of the present invention, a sample for calculating a direction representing a feature regarding each of a plurality of sample points within a predetermined range including a target point of interest selected from one or more points of interest detected from an image. A cluster that detects a cluster of the sample points for each of the one or more points of interest based on a point direction calculation unit and a direction that represents a feature relating to each of the plurality of sample points calculated by the sample point direction calculation unit a detection unit determines the interest points to be excluded among the pre SL 1 or more of the point of interest based on a detection result of the cluster of the sample points for each of the interest points by the cluster detection unit, the determination result Accordingly, the information processing apparatus includes: a point of interest exclusion unit that excludes the point of interest to be excluded from the one or more points of interest.

( 5 ) According to another aspect of the present invention, a direction in which a characteristic regarding each of a plurality of sample points within a predetermined range including a target point of interest selected from one or more points of interest detected from an image is displayed on a computer. A sample point direction calculating step; and a sample point cluster for each of the one or more points of interest based on a direction representing a characteristic regarding each of the plurality of sample points calculated by the sample point direction calculating step. determining a cluster detection step of detecting, the interest points to be excluded among the pre SL 1 or more of the point of interest on the basis of the cluster detection the clusters of the sample points for each interest point detection result of the step, the An interest point exclusion step of excluding the interest points to be excluded from the one or more interest points according to a determination result. It is because of the program.

  According to one embodiment of the present invention, it is possible to provide an information processing method, an information processing apparatus, and a program that describe a point of interest indicating a feature of an image that can accurately detect an image corresponding to a target image.

It is a figure showing an example of functional composition of information processor 1 concerning this embodiment. 12 is a flowchart illustrating an example of a flow of processing in which a control unit 16 detects an image similar to a target image from a candidate image 121. It is a figure which illustrates a part of target image including a part with high possibility that noise is detected. Wavelet response (dx, dy) calculated for each of the direction calculation ranges BF related to a plurality of sample points, and the direction of the sample points related to the direction calculation range BF calculated based on the wavelet responses (dx, dy) FIG.

<Overview>
First, an outline of the information processing apparatus 1 according to the embodiment described below will be described, and then a more detailed embodiment will be described. The information processing apparatus 1 according to the present embodiment determines the characteristics of a digital image based on characteristic points (interest points) among a plurality of points (for example, pixels) constituting the digital image. As an example, the information processing apparatus 1 uses image characteristics of a digital image (hereinafter referred to as a target image) of a plurality of digital images and other digital images (hereinafter referred to as candidate images). Are compared between the digital images, an image corresponding to the target image is extracted from the plurality of candidate images.

  In the following, for convenience of explanation, when the target image is displayed on the display unit, the thing that is displayed on the display unit is referred to as the thing on the target image, and is a part where the thing is not displayed and is a predetermined part. A portion where a single color (solid color) is projected within the allowable range will be referred to as a plain background. A plain background is an intermediate color (a composite color in which multiple colors, for example, white and black are arranged in a fine checkered pattern, etc.) instead of a portion where a single color is projected within a predetermined allowable range. It may be a portion in which a single gray color illusion is projected).

  In this specific example, the information processing apparatus 1 detects a point in the digital image as a point of interest indicating the feature of the image, and corresponds to the target image based on the direction of the image at the detected point of interest (interesting point direction). Images to be extracted from a plurality of candidate images. The selection of the interest point is performed based on various conditions, for example, by the interest point detection step. Hereinafter, with respect to a certain target image and a plurality of candidate images, the image direction at the point of interest selected based on some condition, that is, an image corresponding to the target image based on the direction of the point of interest is extracted from the plurality of candidate images. In the processing to be performed, processing for removing a point of interest that is likely to cause noise will be described. The information processing apparatus 1 can extract the candidate image corresponding to the target image with high accuracy by the process of removing the points of interest that are likely to be noise.

  Before explaining the outline of the process for removing the points of interest that are likely to cause noise, first, a basic process for extracting an image corresponding to the target image from a plurality of candidate images based on the direction of the points of interest. Explain the matter. The information processing apparatus 1 detects one or more points of interest from the target image based on some condition, and selects one point of interest as the target point of interest one by one from the one or more points of interest related to the detected target image. The information processing apparatus 1 calculates the direction of each of a plurality of sample points included in the vicinity of the target interest point for each selected target interest point.

  Here, the target image indicates an image selected by the information processing apparatus 1 by accepting an operation from the user. The operation from the user indicates, for example, an operation for selecting one image from a plurality of images, an operation for reading an image captured by an imaging device, an operation for acquiring an image via the Internet, or the like. The sample points indicate a plurality of points included in a predetermined range indicating the vicinity of the point of interest. As an example of the sample point, when the target interest point is a pixel, a plurality of pixels included in the vicinity of the target interest point may be mentioned, but instead of this, some other plural plurality of pixels included in the vicinity of the target interest point are included. It may be a point or the like.

  These sample points are arranged in a grid in the x-axis direction and the y-axis direction in the vicinity of the target point of interest, for example. Note that these sample points may be arranged in other shapes instead of being arranged in a grid. The interval between the sample points arranged in a grid is, for example, 1 s (s indicates a scale) that is predetermined for each point of interest. Note that these interest points and sample points are preferably pixel positions in the image, but are not necessarily pixel positions. If the pixel positions are integer positions, they may be decimal positions.

  The vicinity of the target interest point including these sample points indicates, for example, a circular range with a radius of 6 s centered on the target interest point. Alternatively, the vicinity of the target interest point may be configured to show a circular range having a radius smaller than the circular range having a radius of 6 s centered on the target interest point, or a circular range having a radius of 6 s. The structure which shows the circular range of a larger radius may be sufficient. Further, in the vicinity of the target interest point, instead of a configuration showing a circular range centered on the target interest point, a shape including other target interest points such as a rectangular range including the target interest point, a square range, a rhombus range, etc. The structure which shows a range may be sufficient.

  In this example, the plurality of sample points included in the vicinity of the target interest point indicates 113 sample points included in a circular range having a radius of 6 s centered on the target interest point. Hereinafter, for convenience of explanation, the plurality of sample points will be collectively referred to as a target sample point group. Further, each sample point included in the target sample point group will be described as a sample point of the target sample point group.

  The direction of the sample points of the target sample point group is determined by a wavelet response (dx, dy) based on the gradient of the luminance value within a predetermined direction calculation range (area) associated with each sample point of the target sample point group. Calculated. Note that the direction of the sample point is a wavelet response (dx, dy) based on a gradient of some other value in the direction calculation range such as a hue gradient or brightness gradient in the direction calculation range associated with each sample point. May be calculated. In addition, the information processing apparatus 1 substitutes the calculated wavelet response (dx, dy) related to the sample point into the following equation (1), so that the angle θ with respect to the sample point is determined for each sample point. Calculate as direction.

θ = tan ⁻¹ (dy / dx) (1)

  As described above, the information processing apparatus 1 can calculate the direction of each sample point of the target sample point group, and can calculate the direction of interest based on the calculated direction. Now that the basic matters have been described, an outline of processing for removing points of interest that are likely to cause noise will be described. Based on the direction of each sample point of the target sample point group calculated as described above, the information processing apparatus 1 determines an interest point that is likely to be noise among one or more interest points related to the target image. To do. Here, the noise indicates a plurality of points of interest that are calculated around a certain point of interest after determining the direction of the point of interest and that have similar directions. Such a plurality of points of interest are likely to have similar feature descriptors, and thus may not be distinguished from each other.

  In such a case, for example, a plurality of points of interest in the target image may be similar to one point of interest in the candidate image (that is, erroneous matching occurs in matching of images based on points of interest). There may be a problem that an image corresponding to the target image cannot be accurately retrieved from the image. Therefore, hereinafter, a plurality of points of interest that cause such a problem will be referred to as noise. Such noise is, for example, on the edge indicating the boundary between the thing reflected in the target image and the plain background, the position of a corner (corner), a blob, a T-branch, etc. (hereinafter referred to as an edge). Often detected.

  Here, among the sample points included in the vicinity of the point of interest detected on the position of these edges and the like, if the edge or the like is surrounded by a plain background within the direction calculation range associated with the sample point, The directions of the sample points included in the direction calculation range tend to face the same direction within a predetermined angle range (for example, a ± 0.1 ° range in the angle range indicating the direction). By using this tendency, the information processing apparatus 1 determines whether or not the target interest point related to the target sample point group is likely to be noise based on the direction of the sample points.

  When it is determined that the target interest point is likely to be noise, the information processing apparatus 1 excludes the target interest point from one or more interest points related to the target image as an exclusion target. In this way, the information processing apparatus 1 can remove points of interest that are likely to be noise from one or more points of interest related to the target image.

  The information processing apparatus 1 performs the same processing as the processing for the target image described so far, and the processing for removing the points of interest that are highly likely to be noise, on each of the plurality of candidate images. Based on the direction of each of a plurality of sample points included in the vicinity of the target interest point, an interest point that is likely to be noise is excluded from the interest points related to the candidate image. Then, the information processing apparatus 1 corresponds to the target image based on one or more points of interest related to the target image remaining without being excluded and one or more points of interest related to the candidate image remaining not excluded. An image is detected from a plurality of candidate images (a group of candidate images).

  Since the information processing apparatus 1 performs this detection based on the interest points remaining after the interest points that are likely to be noise are removed, the image corresponding to the target image can be accurately detected from a plurality of candidate images. Can do. Further, the information processing apparatus 1 can reduce the amount of calculation related to the point of interest that is likely to be noise as a result of removing the point of interest that is likely to be noise, thereby suppressing the calculation cost. it can. Note that the image corresponding to the target image is, for example, an image similar to the target image. The image similar to the target image indicates a candidate image that includes (shows) some or all of a plurality of things shown in the target image. For example, an image similar to the target image is a candidate image in which a specific person A among a plurality of persons on the target image is shown.

  As described above, the information processing apparatus 1 calculates the direction of each of a plurality of sample points in the vicinity of the target interest point selected from the one or more interest points detected from the target image, and calculates the calculated one or more interests. Based on the direction of each of a plurality of sample points related to each point, an interest point that is likely to be noise is determined among one or more interest points, and an interest point that is likely to be noise is determined according to the determination result. Exclude from one or more points of interest detected from the target image. Thereby, the information processing apparatus 1 can accurately detect an image corresponding to the target image.

<Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a functional configuration of the information processing apparatus 1 according to the present embodiment. The information processing apparatus 1 includes a storage unit 12 that stores candidate images 121, an input reception unit 13, a display unit 14, a communication unit 15, and a control unit 16. The candidate image 121 is a generic name for a plurality of candidate images.

  The storage unit 12 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), a RAM (Random Access Memory), and the like. Various information, images, programs, and candidate images 121 processed by the control unit 16 are stored. The storage unit 12 may be an external storage device connected via a digital input / output port such as a USB instead of the one built in the information processing apparatus 1.

The input receiving unit 13 is, for example, a keyboard, mouse, touch pad, or other input device. The input receiving unit 13 may function as a display unit, and may be configured as a touch panel.
The display unit 14 is a display of the information processing apparatus 1 and is, for example, a liquid crystal display panel or an organic EL (ElectroLuminescence) display panel.
The communication unit 15 includes, for example, a digital input / output port such as a USB, an Ethernet port, and the like.

  The control unit 16 includes a communication control unit 20, an interest point detection unit 21, a sample point direction calculation unit 22, a cluster detection unit 23, an interest point exclusion unit 24, a descriptor calculation unit 25, and an image search unit 26. The display control unit 27 is provided. Some or all of these functional units included in the control unit 16 are realized by, for example, a CPU (not shown) executing various programs stored in the storage unit 12. Some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

The communication control unit 20 controls the communication unit 15 so as to acquire various types of information, images, and the like from other devices. Further, the communication control unit 20 controls the communication unit 15 so as to output various information, images, and the like to other devices.
The interest point detection unit 21 detects one or more interest points from the target image selected based on the operation from the user received from the input reception unit 13. Further, the point of interest detection unit 21 detects one or more points of interest from each of the candidate images included in the candidate image 121 based on the candidate image 121 stored by the storage unit 12. In the present embodiment, the information processing apparatus 1 will be described assuming that target image candidates are stored in the storage unit 12 in advance, but instead, for example, another apparatus (via the communication unit 15) ( The target image may be acquired from an imaging device, a server, or the like.

  The sample point direction calculation unit 22 selects one point of interest as the target point of interest one by one from one or more points of interest detected from the target image by the point of interest detection unit 21. Then, the sample point direction calculation unit 22 repeatedly calculates the direction of each of the plurality of sample points (sample points of the target sample point group) included in the vicinity of the selected target interest point for each selected target interest point. . Hereinafter, this calculation process will be described as a sample point direction calculation process. In addition, the sample point direction calculation unit 22 selects candidate images one by one from the candidate images 121, and repeatedly performs the sample point direction calculation process for each selected candidate image, whereby each interest point detected from the candidate images is detected. The direction of each of a plurality of sample points included in each neighborhood is calculated for all candidate images.

  The cluster detection unit 23 detects a cluster of sample points based on the direction of each sample point of the target sample point group related to the target interest point calculated by the sample point direction calculation unit 22. In the following, a cluster of sample points is simply referred to as a cluster, and a process for detecting a cluster is referred to as a cluster detection process. In addition, the cluster detection unit 23 repeatedly performs the cluster detection process for each candidate image selected by the sample point direction calculation unit 22, so that each sample point related to the interest point detected in the selected candidate image is directed in the respective directions. A based cluster is detected for all candidate images.

  The point-of-interest exclusion unit 24 determines a point of interest that is likely to be noise from one or more points of interest related to the target image based on the cluster detection result by the cluster detection unit 23, and is likely to be noise. Are excluded from one or more points of interest related to the target image. Hereinafter, the process of excluding this exclusion target will be described as a noise exclusion process. Further, the interest point excluding unit 24 repeatedly performs noise excluding processing for each candidate image selected by the sample point direction calculating unit 22, thereby excluding an exclusion target related to the candidate image with respect to all candidate images. An interest point that is likely to be noise is an example of an exclusion target.

  The descriptor calculation unit 25 is included in the vicinity of the interest point for each interest point that is not excluded by the interest point exclusion unit 24 among the one or more interest points detected from the target image by the interest point detection unit 21. The direction of interest is calculated based on the direction of each sample point. Then, the descriptor calculation unit 25 calculates a feature descriptor (hereinafter simply referred to as a descriptor) using the contrast information in the vicinity of the point of interest based on the calculated direction of interest point. Hereinafter, the process for calculating the descriptor will be referred to as a descriptor calculation process. In addition, the descriptor calculation unit 25 selects candidate images one by one from the candidate images 121, and performs a descriptor calculation process for each selected candidate image, so that the interest point detected from the selected candidate images is obtained. The descriptors in the vicinity of the interest points that are not excluded by the interest point exclusion unit 24 are calculated for all candidate images.

  The image search unit 26 is a descriptor calculated by the descriptor calculation unit 25 for each interest point of the target image remaining without being excluded by the interest point exclusion unit 24 (hereinafter referred to as a descriptor group related to the target image). , A descriptor calculated by the descriptor calculation unit 25 for each one or more points of interest that are interest points of each candidate image and are not excluded by the interest point exclusion unit 24 (hereinafter referred to as descriptor groups related to candidate images) The candidate image 121 is searched for an image similar to the target image.

  In this search, for example, the image search unit 26 performs matching between the descriptor group related to the target image and the descriptor group related to each candidate image, and the candidate images included in the candidate image 121 in descending order of matching degree. And the top candidate image is searched for as an image similar to the target image. The degree of matching refers to, for example, the nearest neighbor search method (Nearest) when comparing each descriptor included in the descriptor group related to the target image with each descriptor included in the descriptor group related to the candidate image. It is represented by the number of votes obtained by selecting a descriptor having a short distance between descriptors by Neighbor Method) and counting up the number of descriptors of candidate images to which the selected descriptor belongs. In addition, instead of the configuration in which the image search unit 26 searches for the highest candidate image as an image similar to the target image, for example, a configuration in which candidate images from the top to a predetermined order are searched as images similar to the target image, etc. Alternatively, a configuration may be used in which an image similar to the target image is searched based on the order of high matching degree.

  The display control unit 27 controls the display unit 14 to display an image similar to the target image searched by the image search unit 26.

  Hereinafter, the flow of processing in which the control unit 16 detects an image similar to the target image from the candidate image 121 will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of a process flow in which the control unit 16 detects an image similar to the target image from the candidate image 121. Hereinafter, a process after the control unit 16 reads a plurality of images for selecting a target image from the storage unit 12 will be described. Further, in the following, one or more points of interest are detected in advance from each of a plurality of candidate images stored in advance in the storage unit 12, and further, descriptors related to the points of interest are calculated. Explain that it is. The control unit 16 may be configured to detect one or more points of interest from the candidate image when detecting one or more points of interest from the target image. In that case, when calculating the descriptors related to each of the points of interest detected from the target image, the descriptors related to each of the points of interest detected from the candidate image may also be calculated.

  First, the control unit 16 selects a target image from a plurality of images read from the storage unit 12 based on an operation from the user received by the input receiving unit 13 (step S100). Next, the point of interest detector 21 detects one or more points of interest from the target image (step S110). Next, the sample point direction calculation unit 22 selects a point of interest as a target point of interest one by one from the points of interest of the target image detected in step S110, and from step S130 to step S170 for each selected target point of interest. This process is repeated (step S120).

  After the target point of interest is selected in step S120, the sample point direction calculation unit 22 calculates the direction of each sample point included in the vicinity of the selected target point of interest (step S130). Next, the cluster detection unit 23 detects a cluster in which a predetermined number or more of sample points are gathered based on the direction of each sample point included in the vicinity of the target interest point calculated in step S130 (step S140). . This cluster detection may be performed by any method capable of detecting the cluster.

  Further, when one or more clusters are detected, the cluster detection unit 23 generates cluster information for each detected cluster. The cluster information includes, for example, an ID for identifying the cluster, information indicating the size of the cluster (for example, the number of sample points forming the cluster), information indicating the direction of the cluster, and the like. Here, the cluster detection unit 23 calculates, for each detected cluster, a statistic based on the direction of each sample point included in the cluster as the direction of the cluster. In this example, the statistic based on the direction of each sample point included in the cluster is the average value of the direction of each sample point included in the cluster. Instead, the median, mode, etc. It may be.

  After the cluster detection unit 23 detects the cluster in step S140, the interest point exclusion unit 24 selects the target interest point selected in step S120 based on the cluster detection result by the cluster detection unit 23 in step S140. It is determined whether or not there is a high possibility of noise (step S150). Here, with reference to FIG. 3 and FIG. 4, the process of calculating the direction of the point of interest and the characteristics of the direction of the sample point included in the vicinity of the point of interest that is likely to cause noise will be described. A description will be given of whether such a point of interest is likely to be noise.

  FIG. 3 is a diagram illustrating a portion of the target image including a portion where noise is likely to be detected. In FIG. 3, an image Q <b> 1 is a part of the target image including a portion that is likely to be noise. The image Q1 shows a line segment (edge or boundary line) with a sharp contrast change between a plain background portion and a thing portion representing a picture such as a poster in the target image.

  Interest points detected in the vicinity of such edges are likely to be noise. Here, the point of interest PC is an example of the points of interest detected in step S110. Hereinafter, this interest point PC will be referred to as the target interest point PC selected by the sample point direction calculation unit 22 in step S120. The range NA is a range indicating the vicinity of the target interest point PC. As described above, the range NA is a circular range having a radius of 6 s with the target interest point as the center. That is, the range NA includes 113 sample points as the target sample point group. In FIG. 3, the sample points of these target sample point groups are represented by diamonds in the range NA for convenience of explanation.

  The sample point direction calculation unit 22 selects sample points one by one from the target sample point group included in the vicinity of the target interest point. In FIG. 3, description will be made assuming that the selected sample point is the sample point PS. The sample point direction calculation unit 22 sets a square range direction calculation range BF as shown in FIG. 3 around the sample point PS selected on the image Q1. The sample point direction calculation unit 22 calculates the direction of the sample point PS based on the wavelet response (dx, dy) based on the gradient of the luminance value within the direction calculation range BF (area). The direction of the sample point PS is an example of a direction that represents the characteristics of the sample point.

  Here, the wavelet transform (digital filter) used for calculating the wavelet response (dx, dy) related to each sample point of the target sample point group included in the vicinity of the target interest point is applied to the direction calculation range BF. It is a range. In the example shown in FIG. 3, 113 direction calculation ranges BF corresponding to the 113 sample points included in the target sample point group are set by the sample point direction calculation unit 22 in the vicinity of the target interest point. The sample point direction calculation unit 22 calculates a wavelet response (dx, dy) for each of these direction calculation ranges BF, and based on the calculated wavelet response (dx, dy), each sample point of the target sample point group ( Direction) (including the sample point PS).

Here, in the direction of the sample point related to the interest point that is likely to be noise and calculated based on the wavelet response (dx, dy), for example, (A) to (D) described below are used. ) Is seen.
(A) When only the plain background is included in the direction calculation range BF, the direction of the sample point calculated based on the wavelet response (dx, dy) is not limited to the brightness of the plain background, and is 0 °. .

(B) Here, for convenience of explanation, the processing is not actually performed by the information processing apparatus 1 according to the present embodiment, but description will be made using values and the like when the direction calculation range BF is divided into four 2s blocks. When 3 blocks among the divided blocks are a plain background and the remaining 1 block is a mixture of a thing and a plain background, the direction of the sample point PS calculated based on the wavelet response (dx, dy) is 90 °. It is a multiple of 45 ° excluding multiples. Note that the information processing apparatus 1 may be configured to perform processing for dividing the direction calculation range BF into four 2s blocks.

(C) When a line segment extending from the upper end to the lower end appears in the direction calculation range BF, the direction of the sample point calculated based on the wavelet response (dx, dy) is either 0 ° or 180 °. . Note that the determining factor governing which of 0 ° and 180 ° is either a plain background or a bright line segment, and whether the line segment is right or left in the direction calculation range BF. Is there a point?

(D) When a line segment extending from the left end to the right end is reflected in the direction calculation range BF, the direction of the sample point calculated based on the wavelet response (dx, dy) is either 90 ° or 270 °. . In addition, the determining factor governing which of 90 ° and 270 ° is selected is that the solid background or the line segment is brighter, and the line segment is on the upper side or lower side in the direction calculation range BF. It is the point of either.

  Here, with reference to FIG. 4, when the direction of each of a plurality of sample points related to the point of interest on the edge where the point of interest that is likely to be noise is easily detected is calculated, the above (A) to (D) ) Explain how the trend shown in) appears. FIG. 4 shows a wavelet response (dx, dy) calculated for each of the direction calculation ranges BF related to a plurality of sample points, and a sample related to the direction calculation range BF calculated based on the wavelet responses (dx, dy). It is a figure which illustrates the direction of a point.

  First, a process for calculating a wavelet response (dx, dy) by the information processing apparatus 1 according to the present embodiment will be described. The response dx of the wavelet response (dx, dy) is obtained by dividing the direction calculation range BF into two blocks dxm and dxp as in the range BFdx shown in FIG. 4 and the luminance of the image included in the range of the block dxm. It is calculated based on the accumulated value and the accumulated luminance value of the image included in the range of the block dxp. Further, the response dy of the wavelet response (dx, dy) is obtained by dividing the direction calculation range BF into two blocks dym and dyp as in the range BFdy shown in FIG. Is calculated on the basis of the cumulative value of the luminance values and the cumulative value of the luminance values of the images included in the range of the block dyp.

  More specifically, the response dx is obtained by multiplying the cumulative value related to the block dxm by “−1” shown in the block dxm, and multiplying the cumulative value related to the block dxp by “+1” shown in the block dxp. It is calculated by taking the sum of In addition, the response dy multiplies the accumulated value related to the block dym by “−1” shown in the block dym, multiplies the accumulated value related to the block dyp by “+1” shown in the block dyp, and takes the sum thereof. Is calculated by

  Based on the fact that the wavelet response (dx, dy) is calculated by such processing, the direction of each of a plurality of sample points related to the point of interest on the edge where the point of interest that is likely to be noise is likely to be detected will be described below. The following describes how the tendencies shown in the above (A) to (D) appear. In FIG. 4, each direction calculation range BF is an edge where a point of interest that is likely to be noise is likely to be detected, and a plain background (a region in which dots in the direction calculation range BF shown in FIG. 4 are sparse. ) And some thing (a region where dots are dense in the direction calculation range BF shown in FIG. 4).

  Here, as in the case of (B) above, for the sake of convenience of explanation, it is not a process actually performed by the information processing apparatus 1 in the present embodiment, but a value when the direction calculation range BF is divided into four 2s blocks, etc. Will be described. Four numerical values in each direction calculation range BF shown in FIG. 4 are cumulative values (d1, d2, d3, d4) of luminance values related to images included in each of the four 2s blocks. As shown in FIG. 4, the cumulative value d1 is the cumulative value of the upper left block in the direction calculation range BF, and the cumulative value d2 is the cumulative value of the upper right block in the direction calculation range BF. The accumulated value d3 is the accumulated value of the lower left block in the direction calculation range BF, and the accumulated value d4 is the accumulated value of the lower right block in the direction calculation range BF. These cumulative values are shown by rounding values to integer values, but are not necessarily integer values. Further, in FIG. 4, in order to simplify the drawing, the density of the dots in the direction calculation range BF is drawn only for distinguishing between a thing and a plain background, and represents a large number of accumulated values (that is, luminance values). is not. Calculation of the wavelet response (dx, dy) is equivalent to applying the following expressions (2) and (3) to these accumulated values (d1, d2, d3, d4).

dx = (d2 + d4) − (d1 + d3) (2)

dy = (d3 + d4) − (d1 + d2) (3)

  The above equations (2) and (3) simply represent the calculation of the wavelet response (dx, dy) based on the accumulated values relating to the blocks dxm, dxp, dym, and dyp. A value obtained by calculating the direction of each sample point based on the calculated wavelet response (dx, dy) and the above equation (1) is the angle described on the right side of each arrow shown in FIG. As shown in FIG. 4, in the case of a sample point related to a point of interest that is highly likely to be noise and detected on the edge, calculation is performed based on the wavelet response (dx, dy) related to the sample point. It is considered that there is a tendency shown at least in the above (A) to (D) in the direction indicated. In addition, some or all of the plurality of sample points included in the vicinity of the interest point that becomes noise tends to be directed in substantially the same direction. For these reasons, the information processing apparatus 1 is configured such that when one or more clusters are detected by the cluster detection unit 23 (that is, a predetermined number or more of the sample points related to the target interest point are directed in substantially the same direction. If the target interest point is likely to be noise, it can be determined based on the tendencies shown in the above (A) to (D).

  Returning to FIG. Here, a process of determining whether or not the target point of interest by the point of interest exclusion unit 24 is a point of interest that is likely to be noise will be described. When one or more clusters are detected based on the cluster information generated by the cluster detection unit 23 in step S140, the interest point exclusion unit 24 determines the direction of the clusters included in the cluster information from (A) to (A). When the tendency shown in D) is satisfied, it is determined that the target point of interest selected in step S120 is a point of interest that is highly likely to be noise. On the other hand, the interest point exclusion unit 24 determines that the cluster detection unit 23 does not detect a cluster in step S140 or if one or more clusters are detected, and the direction of the cluster included in the cluster information is the above (A). When it does not correspond to the tendency shown to (D), it determines with the object interest point selected by step S120 being an interest point with low possibility of becoming noise.

  Such determination is merely an example. For example, when the interest point exclusion unit 24 detects one or more clusters, the directions of the clusters included in the cluster information are the above (A) to (D). In addition to the case where the trend shown in (1) is satisfied, when one or more clusters are detected, and the direction of the cluster included in the cluster information does not correspond to the tendency shown in (A) to (D) above, The configuration may be such that the target point of interest selected in step S120 is determined to be a point of interest that is likely to be noise. In addition, when one or more clusters are detected and the direction of the clusters included in the cluster information does not correspond to the tendency shown in the above (A) to (D), the interest point excluding unit 24 (A) Based on another tendency different from (D), it may be determined whether or not the target interest point selected in step S120 is an interest point that is likely to be noise.

  If it is determined that the target point of interest selected in step S120 is a point of interest that is likely to be noise (step S150-Yes), the point of interest exclusion unit 24 selects the target point of interest selected in step S120. Exclusion processing to exclude as an exclusion target is performed (step S160). On the other hand, when it is determined that the target point of interest selected in step S120 is a point of interest that is unlikely to be noise (step S150-No), the point of interest exclusion unit 24 returns to step S120 and returns to the next interest. Select a point as the target point of interest. In this way, through the processing from step S130 to step S170, the control unit 16 excludes an exclusion target (that is, a point of interest that is likely to be noise) from one or more points of interest related to the target image.

  Next, the descriptor calculation unit 25 determines the direction of interest of each of the points of interest based on the direction of the sample points included in the vicinity of each of the points of interest that are not excluded by the point of interest exclusion unit 24 in step S150. Based on the calculated interest point direction, a descriptor for each interest point is calculated (step S170). Next, the image search unit 26 matches each of the plurality of candidate images with the target image based on the descriptor calculated in step S170, and arranges the candidate images in the order of the highest degree of matching. Then, the image search unit 26 searches for a candidate image having the highest degree of matching as an image similar to the target image (step S180). Next, the display control unit 27 controls the display unit 14 to display the image retrieved as an image similar to the target image by the image retrieval unit 26 in step S180 (step S190).

  The interest point exclusion unit 24 determines whether or not the target interest point is likely to be noise based on the cluster information when the cluster detection unit 23 detects two or more clusters having a predetermined number of sample points or more. It may be configured to determine whether or not. In this case, for example, when the cluster detection unit 23 detects two or more clusters having a predetermined number of sample points or more by the cluster detection unit 23, and based on the cluster information, the clusters include a plurality of When it corresponds to at least one of the patterns, it is determined that there is a high possibility that the target interest point is noise. The pattern shown below is an example, and other patterns may be included. The patterns shown below may be any combination of some or all of the patterns shown below, and the combinations may include the other patterns described above.

Pattern 1) When the direction of the cluster is a multiple of 45 ° excluding a multiple of 90 ° (an angle that can be determined to be 45 ° within a predetermined range is not strictly 45 °) 2) 2 When a part of the directions of each cluster is directed in a certain direction K and the remaining directions not directed in the direction K are directed in a direction J opposite to the direction K, pattern 3) each of the two or more clusters Some of the directions are directed in the direction K, some of the remaining directions not directed in the direction K are directed in the direction J opposite to the direction K, and the remaining directions are the directions K and J. Pattern 4 when facing different directions I) The direction of each cluster is 0 ° or 180 ° (not strictly 0 ° or 180 °, it can be determined as 0 ° or 180 ° within a predetermined range) Angle)

  In the case of the above pattern 1, since it applies to at least one of the above trends (A) to (D), the target point of interest is likely to be noise. Therefore, the interest point exclusion unit 24 excludes such target interest points as exclusion targets. In the case of the above-described pattern 2, it is likely to occur when, for example, a thick line segment extending from the upper end to the lower end crosses diagonally in the direction calculation range BF, and the target interest point may also be a noise interest point. high. Therefore, the interest point exclusion unit 24 excludes such target interest points as exclusion targets. In the case of the pattern 3 described above, there is a high possibility that the target interest point is detected from some characteristic place different from a simple straight line in the direction calculation range BF. Therefore, the interest point exclusion unit 24 does not exclude such target interest points as exclusion targets. In the case of the above pattern 4, as in the case of the pattern 1, since it applies to at least one of the above trends (A) to (D), there is a high possibility that the target interest point becomes noise. Therefore, the interest point exclusion unit 24 excludes such target interest points as exclusion targets. The patterns 1 to 4 are examples of predetermined conditions.

  As described above, when two or more clusters are detected, it is determined whether or not the pattern corresponds to at least one of the patterns 1 to 4, and the target point of interest may become noise based on the determination result. By determining whether or not the interest point is highly likely, it is possible to prevent a characteristic interest point that does not become noise from being excluded as an interest point that is likely to be noise.

  As described above, the information processing apparatus 1 calculates a direction that represents a feature relating to each of a plurality of sample points within a predetermined range including a target interest point selected from one or more interest points detected from the target image. Based on the calculated direction representing the characteristics of each of the plurality of sample points related to each of the one or more points of interest, the point of interest that is likely to be noise is determined among the one or more points of interest, and according to the determination result The points of interest that are likely to become noise are excluded from one or more points of interest. Thereby, the information processing apparatus 1 can accurately detect an image corresponding to the target image.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and changes, substitutions, deletions, and the like are possible without departing from the gist of the present invention. May be.

  It should be noted that a program for realizing the function of an arbitrary component in the above-described apparatus (for example, information processing apparatus 1) is recorded on a computer-readable recording medium, and the program is read into a computer system and executed. You may make it do. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. “Computer-readable recording medium” means a portable disk such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a CD (Compact Disk) -ROM, or a hard disk built in a computer system. Refers to the device. Further, the “computer-readable recording medium” means a volatile memory (RAM: Random Access) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Memory that holds a program for a certain period of time, such as Memory).

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the above functions. Further, the program may be a so-called difference file (difference program) that can realize the above functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus, 12 Storage part, 13 Input reception part, 14 Display part, 15 Communication part, 16 Control part, 20 Communication control part, 21 Interest point detection part, 22 Sample point direction calculation part, 23 Cluster detection part, 24 Interest point exclusion unit, 25 descriptor calculation unit, 26 image search unit, 27 display control unit, 121 candidate image

Claims (5)

A sample point direction calculating step for calculating a direction representing a feature relating to each of a plurality of sample points within a predetermined range including a target point of interest selected from one or more points of interest detected from the image;
A cluster detection step of detecting a cluster of the sample points for each of the one or more points of interest, based on a direction representing a characteristic regarding each of the plurality of sample points calculated by the sample point direction calculation step;
Determining the interest points to be excluded among the pre SL 1 or more of the point of interest based on a detection result of the cluster of the sample points for each of the interest points by the cluster detection step, the excluded in accordance with the determination result A point of interest exclusion step of excluding the target point of interest from the one or more points of interest;
An information processing method characterized by comprising: The interest point exclusion step determines that the interest point from which one or more of the clusters are detected by the cluster detection step is the interest point to be excluded.
The information processing method according to claim 1 . The point of interest exclusion step is the point of interest to be excluded when the two or more clusters satisfy a predetermined condition with respect to the point of interest in which two or more clusters are detected by the cluster detection step. To determine,
The information processing method according to claim 1 or 2 . A sample point direction calculation unit that calculates a direction representing a feature relating to each of a plurality of sample points within a predetermined range including a target point of interest selected from one or more points of interest detected from an image;
A cluster detection unit that detects a cluster of the sample points for each of the one or more points of interest, based on a direction that represents a characteristic regarding each of the plurality of sample points calculated by the sample point direction calculation unit;
Determining the interest points to be excluded among the pre SL 1 or more of the point of interest based on a detection result of the cluster of the sample points for each of the interest points by the cluster detection unit, the excluded in accordance with the determination result A point of interest exclusion unit that excludes the target point of interest from the one or more points of interest;
An information processing apparatus comprising: On the computer,
A sample point direction calculating step for calculating a direction representing a feature relating to each of a plurality of sample points within a predetermined range including a target point of interest selected from one or more points of interest detected from the image;
A cluster detection step of detecting a cluster of the sample points for each of the one or more points of interest, based on a direction representing a characteristic regarding each of the plurality of sample points calculated by the sample point direction calculation step;
Determining the interest points to be excluded among the pre SL 1 or more of the point of interest based on a detection result of the cluster of the sample points for each of the interest points by the cluster detection step, the excluded in accordance with the determination result A point of interest exclusion step of excluding the target point of interest from the one or more points of interest;
A program for running

Images