JP6904619B1 - retrieval method - Google Patents

Abstract

When searching for similar images, it may be difficult to quickly grasp highly relevant images based on the search results. SOLUTION: A search device narrows down a target image to be a similar search target based on a search condition including position information according to a shooting position of an image for similar image search, and a target image specified as a result of the narrowing down. A similar image similar to the image for searching for a similar image is searched from among them. [Selection diagram] Fig. 2

Description

The present invention relates to a search method, a search device, and a program.

A technique for searching for a similar image similar to the image based on the image is known.

For example, Patent Document 1 is one of the techniques used when searching for similar images. Patent Document 1 describes a means for receiving a search request for a similar image for which an auction ID is specified, and feature data associated with the auction ID specified for the search request from a feature database in which feature data for each product image is recorded. The means for acquiring the auction ID, the means for searching the feature database, and the means for extracting the auction ID associated with the feature data similar to the extracted feature data, and the image data corresponding to the extracted auction ID from the image database. A system including means for acquiring and means for transmitting acquired image data to a user computer is described.

Japanese Unexamined Patent Publication No. 2003-303188

For example, in an investigation by the police, it may be desired to search for similar images taken in other cases based on the photographic images of the leftover items left on the scene. In such crimes and cases, there may be a correlation in space and time, but the technique described in Patent Document 1 does not make adjustments according to the shooting position and the like. Therefore, when a simple search for similar images is performed, similar images that are assumed to have low relevance, such as similar images in cases that are far away from each other in position, are also searched. For example, as in the above example, if you simply search for similar images in a situation where positional relevance is expected, you can also search for images that are expected to have low relevance, such as similar images that are far away from each other. Will be done. As a result, there has been a problem that it is difficult to quickly grasp highly relevant images based on the search results.

Therefore, an object of the present invention is a search method, a search device, and a search device for solving a problem that it may be difficult to quickly grasp a highly relevant image based on a search result when searching for a similar image. To provide a program.

A search method, which is an embodiment of the present invention, for achieving such an object
The search device
Narrow down the target images to be searched for similar images based on the search conditions including the position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image similar to the image for the similar image search is searched for.

Further, the search device according to another embodiment of the present invention is
A narrowing section that narrows down the target images to be searched for similar images based on search conditions that include position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image search unit that searches for similar images similar to the image for similar image search, and a similar image search unit.
It has a structure of having.

In addition, the program which is another form of the present invention
In the search device
A narrowing section that narrows down the target images to be searched for similar images based on search conditions that include position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image search unit that searches for similar images similar to the image for similar image search, and a similar image search unit.
It is a program to realize.

The present invention provides a search method, a search device, and a program capable of quickly grasping highly relevant images based on the search results when searching for similar images by being configured as described above. It will be possible to provide.

It is a figure for demonstrating the outline of the search apparatus in 1st Embodiment of this disclosure. It is a block diagram which shows the configuration example of the search apparatus. It is a figure for demonstrating the processing example of the object detection part. It is a figure for demonstrating an example of narrowing down by a narrowing-down part. It is a flowchart which shows the operation example of a search device. It is a figure which shows the hardware configuration example of the search apparatus in the 2nd Embodiment of this disclosure. It is a block diagram which shows the configuration example of the search apparatus.

[First Embodiment]
The first embodiment of the present invention will be described with reference to FIGS. 1 to 5. FIG. 1 is a diagram for explaining an outline of the search device 100. FIG. 2 is a block diagram showing a configuration example of the search device 100. FIG. 3 is a diagram for explaining a processing example of the object detection unit 152. FIG. 4 is a diagram for explaining an example of narrowing down by the narrowing down unit 153. FIG. 5 is a flowchart showing an operation example of the search device 100.

In the first embodiment of the present invention, a search device 100 for searching for similar images similar to images will be described. As shown in FIG. 1, an image for searching for similar images is input to the search device 100 by a searcher or the like. At this time, as will be described later, the search device 100 can input search conditions including position information together with the image. When the image and the search condition including the position information are input, the search device 100 narrows down the target image to be searched based on the search condition including the position information. After that, the search device 100 searches for a similar image similar to the image for searching for a similar image from the narrowed-down target images.

FIG. 2 shows a configuration example of the search device 100. Referring to FIG. 2, the search device 100 has, as main components, for example, an operation input unit 110, a screen display unit 120, a communication I / F unit 130, a storage unit 140, and an arithmetic processing unit 150. have.

The operation input unit 110 includes an operation input device such as a keyboard and a mouse. The operation input unit 1110 detects an operation of a user or the like using the search device 100 and outputs the operation to the arithmetic processing unit 150.

The screen display unit 120 includes a screen display device such as an LCD (Liquid Crystal Display). The screen display unit 120 can display on the screen information according to the result of the similar image search by the similar image search unit 154, various information stored in the storage unit 140, and the like in response to an instruction from the arithmetic processing unit 150. ..

The communication I / F unit 130 includes a data communication circuit. The communication I / F unit 130 performs data communication with an external device connected by wireless communication or the like.

The storage unit 140 is a storage device such as a hard disk or a memory. The storage unit 140 stores processing information and a program 146 necessary for various processes in the arithmetic processing unit 150. The program 146 realizes various processing units by being read and executed by the arithmetic processing unit 150. The program 146 is read in advance from an external device or a recording medium via a data input / output function such as the communication I / F unit 130, and is stored in the storage unit 140. The main information stored in the storage unit 140 includes, for example, image data 141, image index 142, classification index 143, position index 144, type index 145, and the like.

The image data 141 includes an image (target image) to be searched in the similar image search by the similar image search unit 154, which will be described later. The image included in the image data 141 may correspond to each object region detected by the object detection unit 152, for example. The image included in the image data 141 is read by the object detection unit 152 from an external device, a recording medium, or the like via a data input / output function such as the communication I / F unit 130, based on the image input to the search device 100. It is acquired in advance by a method such as detection, and is stored in the storage unit 140.

The image index 142 is an index for storing feature information, which is information obtained by extracting features of an image included in image data 141. The feature information is read together with the image via a data input / output function such as the communication I / F unit 130, and the feature information is read from the image input to the search device 100 by the search device 100 (or by the object detection unit 152). It has been acquired in advance by a method such as extraction (using the detection result). The image index 142 can be used when the similar image search unit 154, which will be described later, performs a similar image search.

For example, the feature information includes at least one of information indicating the features of the image, such as the color, pattern, and contour feature amount extracted from the image. The features of the image included in the feature information may be those extracted from the image using a known technique such as a global feature amount or a local feature amount such as SIFT (Scale-Invariant Feature Transform). The feature information may include information indicating the features of the image other than those illustrated above.

The image data 141 may include an image in which color information is unnecessary for searching, such as how the glass is broken and footprints. The feature information corresponding to an image containing features unnecessary for such a search is subjected to predetermined processing such as features extracted as a result of grayscale conversion or a filter specialized for the classification to which the image belongs. It can include information indicating the characteristics of the image extracted after performing. In other words, the feature information may include information indicating the feature extracted after processing according to the classification to which the corresponding image belongs.

The classification index 143 is an index for storing classification information, which is information indicating the classification to which the image included in the image data 141 belongs. The classification information indicates, for example, the type of an object indicated by the image, such as a personal computer, a mouse, a car, a chair, a desk, or glass. The classification information may indicate a more subdivided classification than in the above example, such as an unbroken personal computer, broken glass, a chair backrest, etc. It may be included. The classification information is read together with the image via a data input / output function such as the communication I / F unit 130, by an operation on the operation input unit 110 after reading the image (or after detection by the object detection unit 152). It has been acquired in advance by a method such as input, image data input to the search device 100 is input to a pre-learned classification model or the like, and the search device 100 classifies the data. The classification index 143 can be used when the similar image search unit 154, which will be described later, performs a similar image search.

The classification model can be generated, for example, by performing learning using teacher data generated by labeling by humans. For example, in the classification model, teacher data is generated by human labeling the classification to which the image belongs to an image (feature amount of the image) prepared in advance, and machine learning is performed using the generated teacher data. It can be generated by things such as.

The position index 144 is an index for storing position information, which is information indicating the position where the image included in the image data 141 is taken. The position information is information that can specify the shooting position, such as GPS (Global Positioning System) coordinates. The position information is read together with the image via a data input / output function such as the communication I / F unit 130, by an operation on the operation input unit 110 after reading the image (or after detection by the object detection unit 152). It has been acquired in advance by a method such as input. The position index 144 can be used when the similar image search unit 154, which will be described later, performs a similar image search.

The type index 145 is an index for storing category information, which is information indicating an image category or the like included in the image data 141. The category information is, for example, what kind of crime scene was taken (or which case is applicable), at what time the image was taken, which event is shown, and so on. , The category according to the situation where the image was taken is shown. The category information is read together with the image via a data input / output function such as the communication I / F unit 130, by an operation on the operation input unit 110 after reading the image (or after detection by the object detection unit 152). It has been acquired in advance by a method such as input. The type index 145 can be used when the similar image search unit 154, which will be described later, performs a similar image search.

The arithmetic processing unit 150 includes a microprocessor such as an MPU and its peripheral circuits. The arithmetic processing unit 150 reads the program 146 from the storage unit 140 and executes it, thereby realizing various processing units in cooperation with the hardware and the program 146. The main processing units realized by the arithmetic processing unit 150 include, for example, an acquisition unit 151, an object detection unit 152, a narrowing down unit 153, a similar image search unit 154, and an output unit 155.

The acquisition unit 151 acquires an image to be stored in the storage unit 140, an image for performing a similar search, and the like via the communication I / F unit 130 and the like.

For example, the acquisition unit 151 acquires an image together with an instruction indicating that the image is stored in the storage unit 140. The acquisition unit 151 may acquire feature information, classification information, position information, category information, and the like together with the image. Then, the acquisition unit 151 stores the acquired image or the like in the storage unit 140. In other words, the acquisition unit 151 can store the acquired image as image data 141 in the storage unit 140, and can generate various indexes based on the information acquired together with the image and store the acquired image in the storage unit 140.

In addition, the acquisition unit 151 acquires an image for searching for similar images. At this time, the acquisition unit 151 can acquire information indicating search conditions used when searching for similar images, together with the above-mentioned image for searching for similar images. Here, in the case of the present embodiment, the search condition includes at least location information. The position information is GPS coordinates or the like, and is acquired together with the image as information indicating the shooting position of the image when the image is acquired by using an image pickup device, for example. The search condition may include information other than location information, such as classification information and category information. For example, as described above, the acquisition unit 151 acquires the image to be searched and the information indicating the search condition including at least the position information together with the search instruction of the similar image.

The object detection unit 152 detects an object region in the image acquired by the acquisition unit 151. In addition, the object detection unit 152 can determine the classification to which the detected object region (image of the object region portion) belongs.

For example, the object detection unit 152 detects an object region in an image for similar image search acquired by the acquisition unit 151 by using an object detection model learned in advance. FIG. 3 is a diagram for explaining a processing example by the object detection unit 152. In the case illustrated in FIG. 3, the object detection unit 152 detects, for example, three object regions by inputting an image into the object detection model.

The object detection model used by the object detection unit 152 may be one generated by using a known technique. For example, the object detection model can be generated by machine learning using deep learning, local features, or the like (for example, machine learning using teacher data as in the classification model).

In addition, the object detection unit 152 determines the classification to which the detected object region belongs. For example, the object detection unit 152 determines the classification to which the detected object region belongs by using the classification model described above. In the case of the image shown in FIG. 3, as described above, the object detection unit 152 detects three object regions. The object detection unit 152 determines the classification to which the three detected object regions belong by using the classification model. For example, in the case of FIG. 3, the object detection unit 152 determines that the three object regions belong to a person, a desk, and a chair, respectively. The object detection unit 152 may calculate a value indicating the certainty of the classification to which each object region belongs together with the above determination.

For example, as described above, the object detection unit 152 detects the object region and determines the classification to which the object region belongs. The object detection unit 152 may perform the above-described processing using a detection / classification model that simultaneously detects and classifies an object. The detection / classification model can be generated in advance by performing machine learning or the like in the same manner as the classification model and the object detection model.

The narrowing-down unit 153 is a target image that is a target image for which the similar image search unit 154 searches for similar images based on the search conditions including the position information acquired by the acquisition unit 151 and various indexes stored in the storage unit 140. To narrow down. In other words, the narrowing-down unit 153 narrows down the images included in the image data 141 to be searched by the similar image search based on the search condition including the position information.

The narrowing down unit 153 narrows down based on the position information included in the search condition with reference to the position index 144. For example, the narrowing-down unit 153 calculates the search target range based on the position information included in the search condition and the predetermined first condition. Here, the first condition is a condition used when designating a search range based on location information, such as a radius xkm, the same prefecture, the same city, and the like. The narrowing-down unit 153 calculates a search target range such as a radius xkm from a position indicated by the position information included in the search condition by using the position information included in the search condition and a predetermined first condition. Then, the narrowing-down unit 153 refers to the position index 144 and identifies an image whose position information is within the search target range among the images included in the image data 141. That is, the narrowing-down unit 153 sets the image whose position information is within the search target range among the images included in the image data 141 as the image to be searched.

FIG. 4 shows an example of the narrowing down process using the position information by the narrowing down unit 153. As shown in FIG. 4, by performing the narrowing process using the position information, the image taken within the search target range set based on the image shooting location indicated by the position information becomes the image to be searched. In other words, an image taken outside the search target range set based on the image shooting location indicated by the position information is an image that is not the search target.

When the search condition includes information other than the position information, the narrowing unit 153 can narrow down the information so as to satisfy each information included in the search condition. For example, suppose that the search condition includes location information and classification information. In this case, among the images included in the image data 141, the narrowing-down unit 153 sets the image whose position information is within the search target range and whose classification information is indicated by the search conditions as the image to be searched. For example, as described above, the narrowing down unit 153 can narrow down the target image by using a plurality of information included in the search conditions. The same applies when the search condition includes three or more pieces of information.

Further, the narrowing down unit 153 can perform re-narrowing based on the result of the above-mentioned narrowing down and a predetermined upper limit number. For example, it is assumed that the number of images to be searched narrowed down by the above processing exceeds the upper limit. In such a case, the narrowing down unit 153 can change the first condition and perform re-narrowing down. For example, the narrowing section 153 has a narrower range than the first condition, such as narrowing the radius by a predetermined value, changing from the same prefecture to an adjacent city, changing from the same city to a radius of zkm, and so on. In addition, the first condition is changed according to a predetermined standard. Then, the narrowing-down unit 153 re-narrows down based on the changed first condition. How to change the first condition at the time of re-narrowing may be arbitrarily set.

When the search condition includes information other than the position information, the narrowing unit 153 can perform re-narrowing after changing the conditions indicated by each information included in the search condition according to a predetermined standard. For example, the narrowing-down unit 153 can change the conditions by a method such as making a finer classification or specifying a category more. When the search condition includes information other than the position information, the narrowing unit 153 first changes only the first condition corresponding to the position information and re-narrows down, and when the result of the re-narrowing also exceeds the upper limit, It may be configured to change the conditions indicated by other information.

The similar image search unit 154 searches for an image similar to the image for similar image search from the target images specified as a result of the narrowing down unit 153. When the object detection unit 152 has detected a plurality of object regions from the image for similar image search, the similar image search unit 154 has each of the plurality of object regions (images of the object region portion) detected by the object detection unit 152. , Similar image search can be performed.

The similar image search unit 154 can perform a similar image search using a known technique such as CBIR (Content-based Image Retrieval) that performs a similar image search based on the content of the image. For example, the similar image search unit 154 calculates a feature amount from an image for similar image search (for example, an image of an object region portion). At this time, the similar image search unit 154 calculates the feature amount after performing predetermined preprocessing on the image for similar image search, such as grayscale conversion and application of a filter specialized for the classification to which the image belongs. Can be done. Further, the similar image search unit 154 calculates the degree of similarity between the image for similar image search and each target image based on the calculated feature amount and the feature amount of the target image indicated by the image index 142. Then, the similar image search unit 154 identifies the target image whose calculated similarity is equal to or less than the similarity threshold value as a similar image similar to the image for the similar image search. For example, as described above, the similar image search unit 154 searches for similar images similar to the image for similar image search from the target images narrowed down by the narrowing unit 153 based on the content of the image such as the feature amount of the image. do.

When the object detection unit 152 has detected a plurality of object regions from the image, the similar image search unit 154 may perform a similar image search on all of the plurality of object regions detected by the object detection unit 152. Alternatively, a similar image search may be performed on a part of a plurality of object regions detected by the object detection unit 152. For example, the similar image search unit 154 refers only to the object area designated by the operation of the operation input unit 110 or the object area designated by the predetermined conditions among the plurality of object areas detected by the object detection unit 152. , Similar image search may be performed.

Further, the similar image search unit 154 can be configured to determine whether or not to perform a predetermined preprocessing based on the classification determined by the object detection unit 152 or the like. For example, the similar image search unit 154 can be configured to perform the corresponding preprocessing when the classification to which the image to be searched for the similar image belongs, such as broken glass or footprints, is predetermined.

The output unit 155 outputs according to the result of the similar image search by the similar image search unit 154. For example, the output unit 155 displays the similar image searched by the similar image search unit 154 on the screen display unit 120 or transmits it to an external device. When the object detection unit 152 performs a similar image search for each of a plurality of object regions, the output unit 155 displays a similar image similar to the image of each object region portion on the screen so that the corresponding object region can be discriminated. Can be done. The output unit 155 may output other than those exemplified above, such as the number of images (the number of images of the target image) according to the result narrowed down by the narrowing unit 153.

The above is a configuration example of the search device 100.

When the acquisition unit 151 acquires an image together with an instruction indicating that the image is to be stored in the storage unit 140, the search device 100 may generate information for various indexes. For example, the search device 100 can detect the object area by the object detection unit 152 and save the image for each detected object area in the image data 141. Further, the search device 100 can determine the classification to which the object region belongs by the object detection unit 152, and save the determination result in the classification index 143. Further, the search device 100 can calculate the feature amount by the similar image search unit 154 and save the calculated result in the image index 142. Further, as described above, the search device 100 can save the position index and the type index based on the position information and the category information acquired together with the image. When the object detection unit 152 detects a plurality of object regions based on one image, the position information indicated by the position index 144 corresponding to each object region is, for example, the same (may be different). do not have). For example, as described above, when the acquisition unit 151 acquires an image to be stored in the storage unit 140, the search device 100 generates information for various indexes and stores the generated information in each index. Can be done.

Further, the search device 100 stores the image acquired for the similar image search, the search condition, the classification determined at the time of the similar image search, the feature amount, and the like in the storage unit 140 in order to use it at the time of another similar image search. It can be configured to store. That is, the search device 100 can be configured to search for similar images from the images stored in advance and the images used in the past similar image search.

Subsequently, an operation example of the search device 100 will be described with reference to FIG. With reference to FIG. 5, the acquisition unit 151 acquires the image to be searched and the information indicating the search condition including at least the position information together with the search instruction of the similar image (step S101).

The object detection unit 152 detects an object region in the image for similar image search acquired by the acquisition unit 151. Further, the object detection unit 152 determines the classification to which the detected object region (image of the object region portion) belongs (step S102). For example, the object detection unit 152 detects an object region in an image for similar image search acquired by the acquisition unit 151 by using an object detection model learned in advance. Further, the object detection unit 152 determines the classification to which the detected object region belongs by using the classification model learned in advance.

The narrowing-down unit 153 narrows down the images to be searched by the similar image search unit 154 based on the search conditions including the position information acquired by the acquisition unit 151 (step S103).

When the number of target images narrowed down by the process of step S103 exceeds the upper limit (step S104, No), the narrowing down unit 153 changes the first condition and re-filters (step S105).

The process of step S103 may be performed before the process of step S102 or in parallel with the process of step S102.

When the number of target images narrowed down by the process of step S103 is not more than the upper limit (step S104, Yes), the similar image search unit 154 performs a similar image search (step S106). The similar image search unit 154 can perform a similar image search using a known technique such as CBIR (Content-based Image Retrieval) that performs a similar image search based on the content of the image. When the object detection unit 152 detects a plurality of object regions from the image by the process of step S102, the similar image search unit 154 determines each of the plurality of object regions (images of the object region portion) detected by the object detection unit 152. , Similar image search can be performed.

The output unit 155 outputs according to the result of the similar image search by the similar image search unit 154 (step S107). For example, the output unit 155 can display a similar image on the screen display unit 120 or transmit the similar image to an external device.

The above is an operation example of the search device 100.

As described above, the search device 100 has a narrowing-down unit 153 and a similar image search unit 154. With such a configuration, the similar image search unit 154 can perform a similar image search from the results narrowed down by the narrowing down unit 153 based on the search conditions including the position information. As a result, a faster search can be realized. In addition, since images that are judged to have low relevance based on the search conditions are excluded from the search target in advance, similar images that are presumed to have high relevance can be searched more accurately. As a result, when searching for similar images, it is possible to quickly grasp highly relevant images based on the search results.

In the present embodiment, the search device 100 is determined to perform a similar image search by the similar image search unit 154 after narrowing down by the narrowing unit 153. However, the search device 100 may be configured such that the similar image search unit 154 searches for similar images from the entire image data 141, and then the narrowing unit 153 narrows down the search results by the similar image search unit 154. It is more preferable to perform a similar image search by the similar image search unit 154 after the narrowing down by the narrowing unit 153 because the load on the similar image search can be reduced. However, the object of the present invention can also be achieved by a method of narrowing down after performing a similar image search.

Further, the search device 100 may be configured to perform a similar image search on the entire image without detecting the object area. In this configuration, the search device 100 may have an object classification unit instead of the object detection unit 152.

Further, in the present embodiment, a case where the function as the search device 100 is realized by one information processing device is illustrated. However, the function as the search device 100 may be realized by a plurality of information processing devices connected via a network. For example, the search device 100 may be realized by using a cloud or the like.

[Second Embodiment]
Next, a second embodiment of the present disclosure will be described with reference to FIGS. 6 and 7. In the second embodiment, the outline of the configuration of the search device 200 will be described.

FIG. 6 shows a hardware configuration example of the search device 200. Referring to FIG. 6, the search device 200 has the following hardware configuration as an example.
-CPU (Central Processing Unit) 201 (arithmetic unit)
-ROM (Read Only Memory) 202 (storage device)
-RAM (Random Access Memory) 203 (storage device)
-Program group 204 loaded in RAM 203
-Storage device 205 that stores the program group 204
-Drive device 206 that reads and writes the recording medium 210 external to the information processing device
-Communication interface 207 that connects to the communication network 211 outside the information processing device
-I / O interface 208 for inputting / outputting data
-Bus 209 connecting each component

Further, the search device 200 can realize the functions as the narrowing-down unit 221 and the similar image search unit 222 shown in FIG. 7 by the CPU 201 acquiring the program group 204 and executing the program group 204. The program group 204 is stored in the storage device 205 or the ROM 202 in advance, for example, and the CPU 201 loads the program group 204 into the RAM 203 or the like and executes the program group 204 as needed. Further, the program group 204 may be supplied to the CPU 201 via the communication network 211, or may be stored in the recording medium 210 in advance, and the drive device 206 may read the program and supply the program to the CPU 201.

Note that FIG. 6 shows an example of the hardware configuration of the search device 200. The hardware configuration of the search device 200 is not limited to the above case. For example, the search device 200 may be configured from a part of the above-described configuration, such as not having the drive device 206.

The narrowing-down unit 221 narrows down the target images to be searched for similar images based on the search conditions including the position information according to the shooting position of the images for searching similar images. For example, the search device 200 acquires an image for searching for similar images and a search condition including position information. Then, the search device 200 narrows down the target images to be similar search targets based on the search conditions including the position information.

The similar image search unit 222 searches for a similar image similar to the image for similar image search from the target images identified as the result of narrowing down.

As described above, the search device 200 has a narrowing-down unit 221 and a similar image search unit 222. With such a configuration, the similar image search unit 222 can search for a similar image similar to the image for similar image search from the target images specified as a result of the narrowing down unit 221. This makes it possible to search for similar images that are assumed to be highly relevant with higher accuracy. As a result, when searching for similar images, it is possible to quickly grasp highly relevant images based on the search results.

The above-mentioned search device 200 can be realized by incorporating a predetermined program into the search device 200. Specifically, the program according to another embodiment of the present invention causes the search device 200 to search for a target image to be a similar search target based on a search condition including position information according to the shooting position of the image for similar image search. To realize a narrowing down unit 221 and a similar image search unit 222 that searches for a similar image similar to the image for the similar image search from the target images specified as a result of the narrowing down by the narrowing down unit 221. It is a program.

Further, in the search method realized by the above-mentioned search device 200, the search device 200 searches for a target image to be a similar search target based on a search condition including position information according to the shooting position of the image for similar image search. It is a method of narrowing down and searching for a similar image similar to the image for searching for a similar image from the target images specified as a result of the narrowing down.

Even the invention of the program (or recording medium) or the search method having the above-mentioned configuration achieves the above-mentioned object of the present invention in order to have the same operation and effect as the above-mentioned search device 200. Can be done.

<Additional notes>
Part or all of the above embodiments may also be described as in the appendix below. Hereinafter, the outline of the search method and the like in the present invention will be described. However, the present invention is not limited to the following configurations.

(Appendix 1)
The search device
Narrow down the target images to be searched for similar images based on the search conditions including the position information according to the shooting position of the images for similar image search.
A search method for searching for similar images similar to the image for searching for similar images from the target images specified as a result of narrowing down.
(Appendix 2)
The search method described in Appendix 1
A search method in which a search target range is calculated based on the location information and a target image is narrowed down based on the calculated search target range.
(Appendix 3)
The search method described in Appendix 1 or Appendix 2,
A search method that narrows down the target image by using at least one of the classification information indicating the type of the object or the category information indicating the category according to the shooting situation.
(Appendix 4)
The search method described in any one of Appendix 1 to Appendix 3, which is the search method.
A search method that re-filters the target images according to the number of target images identified as the result of narrowing down.
(Appendix 5)
The search method described in Appendix 4
A search method for re-narrowing down the target images when the number of target images specified as a result of narrowing down exceeds a predetermined upper limit.
(Appendix 6)
The search method described in Appendix 4 or Appendix 5.
A search method in which the conditions used when narrowing down using the position information are changed, the search target range is calculated using the changed conditions, and the target image is re-narrowed based on the calculated search target range.
(Appendix 7)
The search method according to any one of Supplementary Note 1 to Supplementary Note 6.
A search method that detects an object area in an image for similar image search and performs a similar image search on at least a part of the detected object area.
(Appendix 8)
The search method according to any one of Supplementary note 1 to Supplementary note 7.
A search method that outputs similar images identified as the result of a search.
(Appendix 9)
The search device
Search for similar images that are similar to the images for similar image search,
A search method that narrows down the search results based on search conditions that include position information according to the shooting position of the image for similar image search.
(Appendix 10)
A narrowing section that narrows down the target images to be searched for similar images based on search conditions that include position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image search unit that searches for similar images similar to the image for similar image search, and a similar image search unit.
Search device with.
(Appendix 11)
In the search device
A narrowing section that narrows down the target images to be searched for similar images based on search conditions that include position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image search unit that searches for similar images similar to the image for similar image search, and a similar image search unit.
A program to realize.

The programs described in each of the above embodiments and appendices may be stored in a storage device or recorded in a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Although the present invention has been described above with reference to each of the above embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

100 Search device 110 Operation input unit 120 Screen display unit 130 Communication I / F unit 140 Storage unit 141 Image data 142 Image index 143 Classification index 144 Position index 145 Type index 146 Program 150 Calculation processing unit 151 Acquisition unit 152 Object detection unit 153 Narrowing down Unit 154 Similar image search unit 155 Output unit 200 Search device 201 CPU
202 ROM
203 RAM
204 Program group 205 Storage device 206 Drive device 207 Communication interface 208 Input / output interface 209 Bus 210 Recording medium 211 Communication network 221 Filtering unit 222 Similar image search unit

Claims (8)

The search device
Narrow down the target images to be searched for similar images based on the search conditions including the position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image similar to the image for the similar image search is searched for, and the image is searched .
A search method for re-narrowing down target images when the number of target images specified as a result of narrowing down exceeds a predetermined upper limit. The search method according to claim 1.
A search method in which a search target range is calculated based on the location information and a target image is narrowed down based on the calculated search target range. The search method according to claim 1 or 2.
A search method that narrows down the target image by using at least one of the classification information indicating the type of the object or the category information indicating the category according to the shooting situation. The search method according to any one of claims 1 to 3.
A search method in which the conditions used when narrowing down using the position information are changed, the search target range is calculated using the changed conditions, and the target image is re-narrowed based on the calculated search target range. The search method according to any one of claims 1 to 4.
A search method that detects an object area in an image for similar image search and performs a similar image search on at least a part of the detected object area. The search method according to any one of claims 1 to 5.
A search method that outputs similar images identified as the result of a search. A narrowing section that narrows down the target images to be searched for similar images based on search conditions that include position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image search unit that searches for similar images similar to the image for similar image search, and a similar image search unit.
Have a,
The narrowing unit is a search device that re-narrows down the target images when the number of target images specified as a result of narrowing down exceeds a predetermined upper limit. In the search device
A narrowing section that narrows down the target images to be searched for similar images based on search conditions that include position information according to the shooting position of the images for similar image search.
From the target images identified as the result of narrowing down, a similar image search unit that searches for similar images similar to the image for similar image search, and a similar image search unit.
Realized ,
The narrowing down unit is a program that re-narrows down the target images when the number of target images specified as a result of narrowing down exceeds a predetermined upper limit.

Images