JP2018124740A - Image retrieval system, image retrieval method and image retrieval program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image retrieval system capable of performing image retrieval at high speed.SOLUTION: The image retrieval system generates an attention level image (83) representing a degree of attention indicative of a possibility that an object belonging to a specific category exists at each position of a query image (81) from the query image (81), generates a region of interest (D) from a saliency image (83) on the basis of the degree of attention, cuts out a region (D ') corresponding to the region of interest (D) from the query image (81) to generate a specific category query image (85), and searches for a reference image corresponding to the specific category query image (85) from a database in which multiple reference images are recorded.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image search system, an image search method, and an image search program for searching a reference image corresponding to a query image from a large number of reference images.

  Conventionally, specific object recognition is performed in which a reference image corresponding to a query image is searched from a large number of reference images, and an object photographed in the query image is specified. In the image search, as a preliminary preparation, a large number of images obtained by photographing a specific object are prepared for various objects and used as reference images, and the reference images are recorded in a database. Further, for each reference image, a feature amount indicating the feature of the reference image is calculated, and the feature amount is recorded in the database in association with each reference image. When performing an image search, an image captured by the imaging device is used as a query image. Then, the feature amount is calculated for the query image, the feature amount of the query image is compared with the feature amount of each reference image, and the reference image corresponding to the reference image having the feature amount having the highest degree of coincidence with the feature amount of the query image is compared. Select as an image. It can be said that the object photographed in the selected reference image is the object photographed in the query image, and the object photographed in the query image is specified (for example, see Patent Document 1).

Japanese Unexamined Patent Publication No. 2015-111339

In recent years, the size of images that can be photographed by a photographing apparatus has increased. When performing an image search using such a large-sized image as a query image, it takes time to calculate a feature amount or to select a reference image, and it is difficult to execute the image search within a practical time. .
An object of the present invention is to provide an image search system, an image search method, and an image search program that can perform an image search at high speed.

  The first embodiment of the present invention includes an attention level image generation unit that generates an attention level image indicating a degree of attention indicating the possibility that an object belonging to a specific category exists at each position of the query image from a query image; An interest region generating unit that generates an interest region based on the attention level from the attention level image, a specific category query image generating unit that generates a specific category query image by cutting out a region corresponding to the interest region from the query image, A reference image search unit that searches a reference image corresponding to the specific category query image from a database in which a plurality of reference images are recorded.

  In the present embodiment, a region of interest is generated based on the degree of attention indicating the possibility that an object belonging to a specific category exists at each position of the query image, and a region corresponding to the region of interest is cut out from the query image to generate a specific category query. An image is generated, and a reference image is searched based on the specific category query image. Therefore, the reference image search process can be performed at high speed, and the image search can be performed at high speed.

  In a second embodiment of the present invention, the image search system further includes a query image reduction unit that reduces the size of the query image, and the attention level image generation unit is reduced by the query image reduction unit. An image search system that generates the attention level image from a query image.

  In this embodiment, the attention level image is generated based on the reduced query image having a small size. Therefore, the attention level image generation process can be performed at high speed, and the image search can be performed at higher speed.

In a third embodiment of the present invention, an attention level image generation step for generating an attention level image indicating a degree of attention indicating a possibility that an object belonging to a specific category exists at each position of the query image from the query image; A region of interest generation step of generating a region of interest from the attention level image based on the degree of attention; a specific category query image generation step of generating a specific category query image by cutting out a region corresponding to the region of interest from the query image; And a reference image search step of searching for a reference image corresponding to the specific category query image from a database in which a plurality of reference images are recorded.
In this embodiment, the same effects as in the first embodiment can be obtained.

The fourth embodiment of the present invention is a degree-of-interest image generation function for generating a degree-of-interest image that represents the degree of attention indicating the possibility that an object belonging to a specific category exists at each position of the query image from the query image. A region of interest generation function for generating a region of interest from the attention level image based on the degree of attention; and a specific category query image generation for cutting out a region corresponding to the region of interest from the query image and generating a specific category query image An image search program for realizing a function and a reference image search function for searching a reference image corresponding to the specific category query image from a database in which a plurality of reference images are recorded.
In this embodiment, the same effects as in the first embodiment can be obtained.

  In the present invention, it is possible to perform image retrieval at high speed.

The schematic diagram which shows the outline | summary of the image search method of each embodiment of this invention. 1 is a block diagram showing an image search system according to a first embodiment of the present invention. The schematic diagram which shows the neural network part of 1st Embodiment of this invention. The flowchart which shows the image search method of 1st Embodiment of this invention. The flowchart which shows the image search method of 1st Embodiment of this invention. The flowchart which shows the region of interest production | generation step of 2nd Embodiment of this invention. The flowchart which shows the region of interest production | generation step of 3rd Embodiment of this invention.

With reference to FIG. 1, the outline of the image search method of each embodiment of the present invention will be described.
For the purpose of contributing to the understanding of each embodiment, the present outline description shows only a basic concept, and the image search method of the present invention can be variously modified. It is not limited to the processing method shown.

  As for the image search method of each embodiment, when searching for a reference image corresponding to a query image from a large number of reference images, pre-processing is performed on the query image so that the reference image search process can be performed at high speed. It is. In the preprocessing, an area where there is a high possibility that an object belonging to a specific category to be recognized exists is extracted from the query image, and a reference image search process is performed based on the extracted image.

  Specifically, as shown in FIG. 1, first, an attention level image 83 representing the attention level indicating the possibility that an object belonging to a specific category exists at each position of the query image 81 is generated from the query image 81. To do. Then, the attention areas a and b are extracted from the generated attention level image 83 based on the attention level, and the interest area D is generated based on the extracted attention areas a and b. Various methods can be used as a method for determining the attention areas a and b and a method for setting the area of interest D, and specific examples will be described in each embodiment. Subsequently, a region D ′ corresponding to the region of interest D is cut out from the query image 81 to generate a specific category query image 85. A reference image search process is performed based on the specific category query image 85 generated in this manner.

A first embodiment of the present invention will be described with reference to FIGS.
The image search system of this embodiment will be described with reference to FIGS.
As shown in FIG. 2, the image search system of the present embodiment includes a photographing unit 10 that captures an image. As the photographing unit 10, a mobile device camera, a vehicle-mounted camera, or the like is used. The imaging unit 10 captures a relatively large image. The size of the image indicates the number of pixels. An image photographed by the photographing unit 10 is a query image 81.

  The image search system includes an arithmetic control unit 12 that searches for a reference image corresponding to the query image 81 from a large number of reference images recorded in the database unit 60. The arithmetic control unit 12 has the following functions. Note that a program for causing the arithmetic control unit 12 to realize each function is also included in the scope of the present invention.

  The arithmetic control unit 12 includes a query image reduction unit 15 that reduces the size of the query image 81. The query image 81 is appropriately reduced to a size suitable for generating the attention level image 83 by the attention level image generation unit 20 described below. In reducing the size of the image, appropriate coarse-graining is performed, such as replacing a plurality of pixels with a single pixel having an average pixel value of the pixel values of the plurality of pixels. For this reason, the resolution is reduced by reducing the size of the image.

  The arithmetic control unit 12 includes a degree-of-interest image generation unit 20 that generates a degree-of-interest image 83 representing the degree of attention about a specific category from the query image 81 reduced by the query image reduction unit 15. The degree of attention indicates the possibility that an object belonging to a predetermined category exists at each position of the query image. For example, B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, `` Learning Deep Features for Discriminative Localization '' Computer Vision and Pattern Recognition ( CVPR), 2016 can be used.

  The attention level image generation unit 20 includes a neural network unit 21 that calculates the attention level for various categories, and an image drawing unit 22 that draws the attention level image 83 that represents the attention level for a specific category.

  As the category, various categories such as a DVD / CD jacket, a poster, and a road sign are used. The specific category is appropriately selected according to the purpose of the image search system. For example, in an image search system that identifies a DVD / CD being shot from a shot image of a DVD / CD jacket and obtains related information such as the content of the DVD / CD, the DVD / CD is a specific category. The jacket category is selected. Also, in an image search system that identifies the road sign being photographed from the captured image of the road sign and obtains the content of the road sign such as speed regulation and one-way traffic, the road sign is selected as a specific category. Is done.

  As shown in FIG. 3, the neural network unit 21 uses a deep convolution neural network. The deep convolutional neural network is formed by stacking an input layer 23, a number of intermediate layers 24, a total coupling layer 28, and an output layer 29. The intermediate layer 24 includes convolution layers 25, activation layers 26, and It is formed by laminating the pooling layer 27.

The deep convolution neural network has an input layer 23 to which an image is input.
The deep convolutional neural network has a number of intermediate layers 24.
The intermediate layer 24 includes a convolution layer 25 that extracts features at each position of the image. In other words, the convolutional layer 25, as shown in the equation (1), for each unit l, with respect to the input x from the previous layer at the predetermined position (i, j) of the image, for all units of the input x A bias β is added to the weighted sum Σαx, and an output y to the next layer is obtained.

The intermediate layer 24 has an activation layer 26 that contributes to improving convergence and learning speed. As shown in Expression (2), the activation layer 26 sets a response from the activation function f to the input x from the convolution layer 25 as an output y to the next layer for each unit l. As the activation function, ReLU (rectified linear unit) is used. As the activation function, other appropriate functions such as a sigmoid function may be used.

The intermediate layer 24 includes a pooling layer 27 that compresses information by discarding local fluctuations in the image. That is, the pooling layer 27 is an average pooling that takes an average value of the input x from the activation layer 26 in the small region M including m elements for each unit l as shown in the equation (3). I do. As a pooling method, other appropriate pooling methods such as maximum pooling may be used.

The deep convolutional neural network has a total connection layer 28 for calculating the attention level for each category c. That is, as shown in Expression (4), the total coupling layer 28 calculates a weighted sum S (= Σωx) for all units of the input x from the previous layer for each category c.
Then, as shown in Expression (5), the total coupling layer 28 sets the response of the soft map function to the input of the weighted sum S as the attention level P for each category c.
Further, the deep convolutional neural network includes an output layer 29 that outputs a degree of attention P for each category c to the image drawing unit 22.

In the deep convolution neural network, the parameters such as the weights α and ω and the bias β described above are determined in advance by learning using a large number of captured images of objects belonging to each category c. That is, the difference between the ideal output Q and the actual output R is measured by the cross entropy E shown in Equation (6).
In order to minimize the cross entropy E, parameters such as weights α, ω and bias β are sequentially updated using the back propagation method as shown in Equation (7) to determine the parameters. .

As shown in FIG. 2, the attention level image generation unit 20 draws the attention level image 83 representing the attention level P for the specific category C from the attention level P for the category c input from the neural network unit 21. The image drawing unit 22 is provided.
The attention level image 83 generated by the attention level image generation unit 20 has the same size as the reduced query image 81.

  The arithmetic control unit 12 includes an attention degree image enlargement unit 35 that enlarges the size of the attention degree image 83 to the same size as the query image 81 before reduction based on the reduction ratio of the query image 81. In enlarging the size of the image, appropriate complementation is performed, such as replacing a single pixel with a plurality of pixels having the same pixel value as the pixel value of the pixel.

The arithmetic control unit 12 includes an interest region generation unit 40 that generates an interest region D based on the attention level from the enlarged attention level image 83.
That is, the region-of-interest generating unit 40 includes a region-of-interest extracting unit 41 that extracts one or more regions having attention degrees equal to or greater than a predetermined threshold as the regions of interest a and b. The region-of-interest generating unit 40 includes a region-of-interest selecting unit 42 that selects one or more regions of interest a, b from one or more regions of interest a, b extracted by the region of interest extraction unit 41. In the present embodiment, the attention area selection unit 42 selects all the attention areas a and b extracted by the attention area extraction unit 41. The region-of-interest generating unit 40 has a region-of-interest setting unit 43 that sets a rectangular region circumscribing all the selected regions of interest a and b as the region of interest D. The region of interest D indicates a region where there is a high possibility that an object belonging to a specific category exists in a corresponding region of the query image 81.

  The arithmetic control unit 12 includes a specific category query image generation unit 45 that generates a specific category query image 85 by cutting out a region corresponding to the region of interest D from the query image 81. Image cropping refers to separating a part of an image without enlarging or reducing the size of the image. For this reason, although the size of the image itself is reduced by cutting out the image, the size of the image is not reduced and the resolution is not reduced.

  The arithmetic control unit 12 includes a specific category query image reduction unit 50 that reduces the specific category query image 85. The size of the specific category query image 85 is appropriately reduced to a size suitable for reference image search by the reference image search unit 52 described below.

The arithmetic control unit 12 includes a reference image search unit 52 that searches a reference image corresponding to the specific category query image 85 from a large number of reference images recorded in the database unit 60.
The reference image search unit 52 includes a feature amount calculation unit 55 that calculates the feature amount of the specific category query image 85. That is, the feature amount calculation unit 55 includes a feature point detection unit 56 that generates an edge image by extracting an edge from the specific category query image 85 and detects a plurality of feature points from the edge image. The feature quantity calculation unit 55 includes a local feature quantity extraction unit 57 that extracts a local feature quantity at each feature point. The SIFT feature value is used as the local feature value, and the SIFT feature value is obtained as an N-dimensional vector. The feature amount calculation unit 55 includes a binary conversion unit 58 that converts the SIFT feature amount into a binary code. The binary conversion unit 58 converts the SIFT feature value v into a binary code h expressed by binary values, as shown in Expression (8). Here, d is the size of the binary code after conversion, w is a matrix of N rows and d columns, and is a vector obtained by random sampling according to a normal distribution from points on a hypersphere of radius 1 in N dimensions. is there.

  On the other hand, the database unit 60 includes a reference image recording unit 61 in which a large amount of reference images are recorded. As a reference image, for each of various categories, a large number of images in which an object belonging to the category is captured are recorded. For example, DVD / CD jackets, posters, road signs, and the like are used as categories. As objects belonging to the category of DVD / CD jackets, speed restrictions as objects belonging to various types of DVD / CD jackets, road sign categories. Various road signs such as one-way streets are used.

  The database unit 60 includes a feature amount recording unit 62 in which the feature amount calculated for each reference image is recorded in association with each reference image. As the feature amount, as with the feature amount calculated by the feature amount calculation unit 55, a binary code obtained by binary conversion of the SIFT feature amount at each feature point is used.

  The database unit 60 includes a related information recording unit 63 in which related information related to each reference image is recorded in association with each reference image. As related information, for example, if the reference image is a DVD / CD jacket, the contents of the DVD / CD, if it is a movie DVD, information on the director or performer, if it is a music CD, the composer or performer Information is used. When the reference image is a traffic sign, the contents of the traffic sign such as a speed limit and one-way traffic are used.

  The reference image search unit 52 includes a reference image selection unit 65 that selects a reference image corresponding to the specific category query image 85 from a large number of reference images recorded in the database unit 60. That is, the reference image selection unit 65 compares the feature amount of each feature point of the specific category query image 85 with the feature amount of all feature points of all the reference images, and the feature amount of the feature point of the specific category query image 85 is the most. A feature point of a reference image having a feature amount with a high degree of coincidence is selected as a feature point corresponding to the feature point of the specific category query image 85. Here, the feature quantity with the smallest Hamming distance indicating the degree of inconsistency between the two binary codes is the feature quantity with the highest degree of coincidence. Then, the reference image selection unit 65 selects a reference image having the largest number of feature points corresponding to the feature points of the specific category query image 85 among all reference images as a reference image corresponding to the specific category query image 85. To do.

It can be said that the object photographed in the query image 81 and the object photographed in the reference image corresponding to the specific category query image 85 are the same type of object, and the type of the object photographed in the reference image is specified in advance. Therefore, the type of the object photographed in the query image 81 is specified.
The image search system includes a related information reading unit 70 that reads related information associated with the selected reference image.

  The image search system includes an output unit 75 that performs output based on the read related information. For example, when a DVD / CD jacket is photographed with a camera of a mobile device, the output unit 75 displays the relevant information of the DVD / CD on the screen of the mobile device, and a road sign is photographed with an in-vehicle camera. In this case, the vehicle is automatically controlled based on the content of the road sign.

The image search method according to the first embodiment of the present invention will be described with reference to FIGS.
Shooting step (S10)
In the photographing step (S10), a relatively large image is photographed. In this embodiment, a DVD / CD jacket is shot by the camera of the mobile device, and the size of the image is 4000 × 3000 pixels. The captured image becomes the query image 81.

Query image reduction step (S15)
In the query image reduction step (S15), the query image 81 is reduced. The query image 81 is appropriately reduced to a size suitable for generating the attention level image 83 in the attention level image generation step (S20) described below. In the present embodiment, the size of the query image 81 is reduced from 4000 × 3000 pixels to 256 × 256 pixels.

Attention level image generation step (S20)
In the attention level image generation step (S20), the attention level image 83 is generated from the reduced query image 81 using the deep convolutional neural network as described above. The size of the attention level image 83 is the same as the size of the reduced query image 81, and is 256 × 256 pixels in this embodiment.

Attention degree image enlargement step (S25)
Attention level image enlargement step (S25), based on the reduction rate of query image 81, attention level image 83 is enlarged to the same size as query image 81 before reduction. In the present embodiment, the size of the attention level image 83 is increased from 256 × 256 pixels to 4000 × 3000 pixels.

Region of interest generation step (S30)
In the region of interest generation step (S30), the region of interest D is generated based on the attention level from the enlarged attention level image 83. That is, in the attention area extraction step (S31), one or more attention areas a and b having an attention degree equal to or higher than a predetermined threshold in the attention degree image 83 are extracted. In the attention area selection step (S32), all the extracted attention areas a and b are selected. In the region of interest setting step (S33), a rectangular region circumscribing all the selected regions of interest a and b is set as the region of interest D.

Specific category query image generation step (S35)
In the specific category query image generation step (S35), a region D ′ corresponding to the region of interest D is cut out from the query image 81 to generate a specific category query image 85. In the present embodiment, the size of the query image 81 is 4000 × 3000 pixels, while the size of the specific category query image 85 is 2000 × 1800 pixels.

Specific category query image reduction step (S40)
In the specific category query image reduction step (S40), the specific category query image 85 is reduced. The size of the specific category query image 85 is appropriately set to a size suitable for reference image search in the reference image search step (S42) described below. In the present embodiment, the size of the specific category query image 85 is reduced from 2000 × 1800 pixels to 320 × 240 pixels.

Reference image search step (S42)
In the reference image search step (S42), the feature amount of the specific category query image 85 is calculated in the feature amount calculation step (S45). That is, in the feature amount calculation step (S46), in the feature point detection step (S46), an edge is extracted from the specific category query image 85 to generate an edge image, and a plurality of feature points are detected from the edge image. In the local feature amount extraction step (S47), SIFT feature amounts are extracted at each feature point. In the binary conversion step (S48), the SIFT feature value is converted into a binary code.

  In the reference image search step (S42), in the reference image selection step (S50), a reference image corresponding to the specific category query image 85 is selected from a large number of reference images recorded in the database. That is, the binary code of each feature point of the search image is compared with the binary code of all the feature points of all reference images recorded in the database, and the feature point of the reference image having the closest Hamming distance is compared with the specific category query. A feature point corresponding to the feature point of the image 85 is selected. Then, the reference image having the largest number of feature points corresponding to the feature points of the specific category query image 85 among all the reference images is selected as the reference image corresponding to the specific category query image 85.

The image search system and image search method of the present embodiment have the following effects.
In the image search system and the image search method of this embodiment, based on the reduced query image 81, the possibility that an object belonging to a specific category exists at each position of the query image 81 using a deep convolution neural network is shown. An attention level image 83 representing the attention level is generated. Then, based on the attention level image 83 enlarged to the same size as the query image 81 before reduction, an interest region D in which there is a high possibility that an object belonging to a specific category exists in the query image 81 is generated. A region D ′ corresponding to the region of interest D is cut out as a specific category query image 85 and a reference image is searched based on the specific category query image 85. For this reason, since the reference image search process is performed based on the specific category query image 85 having a size smaller than that of the query image 81, the image search can be performed at high speed. Furthermore, since the attention level image 83 is generated based on the reduced query image 81 having a small size, the generation process of the attention level image 83 using the deep convolution neural network can be performed at high speed. Image search can be performed at higher speed. Although the size of the specific category query image 85 is smaller than the size of the query image 81, the specific category query image 85 is generated by being cut out from the query image 81, and the resolution of the specific category query image 85 is the query. The resolution of the image 81 is not lowered. For this reason, the same search accuracy as that in the case of performing an image search using the query image 81 as it is is realized.

  In the present embodiment, the attention level image is generated based on the reduced query image. However, when using a computer having sufficient calculation capability, the query image is used as it is without reducing the query image. The attention level image may be generated.

  Further, in the present embodiment, an attention level image is generated from the reduced query image, an interest area is generated from the enlarged attention level image after the attention level image is enlarged, and an area corresponding to the interest area is cut out from the query image The specific category query image is generated. However, the region of interest is generated before the attention level image is expanded, the region of interest is expanded based on the reduction rate of the query image, the region corresponding to the expanded region of interest is cut out from the query image, and the specific category query image is extracted. You may make it produce | generate.

The second embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 6, in the image search method of the present embodiment, in the region of interest generation step (S60), the region of interest extraction step (S61) is the same as in the first embodiment. In the attention area selection step (S62), the attention area a having the maximum area is selected from the one or more attention areas a, b, c extracted in the attention area extraction step (S61). Also, the region of interest setting step (S63) is the same as in the first embodiment.

A third embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 7, in the image search method of the present embodiment, in the region of interest generation step (S65), the region of interest extraction step (S66) is the same as in the first embodiment. In the attention area selection step (S67), the attention areas a and b having an area equal to or larger than a predetermined threshold are selected from the one or more attention areas a, b, and c extracted in the attention area extraction step (S66). . Also, the region of interest setting step (S68) is the same as in the first embodiment.

DESCRIPTION OF SYMBOLS 15 Query image reduction part 20 Attention level image generation part 40 Interest area generation part 41 Attention area extraction part 42 Attention area selection part 43 Interest area setting part 45 Specific category query image generation part 52 Reference image search part 81 Query image 83 Attention degree image 85 Specific category query image a, b, c Region of interest D Region of interest

Claims (9)

A degree-of-interest image generation unit that generates a degree-of-interest image that represents the degree of attention indicating the possibility that an object belonging to a specific category exists at each position of the query image from the query image;
A region of interest generation unit that generates a region of interest from the attention level image based on the degree of attention;
A specific category query image generation unit that generates a specific category query image by cutting out an area corresponding to the region of interest from the query image;
A reference image search unit for searching a reference image corresponding to the specific category query image from a database in which a plurality of reference images are recorded;
An image search system. The image search system further includes a query image reduction unit that reduces the size of the query image,
The attention level image generation unit generates the attention level image from the query image reduced by the query image reduction unit;
The image search system according to claim 1. The attention level image generation unit uses a deep convolution neural network learned using an image in which an object belonging to a plurality of categories including the specific category is captured,
The image search system according to claim 1. The region of interest generation unit
A region-of-interest extraction unit that extracts one or more regions of interest in which the degree of attention is greater than or equal to a predetermined threshold value from the attention level image;
A region of interest selection unit that selects one or more regions of interest from one or more regions of interest extracted by the region of interest extraction unit;
A region-of-interest setting unit that sets, as a region of interest, a rectangular region that circumscribes all the regions of interest selected by the region of interest selection unit;
Comprising
The image search system according to claim 1. The attention area selection unit selects all the attention areas extracted by the attention area extraction unit,
The image search system according to claim 4. The attention area selection unit selects an attention area having the largest area among one or more attention areas extracted by the attention area extraction unit.
The image search system according to claim 4. The attention area selection unit selects an attention area having an area equal to or larger than a predetermined threshold among one or more attention areas extracted by the attention area extraction unit.
The image search system according to claim 4. An attention level image generation step for generating an attention level image indicating the attention level indicating the possibility that an object belonging to a specific category exists at each position of the query image from the query image;
A region of interest generation step for generating a region of interest based on the degree of attention from the attention level image;
A specific category query image generation step of generating a specific category query image by cutting out a region corresponding to the region of interest from the query image;
A reference image search step of searching for a reference image corresponding to the specific category query image from a database in which a plurality of reference images are recorded;
An image search method comprising: On the computer,
A degree-of-interest image generation function that generates a degree-of-interest image that indicates the degree of attention indicating the possibility that an object belonging to a specific category exists at each position of the query image from the query image;
A region of interest generation function for generating a region of interest based on the degree of attention from the attention level image;
A specific category query image generation function for generating a specific category query image by cutting out an area corresponding to the region of interest from the query image;
A reference image search function for searching a reference image corresponding to the specific category query image from a database in which a plurality of reference images are recorded;
An image search program that realizes