JP7519776B2 - Image search device, image search method, and image search program - Google Patents

Description

This disclosure relates to an image search device, an image search method, and an image search program.

When metadata is specified as a search criterion, a technique is available for searching for image data corresponding to the image metadata equivalent to the specified metadata from a storage device that stores image data representing an image in association with image metadata indicating the characteristics of the image.

For example, Patent Document 1 discloses an image search device that includes a search condition input unit that accepts search conditions for searching for images, a query generation unit that generates a first query based on the search conditions, an image search unit that searches for images in a database based on the first query, a relevance estimation unit that estimates the relevance between multiple images selected by a predetermined operation from among images found by the search, a query correction unit that generates a second query based on the relevance between the multiple images, and a function for displaying the second query generated by the query correction unit on an interface.

International Publication No. 2018/159095

The conventional image search device disclosed in Patent Document 1 (hereinafter referred to as the "conventional device") searches for images by further narrowing down the images found in a search based only on the search criteria entered by the user. Therefore, the conventional device has a problem in that if the search criteria entered by the user initially are not accurate, it may not be possible to search for the desired image.

The present disclosure is intended to solve the above-mentioned problems, and aims to provide an image search device that makes it possible to search for a desired image even if the search criteria initially entered are not accurate.

The image search device according to the present disclosure includes an operation acquisition unit that acquires operation information based on a user operation, a first metadata acquisition unit that acquires first metadata from the operation information acquired by the operation acquisition unit, a second metadata acquisition unit that acquires one or more second metadata related to the first metadata based on the first metadata acquired by the first metadata acquisition unit, an image data acquisition unit that acquires, for each second metadata, an image data group that is one or more image data corresponding to image metadata corresponding to each of the second metadata based on the second metadata acquired by the second metadata acquisition unit, a display image generation unit that generates a display image, the display image generation unit generating an image group image representing each of the one or more image data groups acquired by the image data acquisition unit and generating a display image in which the generated one or more image group images are arranged, a display image output unit that outputs the display image generated by the display image generation unit as display image information, a category acquisition unit that acquires category information indicating a category classification, and The metadata acquisition device includes a selection group acquisition unit that acquires, from the operation information acquired by the acquisition unit, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit is a selected image data group; a priority acquisition unit that acquires a priority that is previously assigned to each of the one or more second metadata acquired by the second metadata acquisition unit; and a priority change unit that changes the priority acquired by the priority acquisition unit , wherein the second metadata acquisition unit acquires, as second metadata, one or more metadata that belong to a category classification indicated by the category information among the metadata related to the first metadata, based on the first metadata acquired by the first metadata acquisition unit and the category information acquired by the category acquisition unit, and the priority change unit changes, from the one or more priorities acquired by the priority acquisition unit, a priority that is previously assigned to each of the second metadata corresponding to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit .

According to the present disclosure, it is possible to search for a desired image even if the search criteria initially entered are not accurate.

FIG. 1 is a block diagram showing an example of a configuration of a main part of an image retrieval system to which an image retrieval device according to a first embodiment is applied. FIG. 2 is a block diagram showing an example of a configuration of a main part of the image retrieval device according to the first embodiment. FIG. 3 is an explanatory diagram for explaining an example of contents registered in the metadata database included in the metadata storage device according to the first embodiment. FIG. 4 is an explanatory diagram illustrating an example of a display image generated by the display image generating unit according to the first embodiment. 5A and 5B are diagrams illustrating an example of a hardware configuration of the image retrieval device according to the first embodiment. FIG. 6A is a flowchart illustrating an example of processing performed by the image retrieval device according to the first embodiment. FIG. 6B is a flowchart illustrating an example of processing performed by the image retrieval device according to the first embodiment. FIG. 7 is a block diagram showing an example of a configuration of a main part of an image retrieval device according to the second embodiment. FIG. 8A is a flowchart illustrating an example of processing performed by the image retrieval device according to the second embodiment. FIG. 8B is a flowchart illustrating an example of processing performed by the image retrieval device according to the second embodiment. FIG. 9 is a block diagram showing an example of a configuration of a main part of an image retrieval device according to the third embodiment. In FIG. FIG. 10A is a flowchart illustrating an example of processing performed by the image retrieval device according to the second embodiment. FIG. 10B is a flowchart illustrating an example of processing performed by the image retrieval device according to the second embodiment.

The following describes in detail the embodiments of the present disclosure with reference to the drawings.

Embodiment 1.

The image retrieval device 100 according to the first embodiment will be described with reference to FIGS. 1 to 5 and 6A and 6B.
The configuration of the main parts of an image retrieval system 1 to which an image retrieval device 100 according to the first embodiment is applied will be described with reference to FIG.
FIG. 1 is a block diagram showing an example of a configuration of a main part of an image retrieval system 1 to which an image retrieval device 100 according to the first embodiment is applied.
The image search system 1 includes an imaging device 10 , an image accumulation device 20 , an image storage device 30 , a metadata storage device 40 , an operation input device 50 , a display device 60 , a network 70 , and an image search device 100 .

The network 70 is a communication network such as a LAN (Local Area Network) or a wireless LAN. In the first embodiment, the imaging device 10, the image storage device 20, the image storage device 30, the metadata storage device 40, and the image search device 100 are connected as nodes of the network 70.

The imaging device 10 is an image input device capable of outputting images captured by, for example, a surveillance camera, a digital still camera, or a digital video camera to the outside as image data. The image represented by the image data output by the imaging device 10 may be a still image or a moving image. The image data output by the imaging device 10 may also include audio information representing audio corresponding to the image represented by the image data.
In the first embodiment, the image search system 1 will be described as including, as an example, two imaging devices 10, a first imaging device 11 and a second imaging device 12. Note that in the first embodiment, the first imaging device 11 and the second imaging device 12 will be described as being surveillance cameras. Also, the number of imaging devices 10 is not limited to two, and may be one, or three or more.

The image storage device 20 acquires image data and generates image metadata that indicates the characteristics of the image represented by the acquired image data. The image storage device 20 outputs the acquired image data and the generated image metadata to the image storage device 30, and stores the output image data and image metadata in association with each other in the image storage device 30.

The image storage device 30 is a storage device equipped with a storage medium such as a solid state drive (SSD) or a hard disk drive (HDD).
The image storage device 30 receives the image data and image metadata output by the image storage device 20 from the image database, and stores the image data and image metadata.
The image storage device 30 includes, for example, an image database, and manages the association between the image data output by the image storage device 20 and the image metadata in the image database.

Image metadata includes, for example, shooting date and time information indicating the date and time when the image was taken, shooting location information indicating the shooting location where the image was taken, imaging device information indicating the imaging device 10 that took the image, event information indicating an event such as a sensor event that triggered the image to be taken, or object information indicating the name or type of an object appearing in the image.
Information such as shooting date and time information, shooting location information, imaging device information, and event information is included in, for example, the header of the image data.
For example, the image storage device 20 generates image metadata by extracting information such as shooting date and time information, shooting location information, imaging device information, or event information from the header of image data. Methods for extracting information such as shooting date and time information, shooting location information, imaging device information, or event information from the header of image data are well known, so a description thereof will be omitted.

The name of the object indicated by the object information is, for example, the name of the person when the object is a person.
The type of object indicated by the object information, for example, when the object is a vehicle, is a standard car, a light car, a truck, a bus, a motorcycle, a bicycle, or the like.
For example, the image storage device 20 generates image metadata by analyzing the image indicated by the image data using an image analysis technique and generating object information based on the name or type of an object appearing in the image obtained as a result of the analysis. A method for acquiring the name or type of an object appearing in an image using an image analysis technique is well known, and therefore a detailed description thereof will be omitted.

The operation input device 50 is an input device such as a keyboard or a mouse that accepts an input operation from a user, and upon receiving a user operation that is an input operation from the user, outputs operation information indicating the user operation to the image retrieval device 100.
The display device 60 is a display output device such as a display or projector that displays the display image indicated by the input display image information, and receives the display image information output by the image search device 100 and displays the display image indicated by the display image information.

The image search device 100 acquires operation information based on a user operation from the operation input device 50, and reads image data stored in the image storage device 30 from the image storage device 30 based on the acquired operation information. The image search device 100 generates a display image based on the read image data, and outputs the generated display image to the display device 60 as display image information.

The configuration of the main parts of the image retrieval device 100 according to the first embodiment will be described with reference to FIG.
FIG. 2 is a block diagram showing an example of a configuration of a main part of the image retrieval device 100 according to the first embodiment.
The image search device 100 includes an operation acquisition unit 101 , a first metadata acquisition unit 110 , a second metadata acquisition unit 120 , an image data acquisition unit 130 , a display image generation unit 140 , and a display image output unit 199 .
In addition to the above-mentioned components, the image search device 100 may further include a selection group acquisition unit 150, a priority acquisition unit 160, a maximum number acquisition unit 170, and a priority change unit 180.
In embodiment 1, the image search device 100 is described as including an operation acquisition unit 101, a first metadata acquisition unit 110, a second metadata acquisition unit 120, an image data acquisition unit 130, a display image generation unit 140, a selection group acquisition unit 150, a priority acquisition unit 160, a maximum number acquisition unit 170, a priority change unit 180, and a display image output unit 199.

The operation acquisition unit 101 acquires operation information based on a user operation.
Specifically, for example, the operation acquisition unit 101 acquires operation information output by the operation input device 50 .
The first metadata acquisition unit 110 acquires the first metadata from the operation information acquired by the operation acquisition unit 101 .
The first metadata is information designated by a user operation, and is period information indicating a period, position information indicating a position, imaging device information, event information, object information, or the like.

The second metadata acquisition section 120 acquires, based on the first metadata acquired by the first metadata acquisition section 110, one or more pieces of second metadata related to the first metadata.
Here, the term "related" means "similar" or "nearby."
It should be noted that one of the one or more pieces of second metadata corresponds to the first metadata.

Specifically, for example, when the first metadata is period information, the second metadata acquisition unit 120 acquires a plurality of pieces of period information indicating a plurality of periods that are temporally close to the period indicated by the period information that is the first metadata, and the period information that is the first metadata.
More specifically, for example, when the period indicated by the period information, which is the first metadata, is one day on a certain day, the second metadata acquisition unit 120 acquires seven pieces of second metadata indicating one day three days before the day indicated by the period information, which is the first metadata, one day two days before the day, one day one day before the day, one day on the day, one day one day after the day, one day two days after the day, and one day three days after the day.

The period indicated by the period information is not limited to one day on a certain day. For example, the period indicated by the period information may be one week of a certain week, one month of a certain month, one year of a certain year, or a period from a certain time, day, month, or year to another time, day, month, or year. Furthermore, the number of second metadata items acquired by the second metadata acquisition unit 120 is not limited to seven. The number of second metadata items acquired by the second metadata acquisition unit 120 may be six or less, or eight or more, as long as it is one or more.

Furthermore, for example, when the first metadata is location information, the second metadata acquisition unit 120 acquires a plurality of pieces of location information indicating a plurality of locations spatially close to a location indicated by the location information that is the first metadata, and the location information that is the first metadata. The location referred to here is, for example, latitude and longitude, a city, a region, a building, or an area within a building.
Specifically, for example, when the location indicated by the location information that is the first metadata is a certain city, the second metadata acquisition unit 120 acquires one or more second metadata indicating one or more cities in the vicinity of the city indicated by the location information that is the first metadata. The second metadata acquisition unit 120 may acquire, as one of the one or more second metadata, second metadata corresponding to the location information that is the first metadata.

Furthermore, for example, when the first metadata is imaging device information, the second metadata acquisition section 120 acquires one or more second metadata indicating one or more other imaging devices 10 similar to the imaging device 10 indicated by the imaging device information that is the first metadata, or one or more other imaging devices 10 having uses similar to the use of the imaging device 10 indicated by the imaging device information. The second metadata acquisition section 120 may acquire second metadata corresponding to the imaging device information that is the first metadata, as one of the one or more second metadata.
Also, for example, when the first metadata is imaging device information indicating an imaging device 10 fixed at a predetermined position such as a surveillance camera, the second metadata acquisition unit 120 acquires one or more second metadata indicating each of one or more other imaging devices 10 fixed at positions spatially close to the position at which the imaging device 10 indicated by the imaging device information, which is the first metadata, is fixed. The second metadata acquisition unit 120 may acquire, as one of the one or more second metadata, second metadata corresponding to the imaging device information, which is the first metadata.

Also, for example, when the first metadata is event information, the second metadata acquisition unit 120 acquires one or more second metadata indicating one or more other events similar to the event indicated by the event information that is the first metadata. Specifically, when the event indicated by the event information that is the first metadata is a sensor event based on a door sensor that detects the opening and closing of a certain door, the other event similar to the event is, for example, an event of the same type, such as a sensor event based on a door sensor that detects the opening and closing of another door. The other events similar to the event are not limited to events of the same type.
The second metadata acquisition section 120 may acquire, as one of the one or more pieces of second metadata, second metadata corresponding to event information that is the first metadata.

For example, when the first metadata is object information indicating a type of object, the second metadata acquisition unit 120 acquires one or more second metadata indicating one or more other types similar to the type of object indicated by the object information that is the first metadata. Specifically, for example, when the type of object indicated by the object information that is the first metadata is a standard automobile, since a standard automobile is a type of vehicle, the other object types similar to the type of object are a light automobile, a truck, a bus, a motorcycle, a bicycle, or the like. The second metadata acquisition unit 120 may acquire second metadata corresponding to the object information that is the first metadata as one of the one or more second metadata.

For example, the second metadata acquisition unit 120 acquires one or more pieces of second metadata related to the first metadata acquired by the first metadata acquisition unit 110 by reading them from the metadata storage device 40.
The metadata storage device 40 is a storage device equipped with a storage medium such as an SSD or HDD.
Specifically, for example, the metadata storage device 40 includes a metadata database, and stores one or more pieces of metadata that are related to each other in association with one metadata group.

The metadata database included in the metadata storage device 40 according to the first embodiment will be described with reference to FIG.
FIG. 3 is an explanatory diagram for explaining an example of contents registered in the metadata database included in the metadata storage device 40 according to the first embodiment.
As shown in FIG. 3, metadata groups such as a vehicle group, a surveillance camera group, or a sensor event group are registered in advance in the metadata database.

In addition, the metadata database pre-registers metadata indicating each of vehicles such as passenger cars, light cars, trucks, buses, motorcycles, and bicycles that are related to each other in that they are vehicles, and corresponds to a vehicle group pre-registered in the metadata database.
Similarly, metadata indicating each of the first imaging device 11 or the second imaging device 12, etc., which have in common that they are both surveillance cameras, is pre-registered in the metadata database in association with a surveillance camera group pre-registered in the metadata database.
Similarly, the metadata database pre-registers metadata indicating a first event or a second event, etc., which are common in that they are sensor events that trigger the first imaging device 11 or the second imaging device 12, both of which are surveillance cameras, to capture images, in association with a sensor event group pre-registered in the metadata database.

The contents registered in the meta database shown in FIG. 3 are merely an example, and are not limited to this.
Among the contents registered in the meta database shown in FIG. 3, the priority will be described later.

The second metadata acquisition section 120 searches the metadata database based on the first metadata acquired by the first metadata acquisition section 110, and acquires one or more pieces of second metadata related to the first metadata.
Specifically, when the first metadata is metadata indicating a passenger car, the second metadata acquisition unit 120 acquires, as the second metadata, metadata indicating each of a passenger car, a light car, a truck, a bus, a motorcycle, a bicycle, etc., which is one or more metadata associated with the vehicle group to which the metadata indicating the passenger car is associated.

When the second metadata acquisition unit 120 searches the metadata database based on the first metadata acquired by the first metadata acquisition unit 110 and finds that metadata corresponding to the first metadata has not been registered in advance in the metadata database, the second metadata acquisition unit 120 acquires the first metadata as the second metadata. In this case, the second metadata acquisition unit 120 may newly add the first metadata to the metadata database and register it.

Based on one or more second metadata items acquired by the second metadata acquisition unit 120, the image data acquisition unit 130 acquires, for each second metadata item, a group of image data items that is one or more image data items corresponding to image metadata corresponding to each of the second metadata items.
Specifically, the image data acquisition unit 130 acquires one or more image data corresponding to image metadata corresponding to each of the one or more second metadata by reading each of the second metadata from the image storage device 30, and combines the one or more image data acquired for each of the second metadata into one image data group.

More specifically, for example, if the first metadata is metadata indicating a standard automobile, the second metadata is metadata indicating, for example, a standard automobile, a light automobile, a truck, a bus, a motorcycle, and a bicycle. The image data acquisition unit 130 acquires one or more image data corresponding to the image metadata corresponding to each of the metadata indicating a standard automobile, a light automobile, a truck, a bus, a motorcycle, and a bicycle, for each of the metadata indicating a standard automobile, a light automobile, a truck, a bus, a motorcycle, and a bicycle.

The display image generating unit 140 generates a display image.
Specifically, the display image generating section 140 generates an image group image representing each of the one or more image data groups acquired by the image data acquiring section 130 .
For example, the display image generating unit 140 generates an image group image representing the image data group for each of the one or more image data groups by using one or more image data representative of the image data group among the one or more image data included in the image data group. A method for selecting one or more image data representative of the image data group to be used to generate the image group image among the one or more image data included in the image data group is well known, and therefore a description thereof will be omitted.
Furthermore, the display image generating unit 140 generates a display image by arranging one or more generated image group images.
The display image output unit 199 outputs the display image generated by the display image generating unit 140 as display image information.

By configuring as described above, the image search device 100 can visually provide the user via the display device 60 with a display image in which image group images representing one or more image data corresponding to image metadata corresponding to each of one or more second metadata related to the first metadata input by the user are arranged.
By visually checking the display image displayed on the display device 60, the user can find the desired image even if the first metadata, which is the search condition initially entered, is not accurate.

The display image generation unit 140 may use priorities that are assigned in advance in association with each of the one or more second metadata acquired by the second metadata acquisition unit 120 to determine the appearance of each of the one or more image group images to be generated, or the appearance in which the generated one or more image group images are arranged, and generate a display image in which the one or more image group images are arranged based on the determined appearance.
For example, the display image generation unit 140 generates an image group image representing an image data group corresponding to image metadata corresponding to second metadata having a high priority, the size of the image group image being larger than the size of the image group image representing an image data group corresponding to image metadata corresponding to second metadata having a low priority.
Also, for example, the display image generation unit 140 generates a display image in which one or more image group images are arranged so that an image group image representing an image data group corresponding to image metadata corresponding to second metadata having a high priority is positioned to the left or upper part of the display screen of the display device 60, compared with an image group image representing an image data group corresponding to image metadata corresponding to second metadata having a low priority.

The priority is acquired by the priority acquisition unit 160 .
3, the priority is registered in advance in a metadata database included in the metadata storage device 40 in association with each of one or more pieces of metadata registered in the metadata database. The priority acquisition unit 160 acquires the priority, which is assigned in advance in association with each of one or more pieces of second metadata acquired by the second metadata acquisition unit 120, by searching the metadata database included in the metadata storage device 40.

By configuring as described above, the image search device 100 can provide the user, via the display device 60, with a display image in which a group of image data corresponding to image metadata corresponding to second metadata with a high priority is made visually more prominent by comparing the group of image data corresponding to image metadata corresponding to second metadata with a low priority.

The display image generating unit 140 may use the priority acquired by the priority acquiring unit 160 and the maximum number of image group images to be arranged in the display image to determine the manner in which each of the one or more image group images to be generated, or the manner in which the generated one or more image group images are arranged, and generate a display image in which the one or more image group images are arranged based on the determined manner.

For example, when the number of generated image group images is greater than the maximum number of image group images to be arranged in the display image, the display image generating unit 140 generates the maximum number of image group images in order of the highest priority acquired by the priority acquiring unit 160. For example, when generating image group images in this case, the display image generating unit 140 may generate an image group image representing an image data group corresponding to image metadata corresponding to the second metadata with a high priority that is larger than the size of an image group image representing an image data group corresponding to image metadata corresponding to the second metadata with a low priority.

In addition, for example, in this case, when the display image generating unit 140 selects the maximum number of image group images in order of the highest priority acquired by the priority acquisition unit 160 and generates a display image in which the selected maximum number of image group images are arranged, the display image generating unit 140 may generate a display image in which the maximum number of image group images are arranged so that the image group images representing the image data groups corresponding to the image metadata corresponding to the second metadata with a high priority are positioned to the left or upper part of the display screen of the display device 60, compared with the image group images representing the image data groups corresponding to the second metadata with a low priority.

The maximum number is acquired by the maximum number acquisition unit 170 .
Specifically, for example, the maximum number is stored in advance in the metadata storage device 40. The maximum number acquisition section 170 acquires, for example, the maximum number of image group images to be arranged in the display image generated by the display image generation section 140 by reading it from the metadata storage device 40. The maximum number may be stored in advance in the maximum number acquisition section 170.

FIG. 4 is an explanatory diagram showing an example of a display image 200 generated by the display image generating unit 140 according to the first embodiment.
The display image 200 shown in Figure 4 is generated by the display image generation unit 140 when, as an example, the metadata and priorities registered in the metadata database are as shown in Figure 3, the first metadata is metadata indicating a passenger car, and the maximum number acquired by the maximum number acquisition unit 170 is 5.
Since the first metadata is metadata indicating a standard automobile, the second metadata is, for example, metadata indicating each of a standard automobile, a light automobile, a truck, a bus, a motorcycle, and a bicycle, as shown in FIG. 3.
The display image 200 shown in Figure 4 has five image group images 201, 202, 203, 204, and 205 that represent image data groups corresponding to image metadata corresponding to each of the five metadata items, from a standard car, which has the highest priority, to a truck, which has the fifth highest priority, among the metadata indicating each of the following: a regular car, a light car, a truck, a bus, a motorcycle, and a bicycle.

For example, image group image 201 is an image group image representing an image data group corresponding to image metadata corresponding to metadata indicating a passenger car, which has the highest priority, and is generated based on three image data included in the image data group.
In addition, image group image 202 is an image group image representing an image data group corresponding to image metadata corresponding to metadata indicating a motorcycle with the second highest priority, and is generated based on one image data piece included in the image data group.
Similarly, image group image 203 is an image group image representing a group of image data corresponding to image metadata corresponding to metadata indicating a bicycle, which has the third highest priority; image group image 204 is an image group image representing a group of image data corresponding to image metadata corresponding to metadata indicating a truck, which has the fourth highest priority; and image group image 205 is an image group image representing a group of image data corresponding to image metadata corresponding to metadata indicating a light vehicle, which has the fourth highest priority.

Image group image 201 shown in FIG. 4 was generated based on three pieces of image data included in the image data group, but may be generated based on two or fewer pieces of image data included in the image data group, or may be generated based on four or more pieces of image data included in the image data group. Also, image group images 202, 203, 204, and 205 shown in FIG. 4 were generated based on one piece of image data included in the image data group, but may be generated based on two or more pieces of image data included in the image data group.

By configuring as described above, the image search device 100 can provide the user via the display device 60 with a display image 200 in which only a predetermined number of image group images are displayed in descending order of priority for the image group images 201, 202, 203, 204, and 205 that represent image data groups corresponding to image metadata that corresponds to the second metadata with a high priority.

In addition, the image data acquisition unit 130 may select, from the one or more second metadata acquired by the second metadata acquisition unit 120, second metadata up to a number equivalent to the maximum number acquired by the maximum number acquisition unit 170, in order of highest priority acquired by the priority acquisition unit 160, and acquire, for each second metadata, only a group of image data which is one or more image data corresponding to the image metadata corresponding to each of the selected second metadata.
With this configuration, the image search device 100 does not need to acquire image data groups corresponding to image metadata corresponding to the second metadata for which the display image generating unit 140 does not need to generate an image group image. Therefore, the image search device 100 can reduce the time it takes to output display image information to the display device 60.

The selection group acquisition unit 150 acquires, from the operation information acquired by the operation acquisition unit 101, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit 130 is the selected image data group.
Specifically, for example, the user uses the operation input device 50 to perform a user operation to select one or more image group images from five image group images 201, 202, 203, 204, and 205, which are one or more image group images contained in the display image 200 shown in Fig. 4 and provided to the user via the display device 60. The operation acquisition unit 101 acquires operation information indicating the user operation from the operation input device 50. The selection group acquisition unit 150 acquires selection group information from the operation information acquired by the operation acquisition unit 101 in this manner.

The display image generation unit 140 arranges images indicated by one or more image data included in the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, and generates a display image capable of displaying the one or more images in a list.
With this configuration, the image search device 100 can enable the user to search for a desired image even if the search criteria initially input by the user are not accurate.

The image search device 100 may change the priority that is previously assigned in association with the second metadata corresponding to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150. For example, the change is performed by the priority change unit 180 changing the priority acquired by the priority acquisition unit 160.
Specifically, the priority change unit 180 changes a priority that has been previously assigned in association with second metadata equivalent to image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, from among one or more priorities acquired by the priority acquisition unit 160.

More specifically, for example, the priority change unit 180 changes a priority associated with second metadata equivalent to image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, from among one or more priorities acquired by the priority acquisition unit 160, which has been pre-registered in a metadata database possessed by the metadata storage device 40, to a higher priority.
By configuring in this manner, the image search device 100 can enable the user to search for the desired image even if the search criteria initially input by the user are not accurate, and can make it easier for the user to search for the desired image by increasing the priority of metadata equivalent to image metadata corresponding to a group of image data that the user has previously displayed in a list.

5A and 5B are diagrams showing an example of a hardware configuration of the image retrieval device 100 according to the first embodiment.
With reference to this figure, the hardware configuration of the main parts of the image retrieval device 100 according to the first embodiment will be described.

As shown in FIG. 5A, the image search device 100 is configured by a computer, and the computer has a processor 501 and a memory 502. The memory 502 stores a program for causing the computer to function as the operation acquisition unit 101, the first metadata acquisition unit 110, the second metadata acquisition unit 120, the image data acquisition unit 130, the display image generation unit 140, the display image output unit 199, the selection group acquisition unit 150, the priority acquisition unit 160, the maximum number acquisition unit 170, and the priority change unit 180. The processor 501 reads and executes the program stored in the memory 502, thereby realizing the functions of the operation acquisition unit 101, the first metadata acquisition unit 110, the second metadata acquisition unit 120, the image data acquisition unit 130, the display image generation unit 140, the display image output unit 199, the selection group acquisition unit 150, the priority acquisition unit 160, the maximum number acquisition unit 170, and the priority change unit 180.

Also, as shown in FIG. 5B, the image search device 100 may be configured with a processing circuit 503. In this case, the functions of the operation acquisition unit 101, the first metadata acquisition unit 110, the second metadata acquisition unit 120, the image data acquisition unit 130, the display image generation unit 140, the display image output unit 199, the selection group acquisition unit 150, the priority acquisition unit 160, the maximum number acquisition unit 170, and the priority change unit 180 may be realized by the processing circuit 503.

The image search device 100 may also be configured with a processor 501, a memory 502, and a processing circuit 503 (not shown). In this case, some of the functions of the operation acquisition unit 101, the first metadata acquisition unit 110, the second metadata acquisition unit 120, the image data acquisition unit 130, the display image generation unit 140, the display image output unit 199, the selection group acquisition unit 150, the priority acquisition unit 160, the maximum number acquisition unit 170, and the priority change unit 180 may be realized by the processor 501 and the memory 502, and the remaining functions may be realized by the processing circuit 503.

The processor 501 may be, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor).

The memory 502 is, for example, a semiconductor memory or a magnetic disk. More specifically, the memory 502 is, for example, a random access memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an SSD, or an HDD.

The processing circuit 503 is, for example, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration).

The operation of the image retrieval device 100 according to the first embodiment will be described with reference to FIGS. 6A and 6B.
Figures 6A and 6B are flowcharts illustrating an example of the processing of the image retrieval device 100 relating to embodiment 1, where Figure 6A shows a part of the processing of the image retrieval device 100 relating to embodiment 1 and Figure 6B shows the remainder of the processing of the image retrieval device 100 relating to embodiment 1.
The image search device 100 repeatedly executes the processes of the flowcharts shown in FIGS. 6A and 6B.

First, in step ST601, the operation acquisition section 101 acquires operation information.
Next, in step ST602, the first metadata acquisition section 110 decides whether or not the operation information is operation information that designates the first metadata.
A case where the first metadata acquisition section 110 has determined in step ST602 that the operation information is not operation information that designates the first metadata will be described later.
If the first metadata acquisition section 110 determines in step ST602 that the operation information is operation information that specifies the first metadata, then in step ST603, the first metadata acquisition section 110 acquires the first metadata from the operation information.

After step ST603, in step ST611, the second metadata acquisition section 120 searches for the second metadata based on the first metadata, and determines whether or not the second metadata corresponding to the first metadata exists.
In step ST611, when the second metadata acquisition section 120 determines that there is no second metadata corresponding to the first metadata as a result of searching for the second metadata by the second metadata acquisition section 120, in step ST621, the second metadata acquisition section 120 newly registers metadata corresponding to the first metadata in the metadata database of the metadata storage device 40. Note that, when the second metadata acquisition section 120 newly registers metadata, it may register the priority corresponding to the metadata as a predetermined value such as 0.
After step ST621, in step ST622, the second metadata acquisition section 120 acquires the first metadata as the second metadata.
In step ST611, if the second metadata acquisition unit 120 searches for the second metadata and determines that one or more pieces of second metadata corresponding to the first metadata exist, in step ST612, the second metadata acquisition unit 120 acquires the one or more pieces of second metadata.

After step ST612 or after step ST622, in step ST631, the priority acquisition section 160 acquires priorities corresponding to the one or more pieces of second metadata.
After step ST631, in step ST632, the maximum number acquisition section 170 acquires the maximum number of image group images to be arranged in the display image.
After step ST632, in step ST633, the image data acquisition section 130 acquires, for each of the second metadata, an image data group which is one or more pieces of image data corresponding to the image metadata corresponding to each of the one or more pieces of second metadata. Note that, in step ST633, the image data acquisition section 130 may acquire an image data group for the second metadata up to the maximum number acquired by the maximum number acquisition section 170 based on the priority acquired by the priority acquisition section 160.

After step ST633, in step ST634, the display image generating unit 140 generates an image group image representing each of the one or more image data groups acquired by the image data acquiring unit 130. Note that in step ST634, the display image generating unit 140 may generate an image group image for an image data group up to the maximum number acquired by the maximum number acquiring unit 170, based on the priority acquired by the priority acquiring unit 160.
After step ST634, in step ST635, the display image generating unit 140 generates a display image in which one or more image group images generated by the display image generating unit 140 in step ST634 are arranged. Note that in step ST635, the display image generating unit 140 may generate a display image in which image group images are arranged up to the maximum number acquired by the maximum number acquiring unit 170, based on the priority acquired by the priority acquiring unit 160.

After step ST635, in step ST636, the display image output section 199 outputs the display image generated by the display image generating section 140 as display image information.
After step ST636, the image retrieval device 100 ends the process of the flowcharts shown in FIGS. 6A and 6B, and returns to the process of step ST601 to repeatedly execute the process of the flowcharts shown in FIGS. 6A and 6B.

A case where the first metadata acquisition section 110 has determined in step ST602 that the operation information is not operation information that designates the first metadata will be described.
In this case, in step ST641, the selection group acquiring section 150 determines whether or not the display image information most recently output by the display image output section 199 is display image information indicating a display image in which one or more image group images are arranged.
In step ST641, if the selection group acquisition unit 150 determines that the display image information output immediately before is not display image information indicating a display image in which one or more image group images are arranged, the image search device 100 terminates the processing of the flowchart shown in Figures 6A and 6B, and the image search device 100 returns to the processing of step ST601 and repeatedly executes the processing of the flowchart shown in Figures 6A and 6B.

In step ST641, if the selection group acquisition unit 150 determines that the display image information output immediately before is display image information indicating a display image in which one or more image group images are arranged, in step ST642, the selection group acquisition unit 150 determines whether the operation information is selection group information.
If, in step ST642, the selection group acquisition unit 150 determines that the operation information is not selection group information, the image search device 100 terminates the processing of the flowchart shown in Figures 6A and 6B, and the image search device 100 returns to the processing of step ST601 and repeatedly executes the processing of the flowchart shown in Figures 6A and 6B.

If the selection group acquisition section 150 determines in step ST642 that the operation information is selection group information, the selection group acquisition section 150 acquires the selection group information in step ST643.
After step ST643, in step ST644, the priority change section 180 changes the priority previously assigned in association with the second metadata corresponding to the image metadata corresponding to the image data group indicated by the selection group information.

After step ST644, in step ST645, the display image generating section 140 generates a display image in which images indicated by one or more image data included in the image data group indicated by the selection group information are arranged.
After step ST645, in step ST646, the display image output section 199 outputs the display image generated by the display image generating section 140 as display image information.
After step ST646, the image retrieval device 100 ends the process of the flowchart shown in FIGS. 6A and 6B, and returns to the process of step ST601 to repeatedly execute the process of the flowchart shown in FIGS. 6A and 6B.

As described above, the image search device 100 includes the operation acquisition unit 101 that acquires operation information based on a user operation, the first metadata acquisition unit 110 that acquires first metadata from the operation information acquired by the operation acquisition unit 101, the second metadata acquisition unit 120 that acquires one or more second metadata related to the first metadata based on the first metadata acquired by the first metadata acquisition unit 110, the image data acquisition unit 130 that acquires, for each second metadata, based on the one or more second metadata acquired by the second metadata acquisition unit 120, an image data group that is one or more image data corresponding to image metadata corresponding to each of the second metadata, the display image generation unit 140 that generates a display image, the display image generation unit 140 generating image group images representing each of the one or more image data groups acquired by the image data acquisition unit 130 and generating a display image in which the generated one or more image group images are arranged, and the display image output unit 199 that outputs the display image generated by the display image generation unit 140 as display image information.
With this configuration, the image search device 100 can search not only for images that match the initially input search criteria, but also for related images. Therefore, the image search device 100 can search for a desired image even if the initially input search criteria are not accurate. Therefore, a user can search for a desired image even if the initially input search criteria are not accurate.

As described above, in addition to the configuration described above, the image search device 100 is equipped with a selection group acquisition unit 150 that acquires selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit 130 is a selected image data group from the operation information acquired by the operation acquisition unit 101, and the display image generation unit 140 is configured to generate a display image in which images indicated by one or more image data included in the image data group indicated by the selection group information acquired by the selection group acquisition unit 150 are arranged.
With this configuration, the image search device 100 can search for a desired image even if the initially input search criteria are not accurate. Therefore, a user can search for a desired image even if the initially input search criteria are not accurate.

As described above, in addition to the configuration described above, the image search device 100 is equipped with a priority acquisition unit 160 that acquires a priority that has been assigned in advance in association with each of the one or more second metadata acquired by the second metadata acquisition unit 120, and the display image generation unit 140 is configured to use the priority acquired by the priority acquisition unit 160 to determine the appearance of each of the one or more image group images to be generated, or the appearance in which the generated one or more image group images are arranged, and to generate a display image in which the one or more image group images are arranged based on the determined appearance.
By configuring in this manner, the image search device 100 is able to search for a desired image even if the search conditions initially input are not accurate, and can provide the user via the display device 60 with a display image in which a group of image data corresponding to image metadata corresponding to second metadata with a high priority is made visually more prominent than a group of image data corresponding to image metadata corresponding to second metadata with a low priority.

As described above, in addition to the configuration described above, the image search device 100 also includes a priority acquisition unit 160 that acquires a priority that has been assigned in advance in association with each of the one or more second metadata acquired by the second metadata acquisition unit 120, and a maximum number acquisition unit 170 that acquires the maximum number of image group images to be arranged in the display image generated by the display image generation unit 140.The display image generation unit 140 is configured to use the priority acquired by the priority acquisition unit 160 and the maximum number acquired by the maximum number acquisition unit 170 to determine the appearance of each of the one or more image group images to be generated, or the appearance in which the generated one or more image group images will be arranged, and to generate a display image in which the one or more image group images are arranged based on the determined appearance.
By configuring in this manner, the image search device 100 is able to search for a desired image even if the search conditions initially input are not accurate, and can provide the user via the display device 60 with a predetermined number of display images of image group images representing image data groups corresponding to image metadata that corresponds to the second metadata having a high priority.

As described above, in addition to the configuration described above, the image search device 100 is equipped with a priority acquisition unit 160 that acquires a priority that has been assigned in advance in association with each of the one or more second metadata acquired by the second metadata acquisition unit 120, and a maximum number acquisition unit 170 that acquires the maximum number of image group images to be arranged in the display image generated by the display image generation unit 140.The image data acquisition unit 130 is configured to select, from the one or more second metadata acquired by the second metadata acquisition unit 120, second metadata in order of highest priority acquired by the priority acquisition unit 160, a number of second metadata up to the maximum number acquired by the maximum number acquisition unit 170, and to acquire, for each second metadata, an image data group that is one or more image data corresponding to the image metadata corresponding to each of the selected second metadata.
With this configuration, the image search device 100 does not need to acquire image data groups corresponding to image metadata corresponding to the second metadata for which the display image generating unit 140 does not need to generate an image group image. Therefore, the image search device 100 can reduce the time it takes to output display image information to the display device 60.

As described above, in addition to the above-mentioned configuration, the image search device 100 is equipped with a selection group acquisition unit 150 that acquires, from the operation information acquired by the operation acquisition unit 101, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit 130 is a selected image data group; a priority acquisition unit 160 that acquires priorities that are assigned in advance in correspondence with each of the one or more second metadata acquired by the second metadata acquisition unit 120; and a priority change unit 180 that changes the priorities acquired by the priority acquisition unit 160, and the priority change unit 180 is configured to change the priorities that are assigned in advance in correspondence with the second metadata equivalent to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, among the one or more priorities acquired by the priority acquisition unit 160.
By configuring in this manner, the image search device 100 can enable the user to search for the desired image even if the search criteria initially input by the user are not accurate, and can make it easier for the user to search for the desired image by increasing the priority of metadata equivalent to image metadata corresponding to a group of image data that the user has previously displayed in a list.

Embodiment 2.
An image retrieval device 100a according to the second embodiment will be described with reference to FIG. 7, FIG. 8A, and FIG. 8B.
An image retrieval system 1a (not shown) to which the image retrieval device 100a according to the second embodiment is applied is merely a system in which the image retrieval device 100 included in the image retrieval system 1 according to the first embodiment shown as an example in Fig. 1 is replaced with the image retrieval device 100a. Therefore, a description of a block diagram showing an example of the configuration of the main parts of the image retrieval system 1a according to the second embodiment will be omitted. Also, descriptions of the imaging device 10, image storage device 20, image storage device 30, metadata storage device 40, operation input device 50, display device 60, and network 70 included in the image retrieval system 1a in the same way as in the image retrieval system 1 according to the first embodiment will be omitted.
In the first embodiment, the case where the first metadata, which is the search condition initially input by the user, is one item has been described. However, the first metadata initially input by the user is not limited to one item, and may be two or more items.
In the second embodiment, as an example, a case will be described in which the number of first metadata items input by the user at first is two.

The configuration of the main parts of an image retrieval device 100a according to the second embodiment will be described with reference to FIG.
FIG. 7 is a block diagram showing an example of a configuration of a main part of an image retrieval device 100a according to the second embodiment.
The image search device 100 a includes an operation acquisition unit 101 , a first metadata acquisition unit 110 a , a second metadata acquisition unit 120 a , an image data acquisition unit 130 a , a display image generation unit 140 a , and a display image output unit 199 .
In addition to the above-mentioned components, the image search device 100a may further include a selection group acquisition unit 150, a priority acquisition unit 160a, a maximum number acquisition unit 170, and a priority change unit 180a.
The image search device 100a according to embodiment 2 will be described as including an operation acquisition unit 101, a first metadata acquisition unit 110a, a second metadata acquisition unit 120a, an image data acquisition unit 130a, a display image generation unit 140a, a selection group acquisition unit 150, a priority acquisition unit 160a, a maximum number acquisition unit 170, a priority change unit 180a, and a display image output unit 199.

Compared to the image retrieval device 100 of embodiment 1, the image retrieval device 100a differs in that, of the operation acquisition unit 101, first metadata acquisition unit 110, second metadata acquisition unit 120, image data acquisition unit 130, display image generation unit 140, selection group acquisition unit 150, priority acquisition unit 160, maximum number acquisition unit 170, priority change unit 180, and display image output unit 199 provided in the image retrieval device 100 of embodiment 1, the first metadata acquisition unit 110, the second metadata acquisition unit 120, the image data acquisition unit 130, the display image generation unit 140, the priority acquisition unit 160, and the priority change unit 180 are changed to first metadata acquisition unit 110a, second metadata acquisition unit 120a, image data acquisition unit 130a, display image generation unit 140a, priority acquisition unit 160a, and priority change unit 180a.
In the configuration of the image retrieval device 100a, the description of the configuration similar to that of the image retrieval device 100 according to the first embodiment will be omitted.

While the first metadata acquisition section 110 according to the first embodiment acquires one piece of first metadata, the first metadata acquisition section 110a acquires two pieces of first metadata.
In the second embodiment, it is assumed that the two pieces of first metadata acquired by the first metadata acquisition unit 110a are an AND condition.
Furthermore, the number of first metadata items acquired by the first metadata acquisition unit 110a is not limited to two, and may be three or more.
In the second embodiment, the first metadata acquisition unit 110a will be described as acquiring, as two pieces of first metadata, metadata indicating the first imaging device 11 and metadata indicating a first event.

The second metadata acquisition section 120a acquires, based on the plurality of first metadata acquired by the first metadata acquisition section 110, one or more second metadata related to the first metadata for each of the plurality of first metadata.
Specifically, for example, the second metadata acquisition section 120a acquires, for each of the two first metadata items acquired by the first metadata acquisition section 110, one or more second metadata items related to the first metadata.
The method by which the second metadata acquisition unit 120a acquires one or more second metadata related to the first metadata is similar to the method by which the second metadata acquisition unit 120 in embodiment 1 acquires one or more second metadata related to the first metadata, so a description thereof will be omitted.
In the following, in the second embodiment, it is assumed that the metadata registered in the metadata database included in the metadata storage device 40 is the same as the metadata shown in FIG.
The second metadata acquisition unit 120a acquires metadata indicating the first imaging device 11 and metadata indicating the second imaging device 12 as second metadata related to the first metadata indicating the first imaging device 11. In addition, the second metadata acquisition unit 120a acquires metadata indicating the first event and metadata indicating the second event as second metadata related to the first metadata indicating the first event.

The image data acquisition unit 130a generates one or more second metadata sets by sequentially combining one of the one or more second metadata related to one of the first metadata acquired by the second metadata acquisition unit 120a with one of the one or more second metadata related to the other first metadata acquired by the second metadata acquisition unit 120a. Based on the one or more generated second metadata sets, the image data acquisition unit 130a acquires, for each second metadata set, an image data group that is one or more image data corresponding to the image metadata corresponding to each of the second metadata sets.

Specifically, for example, when the second metadata acquisition unit 120a acquires metadata indicating the first imaging device 11 and metadata indicating the second imaging device 12 as second metadata related to the first metadata indicating the first imaging device 11, and acquires metadata indicating the first event and metadata indicating the second event as second metadata related to the first metadata indicating the first event, the image data acquisition unit 130a generates four second metadata sets: a second metadata set that combines the second metadata indicating the first imaging device 11 and the second metadata indicating the first event, a second metadata set that combines the second metadata indicating the first imaging device 11 and the second metadata indicating the second event, a second metadata set that combines the second metadata indicating the second imaging device 12 and the second metadata indicating the first event, and a second metadata set that combines the second metadata indicating the second imaging device 12 and the second metadata indicating the second event.
The image data acquisition section 130a acquires, for each of the four generated second metadata sets, a group of image data corresponding to the image metadata corresponding to each of the four second metadata sets.

The display image generating section 140a generates image group images representing each of the four image data groups acquired by the image data acquiring section 130a for each second metadata set.
Furthermore, the display image generating unit 140a generates a display image by arranging one or more generated image group images.

The priority acquisition unit 160a acquires the priority of each of the multiple second metadata acquired by the second metadata acquisition unit 120a. The priority acquisition unit 160a acquires the priority of the second metadata set by calculating the priority of the second metadata set using the priority of each of the multiple acquired second metadata. For example, the priority acquisition unit 160a acquires the priority of the second metadata set by adding up the priorities of two or more second metadata included in the second metadata set. The method of calculating the priority of the second metadata set is not limited to the method of adding up the priorities of two or more second metadata included in the second metadata set. The display image generation unit 140a may acquire the priority of the second metadata set by multiplying the priorities of two or more second metadata included in the second metadata set, or may calculate and acquire the priority of the second metadata set based on a predetermined calculation formula that uses the priorities of two or more second metadata included in the second metadata set as arguments.

The display image generation unit 140a may use the priority of the second metadata set based on the priority of the second metadata acquired by the priority acquisition unit 160a to determine the appearance of each of the one or more image group images to be generated, or the appearance in which the generated one or more image group images are arranged, and generate a display image in which the one or more image group images are arranged based on the determined appearance.
In addition, the display image generation unit 140a may use the priority of the second metadata set based on the priority of the second metadata acquired by the priority acquisition unit 160a and the maximum number acquired by the maximum number acquisition unit 170 to determine the appearance of each of the one or more image group images to be generated, or the appearance in which the generated one or more image group images are arranged, and generate a display image in which the one or more image group images are arranged based on the determined appearance.
The aspects of the one or more image group images and the aspect of arranging the one or more image group images have been described in the first embodiment, and therefore description thereof will be omitted.

The image data acquisition unit 130a may also select, from among the one or more second metadata sets generated by the image data acquisition unit 130a, second metadata sets up to a number equivalent to the maximum number acquired by the maximum number acquisition unit 170 in order of priority of the second metadata sets based on the priority of the second metadata acquired by the priority acquisition unit 160a, and acquire, for each second metadata set, a group of image data that is one or more image data corresponding to the image metadata corresponding to each of the selected second metadata sets.

The priority change unit 180a changes the priorities previously assigned to each of the multiple second metadata corresponding to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, among the priorities of each of the multiple second metadata acquired by the priority acquisition unit 160a.
More specifically, for example, the priority change unit 180a changes the priority associated with each of the multiple second metadata corresponding to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, which has been pre-registered in the metadata database of the metadata storage device 40, to a higher priority among the multiple second metadata acquired by the priority acquisition unit 160a.

The hardware configuration of the main parts of the image search device 100a is the same as that described with reference to Figures 5A and 5B in embodiment 1, and therefore will not be illustrated or described. That is, the functions of the operation acquisition unit 101, the first metadata acquisition unit 110a, the second metadata acquisition unit 120a, the image data acquisition unit 130a, the display image generation unit 140a, the selection group acquisition unit 150, the priority acquisition unit 160a, the maximum number acquisition unit 170, the priority change unit 180a, and the display image output unit 199 may be realized by the processor 501 and the memory 502, or may be realized by the processing circuit 503.

The operation of the image retrieval device 100a according to the second embodiment will be described with reference to FIGS. 8A and 8B.
Figures 8A and 8B are flowcharts illustrating an example of the processing of the image retrieval device 100a relating to embodiment 2, where Figure 8A shows a part of the processing of the image retrieval device 100a relating to embodiment 2 and Figure 8B shows the remainder of the processing of the image retrieval device 100a relating to embodiment 2.
The image search device 100a repeatedly executes the processes of the flowcharts shown in FIGS. 8A and 8B.

First, in step ST801, the operation acquisition section 101 acquires operation information.
Next, in step ST802, the first metadata acquisition section 110a decides whether or not the operation information is operation information that designates the first metadata.
A case where the first metadata acquisition section 110a has determined in step ST802 that the operation information is not operation information that designates the first metadata will be described later.
If the first metadata acquisition unit 110a determines in step ST802 that the operation information is operation information that specifies the first metadata, then in step ST803, the first metadata acquisition unit 110a acquires the first metadata from the operation information.
In the second embodiment, the first metadata acquisition unit 110a will be described as acquiring a plurality of pieces of first metadata. Note that the case where the first metadata acquisition unit 110a acquires one piece of first metadata will be omitted since it has been described in the first embodiment.

After step ST803, the image retrieval device 100a performs the following processes of steps ST811 and ST812, or the processes of steps ST811, ST821, and ST822, for each of all the first metadata.
In step ST811, the second metadata acquisition section 120a searches for the second metadata based on the first metadata, and determines whether or not the second metadata corresponding to the first metadata exists.
In step ST811, when the second metadata acquisition section 120a determines that there is no second metadata corresponding to the first metadata as a result of searching for the second metadata, in step ST821, the second metadata acquisition section 120a newly registers metadata corresponding to the first metadata in the metadata database of the metadata storage device 40. When the second metadata acquisition section 120a newly registers metadata, the second metadata acquisition section 120a may register the priority corresponding to the metadata as a predetermined value such as 0.
After step ST821, in step ST822, the second metadata acquisition section 120a acquires the first metadata as the second metadata.
In step ST811, if the second metadata acquisition unit 120a searches for the second metadata and determines that one or more pieces of second metadata corresponding to the first metadata exist, in step ST812, the second metadata acquisition unit 120a acquires the one or more pieces of second metadata.

After performing the process of step ST812 or the process of step ST822 for all the multiple pieces of first metadata, the image retrieval device 100a performs the process of step ST813.
In step ST813, the image data acquisition section 130a generates one or more second metadata sets.
After step ST813, in step ST831, the priority acquisition section 160a acquires a priority corresponding to each of the plurality of second metadata, and calculates a priority of each of the one or more second metadata sets.
After step ST831, in step ST832, the maximum number acquisition section 170 acquires the maximum number of image group images to be arranged in the display image.
After step ST832, in step ST833, the image data acquisition unit 130a acquires, for each second metadata set, an image data group that is one or more pieces of image data corresponding to image metadata corresponding to each of the one or more second metadata sets. Note that, in step ST833, the image data acquisition unit 130a may acquire an image data group for the second metadata sets up to the maximum number acquired by the maximum number acquisition unit 170 based on the priority acquired by the priority acquisition unit 160a.

After step ST833, in step ST834, the display image generating unit 140a generates an image group image representing each of the one or more image data groups acquired by the image data acquiring unit 130a. Note that in step ST834, the display image generating unit 140a may generate an image group image for an image data group up to the maximum number acquired by the maximum number acquiring unit 170, based on the priority acquired by the priority acquiring unit 160a.
After step ST834, in step ST835, the display image generating unit 140a generates a display image in which one or more image group images generated by the display image generating unit 140a in step ST834 are arranged. Note that in step ST835, the display image generating unit 140a may generate a display image in which image group images are arranged up to the maximum number acquired by the maximum number acquiring unit 170, based on the priority acquired by the priority acquiring unit 160a.

After step ST835, in step ST836, the display image output section 199 outputs the display image generated by the display image generating section 140a as display image information.
After step ST836, the image retrieval device 100a ends the process of the flowcharts shown in FIGS. 8A and 8B, and returns to the process of step ST801 to repeatedly execute the process of the flowcharts shown in FIGS. 8A and 8B.

A case where the first metadata acquisition section 110a has determined in step ST802 that the operation information is not operation information that designates the first metadata will be described.
In this case, in step ST841, the selection group acquiring section 150 determines whether or not the display image information most recently output by the display image output section 199 is display image information indicating a display image in which one or more image group images are arranged.
In step ST841, if the selection group acquisition unit 150 determines that the display image information output immediately before is not display image information indicating a display image in which one or more image group images are arranged, the image search device 100a terminates the processing of the flowchart shown in Figures 8A and 8B, and the image search device 100a returns to the processing of step ST801 and repeatedly executes the processing of the flowchart shown in Figures 8A and 8B.

In step ST841, if the selection group acquisition unit 150 determines that the display image information output immediately before is display image information indicating a display image in which one or more image group images are arranged, in step ST842, the selection group acquisition unit 150 determines whether the operation information is selection group information.
If, in step ST842, the selection group acquisition unit 150 determines that the operation information is not selection group information, the image search device 100a terminates the processing of the flowcharts shown in Figures 8A and 8B, and the image search device 100a returns to the processing of step ST801 and repeatedly executes the processing of the flowcharts shown in Figures 8A and 8B.

If the selection group acquiring section 150 determines in step ST842 that the operation information is selection group information, the selection group acquiring section 150 acquires the selection group information in step ST843.
After step ST843, in step ST844, the priority change unit 180a changes the priority that was previously assigned in association with the second metadata included in the second metadata set corresponding to the image metadata corresponding to the image data group indicated by the selection group information.

After step ST844, in step ST845, the display image generating section 140a generates a display image in which images indicated by one or more image data included in the image data group indicated by the selection group information are arranged.
After step ST845, in step ST846, the display image output section 199 outputs the display image generated by the display image generating section 140a as display image information.
After step ST846, the image retrieval device 100a ends the process of the flowchart shown in FIG. 8A and FIG. 8B, and returns to the process of step ST801 to repeatedly execute the process of the flowchart shown in FIG. 8A and FIG. 8B.

Embodiment 3.
An image retrieval device 100b according to the third embodiment will be described with reference to FIG. 9, FIG. 10A, and FIG. 10B.
An image retrieval system 1b (not shown) to which the image retrieval device 100b according to the third embodiment is applied is merely a system in which the image retrieval device 100 included in the image retrieval system 1 according to the first embodiment shown as an example in Fig. 1 is replaced with the image retrieval device 100b. Therefore, a description of a block diagram showing an example of the configuration of the main parts of the image retrieval system 1b according to the third embodiment will be omitted. Also, a description of the imaging device 10, the image storage device 20, the image storage device 30, the metadata storage device 40, the operation input device 50, the display device 60, and the network 70 included in the image retrieval system 1b in the same way as in the image retrieval system 1 according to the first embodiment will be omitted.

The configuration of the main parts of an image retrieval device 100b according to the third embodiment will be described with reference to FIG.
FIG. 9 is a block diagram showing an example of a configuration of a main part of an image retrieval device 100b according to the third embodiment.
The image search device 100b includes an operation acquisition unit 101, a first metadata acquisition unit 110, a second metadata acquisition unit 120b, an image data acquisition unit 130, a display image generation unit 140, a selection group acquisition unit 150, a priority acquisition unit 160b, a maximum number acquisition unit 170, a priority change unit 180b, a category acquisition unit 190, and a display image output unit 199.

Compared to the image retrieval device 100 of embodiment 1, the image retrieval device 100b has a category acquisition unit 190 that is not provided in the image retrieval device 100 of embodiment 1, and the second metadata acquisition unit 120, priority acquisition unit 160, and priority change unit 180 provided in the image retrieval device 100 of embodiment 1 have been changed to a second metadata acquisition unit 120b, a priority acquisition unit 160b, and a priority change unit 180b.
In the configuration of the image retrieval device 100b, the description of the configuration similar to that of the image retrieval device 100 according to the first embodiment will be omitted.
In the third embodiment, the image retrieval device 100b is described as a modification of the image retrieval device 100 according to the first embodiment, but may be a modification of the image retrieval device 100a according to the second embodiment. That is, compared to the image retrieval device 100a according to the second embodiment, the image retrieval device 100b may include a category acquisition unit 190 that is not included in the image retrieval device 100a according to the second embodiment, and the second metadata acquisition unit 120a, the priority acquisition unit 160a, and the priority change unit 180a included in the image retrieval device 100a according to the second embodiment may be changed to a second metadata acquisition unit 120b, a priority acquisition unit 160b, and a priority change unit 180b.

The category acquisition unit 190 acquires category information indicating a category classification, such as a user classification based on user identification information indicating each of one or more users who perform an input operation on the operation input device 50, an imaging device classification based on imaging device identification information indicating each of one or more imaging devices 10, or an imaging device group classification based on imaging device group identification information indicating each of one or more imaging device groups constituted by one or more imaging devices 10.
The above-mentioned user classification, imaging device classification, imaging device group classification, etc. are merely examples of category classification, and category classification is not limited to user classification, imaging device classification, imaging device group classification, etc. For example, when the imaging device 10 is a surveillance camera, the category classification may be surveillance area classification based on surveillance area identification information indicating each of one or more surveillance areas.
The category acquiring unit 190 acquires category information from, for example, the operation information acquired by the operation acquiring unit 101 .

Based on the first metadata acquired by the first metadata acquisition unit 110 and the category information acquired by the category acquisition unit 190, the second metadata acquisition unit 120b acquires, as second metadata, one or more pieces of metadata related to the first metadata that belong to a category classification indicated by the category information.

Specifically, for example, the second metadata acquisition unit 120b acquires one or more pieces of second metadata related to the first metadata based on the first metadata acquired by the first metadata acquisition unit 110 by reading them from the metadata storage device 40. The metadata storage device 40 includes, for example, metadata databases corresponding to one or more category classifications, and each of the metadata databases stores one or more pieces of metadata related to each other in association with one metadata group.
The second metadata acquisition unit 120b acquires one or more second metadata items related to the first metadata acquired by the first metadata acquisition unit 110 by reading them from the metadata storage device 40, using a metadata database corresponding to the category classification indicated by the category information acquired by the category acquisition unit 190.

For example, when searching for an image, the image desired by each user is different. For example, a metadata base corresponding to each user is prepared in advance, and the image search device 100b acquires the second metadata using the metadata base corresponding to the user who uses the image search device 100b, so that the user who uses the image search device 100b can easily find the desired image.
Furthermore, for example, a metadata database corresponding to each imaging device 10 is prepared in advance, and the image search device 100b acquires the second metadata using the metadata database corresponding to the imaging device 10 whose category classification is indicated by the category classification information, thereby making it easier for a user of the image search device 100b to find a desired image.
Furthermore, for example, a metadata database corresponding to each set of imaging devices may be prepared in advance, and the image search device 100b may acquire the second metadata using the metadata database corresponding to the set of imaging devices that is the category classification indicated by the category classification information, thereby making it easier for a user of the image search device 100b to find a desired image.

The priority acquisition unit 160b acquires the priority that has been assigned in advance to each of the one or more second metadata items acquired by the second metadata acquisition unit 120b by searching a metadata database corresponding to the category classification indicated by the category information.
The priority change unit 180b changes the priority associated with the second metadata equivalent to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit 150, which has been pre-registered in the metadata database corresponding to the category classification indicated by the category information, to a higher priority.

The hardware configuration of the main parts of the image search device 100b is the same as that described with reference to Figures 5A and 5B in embodiment 1, and therefore will not be illustrated or described. That is, the functions of the operation acquisition unit 101, the first metadata acquisition unit 110, the second metadata acquisition unit 120b, the image data acquisition unit 130, the display image generation unit 140, the selection group acquisition unit 150, the priority acquisition unit 160b, the maximum number acquisition unit 170, the priority change unit 180b, the category acquisition unit 190, and the display image output unit 199 may be realized by the processor 501 and the memory 502, or may be realized by the processing circuit 503.

The operation of the image retrieval device 100b according to the third embodiment will be described with reference to FIGS. 10A and 10B.
Figures 10A and 10B are flowcharts illustrating an example of the processing of the image retrieval device 100b relating to embodiment 3, where Figure 10A shows a part of the processing of the image retrieval device 100b relating to embodiment 3 and Figure 10B shows the remainder of the processing of the image retrieval device 100b relating to embodiment 3.
The image search device 100b repeatedly executes the processes of the flowcharts shown in FIGS. 10A and 10B.

First, in step ST1000, the category acquiring section 190 acquires category information.
After step ST1000, in step ST1001, operation acquisition section 101 acquires operation information.
Next, in step ST1002, first metadata acquisition section 110 decides whether or not the operation information is operation information that designates the first metadata.
A case where first metadata acquisition section 110 has determined in step ST1002 that the operation information is not operation information that designates the first metadata will be described later.
If, in step ST1002, the first metadata acquisition section 110 determines that the operation information is operation information that specifies the first metadata, in step ST1003, the first metadata acquisition section 110 acquires the first metadata from the operation information.

After step ST1003, in step ST1011, the second metadata acquisition unit 120b searches for second metadata based on the category information and the first metadata, and determines whether or not there is second metadata that corresponds to the first metadata and belongs to the category classification indicated by the category information.
In step ST1011, when the second metadata acquisition section 120b searches for the second metadata and determines that there is no second metadata that corresponds to the first metadata and belongs to the category classification, in step ST1021, the second metadata acquisition section 120b newly registers metadata corresponding to the first metadata in the metadata database corresponding to the category classification. Note that, when the second metadata acquisition section 120b newly registers metadata, it may register the priority corresponding to the metadata as a predetermined value such as 0.
After step ST1021, in step ST1022, the second metadata acquisition section 120b acquires the first metadata as the second metadata.
In step ST1011, if the second metadata acquisition unit 120b searches for second metadata and determines that there is one or more pieces of second metadata that correspond to the first metadata and belong to the category classification, then in step ST1012, the second metadata acquisition unit 120b acquires the one or more pieces of second metadata.

After step ST1012 or after step ST1022, in step ST1031, the priority acquisition section 160b acquires a priority corresponding to each of the one or more pieces of second metadata.
After step ST1031, in step ST1032, the maximum number acquisition section 170 acquires the maximum number of image group images to be arranged in the display image.
After step ST1032, in step ST1033, the image data acquisition unit 130 acquires, for each of the second metadata, an image data group that is one or more pieces of image data corresponding to the image metadata corresponding to each of the one or more pieces of second metadata. Note that, in step ST1033, the image data acquisition unit 130 may acquire an image data group for the second metadata up to the maximum number acquired by the maximum number acquisition unit 170 based on the priority acquired by the priority acquisition unit 160b.

After step ST1033, in step ST1034, the display image generating unit 140 generates an image group image representing each of the one or more image data groups acquired by the image data acquiring unit 130. Note that in step ST1034, the display image generating unit 140 may generate an image group image for an image data group whose upper limit is the maximum number acquired by the maximum number acquiring unit 170, based on the priority acquired by the priority acquiring unit 160b.
After step ST1034, in step ST1035, the display image generating unit 140 generates a display image in which one or more image group images generated by the display image generating unit 140 in step ST1034 are arranged. Note that in step ST1035, the display image generating unit 140 may generate a display image in which image group images are arranged up to the maximum number acquired by the maximum number acquiring unit 170, based on the priority acquired by the priority acquiring unit 160b.

After step ST1035, in step ST1036, the display image output section 199 outputs the display image generated by the display image generating section 140 as display image information.
After step ST1036, the image retrieval device 100b ends the process of the flowcharts shown in FIGS. 10A and 10B, and returns to the process of step ST1001 to repeatedly execute the process of the flowcharts shown in FIGS. 10A and 10B.

A case where first metadata acquisition section 110 has determined in step ST1002 that the operation information is not operation information that designates the first metadata will be described.
In this case, in step ST1041, the selection group acquiring section 150 determines whether or not the display image information most recently output by the display image output section 199 is display image information indicating a display image in which one or more image group images are arranged.
In step ST1041, if the selection group acquisition unit 150 determines that the display image information output immediately before is not display image information indicating a display image in which one or more image group images are arranged, the image search device 100b terminates the processing of the flowcharts shown in Figures 10A and 10B, and the image search device 100b returns to the processing of step ST1001 and repeatedly executes the processing of the flowcharts shown in Figures 10A and 10B.

In step ST1041, if the selection group acquisition unit 150 determines that the display image information output immediately before is display image information indicating a display image in which one or more image group images are arranged, in step ST1042, the selection group acquisition unit 150 determines whether the operation information is selection group information.
If, in step ST1042, the selection group acquisition unit 150 determines that the operation information is not selection group information, the image search device 100b terminates the processing of the flowcharts shown in Figures 10A and 10B, and the image search device 100b returns to the processing of step ST1001 and repeatedly executes the processing of the flowcharts shown in Figures 10A and 10B.

If the selection group acquisition section 150 determines in step ST1042 that the operation information is selection group information, the selection group acquisition section 150 acquires the selection group information in step ST1043.
After step ST1043, in step ST1044, the priority change unit 180b changes the priority that has been previously assigned in association with second metadata that corresponds to the image metadata corresponding to the image data group indicated by the selection group information and belongs to the category classification indicated by the category information.

After step ST1044, in step ST1045, the display image generating section 140 generates a display image in which images indicated by one or more image data included in the image data group indicated by the selection group information are arranged.
After step ST1045, in step ST1046, the display image output section 199 outputs the display image generated by the display image generating section 140 as display image information.
After step ST1046, the image retrieval device 100b ends the process of the flowcharts shown in FIGS. 10A and 10B, and returns to the process of step ST1001 to repeatedly execute the process of the flowcharts shown in FIGS. 10A and 10B.

As described above, the image search device 100b includes an operation acquisition unit 101 that acquires operation information based on a user operation, a first metadata acquisition unit 110 that acquires first metadata from the operation information acquired by the operation acquisition unit 101, a second metadata acquisition unit 120b that acquires one or more second metadata related to the first metadata based on the first metadata acquired by the first metadata acquisition unit 110, an image data acquisition unit 130 that acquires, for each second metadata, an image data group that is one or more image data corresponding to image metadata corresponding to each of the second metadata based on the one or more second metadata acquired by the second metadata acquisition unit 120b, and a display image generation unit 140 that generates a display image, The device includes a display image generating unit 140 that generates an image group image representing each of the one or more image data groups acquired by the data acquiring unit 130 and generates a display image by arranging the one or more generated image group images, a display image output unit 199 that outputs the display image generated by the display image generating unit 140 as display image information, and a category acquiring unit 190 that acquires category information indicating a category classification, and the second metadata acquiring unit 120b is configured to acquire, as second metadata, one or more metadata belonging to a category classification indicated by the category information, among metadata related to the first metadata, based on the first metadata acquired by the first metadata acquiring unit 110 and the category information acquired by the category acquiring unit 190.

With this configuration, the image search device 100b can search not only for images that match the initially input search criteria, but also for related images. Therefore, the image search device 100b can search for a desired image even if the initially input search criteria are not accurate. Therefore, a user can search for a desired image even if the initially input search criteria are not accurate.
In addition, when acquiring the second metadata, the image search device 100b acquires as the second metadata one or more pieces of metadata related to the first metadata that belong to the category classification indicated by the category information, thereby making it easier for the user to find the desired image.

In addition, within the scope of the invention, this disclosure allows for free combination of each embodiment, modification of any component of each embodiment, or omission of any component of each embodiment.

1, 1a, 1b Image search system, 10 Imaging device, 11 First imaging device, 12 Second imaging device, 20 Image storage device, 30 Image storage device, 40 Metadata storage device, 50 Operation input device, 60 Display device, 70 Network, 100, 100a, 100b Image search device, 101 Operation acquisition unit, 110, 110a First metadata acquisition unit, 120, 120a, 120b Second metadata acquisition unit, 130, 130a Image data acquisition unit, 140, 140a Display image generation unit, 150 Selection group acquisition unit, 160, 160a, 160b Priority acquisition unit, 170 Maximum number acquisition unit, 180, 180a, 180b Priority change unit, 190 Category acquisition unit, 199 Display image output unit, 200 Display image, 201, 202, 203, 204, 205 image group image, 501 processor, 502 memory, 503 processing circuit.

Claims (8)

an operation acquisition unit that acquires operation information based on a user operation;
a first metadata acquisition unit that acquires first metadata from the operation information acquired by the operation acquisition unit;
a second metadata acquisition unit that acquires one or more pieces of second metadata related to the first metadata based on the first metadata acquired by the first metadata acquisition unit;
an image data acquisition unit that acquires, for each of the second metadata, an image data group that is one or more image data corresponding to image metadata corresponding to each of the second metadata, based on the one or more second metadata acquired by the second metadata acquisition unit;
a display image generating unit that generates a display image, the display image generating unit generating an image group image representing each of the one or more image data groups acquired by the image data acquiring unit, and generating the display image by arranging the generated one or more image group images;
a display image output unit that outputs the display image generated by the display image generation unit as display image information;
a category acquisition unit that acquires category information indicating a category classification;
a selection group acquisition unit that acquires, from the operation information acquired by the operation acquisition unit, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit is selected;
a priority acquisition unit that acquires a priority level that is assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition unit;
a priority change unit that changes the priority acquired by the priority acquisition unit,
the second metadata acquisition unit acquires, as the second metadata, one or more pieces of metadata that belong to the category classification indicated by the category information, among pieces of metadata related to the first metadata, based on the first metadata acquired by the first metadata acquisition unit and the category information acquired by the category acquisition unit;
The image search device is characterized in that the priority change unit changes the priority that has been previously assigned in association with the second metadata, which corresponds to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit, among the one or more priorities acquired by the priority acquisition unit. a selection group acquisition unit that acquires, from the operation information acquired by the operation acquisition unit, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit is selected;
The image search device according to claim 1 , wherein the display image generation unit generates the display image by arranging images represented by one or more of the image data included in the image data group indicated by the selection group information acquired by the selection group acquisition unit. a priority acquisition unit that acquires a priority level that is assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition unit;
The image search device of claim 1, wherein the display image generation unit uses the priority acquired by the priority acquisition unit to determine a configuration for each of the one or more image group images to be generated, or a configuration for arranging the one or more generated image group images, and generates the display image in which the one or more image group images are arranged based on the determined configuration. a priority acquisition unit that acquires a priority level that is assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition unit;
a maximum number acquisition unit that acquires a maximum number of the image group images to be arranged in the display image generated by the display image generation unit;
The image search device of claim 1, wherein the display image generation unit determines a manner in which each of the one or more image group images to be generated, or a manner in which the one or more generated image group images are arranged, using the priority acquired by the priority acquisition unit and the maximum number acquired by the maximum number acquisition unit, and generates the display image in which the one or more image group images are arranged based on the determined manner. a priority acquisition unit that acquires a priority level that is assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition unit;
a maximum number acquisition unit that acquires a maximum number of the image group images to be arranged in the display image generated by the display image generation unit;
Equipped with
The image search device of claim 1, wherein the image data acquisition unit selects, from the one or more second metadata acquired by the second metadata acquisition unit, a number of second metadata up to the maximum number acquired by the maximum number acquisition unit, in order of the highest priority acquired by the priority acquisition unit, and acquires, for each of the selected second metadata, the image data group being one or more image data corresponding to the image metadata corresponding to each of the selected second metadata. a selection group acquisition unit that acquires, from the operation information acquired by the operation acquisition unit, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit is selected;
a priority acquisition unit that acquires a priority level that is assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition unit;
a priority change unit that changes the priority acquired by the priority acquisition unit;
Equipped with
The image search device of claim 1, wherein the priority change unit changes the priority that is pre-assigned in association with the second metadata corresponding to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit, among the one or more priorities acquired by the priority acquisition unit. an operation acquiring step in which an operation acquiring unit acquires operation information based on a user operation;
a first metadata acquisition step of acquiring first metadata from the operation information acquired by the operation acquisition unit by a first metadata acquisition unit;
a second metadata acquisition step of acquiring one or more pieces of second metadata related to the first metadata based on the first metadata acquired by the first metadata acquisition step;
an image data acquisition step in which an image data acquisition unit acquires, for each of the second metadata, an image data group which is one or more image data corresponding to image metadata corresponding to each of the second metadata, based on the second metadata acquired by the second metadata acquisition unit;
a display image generating step in which a display image generating unit generates a display image, the display image generating step generating an image group image representing each of the one or more image data groups acquired by the image data acquiring unit, and generating the display image by arranging the generated one or more image group images;
a display image output step in which a display image output unit outputs the display image generated by the display image generation unit as display image information;
a category acquisition step in which a category acquisition unit acquires category information indicating a category classification;
a selection group acquisition unit acquiring, from the operation information acquired by the operation acquisition unit, selection group information indicating which of the one or more image data groups acquired by the image data acquisition unit is selected;
A priority acquisition unit acquires a priority level that is assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition unit;
a priority change unit changing the priority acquired by the priority acquisition unit;
the second metadata acquisition unit acquires, as the second metadata, one or more pieces of metadata that belong to the category classification indicated by the category information, among pieces of metadata related to the first metadata, based on the first metadata acquired by the first metadata acquisition unit and the category information acquired by the category acquisition unit;
The image search method is characterized in that the priority change unit changes the priority that has been previously assigned in association with the second metadata, which corresponds to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition unit, among the one or more priorities acquired by the priority acquisition unit. On the computer,
an operation acquisition function for acquiring operation information based on a user operation;
a first metadata acquisition function that acquires first metadata from the operation information acquired by the operation acquisition function;
a second metadata acquisition function that acquires one or more second metadata items related to the first metadata based on the first metadata acquired by the first search and acquisition function;
an image data acquisition function that acquires, for each of the second metadata, an image data group that is one or more image data corresponding to image metadata corresponding to each of the second metadata, based on the second metadata acquired by the second metadata acquisition function;
a display image generating function for generating a display image, the display image generating function generating an image group image representing each of the one or more image data groups acquired by the image data acquiring function, and generating the display image by arranging the generated one or more image group images;
a display image output function that outputs the display image generated by the display image generating function as display image information;
A category acquisition function for acquiring category information indicating a category classification;
a selection group acquisition function that acquires, from the operation information acquired by the operation acquisition function, selection group information indicating which of the one or more image data groups acquired by the image data acquisition function is selected;
a priority acquisition function that acquires a priority assigned in advance in association with each of the one or more pieces of second metadata acquired by the second metadata acquisition function;
a priority change function for changing the priority acquired by the priority acquisition function,
the second metadata acquisition function acquires, based on the first metadata acquired by the first metadata acquisition function and the category information acquired by the category acquisition function, one or more pieces of metadata related to the first metadata, the one or more pieces of metadata belonging to the category classification indicated by the category information, as the second metadata;
The priority change function of the image search program is characterized in that it changes the priority that has been previously assigned in association with the second metadata, which corresponds to the image metadata corresponding to the image data group indicated by the selection group information acquired by the selection group acquisition function, out of the one or more priorities acquired by the priority acquisition function.

Images