KR20200093321A - Image search apparatus and method - Google Patents

Abstract

According to one embodiment, disclosed is an image searching device capable of easily allocating attribute tags to images stored in a database, which comprises the steps of: obtaining reference feature vectors corresponding to a plurality of reference images including the same attribute; obtaining a representative feature vector corresponding to the same attribute from a trained generation model based on the obtained reference feature vectors; and comparing the feature vectors of target images stored in the database with the representative feature vector, and allocating an attribute tag corresponding to the same attribute to at least some of the target images.

Description

Image search device and method {IMAGE SEARCH APPARATUS AND METHOD}

This disclosure relates to the field of search. More specifically, the present disclosure relates to an image-based search apparatus and method.

With the development of information and communication technology, information retrieval using various terminals such as smart phones and personal computers is widely used. Conventionally, a document or the like has been searched and provided based on text input by a user, but in recent years, a user has increased the desire to search for an image such as a picture or a picture.

However, it is not easy to search for images such as clothing and landscapes using only text. For example, even when a user wants to search for clothing in a style desired by a user, it is sometimes difficult to find an appropriate image based on a text search.

Recently, a service that receives a search image rather than text from a user and searches for an image similar to the search image has been attempted, but for this, there is a difficulty in manually tagging numerous images stored in a database.

An image retrieval apparatus and method according to an embodiment of the present invention is to make it easy to allocate attribute tags to images stored in a database.

In addition, an image search apparatus and method according to an embodiment of the present invention make it a technical task to enable a user to more accurately search a desired type of image using a representative feature vector.

An image search method according to an embodiment may include obtaining reference feature vectors corresponding to a plurality of reference images including the same attribute; Obtaining a representative feature vector corresponding to the same attribute from a trained generation model based on the obtained reference feature vectors; And comparing the feature vectors of target images stored in a database with the representative feature vector, and assigning an attribute tag corresponding to the same attribute to a target image of at least some of the target images.

An image search method according to an embodiment may include receiving a plurality of search images from a user terminal; Obtaining feature vectors corresponding to the plurality of search images; Training a generation model with the acquired feature vectors; Determining a target image corresponding to the plurality of search images by comparing a representative feature vector output from the generation model trained with the feature vectors and feature vectors of target images stored in a database; And transmitting the determined target image to the user terminal.

An image search apparatus according to an embodiment includes a processor; A memory storing at least one program and generation model; And a database storing target images, wherein the processor acquires reference feature vectors corresponding to a plurality of reference images including the same attribute as the at least one program is executed, and the obtained reference feature vectors Obtaining a representative feature vector corresponding to the same attribute from the generation model trained based on the fields, comparing the feature vectors of the target images with the representative feature vector, to a target image of at least some of the target images An attribute tag corresponding to the same attribute may be assigned.

An image search method according to another embodiment may include accessing an image search apparatus and selecting an attribute; And receiving, from the image search apparatus, a target image assigned with an attribute tag corresponding to the selected attribute among target images stored in the image search apparatus, wherein the target images stored in the image search apparatus include a plurality of attributes. At least some attribute tags among a plurality of attribute tags corresponding to the plurality of attributes may be assigned based on a comparison result of representative feature vectors corresponding to each of the feature vectors and the feature vectors of the target images.

The image retrieval apparatus and method according to an embodiment may simplify the assignment of attribute tags to images stored in a database.

In addition, the image search apparatus and method according to an embodiment may allow a user to more accurately search a desired type of image using a representative feature vector.

However, the effects that the image retrieval apparatus and method according to an embodiment can achieve are not limited to those mentioned above, and other effects not mentioned are common knowledge in the art to which the present disclosure pertains from the following description. It can be clearly understood by those who have.

Brief descriptions of each drawing are provided in order to better understand the drawings cited herein.
1 is a diagram illustrating an image search apparatus and a user terminal according to an embodiment.
2 is a flowchart illustrating an attribute tag allocation method according to an embodiment.
3 is a diagram for explaining a method of obtaining a representative feature vector from a generation model.
4 is an exemplary diagram for explaining a method of assigning an attribute tag to a target image.
5 is a diagram illustrating a UI screen that allows a user to select an attribute.
6 is a flowchart illustrating an image search method according to an embodiment.
7 is a flowchart illustrating an image search method according to another embodiment.
8 is a diagram illustrating feature vectors for each attribute extracted from a search image.
9 is a view for explaining a method of exchanging at least one of the feature vectors for each attribute of the search image into a representative feature vector.
10 is a view for explaining a method of comparing a feature vector for each attribute of a search image and a feature vector for each attribute of the target images.
11 is an exemplary diagram illustrating a search image including a plurality of objects.
FIG. 12 is a diagram illustrating a method of exchanging at least one of the feature vector sequences of the search image shown in FIG. 11 with a representative feature vector.
13 is an exemplary diagram illustrating search result images corresponding to the search image shown in FIG. 11.
14 is a block diagram showing the configuration of an image search apparatus according to an embodiment.

The present disclosure may be variously modified and may have various embodiments, and specific embodiments are illustrated in the drawings and will be described through detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that all modifications, equivalents, and substitutes included in the spirit and scope of the present disclosure are included.

In describing the embodiments, when it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter, the detailed description is omitted. In addition, the numbers used in the description process of the embodiment (for example, first, second, etc.) are only identification symbols for distinguishing one component from other components.

In addition, when one component is referred to as “connected” or “connected” with another component in the present specification, the one component may be directly connected to the other component, or may be directly connected, but is specifically opposed. It should be understood that, as long as there is no description to be made, it may be connected or connected through another component in the middle.

In addition, in this specification, two or more components are expressed as'~ unit (unit)' or'module', or two or more components are combined into one component, or two or more components are divided into more detailed functions. It may be differentiated into. In addition, each of the components to be described below may additionally perform some or all of the functions of other components in addition to the main functions in charge of them, and some of the main functions of each component are different. Needless to say, it may also be carried out exclusively by components.

Hereinafter, embodiments according to the technical spirit of the present disclosure will be described in detail.

1 is a diagram illustrating an image search apparatus 100 and a user terminal 10 according to an embodiment.

The user terminal 10 selects or inputs a search condition (for example, a search image, an attribute, etc.) to the image search apparatus 100 and transmits it, and the image search apparatus 100 transmits a search result image corresponding to the search condition. Transmit to the user terminal 10. In one embodiment, the image search apparatus 100 may store target images for determining a search result image corresponding to a search condition in a database. The image search apparatus 100 may transmit the target image corresponding to the search condition among the target images to the user terminal 10 as a search result image.

The user terminal 10 may include, but is not limited to, a desktop PC, tablet, smartphone, laptop, etc. that can access the image retrieval device 100 through a network.

In one embodiment, the image search apparatus 100 and the user terminal 10 are connected to each other through a network to transmit and receive data between each other. Here, the network may include a wired network and a wireless network, and specifically, various networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). It may include. In addition, the network 10 may include a known World Wide Web (WWW). However, the network 10 according to the present invention is not limited to the networks listed above, and may include at least a part of a known wireless data network, a known telephone network, and a known wired/wireless television network.

The image retrieval apparatus 100 according to an embodiment may assign attribute tags to pre-stored target images. In the present specification,'attribute' may mean characteristics of an object included in an image. The attributes may be classified by category. For example, some clothing may have red attributes belonging to the color category, striped attributes belonging to the pattern category, and dress attributes belonging to the shape category. Also, in this specification, the'attribute tag' may mean information allocated to an image to indicate characteristics of an object in the image. For example, an attribute tag of a garment may indicate that the color of the garment is red, the pattern is a stripe, and the shape is a dress.

A method in which the image search apparatus 100 assigns attribute tags to target images will be described with reference to FIG. 2.

2 is a flowchart illustrating an attribute tag allocation method according to an embodiment.

In step S210, the image search apparatus 100 obtains reference feature vectors corresponding to a plurality of reference images having the same attribute or assigned the same attribute tag. In an embodiment, the administrator may assign the same attribute tag to multiple reference images. For example, the same attribute tag may be assigned to a plurality of reference images, such as'red','comb', or'clothes popular with women'.

Reference feature vectors corresponding to a plurality of reference images may be output by a machine learning model. The machine learning model may include, for example, a convolution nueral network (CNN). Also, the reference feature vectors may be N (N is an integer of 1 or more) dimension data. In one embodiment, reference feature vectors corresponding to a plurality of reference images may be generated by post processing after being output by a machine learning model. The post-processing, for example, after sorting a plurality of elements included in the feature vector output from the machine learning model by size, obtains an index indicating the position of the top k elements in the feature vector, and obtains the obtained And generating an index feature vector having an index as an element as a reference feature vector.

In step S220, the image search apparatus 100 obtains a representative feature vector corresponding to the same attribute or the same attribute tag from a generated model trained based on reference feature vectors corresponding to a plurality of reference images. The generation model can be trained based on reference feature vectors of a plurality of reference images, thereby generating a representative feature vector similar to reference feature vectors. In one embodiment, the generation model may include a GAN (generative adversarial network).

Referring to FIG. 3, the generation model 300 is trained by reference feature vectors 320a, 320b, and 320c corresponding to the reference images 310a, 310b, 310c, and then input random noise, for example. When input as data, representative feature vectors similar to the reference feature vectors 320a, 320b, and 320c may be generated. In one embodiment, the GAN corresponding to the generation model 300 may consist of a plurality of fully connected layers (FCLs). FCL is a network structure in which each layer is composed of at least one node, and each node of a layer is individually connected to all nodes included in a previous layer and a next layer. Since the GAN is trained by the reference feature vector, not the image, it is not necessary to include the CNN, and only the FCL can generate a representative feature vector similar to the reference feature vector. That is, instead of a GAN for image generation, a GAN composed of a plurality of FCLs for generation of a representative feature vector is used, so that training can be quickly performed and processing speed can also be very fast.

In one embodiment, the generation model may obtain an average value of the elements of the reference feature vectors, and generate a representative feature vector having the obtained average values as elements.

Referring back to FIG. 2, in step S230, the image search apparatus 100 compares the feature vectors of the target images stored in the database with the representative feature vector. In one example, the image search apparatus 100 may calculate a distance (eg, a cosine distance) between each of the feature vectors of the target images and the representative feature vector. The distance calculated to determine the similarity between the feature vectors is well known, and thus detailed description is omitted herein.

In step S240, the image search apparatus 100 assigns an attribute tag corresponding to the same attribute of the plurality of reference images to at least some of the target images. In one embodiment, the image retrieval apparatus 100 may assign an attribute tag corresponding to the same attribute to a target image corresponding to a feature vector having a distance equal to or less than a predetermined value compared to a representative feature vector.

FIG. 4 is a diagram for explaining a method of assigning an attribute tag to a target image. Referring to FIG. 4, feature vectors 410b, 420b, and 430b of the target images 410a, 420a, and 430a, respectively, and representative feature vectors According to the result of comparison, the attribute tags 410c and 420c of the'zigzag pattern' may be assigned to the first target image 410a and the second target image 420a, and the'Burgundy color' may be assigned to the third target image 430a. The attribute tag 430c of 'may be assigned. 4 illustrates that one attribute tag is assigned to one target image, but a plurality of attribute tags may be assigned to one target image.

According to an embodiment, the attribute tag can be assigned to a large amount of target images based on a relatively small number of reference images, so an administrator needs to assign the attribute tag to target images stored in the database one by one. Will disappear.

In addition, according to an embodiment, even when a need to allocate a new attribute tag to target images occurs due to development, fashion, etc. of a new attribute, the new attribute tag can be quickly assigned to target images. For example, if a small number of reference images having new attributes are selected by the administrator, the image retrieval apparatus 100 trains a generation model with reference feature vectors of the selected reference images, and a representative feature vector output from the generation model. According to the comparison result of the feature vectors of the target images, an attribute tag corresponding to a new attribute may be assigned to the target images.

5 is a diagram illustrating a UI screen 500 that allows a user to select an attribute.

As described above, when the attribute tag is assigned to target images stored in the database, the user can easily search for an image by simply selecting a specific attribute on the UI screen 500 displayed on the display of the user terminal 10. .

Referring to FIG. 5, when the search screen 500 is displayed on the display of the user terminal 10, the user searches the search screen 500 for various properties 530 belonging to a specific category (eg, pattern) 520. By selecting some (for example, check), target images to which attribute tags corresponding to corresponding attributes are assigned may be received as a search result.

The image search apparatus 100 may transmit the target image to which the tag corresponding to the attribute selected by the user terminal 10 among the target images stored in the database is assigned as a search result image to the user terminal 10.

Depending on the implementation, the user may input the search word 510 as an attribute on the search screen 500. The image search apparatus 100 may transmit a target image having an attribute tag related to an attribute corresponding to the search word 510 among target images stored in the database as the search result image to the user terminal 10.

In one embodiment, the image search apparatus 100 compares the feature vector corresponding to the attribute selected by the user terminal 10 with the feature vectors of target images stored in the database and compares the feature vector with the feature vector identical or similar to the representative feature vector. The corresponding target image may be transmitted to the user terminal 10 as a search result image. The image search apparatus 100 may transmit a target image corresponding to a feature vector having a distance equal to or less than a predetermined value between the representative feature vectors to the user terminal 10 as a search result image.

The feature vectors to be compared with the representative feature vectors may vary depending on what kind of machine learning model outputs the reference feature vectors used to train the generation model that outputs the representative feature vectors. For example, when a reference feature vector used to train a generation model is output by a dependent model, feature vectors of target images to be compared may also be output by a dependent model. The dependent model may refer to a machine learning model trained to output feature vectors corresponding to attributes of a specific category. For example, the dependent model may include a color category-only model, a pattern category-only model, and the like. When the administrator wants to assign a tag of a color attribute to target images, the reference images are input to the color category-specific model to obtain reference feature vectors corresponding to the reference images, and the generated model is generated from the obtained reference feature vectors. You can train.

In addition, when the feature vector used to train the generation model is output by an independent model, feature vectors of target images to be compared may also be output by the independent model. The independent model may refer to a machine learning model trained to output a feature vector independent of the attribute category.

6 is a flowchart illustrating an image search method according to an embodiment.

In step S610, the image search apparatus 100 receives a plurality of search images from the user terminal 10. The image search apparatus 100 may directly receive a plurality of search images from the user terminal 10 through a network, and information (eg, a plurality of search images) that can specify a plurality of search images from the user terminal 10. A URL in which search images are stored, identification information of a plurality of search images, and the like) may be received, and a plurality of search images may be obtained based on the received information.

In step S620, the image search apparatus 100 acquires feature vectors corresponding to a plurality of search images. The image search apparatus 100 may input a plurality of search images into the machine learning model and obtain feature vectors output from the machine learning model. The machine learning model may be a dependent model or an independent model.

In step S630, the image search apparatus 100 trains the generation model with feature vectors corresponding to a plurality of search images. When the generation model is GAN, the GAN may be trained to generate data similar to the training data based on the interaction of the generator and the discriminator.

In step S640, the image search apparatus 100 compares the representative feature vectors output from the generation model trained with the feature vectors of the plurality of search images and the feature vectors of the target images. The image search apparatus 100 may calculate the distance between each of the feature vectors of the target images and the representative feature vector.

As described above, the feature vectors to be compared with the representative feature vectors may vary depending on what kind of machine learning model the feature vectors used to train the generation model that outputs the representative feature vectors.

In step S650, the image search apparatus 100 transmits a target image corresponding to a plurality of search images among target images stored in the database to the user terminal 10 as a search result image.

The image search apparatus 100 may transmit a target image corresponding to a feature vector having a distance equal to or less than a predetermined value between the representative feature vectors to the user terminal 10 as a search result image.

According to the embodiment illustrated in FIG. 6, for example, when a user selects a plurality of search images matching his/her preference, the image search apparatus 100 may image other than search images matching the user's preference You can provide them as search results.

7 is a flowchart illustrating an image search method according to another embodiment.

In step S710, the image search apparatus 100 receives the search image from the user terminal 10. The image search apparatus 100 may directly receive a search image from the user terminal 10 through a network, or information that can specify a search image from the user terminal 10 (eg, a URL in which the search image is stored, The identification information of the search image, etc.) may be received, and the search image may be obtained based on the received information.

In step S720, the image search apparatus 100 obtains a feature vector for each attribute of the search image. The image search apparatus 100 may obtain feature vectors output from a plurality of dependent models by inputting a search image into each of the plurality of dependent models. For example, as shown in FIG. 8, the search image 810 is input to the first dependent model 821 to obtain a feature vector 831 corresponding to a first attribute (eg, color attribute), and , By inputting the search image 810 as the second dependent model 823 to obtain a feature vector 833 corresponding to the second attribute (eg, pattern attribute), and obtaining the search image 810 as the third dependent model It is possible to obtain a feature vector 835 corresponding to a third attribute (eg, shape attribute) by inputting it at 825.

In one embodiment, when there are two search images received from the user terminal 10, one of the search images is input to the first dependent model 821 to obtain a feature vector corresponding to the first attribute, and the other one The search images of can be input to the second dependent model 823 and the third dependent model 825 to obtain a feature vector corresponding to the second attribute and a feature vector corresponding to the third attribute.

In another embodiment, if there are three search images received from the user terminal 10, one of the search images is input to the first dependent model 821 to obtain a feature vector corresponding to the first attribute, and the other one A feature image corresponding to the second attribute is obtained by inputting the search image of the second dependency model 823, and another feature of the search image is input to the third dependency model 825 to correspond to the third attribute. Vectors can also be obtained.

That is, the image search apparatus 100 may obtain a feature vector for each attribute by combining feature vectors corresponding to a plurality of search images.

8 shows that three dependent models 821, 823, and 825 are used to extract feature vectors for each attribute, but this is only an example, and extracting feature vectors of various numbers of attributes from a search image Various number of dependent models for this can be used.

Referring back to FIG. 7, in step S730, the image retrieval apparatus 100 represents a feature vector of the attribute selected by the user terminal 10 among the feature vectors for each attribute, and a representative corresponding to the attribute selected by the user terminal 10 Change to a feature vector. The user can select an attribute to be changed to a representative feature vector through the user terminal 10. The user terminal 10 may select an attribute (eg, color attribute) to be changed among attributes of objects included in the search image and other attributes to replace the attribute to be changed. In response to this, the image search apparatus 100 may change the feature vector of the attribute selected by the user terminal 10 among the feature vectors for each attribute to a representative feature vector corresponding to other attributes.

For example, as illustrated in FIG. 9, among the feature vector 831 of the first attribute, the feature vector 833 of the second attribute, and the feature vector 835 of the third attribute, by the user terminal 10 The selected feature vector 833 of the second attribute can be exchanged for a representative feature vector 933. Here, the representative feature vector 933 may be a feature vector output from a generation model trained as reference feature vectors output from the second attribute-only dependent model 823.

In operation S740, the image search apparatus 100 compares a feature vector for each attribute including a representative feature vector and a feature vector for each attribute of target images stored in the database. Here, the feature vector for each attribute of target images stored in the database may be output from a dependent model.

In one embodiment, the image retrieval apparatus 100, as shown in FIG. 10, feature-specific feature vectors 831, 933, and 835 including representative feature vectors 933 and feature-specific features of target images stored in the database In comparing the vectors 1031, 1033, and 1035, it is possible to obtain a distance for each attribute by comparing feature vectors of attributes belonging to the same category. The image search apparatus 100 calculates a distance corresponding to the first attribute, a distance corresponding to the second attribute, and a distance corresponding to the third attribute, for each feature vector 1031, 1033, and 1035 for each attribute of the target images can do.

In step S750, the image search apparatus 100 may determine a target image to be transmitted to the user terminal 10 among target images stored in the database based on the calculated distance for each attribute.

The image retrieval apparatus 100 may determine a target image identical and similar to the feature vector for each attribute including a representative feature vector among target images stored in the database based on a result of combining the distance for each attribute. In one example, the image retrieval apparatus 100 may select a predetermined number of target images in the order in which the result values of the combination of distances for each attribute are large, or, as a result of combining the distances for each attribute, a target image having a value greater than or equal to a preset value You can also choose

In step S760, the image search apparatus 100 transmits the target image selected in step S750 to the user terminal 10.

According to the embodiment shown in FIG. 7, the user transmits a search image to the image search apparatus 100 and receives a search result corresponding to the changed attribute by changing a specific attribute of an object included in the search image to another attribute can do. For example, the user may transmit a search image including red and striped dresses to the image search apparatus 100, and select blue instead of red to receive images containing blue and striped dresses as search results. will be.

11 is an exemplary diagram illustrating a search image 1100 including a plurality of objects 1110, 1130, and 1150.

In one embodiment, the search image 1100 transmitted from the user terminal 10 to the image search apparatus 100 may include a plurality of objects 1110, 1130, and 1150 as shown in FIG. In FIG. 11, the plurality of objects 1110, 1130, and 1150 include an upper object 1110, a lower object 1130, and a bag object 1150. In this case, the image search apparatus 100 crops partial images corresponding to each of the plurality of objects 1110, 1130, and 1150, inputs them to the dependent model, and the properties of each object output from the dependent model Feature vectors can be obtained. For example, the image retrieval apparatus 100 may obtain a feature vector for each attribute of the upper object 1110, a feature vector for each attribute of the lower object 1130, and a bag object 1150 from the search image 1100 illustrated in FIG. 11. A feature vector for each attribute can be obtained.

The image retrieval apparatus 100 identifies the object selected by the user terminal 10 in a feature vector sequence including a feature vector for each attribute of a plurality of objects 1110, 1130, and 1150 included in the search image 1100 The feature vector of the attribute may be changed to a representative feature vector corresponding to the attribute selected by the user terminal 10. For example, when the user selects the bag object 1150 from the search image 1100 illustrated in FIG. 11 and wants to change the color attribute of the bag to another color, the image search device 100 displays the bag object 1150 ), a representative feature vector corresponding to a color to be changed by a user can be included in the feature vector sequence instead of the feature vector of the color attribute.

For example, as illustrated in FIG. 12, the third of the feature vector 1210 for each attribute of the first object, the feature vector 1220 for each attribute of the second object, and the feature vector 1230 for each attribute of the third object. The feature vector 1231 of the first attribute of the object 1230 may be exchanged for a representative feature vector 1241.

The image search apparatus 100 may determine a target image corresponding to the search image 1100 by comparing the feature vector sequence 1250 including the representative feature vector 1241 with the feature vector sequences of target images stored in the database. have. In comparing the feature vector sequence 1250 including the representative feature vector 1241 and the feature vector sequence of target images, the image search apparatus 100 compares feature vectors of attributes of the same category of the same object, and compares each The distance for each attribute of the object may be calculated, and a target image corresponding to the search result may be determined based on the calculated distance. In one embodiment, the image retrieval apparatus 100 combines the feature vector sequence 1250 including the representative feature vector 1241 and the feature-specific distance for each object calculated as a result of comparing the feature vector sequence of the target images , A target image corresponding to a search result may be determined based on the combination result value.

In one embodiment, the image search apparatus 100 may transmit the target image selected as the search result to the user terminal 10.

In another embodiment, the image search apparatus 100 extracts partial images corresponding to the object selected by the user terminal 10 from the target image selected as the search result, and synthesizes the extracted partial images into the search image as a search result It can also be transmitted to the user terminal 10. For example, as illustrated in FIGS. 11 and 13, the image search apparatus 100 may include a portion including the bag object 1150 in the search image 1100 and a partial image including the bag object in the target image ( 1311, 1321, and 1331, the exchanged composite images 1310, 1320, and 1330 may be transmitted to the user terminal 10 as a search result image.

The user transmits a search image including a plurality of objects to the image search apparatus 100, so that a coordination style that matches his or her preference can be easily identified through the search results.

14 is a block diagram showing the configuration of an image search apparatus 100 according to an embodiment.

Referring to FIG. 14, an image retrieval apparatus 100 according to an embodiment may include a memory 1410, a database 1430, a communication unit 1450, and a processor 1470.

The memory 1410 stores at least one program. The memory 1410, the database 1430, the communication unit 1450, and the processor 1470 may operate according to at least one program. Also, the memory 1410 may further store a generation model for generating a representative feature vector. Also, the memory 1410 may further store a machine learning model for extraction of feature vectors.

The database 1430 stores target images and feature vectors of target images. The feature vectors of the target images may be output by a machine learning model. For example, feature vectors of target images may include output by a dependent model, and may include output by a independent model. In addition, for a target image including a plurality of objects, the feature vector may include a feature vector sequence including feature vectors for each attribute of each object. In one embodiment, the database 1430 may be formed of a storage device such as a hard disk (HDD) or SSD.

The communication unit 1450 transmits and receives data to and from the user terminal 10. The communication unit 1450 receives a search condition from the user terminal 10 and transmits a search result corresponding to the search condition, for example, a search result image to the user terminal 10.

The processor 1470 obtains reference feature vectors corresponding to a plurality of reference images having the same attribute or assigned the same attribute tag, and from the generated model trained based on the reference feature vectors to the same attribute to the same attribute tag An attribute tag corresponding to the same attribute may be assigned to a target image of at least some of the target images by obtaining a corresponding representative feature vector and comparing the feature vectors of the target images with the representative feature vector.

Further, the processor 1470 may select a target image corresponding to the search condition received from the user terminal 10. The processor 1470 selects a target image to which an attribute tag corresponding to the attribute selected or input by the user terminal 10 is allocated, or a feature similar to/similar to a representative feature vector of the attribute selected or input by the user terminal 10 You can select a target image with a vector.

Further, the processor 1470 may select a target image corresponding to a plurality of search images received from the user terminal 10. Also, the processor 1470 may replace some of the feature vectors for each attribute of the search image received from the user terminal 10 with representative feature vectors of other attributes, and select a target image based on the substituted results.

Meanwhile, the above-described embodiments of the present disclosure may be written as a program executable on a computer, and the created program may be stored on a medium.

The medium may be a computer that continuously stores executable programs or may be temporarily stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a combination of single or several hardware, and is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks, And program instructions including ROM, RAM, flash memory, and the like. Also, examples of other media include an application store for distributing applications, a site for distributing or distributing various software, and a recording medium or storage medium managed by a server.

As described above, the technical idea of the present disclosure has been described in detail with reference to preferred embodiments, but the technical idea of the present disclosure is not limited to the above embodiments and has ordinary knowledge in the art within the scope of the technical idea of the present disclosure. Various modifications and changes are possible by the ruler.

100: image search device
1410: memory
1430: database
1450: Communication Department
1470: processor

Claims (13)

In the image search method by the image search device,
Obtaining reference feature vectors corresponding to a plurality of reference images including the same attribute;
Obtaining a representative feature vector corresponding to the same attribute from a trained generation model based on the obtained reference feature vectors; And
And comparing the feature vectors of target images stored in a database with the representative feature vector, and assigning an attribute tag corresponding to the same attribute to a target image of at least some of the target images. Way.
According to claim 1,
The image search method,
Selecting an attribute from a user terminal;
Determining a target image to which an attribute tag corresponding to the selected attribute is assigned among the target images; And
And transmitting the determined target image to the user terminal.
According to claim 1,
The image search method,
Selecting an attribute from a user terminal;
Comparing feature vectors of target images stored in the database with a representative feature vector corresponding to the selected attribute; And
And transmitting the target image selected based on the comparison result among the target images to the user terminal.
According to claim 3,
The comparing step,
When the representative feature vector corresponding to the attribute selected by the user terminal is output from the generated model trained based on reference feature vectors output from the dependent model, the representative feature vector corresponding to the selected attribute and the dependent Compare feature vectors of the target images output from the model,
When the representative feature vector corresponding to the attribute selected by the user terminal is output from the generated model trained based on the reference feature vectors output from the independent model, the representative feature vector corresponding to the selected attribute and the independent And comparing feature vectors of the target images output from a model.
According to claim 1,
The image search method,
Receiving a search image from a user terminal;
Obtaining a feature vector for each attribute of the search image;
Changing a feature vector of the attribute selected by the user terminal among the feature vectors for each attribute to a representative feature vector corresponding to the attribute selected by the user terminal;
Comparing a feature vector for each attribute including the representative feature vector and a feature vector for each attribute of target images stored in the database; And
And transmitting the target image selected based on the comparison result among the target images stored in the database to the user terminal.
The method of claim 5,
The comparing step,
Obtaining a distance for each attribute by comparing the feature vector for each attribute including the representative feature vector and the feature vector for each attribute of the target images between attributes of the same category; And
And selecting a target image to be transmitted to the user terminal from among target images stored in the database according to a result of combining the sizes of the distances for each attribute.
According to claim 1,
The image search method,
Receiving a search image including a plurality of objects from a user terminal;
Obtaining a feature vector sequence including a feature vector for each attribute of a plurality of objects included in the search image;
Selecting an object from the user terminal among the plurality of objects;
Changing a feature vector of the attribute selected by the user terminal among the feature vectors for each attribute of the selected object, to a representative feature vector corresponding to the attribute selected by the user terminal;
Comparing a feature vector sequence including the representative feature vector with a feature vector sequence of target images stored in the database; And
And transmitting the target image selected based on the comparison result among the target images stored in the database to the user terminal.
The method of claim 7,
The step of transmitting the selected target image to the user terminal,
Synthesizing partial images corresponding to the object selected by the user terminal among the plurality of objects included in each of the selected target images into the search image; And
And transmitting the search images in which the partial images are synthesized as a search result to the user terminal.
In the image search method by the image search device,
Receiving a plurality of search images from a user terminal;
Obtaining feature vectors corresponding to the plurality of search images;
Training a generation model with the acquired feature vectors;
Determining a target image corresponding to the plurality of search images by comparing a representative feature vector output from the generation model trained with the feature vectors and feature vectors of target images stored in a database; And
And transmitting the determined target image to the user terminal.
The method of claim 1 or 9,
The generation model,
Image retrieval method characterized in that it comprises a GAN (generative adversarial network) including a plurality of fully connected layer (FCL).
A program stored in a medium to execute the image search method of any one of claims 1 to 10 in combination with hardware.
Processor;
A memory storing at least one program and generation model; And
Includes a database that stores target images,
The processor, as the at least one program is executed,
Obtaining reference feature vectors corresponding to a plurality of reference images including the same attribute,
Obtain a representative feature vector corresponding to the same attribute from the generated model trained based on the obtained reference feature vectors,
And comparing the feature vectors of the target images with the representative feature vectors, and assigning an attribute tag corresponding to the same attribute to a target image of at least some of the target images.
In the image search method by the user terminal device,
Selecting an attribute by accessing the image search apparatus; And
And receiving, from the image search apparatus, a target image to which an attribute tag corresponding to the selected attribute is assigned among target images stored in the image search apparatus.
Target images stored in the image search device,
Characterized in that at least some attribute tags among the plurality of attribute tags corresponding to the plurality of attributes are allocated based on a comparison result of representative feature vectors corresponding to each of the plurality of attributes and the feature vectors of the target images. How to search for images.

Images