KR102193571B1 - Electronic device, image searching system and controlling method thereof - Google Patents

Abstract

An electronic device, an image retrieval system, and a control method are disclosed. The electronic device includes an input interface for receiving a keyword and a processor for searching for a candidate image based on the input keyword, the processor extracting a tag and a reliability associated with the tag from one of the searched candidate images, and the input keyword And converting the extracted tag into a vector, calculating a goodness of fit between the converted keyword vector and the converted tag vector based on each converted tag vector, and applying the extracted reliability to each of the calculated goodness-of-fits to give one candidate image A degree of similarity to is calculated, and a candidate image with the highest similarity calculated by comparing the degree of similarity calculated for all candidate images is determined as a recommended image.

Description

Electronic device, image search system and control method {ELECTRONIC DEVICE, IMAGE SEARCHING SYSTEM AND CONTROLLING METHOD THEREOF}

The present disclosure relates to an electronic device, an image retrieval system, and a control method, and more particularly, to an electronic device, an image retrieval system, and a control method for identifying an image having a high similarity to a keyword.

With the development of electronic technology, many people use document creation programs to create various documents such as reports. Documents such as reports are mainly composed of text, but inserting appropriate images can help the understanding of the document and increase its completeness. With the development of communication technology in the modern society, users can easily access a large amount of data, so that an appropriate image can be selected from among numerous images and inserted into a document. However, when a user searches for images by keyword, too many images are searched, so there is a problem in that too much time has to be spent to select an appropriate image for a document.

In order to solve this problem, research is being conducted on a technique to analyze and re-evaluate the characteristics of the searched image candidates. However, since the existing research aims to increase the performance of a search engine, many similar images can be searched, but there is a limitation in the ability to identify appropriate images required by users.

Therefore, there is a need for a technology capable of identifying the most appropriate image required by a user based on a keyword.

The present disclosure is to solve the above problems, and an object of the present disclosure is to provide an electronic device, an image retrieval system, and a control method for retrieving an image based on a keyword and identifying an image having the highest similarity to the keyword among the retrieved images. will be.

According to an embodiment of the present disclosure for achieving the above object, the electronic device includes an input interface for receiving at least one keyword and a processor for searching for at least one candidate image based on the input at least one keyword. Including, the processor extracts at least one tag and the reliability associated with the at least one tag from one candidate image among the searched at least one candidate image, the input at least one keyword and the extracted at least one Converts a tag to a vector, calculates a degree of fit between the converted at least one keyword vector and the converted at least one tag vector based on each converted tag vector, and the extracted reliability for each calculated fit A similarity to the one candidate image may be calculated by applying, and a candidate image having the highest similarity may be determined as a recommended image by comparing the similarity calculated for all of the at least one candidate image.

And, the processor calculates a first cosine distance between the n-th tag vector and the first keyword vector, calculates the m-th cosine distance between the n-th tag vector and the m-th keyword vector, and the calculated first cosine distance and the A goodness of fit for the nth tag vector is calculated by averaging the m cosine distance, and a similarity related to the nth tag is calculated by multiplying the calculated goodness of fit for the nth tag vector by a reliability related to the extracted nth tag, The similarity of the one candidate image may be calculated by summing the calculated similarity of each tag.

Alternatively, the processor generates a keyword subspace based on the at least one keyword vector, and calculates a fitness for each tag vector by calculating a tag vector angle between each of the at least one tag vector and the generated keyword subspace The similarity for each tag is calculated by multiplying the calculated angle of each tag vector by the reliability associated with each tag, and the similarity for the one candidate image is calculated by summing the calculated similarity for each tag. can do.

In addition, the reliability may be proportional to a size of an object related to the tag included in the one image or a matching degree of an object related to the tag.

Meanwhile, the electronic device may further include a display that arranges and displays at least one candidate image in descending order of similarity based on the recommended image or the calculated similarity.

In addition, when one image among the displayed at least one candidate image is selected, the processor may insert the selected one image into text editing software.

Meanwhile, according to an embodiment of the present disclosure for achieving the above object, a method of controlling an electronic device includes receiving at least one keyword, and selecting at least one candidate image based on the inputted at least one keyword. Searching, extracting at least one tag and a reliability associated with the at least one tag from one candidate image among the searched at least one candidate image, the input at least one keyword, and the extracted at least one tag Converting into a vector, calculating a fitness level between the converted at least one keyword vector and the converted at least one tag vector based on each of the converted tag vectors, and the extracted Computing a similarity for the one candidate image by applying a reliability, and determining a candidate image with the highest similarity as a recommended image by comparing the calculated similarity for all of the at least one candidate image. can do.

In the calculating of the fitness, the first cosine distance between the nth tag vector and the first keyword vector is calculated, the mth cosine distance between the nth tag vector and the mth keyword vector is calculated, and the calculated first Averaging the cosine distance and the m-th cosine distance to calculate a goodness-of-fit for the n-th tag vector, and calculating the similarity includes multiplying the calculated goodness-of-fit for the n-th tag vector by a reliability associated with the extracted n-th tag. A degree of similarity related to the nth tag may be calculated, and a degree of similarity for the one candidate image may be calculated by summing the calculated similarity degrees related to each tag.

Alternatively, in the calculating of the fitness level, a keyword subspace is generated based on the at least one keyword vector, and a tag vector angle between each of the at least one tag vector and the generated keyword subspace is calculated, In the step of calculating the degree of suitability for each tag, and calculating the degree of similarity, the calculated degree of similarity related to each tag is calculated by multiplying the calculated angle of each tag vector by the reliability level associated with each tag, and the calculated degree of similarity related to each tag is summed. Thus, the degree of similarity to the one candidate image may be calculated.

In addition, the reliability may be proportional to a size of an object related to the tag included in the one image or a matching degree of an object related to the tag.

Meanwhile, the method of controlling the electronic device may further include arranging and displaying the at least one candidate image in descending order of similarity based on the recommended image or the calculated similarity.

Also, the method of controlling the electronic device may further include inserting the selected one image into text editing software when one of the displayed at least one candidate image is selected.

On the other hand, according to an embodiment of the present disclosure for achieving the above object, the image search system is based on a terminal device that receives at least one keyword and transmits it to a server, and the at least one keyword received from the terminal device. And a server that searches for at least one candidate image, wherein the server extracts at least one tag and a reliability associated with the at least one tag from one candidate image among the searched at least one candidate image, and the input Convert at least one keyword and the extracted at least one tag into a vector, and calculate a suitability between the converted at least one keyword vector and the converted at least one tag vector based on each converted tag vector, The calculated similarity of the one candidate image is calculated by applying the extracted reliability to each calculated goodness of fit, and the calculated similarity is the highest candidate image by comparing the calculated similarity of all of the at least one candidate image Is determined as a recommended image, and at least one image of the determined recommended image or the searched candidate image may be transmitted to the terminal device.

As described above, according to various embodiments of the present disclosure, the electronic device, the image search system, and the control method may search for an image and identify an image having the highest similarity to a keyword among the searched images.

In addition, the electronic device, the image search system, and the control method may automatically select an image having the highest similarity with a keyword and insert it into a document.

The effects of the present invention are not limited to the above-mentioned effects, and other effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram of an electronic device according to an embodiment of the present disclosure.
2 is a diagram illustrating a process of determining a recommended image according to an embodiment of the present disclosure.
3 is a diagram illustrating a process of determining a similarity between a keyword and an image according to an embodiment of the present disclosure.
4 is a diagram illustrating a process of determining a similarity between a keyword and an image according to another embodiment of the present disclosure.
5 is a diagram illustrating a result of determining a similarity of a candidate image according to an embodiment of the present disclosure.
6 is a diagram illustrating an electronic device in which a candidate image is displayed according to a degree of similarity according to an embodiment of the present disclosure.
7 is a diagram illustrating a selection image inserted in a document according to an embodiment of the present disclosure.
8 is a diagram illustrating an image search system according to an exemplary embodiment of the present disclosure.
9 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

Hereinafter, various embodiments will be described in more detail with reference to the accompanying drawings. The embodiments described herein may be variously modified. Certain embodiments may be depicted in the drawings and described in detail in the detailed description. However, specific embodiments disclosed in the accompanying drawings are only intended to facilitate understanding of various embodiments. Therefore, the technical idea is not limited by the specific embodiments disclosed in the accompanying drawings, and it should be understood to include all equivalents or substitutes included in the spirit and scope of the invention.

Terms including ordinal numbers such as first and second may be used to describe various elements, but these elements are not limited by the above-described terms. The above-described terms are used only for the purpose of distinguishing one component from other components.

In the present specification, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance. When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Meanwhile, a "module" or "unit" for a component used in the present specification performs at least one function or operation. And, the "module" or "unit" may perform a function or operation by hardware, software, or a combination of hardware and software. In addition, a plurality of "modules" or a plurality of "units" excluding "module" or "unit" that must be performed in specific hardware or performed by at least one control unit may be integrated into at least one module. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In the description of the present invention, the order of each step should be understood without limitation, unless the preceding step must be performed logically and temporally prior to the subsequent step. That is, except for the above exceptional cases, even if a process described as a subsequent step is performed prior to a process described as a preceding step, the essence of the invention is not affected, and the scope of rights should be defined regardless of the order of the steps. And in the present specification, "A or B" is defined to mean not only selectively indicating any one of A and B, but also including both A and B. In addition, in the present specification, the term "comprising" has the meaning of encompassing further including other components in addition to the elements listed as including.

Encryption/decryption may be applied to the information (data) transmission process performed in this specification as needed, and expressions describing the information (data) transmission process in this specification and claims are all encrypted/decrypted, even if not stated otherwise. It should be interpreted as including cases. In this specification, expressions in the form of "transmitted from A to B (transfer)" or "received from A by B" include those transmitted (transmitted) or received with another medium in the middle, and must be from A to B It does not just express what is directly transmitted (delivered) or received.

In this specification, only essential components necessary for the description of the present invention are described, and components not related to the essence of the present invention are not mentioned. In addition, it should not be interpreted as an exclusive meaning including only the mentioned components, but should be interpreted as a non-exclusive meaning that may also include other components.

In addition, in describing the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be abbreviated or omitted. Meanwhile, each embodiment may be implemented or operated independently, but each embodiment may be implemented or operated in combination.

1 is a block diagram of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 1, the electronic device 100 may include an input interface 110 and a processor 120. For example, the electronic device 100 may include a smart phone, a notebook computer, a tablet PC, a desktop computer, a smart TV, a kiosk, and the like.

The input interface 110 may receive a keyword. The input interface 110 may receive one keyword or a plurality of keywords. In addition, the input interface 110 may receive only a keyword to search for an image, or may receive input as a phrase, sentence, or paragraph including the keyword. If the input interface 110 receives input as a phrase, sentence, or paragraph including a keyword, the processor 120 may extract the keyword from the input phrase, sentence, or paragraph based on parts of speech.

The processor 120 may search for at least one candidate image based on the input keyword. For example, the electronic device 100 may include a memory (not shown), and the processor 120 may search for a candidate image related to a keyword from among images stored in the memory. Alternatively, the electronic device 100 may include a communication interface (not shown), and the processor 120 may control the communication interface to search for a candidate image related to a keyword among images stored in an external server or the like through the communication interface. In the present disclosure, the processor 120 searching for a candidate image refers to a case of searching for a candidate image related to a keyword from among images stored in a memory and a case of searching for a candidate image related to a keyword from among images stored in an external server or the like through a communication interface. It means all inclusive.

In addition, the number of candidate images searched by the processor 120 may be one or a plurality. The processor 120 may extract the tag and the reliability associated with the tag from one searched candidate image. The tag may mean information on an object included in the candidate image. In addition, the reliability level may mean the level of reliability of an object included in the candidate image. For example, if an object A occupying a relatively large area and an object B occupying a relatively small area are included in the candidate image, the reliability of object A may be higher than that of object B. As an example, the reliability of object A may be 0.8, and the reliability of object B may be extracted as a numerical value. One or more tags may be extracted from one candidate image.

The processor 120 may convert each keyword and tag into a vector. The process of converting keywords and tags into vectors can be referred to as a word embedding process, and the word embedding process is a process for calculating similarity.

The processor 120 may calculate a suitability between one or more keyword vector sets based on the converted one tag vector. The degree of suitability may mean a degree of similarity between one tag vector and a set of keyword vectors. When there are a plurality of extracted tags, there are also a plurality of converted tag vectors. Goodness of fit can be calculated for all tag vectors.

The processor 120 calculates a fitness level based on each tag vector, then applies the calculated fitness level for each tag vector and the reliability level corresponding to each tag vector, and adds the fitness level to which the reliability level is applied, and adds a keyword and one candidate image. The degree of similarity between the livers can be calculated. A specific process of calculating the similarity will be described later. If there are a plurality of candidate images, the processor 120 may repeat the above-described process for all candidate images.

The processor 120 may compare the calculated similarity for each of the candidate images, and determine a candidate image with a high possible similarity as the recommended image.

A detailed process of determining a recommended image will be described below.

2 is a diagram illustrating a process of determining a recommended image according to an embodiment of the present disclosure.

Referring to FIG. 2, the electronic device may receive a keyword 10. For example, the keyword 10 may be directly input from the user. Alternatively, the keyword 10 may be automatically extracted from a phrase, sentence, or paragraph. The electronic device may receive one or more keywords. The electronic device may search for a candidate image 20 related to the input keyword 10. For example, the electronic device may search for a candidate image 20 related to the input keyword 10 using a search engine, and there may be one or more searched candidate images 20.

The electronic device may calculate a similarity for each of the searched candidate images 20. Below, a process of calculating the similarity of one candidate image will be described.

The electronic device may extract the tag 30 and reliability from the candidate image 20. For example, the electronic device may extract the tag 30 and reliability using an object detection and recognition module, and additionally may additionally extract the tag 30 from the caption information of the image using a Python morpheme analysis module. . One or more tags 30 to be extracted may be plural, and the reliability may be a value corresponding to each of the extracted tags 30.

The electronic device may perform a word embedding process expressed in a vector form to calculate a similarity between a keyword and a candidate image. That is, the electronic device may convert the input keyword 10 into the keyword vector 40-1 and convert the extracted tag 30 into the tag vector 40-2. For example, the Word2vec method may be used for the word embedding process.

The electronic device may calculate the similarity of the candidate image based on the keyword vector 40-1, the tag vector 40-2, and the reliability. For example, the electronic device may calculate the similarity of the candidate image using a weighted cosine distance method or a weighted subspace distance method. A specific weighted cosine distance method and a weighted subspace distance method will be described later.

Through the above-described process, the electronic device may calculate a degree of similarity for one candidate image. When a plurality of candidate images exist, the electronic device may calculate a similarity for each candidate image by repeating the above-described process for each of the other candidate images.

When a similarity degree is calculated for each of the plurality of candidate images, the electronic device may sort 50 the candidate images in descending order based on the similarity value. In addition, the electronic device may determine the candidate image having the highest similarity as the recommended image.

The electronic device may determine an image having the highest similarity to the keyword input through the above-described process. Hereinafter, a weighted cosine distance method and a weighted subspace distance method for determining similarity will be described.

3 is a diagram illustrating a process of determining a similarity between a keyword and an image according to an embodiment of the present disclosure.

Referring to FIG. 3, a process of determining the similarity of a candidate image using a weighted cosine distance method is illustrated. The electronic device converts the input keyword into keyword vectors (w1, w2, ..., wM), and extracts the tag and the reliability (a1, a2, ..., aN) corresponding to the tag from the searched candidate image, The extracted tags can be converted into tag vectors (v1, v2, ..., vN).

The electronic device may calculate a cosine distance between each keyword vector and the tag vector. For example, the electronic device calculates the cosine distance between the first keyword vector (w1) and the first tag vector (v1), calculates the cosine distance between the second keyword vector (w2) and the first tag vector (v1), and calculates the M-th keyword The cosine distance between the vector wM and the first tag vector v1 may be calculated. That is, the electronic device may calculate a cosine distance between the i-th keyword vector (wi) and the j-th tag vector (vj).

The electronic device may calculate a suitability (μ) between the keyword vector and the tag vector based on the tag vector. The fitness may be calculated by averaging the cosine distances of all keyword vectors calculated based on the tag vector. That is, the first goodness-of-fit (μ1) is a value obtained by summing all the cosine distances between the first keyword vector (w1) and the first tag vector (v1), and the cosine distance between the M-th keyword vector (wM) and the first tag vector (v1) Can be calculated as

The electronic device may repeat the above-described process for all tag vectors to calculate the N-th suitability (μN) from the first suitability (μ1). The electronic device may calculate the similarity of the candidate image by multiplying the calculated fitness by the reliability corresponding to each tag vector and then adding them. Reliability can be applied as a weight to the calculated fitness. Accordingly, the electronic device can more accurately calculate the similarity between the keyword and the candidate image by calculating the similarity to which the weight is applied to the candidate image, and the image matching the input keyword from among a plurality of candidate images or the most appropriate image requested by the user. Can be identified.

4 is a diagram illustrating a process of determining a similarity between a keyword and an image according to another embodiment of the present disclosure.

Referring to FIG. 4, a process of determining the similarity of a candidate image using a weighted subspace distance method is illustrated. The electronic device converts the input keyword into keyword vectors (w1, w2, ..., wM), and extracts the tag and the reliability (a1, a2, ..., aN) corresponding to the tag from the searched candidate image, The extracted tags can be converted into tag vectors (v1, v2, ..., vN).

The electronic device may generate a subspace formed by the converted keyword vectors w1, w2, ..., wM. In addition, the electronic device may calculate an angle (φ) between the tag vector and the subspace for each tag vector (v1, v2, ..., vN). For example, the electronic device may calculate the first angle (φ1) between the first tag vector (v1) and the subspace, and may calculate the Nth angle (φN) between the Nth tag vector (v1) and the subspace. . The calculated angle φ between the tag vector and the subspace may be a degree of fitness between the keyword vector and the tag vector.

The electronic device may calculate the similarity of the candidate image by multiplying the calculated angle (or fitness) by the reliability corresponding to each tag vector and then adding them. As described above, the reliability may be applied as a weight to the calculated fitness. Accordingly, the electronic device can more accurately calculate the similarity between the keyword and the candidate image by calculating the similarity to which the weight is applied to the candidate image, and the image matching the input keyword from among a plurality of candidate images or the most appropriate image requested by the user. Can be identified.

In addition, the weighted subspace distance method described with reference to FIG. 4 has an advantage of calculating a similarity in consideration of mutual relationships between a plurality of keywords. For example,'GALAXY' and'NOTE' are individually completely irrelevant words, but the two words can be connected to mean a specific product. Since the weighted subspace distance method calculates the similarity based on the angle between the subspace formed by the keyword vectors (w1, w2, ..., wM) and the tag vector, it is possible to calculate the similarity in consideration of the mutual relationship between a plurality of keywords. .

5 is a diagram illustrating a result of determining a similarity of a candidate image according to an embodiment of the present disclosure.

Referring to FIG. 5, as an embodiment, a result of determining the similarity of a candidate image for three keywords {NASA, sun, touch} is shown. The electronic device searched for 10 candidate images for the input three keywords. In addition, the electronic device determined the degree of similarity for each of the 10 candidate images. A is the similarity result when only the API tag is used, B is the similarity result when only the caption tag is used, and C is the similarity result when both the API tag and the caption tag are used. D is the average value of similarity for each immigration site obtained by subject evaluation. FIG. 5 shows a result that when the similarity is determined according to the method of the present disclosure using both the API tag and the caption tag, a result that is very close to the average of the subject's subjective judgment can be obtained.

Meanwhile, the electronic device may display the determined recommended image or a plurality of candidate images based on the similarity.

6 is a diagram illustrating an electronic device in which a candidate image is displayed according to a degree of similarity according to an embodiment of the present disclosure.

Referring to FIG. 6, an electronic device 100 in which a candidate image and a degree of similarity are displayed is illustrated. The electronic device 100 may determine a degree of similarity with respect to each of the candidate images, and determine a candidate image with the highest similarity as the recommended image. In addition, the electronic device 100 may display the determined recommended image 11 including the display on the display together with the calculated similarity.

Also, the electronic device 100 may display other candidate images 12 and 13 other than the recommended image 11 on the display in descending order based on the calculated similarity. The electronic device 100 may display the recommendation image 11 and other candidate images 12 and 13 on one screen, and other candidate images in addition to the page screen and the recommendation image 11 displaying only the recommendation image 11 The page screen displaying (12, 13) may be displayed separately.

Meanwhile, the electronic device 100 may insert the image for which the degree of similarity is calculated into the document.

7 is a diagram illustrating a selection image inserted in a document according to an embodiment of the present disclosure.

The electronic device 100 may insert the searched image 21 into a document being created with text editing software. For example, the electronic device 100 may directly insert the determined recommended image, or may insert the recommended image into a document according to a user's selection.

Alternatively, as described above, the electronic device 100 may display other candidate images in addition to the recommended image, and may insert an image selected from among other candidate images in the document according to the user's selection.

Meanwhile, the image search process may be implemented as a system.

8 is a diagram illustrating an image search system according to an exemplary embodiment of the present disclosure.

Referring to FIG. 8, an image search system 1000 including an electronic device 100 and a server 200 is illustrated. For example, the electronic device 100 may be a user terminal device such as a smart phone, a notebook computer, a tablet PC, a desktop computer, a smart TV, and a kiosk. The electronic device 100 may receive a keyword and transmit the input keyword to the server 200. The electronic device 100 may transmit a keyword directly input by a user to the server 200, extract a keyword from a phrase, sentence, or paragraph and transmit the extracted keyword to the server 200. There may be one or more keywords.

The server 200 may search for a candidate image based on a keyword received from the electronic device 100. In addition, the server 200 may extract the tag and the reliability associated with the tag from one candidate image among the searched candidate images. The server 200 may convert the input keyword and the extracted tag into a vector.

In addition, the server 200 may calculate a degree of fitness between the converted keyword vector and the converted tag vector based on the converted tag vector. The server 200 may calculate the similarity of the candidate image by applying the extracted reliability to each calculated fitness. The server 200 may calculate the similarity using a weighted cosine distance method and a weighted subspace distance method. When there are a plurality of candidate images, the server 200 may perform the above-described similarity calculation process for all candidate images.

The server 200 may compare the calculated similarity to determine a candidate image with the highest similarity calculated as a recommendation image, and transmit at least one image of the determined recommendation image or the searched candidate image to the electronic device 100.

The electronic device 100 may arrange and display the received recommended image or candidate image based on similarity, and may insert one of the recommended image or candidate image into the document.

So far, various embodiments of searching for images and determining a recommended image having the highest similarity to a keyword have been described. The following describes the electronic device control method.

9 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 9, the electronic device may receive at least one keyword (S910). The electronic device may search for at least one candidate image based on the input at least one keyword (S920).

The electronic device may extract at least one tag and a reliability associated with the at least one tag from one candidate image among the searched at least one candidate image (S930). The reliability may be a value proportional to the size of an object related to a tag included in one image or a degree of matching of an object related to the tag.

The electronic device may convert the input at least one keyword and the extracted at least one tag into a vector (S940). The electronic device may calculate a fitness level between the converted at least one keyword vector and the converted at least one tag vector based on each converted tag vector (S950). The electronic device may calculate a similarity for one candidate image by applying the extracted reliability to each calculated fitness (S960). For example, the electronic device may calculate the similarity using a weighted cosine distance method and a weighted subspace distance method.

The weighted cosine distance method refers to a method of calculating a similarity based on a cosine distance between a tag vector and a keyword vector. The electronic device calculates a first cosine distance between the n-th tag vector and the first keyword vector, calculates the m-th cosine distance between the n-th tag vector and the m-th keyword vector, and the calculated first cosine distance and m-th cosine distance It is possible to calculate the fit for the nth tag vector by averaging. In addition, the electronic device calculates a similarity related to the nth tag obtained by multiplying the calculated fitness for the nth tag vector by the reliability related to the extracted nth tag, and summing the calculated similarity related to each tag to obtain one candidate image You can calculate the similarity to

The weighted subspace distance method refers to a method of calculating a similarity based on an angle between a subspace generated as a keyword vector and a tag vector. The electronic device may generate a keyword subspace based on at least one keyword vector. The electronic device may calculate a degree of suitability for each tag vector by calculating a tag vector angle between each of the at least one tag vector and the generated keyword subspace. In addition, the electronic device calculates the similarity associated with each tag obtained by multiplying the calculated angle of each tag vector by the reliability associated with each tag, and calculates the similarity for one candidate image by summing the calculated similarity associated with each tag. can do.

The electronic device may determine a candidate image with the highest similarity calculated by comparing the similarity calculated for all of the at least one candidate image as the recommended image (S970).

Meanwhile, the electronic device may arrange and display at least one candidate image in descending order of similarity based on the recommended image or the calculated similarity. In addition, when one of the displayed at least one candidate image is selected, the electronic device may insert the selected one image into the text editing software.

The method for controlling an electronic device according to the various embodiments described above may be provided as a computer program product. The computer program product may include the S/W program itself or a non-transitory computer readable medium in which the S/W program is stored.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, and ROM.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

1000: image search system
100: electronic device 200: server
110: input interface 120: processor

Claims (13)

An input interface for receiving at least one keyword; And
Includes; a processor that searches for a plurality of candidate images based on the input at least one keyword,
The processor,
Extracting at least one tag and a reliability associated with the at least one tag from one candidate image among the searched plurality of candidate images, converting the input at least one keyword and the extracted at least one tag into a vector, The one candidate by calculating a fitness level between the converted at least one keyword vector and the converted at least one tag vector based on each converted tag vector, and applying the extracted reliability to each calculated fitness level A similarity to the image is calculated, and the calculated similarity of all of the plurality of candidate images is compared to determine the candidate image with the highest similarity as a recommended image,
The processor, in calculating the similarity,
Calculate the first cosine distance between the nth tag vector and the first keyword vector, calculate the mth cosine distance between the nth tag vector and the mth keyword vector, and average the calculated first cosine distance and the mth cosine distance Then, a degree of suitability for the n-th tag vector is calculated, a similarity degree related to the n-th tag is calculated by multiplying the calculated degree of suitability for the n-th tag vector by a reliability related to the extracted n-th tag, and the calculated respective tags The electronic device, which calculates a similarity for the one candidate image by summing the similarity related to. delete The method of claim 1,
The processor,
A keyword subspace is generated based on the at least one keyword vector, a tag vector angle between each of the at least one tag vector and the generated keyword subspace is calculated to calculate a fitness for each tag vector, and the calculated An electronic device that calculates a similarity associated with each tag obtained by multiplying each tag vector angle by a reliability associated with each tag, and calculates a similarity with respect to the one candidate image by summing the calculated similarity associated with each tag. The method of claim 1,
The reliability is,
The electronic device, which is proportional to a size of an object related to the tag included in the one image or a degree of matching of an object related to the tag. The method of claim 1,
The electronic device further comprises: a display configured to sort and display the plurality of candidate images in descending order of similarity based on the recommended image or the calculated similarity. The method of claim 5,
The processor,
When one of the displayed plurality of candidate images is selected, inserting the selected one image into text editing software. In the control method of the electronic device performed by the processor of the electronic device,
Receiving at least one keyword input through an input interface;
Searching for a plurality of candidate images based on the input at least one keyword;
Extracting at least one tag and a reliability associated with the at least one tag from one candidate image among the searched plurality of candidate images;
Converting the input at least one keyword and the extracted at least one tag into a vector;
Calculating a fitness degree between the converted at least one keyword vector and the converted at least one tag vector based on each converted tag vector;
Calculating a similarity of the one candidate image by applying the extracted reliability to each of the calculated fitness levels; And
Comprising the calculated similarity for all of the plurality of candidate images and determining the candidate image having the highest similarity as a recommended image; Including,
The step of calculating the fitness is,
Calculate the first cosine distance between the nth tag vector and the first keyword vector, calculate the mth cosine distance between the nth tag vector and the mth keyword vector, and average the calculated first cosine distance and the mth cosine distance To calculate the fitness for the n-th tag vector,
The step of calculating the similarity,
A similarity related to the n-th tag is calculated by multiplying the calculated fitness for the n-th tag vector by a reliability related to the extracted n-th tag, and the calculated similarity related to each tag is summed to obtain the one candidate image. A method of controlling an electronic device for calculating the degree of similarity. delete The method of claim 7,
The step of calculating the fitness is,
A keyword subspace is generated based on the at least one keyword vector, and a tag vector angle between each of the at least one tag vector and the generated keyword subspace is calculated to calculate a fitness for each tag vector,
The step of calculating the similarity,
Calculating a similarity related to each tag obtained by multiplying the calculated angle of each tag vector by a reliability related to each tag, and calculating a similarity to the one candidate image by summing the calculated similarity related to each tag, Control method of electronic devices. The method of claim 7,
The reliability is,
The method of controlling an electronic device, which is proportional to a size of an object related to the tag included in the one image or a matching degree of an object related to the tag. The method of claim 7,
Arranging and displaying the plurality of candidate images in descending order of similarity based on the recommended image or the calculated similarity. The method of claim 11,
When one of the plurality of displayed candidate images is selected, inserting the selected one image into text editing software. A terminal device that receives at least one keyword and transmits it to a server; And
Including; a server for searching for a plurality of candidate images based on the at least one keyword received from the terminal device;
The server,
Extracting at least one tag and a reliability associated with the at least one tag from one candidate image among the searched plurality of candidate images, converting the input at least one keyword and the extracted at least one tag into a vector, The one candidate by calculating a fitness level between the converted at least one keyword vector and the converted at least one tag vector based on each converted tag vector, and applying the extracted reliability to each calculated fitness level Calculates a similarity to an image, compares the calculated similarity for all of the plurality of candidate images to determine the candidate image with the highest similarity as a recommendation image, and at least one of the determined recommendation image or the searched candidate image Transmit the image of to the terminal device,
The server, in calculating the similarity,
Calculate the first cosine distance between the nth tag vector and the first keyword vector, calculate the mth cosine distance between the nth tag vector and the mth keyword vector, and average the calculated first cosine distance and the mth cosine distance Then, a degree of suitability for the n-th tag vector is calculated, a similarity degree related to the n-th tag is calculated by multiplying the calculated degree of suitability for the n-th tag vector by a reliability related to the extracted n-th tag, and the calculated respective tags The image retrieval system for calculating a similarity for the one candidate image by summing the similarity related to.

Images