JP2018156333A - Generation device, generation method, and generation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reflect information similarity on distance in a distributed representation space.SOLUTION: A generation device of the present disclosure comprises: a computation unit configured to compute a first similarity, representing similarity between first information and second information, and a second similarity representing similarity between the first information and third information; and a generation unit configured to generate each distributed representation based on a relationship between the first similarity and the second similarity in such a way that one of distributed representations of the second and third information is similar to a distributed representation of the first information but the other one of the distributed representations is not similar thereto.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a generation device, a generation method, and a generation program.

  Conventionally, a classification technique corresponding to a feature amount of information such as an image or text is known. As an example of such a technique, a method called triplet loss has been proposed. In the triplet loss, the degree of coincidence of tags indicating the contents of each information is calculated, information having the calculated degree of coincidence of tags exceeding the first threshold is set as similar information, and information whose degree of coincidence is below the second threshold is not calculated. Similar information. In triplet loss, for example, the difference between the distributed representation of the first information and the distributed representation of the second information similar to the first information is small, and the first information, the distributed representation, and the first information are not similar. The distributed representation of each information is learned so that the difference from the distributed representation of the three information becomes large. As a result of such processing, the first information and the second information are arranged in the vicinity in the distributed representation space, and the first information and the third information are arranged so as to be separated from each other.

JP 2010-250849 A

"Fashion Style in 128 Floats: Joint Ranking and Classification using Weak Data for Feature Extraction" Edgar Simo-Serra and Hiroshi Ishikawa, Department of Computer Science and Engineering, Waseda University, Tokyo, Japan

  However, in the above-described prior art, there are cases where the similarity of information cannot be reflected on the distance in the distributed expression space.

  For example, in the prior art, information whose tag matching degree with the first information is smaller than the first threshold is treated as information similar to the first information regardless of whether the tag matching degree is high or low. The degree of matching of tags between information is not reflected in the similarity of distributed expressions. As a result, in the search using the distributed expression space, information similar to the first information can be searched, but the accuracy of the ranking according to the similarity to the first information cannot be guaranteed.

  The present application has been made in view of the above, and an object thereof is to reflect the similarity of information to the distance in the distributed expression space.

  The generation device according to the present application calculates a first similarity that is a similarity between the first information and the second information, and a second similarity that is a similarity between the first information and the third information. Based on the relationship between the calculating unit and the first similarity and the second similarity, one of the distributed representation of the second information and the third information is a distributed representation of the first information. It has a generation part which generates each distributed expression so that it may be similar and the other may not be similar.

  According to one aspect of the embodiment, the similarity of information can be reflected in the distance on the distributed representation space.

FIG. 1 is a diagram illustrating an example of processing executed by the information providing apparatus according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. FIG. 3 is a diagram illustrating an example of information registered in the image database according to the embodiment. FIG. 4 is a diagram illustrating an example of information registered in the distributed representation database according to the embodiment. FIG. 5 is a diagram illustrating an example of the similarity calculated by the information providing apparatus according to the embodiment. FIG. 6 is a flowchart illustrating an example of a flow of generation processing executed by the information providing apparatus according to the embodiment. FIG. 7 is a diagram illustrating an example of a hardware configuration.

  Hereinafter, a mode for carrying out a generation device, a generation method, and a generation program according to the present application (hereinafter referred to as “embodiment”) will be described in detail with reference to the drawings. Note that the generation device, the generation method, and the generation program according to the present application are not limited to the embodiment. In the following embodiments, the same parts are denoted by the same reference numerals, and redundant description is omitted.

[Embodiment]
[1. Regarding the processing provided by the information providing device]
First, an example of a generation process executed by an information providing apparatus as an example of a generation apparatus will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of processing executed by the information providing apparatus according to the embodiment. In the following description, as processing executed by the information providing apparatus 10, in order to perform image search using distributed representation, a generation processing that generates a distributed representation reflecting image similarity, and a generation processing are performed. A search process for searching for an image using the generated distributed expression will be described. In the following description, a process in which the information providing apparatus 10 “generates” a distributed representation of each information will be described. However, such processing appropriately “learns” the value of an appropriate distributed representation corresponding to each data. It is assumed that this is a process of “generating” a distributed representation.

  In addition, the following generation processing and search processing generate not only various images such as still images and moving images but also distributed representations of arbitrary contents such as audio, movies, novels, news articles, etc. It can be applied to the process of searching for content using.

[1-1. Overview of information providing device)
The information providing apparatus 10 is an information processing apparatus that can communicate with the user terminal 100 via a predetermined network N (for example, see FIG. 2) such as the Internet, and is realized by, for example, a server apparatus or a cloud system. Is done. The information providing apparatus 10 may be able to communicate with an arbitrary number of user terminals 100 via the network N.

  The user terminal 100 is an information processing device used by a user who requests information retrieval, and is realized by an information processing device such as a PC (Personal Computer), a server device, or a smart device. For example, the user terminal 100 transmits information indicating the image and the content of the image to the information providing apparatus 10 as a search query. In such a case, the information providing apparatus 10 searches for an image similar to the image received as the search query using the distributed expression generated by the generation process described later, and sets the searched image to the similarity with the search query. The data is provided to the user terminal 100 in the corresponding ranking format.

[1-2. About generation processing)
Here, in order to facilitate not only the appearance feature amount of the image but also the image search according to the content of the imaged imaging target, the feature amount indicating the content of the imaged imaging target is generated, and the generated feature A technique for classifying images according to the amount can be considered. For example, a technique is conceivable in which tag information indicating the content of the imaging target captured in the image is held as meta information of the image, and a distributed expression based on the feature amount of the imaging target is generated according to the degree of coincidence of the meta information.

  A triplet loss technique is known as a technique for generating a distributed representation according to the degree of coincidence of meta information. However, in the triplet loss technique, depending on whether or not the degree of coincidence of meta information is higher than a predetermined threshold, a set of a reference image, an image that is correct data, and an image that is incorrect data, that is, Generate information called triples. In the triplet loss technique, the distributed representations of the reference image and the image that is correct data (hereinafter referred to as “correct pair”) are similar, and the image that is the reference image and incorrect data (hereinafter “incorrect data”). The distributed expression of each image is learned so that the distributed expression of “correct answer pair” is dissimilar.

  However, in such triplet loss technology, when the degree of coincidence of meta information is higher than a predetermined threshold value, it is handled as correct answer data, so that the reference image is located near the distributed representation of the reference image in the distributed representation space. Although the distributed representation of the image similar to the reference image is arranged, the distributed representation of the image most similar to the reference image may not be arranged in the vicinity of the distributed representation of the reference image. That is, with the conventional triplet loss technique, the similarity between similar images cannot be reflected in the similarity of the distributed representation.

  Therefore, the information providing apparatus 10 executes the following generation process. First, the information providing apparatus 10 selects three images that are the first information, the second information, and the third information from the group of images to be distributed. In the following description, the three images selected as the first information to the third information are referred to as the first image to the third image, respectively.

  Subsequently, the information providing apparatus 10 includes a first similarity that is a similarity between the first information and the second information, and a second similarity that is a similarity between the first information and the third information. Is calculated. Specifically, the information providing apparatus 10 calculates the similarity between the first image and the second image as the first similarity, and sets the similarity between the first image and the third image as the second similarity. Calculate as degrees. That is, the information providing apparatus 10 uses the first image as a reference image, and calculates the first similarity between the reference image and the second image, and the second similarity between the reference image and the third image.

  Then, based on the relationship between the first similarity and the second similarity, the information providing apparatus 10 has one of the distributed representations of the second image and the third image similar to the distributed representation of the first image. , Generate a distributed representation of each image so that the other is not similar. For example, when the first similarity is greater than the second similarity, the information providing apparatus 10 has a similar distributed representation of the first image and the distributed representation of the second image, and the first and second distributed representations of the first image and the second image. Each distributed representation is generated so that the distributed representation of the three images is not similar. On the other hand, when the second similarity is larger than the first similarity, the information providing apparatus 10 has the distributed representation of the first image similar to the distributed representation of the third image, and the distributed representation of the first image and the first representation. Each distributed representation is generated so that the two images are not similar to the distributed representation.

  That is, the information providing apparatus 10 does not select an image that is correct answer data or incorrect answer data depending on whether the similarity between the reference image and another image is higher than a predetermined threshold value. Based on the similarity between the reference image and the second image and the similarity between the reference image and the third image, images that are relative correct answer data and incorrect answer data are set. The information providing apparatus 10 learns the distributed expression so that the distributed expressions of the correct answer pairs are similar and the distributed expressions of the incorrect answer pairs are not similar.

  That is, even when an image similar to the reference image is selected as the second image and the third image, the information providing apparatus 10 calculates relative correct answer data and incorrect answer data according to the similarity to the reference image. Set and generate a distributed representation of each image. As a result of such processing, the information providing apparatus 10 can reflect the relative similarity between the images similar to the reference image in the distributed expression. Therefore, in the image search using the distributed expression, The ranking of images according to the similarity can be generated.

  Moreover, since the information providing apparatus 10 can reflect the relative similarity of each image in the distributed expression, each image can be dropped into the distributed expression space in which it is guaranteed that a distance function satisfying the distance axiom can be used. it can. For example, in a technology that uses an intermediate representation generated from an image as a feature value by a multistage neural network used for deep learning etc., it is unclear what distance function is applicable, so the feature values can be simply Comparison becomes difficult. On the other hand, since the information providing apparatus 10 generates a distributed expression so as to reflect the relative similarity of each image, learning of the distributed expression is performed so that the distributed expression space becomes the Euclidean space. For this reason, the information providing apparatus 10 easily realizes a similar image search, for example, by calculating the Euclidean distance between the distributed representation of an image serving as a search query and the distributed representation of another image during an image search. be able to.

[1-3. About similarity)
Here, when calculating the degree of similarity between the images, the information providing apparatus 10 may calculate the degree of similarity of each image according to an arbitrary criterion according to the purpose of generating the distributed representation. For example, the information providing apparatus 10 may calculate the first similarity and the second similarity based on various structural (appearance) similarities such as color, pixel, and edge. Further, the information providing apparatus 10 may calculate the semantic similarity (that is, semantic similarity) of each image.

  For example, the information providing apparatus 10 uses a model in which a feature of an imaging target is extracted from an image using a technique in ILSVRC (ImageNet Large Scale Visual Recognition Challenge) or the like, and uses an imaging target in each image. And the first similarity and the second similarity indicating the similarity of the extracted features may be calculated. For example, the information providing apparatus 10 identifies features such as the type and color of the imaging target captured in each image, and is based on the commonality and similarity (hereinafter referred to as “similarity”) of the identified features. Thus, the first similarity and the second similarity may be calculated. Such a model may be realized by a neural network such as a convolutional neural network (CNN). Further, the information providing apparatus 10 classifies the imaging target of each image using a model that classifies the imaging target based on the Fisher vector of each image, and the first similarity indicating the similarity of the classification result and The second similarity may be calculated.

  In addition, the information providing apparatus 10 determines the first based on the semantic similarity of the image, such as the characteristics of the imaging target, the purpose of use of the image such as which transaction target the image is used to describe in the electronic commerce, and the like. The similarity and the second similarity may be calculated. For example, each image may be associated with tag information indicating the meaning of various images such as the characteristics of the imaging target and the purpose of the image. In addition, for example, in web content in which each image is posted, information that can be associated with various images such as the caption of the image, the name of the imaging target (for example, the name of the transaction target), the price of the imaging target, etc. is posted. There may be. Therefore, the information providing apparatus 10 collects various types of information that can be associated with each image as meta information indicating the meaning of the image. Then, the information providing apparatus 10 calculates the first similarity based on the similarity between the meta information of the first image and the meta information of the second image, and the meta information of the first image and the meta information of the third image. The second similarity degree may be calculated based on the similarity degree. That is, the information providing apparatus 10 may calculate the first similarity and the second similarity according to not only the similarity of the image itself but also the similarity of the information associated with the image.

  Further, the information providing apparatus 10 calculates the first similarity based on the meaning or the similarity of the notation between the meta information of the first image and the meta information of the second image, and the meta information of the first image and the third information The second similarity may be calculated based on the meaning of the image meta information or the similarity of the notation. For example, the information providing apparatus 10 calculates the degree of coincidence of the text included in the meta information (that is, the notation similarity), and calculates the first similarity and the second similarity based on the calculated coincidence. Also good. Further, the information providing apparatus 10 calculates the similarity of the meaning of the text included in the meta information (that is, the similarity of meaning) using, for example, w2v, and based on the calculated similarity, the first similarity Alternatively, the second similarity may be calculated.

  In addition to the information included in the tag information attached to the image and various types of information posted on the same web content as the image, the information providing apparatus 10 may be, for example, text posted by the user on the microblog along with the image Any information of any type may be adopted as meta information as long as it is information associated with an image, such as a word included in. That is, the information providing apparatus 10 may adopt arbitrary information as meta information as long as it is information associated with an image and can indicate the semantic content of the image. In addition, the information providing apparatus 10 can store arbitrary types of information on the image meta data according to the criteria for performing similar image search, such as the color of the image, the shape of the imaging target, and the semantic meaning of the image. It may be adopted as information.

[1-4. About search queries)
Further, the information providing apparatus 10 may adopt the content of the search query corresponding to each image as meta information. For example, the information providing apparatus 10 receives a search query input by the user U from a search server or the like (not shown) that performs a web search, and an image displayed as a search result when the search query is input. A search log indicating the image selected by the user U is acquired. And the information provision apparatus 10 specifies the search query which the user U input when each image used as the production | generation object of a distributed expression was selected by the user U as meta information.

  Then, the information providing apparatus 10 includes the search query input by the user U when the first image is selected by the user U and the user U input when the second image is selected by the user U. The first similarity is calculated based on the similarity with the search query. In addition, the information providing apparatus 10 includes a search query input by the user U when the first image is selected by the user U, and a user U input when the third image is selected by the user U. Based on the similarity with the search query, the second similarity is calculated. And the information provision apparatus 10 produces | generates the dispersion | distribution expression according to the relative similarity of each image using the calculated 1st similarity and 2nd similarity.

  Here, it can be said that the search query input when the user U selects an image reflects the search intention when searching for the image. For this reason, the information providing apparatus 10 calculates the similarity of the search query input by each user U when each image is selected, so that the similarity of the search intention of the user U when searching for each image is calculated. Can be calculated. When the distributed representation of each image is generated so as to reflect such similarity, the search intention of the user U when searching for each image can be embedded in the distance in the distributed representation space. As a result, the information providing apparatus 10 can realize an image search that reflects the search intention of the user U using the distributed expression.

  For example, the information providing apparatus 10 acquires “shoes, blue, brandA, sizeA” as a search query input by the user U when the first image is selected, and the user when the second image is selected. “Shoes, blue, brandA, sizeB” is acquired as a search query input by U, and “shoes, red, brandB, sizeB” is acquired as a search query input by the user U when the third image is selected. Shall. In such a case, the information providing apparatus 10 calculates an IoU (Intersection over Union) between the search query corresponding to the first image and the search query corresponding to the second image as the first similarity.

  More specifically, the information providing apparatus 10 determines the number of common token types among the tokens included in the search query corresponding to the first image and the tokens included in the search query corresponding to the second image. Then, a value divided by the number of types of tokens included in each search query is calculated as the first similarity. In the example described above, the search query corresponding to the first image and the search query corresponding to the second image have three similar tokens “shoes”, “blue”, and “brandA”, and “shoes” is included in each query. ”,“ Blue ”,“ brandA ”,“ sizeA ”, and“ sizeB ”have appeared. For this reason, the information providing apparatus 10 makes the first similarity “0.6” obtained by dividing the number “3” of commonly included token types by the number “5” of token types included in each query. Calculate as degrees. Similarly, the information providing apparatus 10 calculates IoU (for example, “0.33”) between the search query corresponding to the first image and the search query corresponding to the third image as the second similarity.

  In such a case, the first similarity value is larger than the second similarity value. For this reason, it can be said that the first image and the second image are more similar than the first image and the third image. Therefore, the information providing apparatus 10 makes the distributed representation of the first image and the distributed representation of the second image similar, and the distributed representation of the first image and the distributed representation of the third image are not similar. Learn distributed representation.

  Here, the information providing apparatus 10 may calculate the first similarity and the second similarity in consideration of the token amount of the search query. For example, when the amount of tokens in the search query is large, it is estimated that the search query shows the search intention of the user U more clearly than other search queries. Therefore, the information providing apparatus 10 extracts a search query in which the amount of tokens exceeds a predetermined threshold from the search queries input by the user U when each image is selected, and sets the similarity of the extracted search queries. Based on this, the first similarity and the second similarity may be calculated. For example, the information providing apparatus 10 may extract a search query including four or more tokens, or may extract only a search query having the largest number of tokens among search queries acquired from an external server or the like. . Further, the information providing apparatus 10 may set the token amount as a threshold so that a search query input by a majority of users who have selected images is included.

[1-5. (Similarity based on multiple types of information)
Here, the information providing apparatus 10 calculates the first similarity based on the similarity for each type of the plurality of types of information associated with the first image and the plurality of types of information associated with the second image. The second similarity may be calculated based on the similarity for each type of the plurality of types of information associated with the first image and the plurality of types of information associated with the third image. That is, the information providing apparatus 10 may calculate the first similarity and the second similarity based on the similarity as the whole meta information when meta information including plural types of information is associated with the image. The first similarity and the second similarity may be calculated based on the similarity for each type.

  For example, the information providing apparatus 10 includes information indicating structural features of an image (hereinafter referred to as “structural information”) and information indicating semantic features of the image such as an imaging target (hereinafter referred to as “semantic information”). ").” Is acquired. In such a case, the information providing apparatus 10 uses the similarity between the structure information of the first image and the structure information of the second image (hereinafter referred to as “structure similarity”), and the semantic information of the first image. The first similarity may be calculated based on the similarity between the second image and the semantic information of the second image (hereinafter referred to as “meaning similarity”).

  Moreover, the information provision apparatus 10 may calculate the 1st similarity which considered weighting for every classification. For example, the information providing apparatus 10 may calculate the first similarity based on a value obtained by integrating the first priority with the structural similarity and a value obtained by integrating the second priority with the semantic similarity. . By setting such priorities, whether the information providing apparatus 10 generates a distributed expression that emphasizes the structural similarity of the image or generates a distributed expression that emphasizes the semantic similarity of the image Can be set flexibly.

  In addition, the information providing apparatus 10 may calculate the first similarity and the second similarity having a predetermined number of digits by combining the similarities for each type in descending order of priority. For example, in the case of calculating a 128-bit value as the first priority, the information providing apparatus 10 calculates a 64-bit structural similarity and a semantic similarity. Then, for example, the information providing apparatus 10 combines the semantic similarity as an upper digit and the structural similarity as a lower digit, thereby obtaining a 128-bit first similarity to which the semantic similarity contributes more preferentially. It may be calculated.

  In addition, when the information providing apparatus 10 calculates the structural similarity and the semantic similarity in decimal numbers, for example, a value obtained by integrating the semantic similarity with 10 to the nth power (n is the number of digits of the structural similarity) is used. The first similarity to which the semantic similarity contributes more preferentially may be calculated by calculating and adding the structural similarity to the calculated value. Further, the information providing apparatus 10 calculates the similarity for each type even when the number of types of information included in the meta information is three or more, and the type used with higher priority among the calculated similarities The first similarity degree and the second similarity degree may be calculated by combining the priorities with the similarity degree in the higher order.

[1-6. (Selecting images)
Here, when the information providing apparatus 10 selects the first image to the third image, the information providing device 10 may select the first image to the third image for all combinations from the images for which the distributed representation is to be generated. In addition, the information providing apparatus 10 intentionally selects information whose similarity with the first image exceeds a predetermined threshold, such as an image belonging to the same field (class) as the first image, so that images between images belonging to the field The relative similarity may be dropped on the distributed expression space.

  In addition, the information providing apparatus 10 may narrow the field that is the selection source of the first image to the third image in stages in order to efficiently advance the learning of the distributed expression. For example, in the initial stage of learning, the information providing apparatus 10 selects first to third images randomly from images belonging to all categories, and learning progresses (for example, the accuracy of distributed representation is a predetermined threshold value). 1) to 3rd image are selected at random from images belonging to a predetermined category. When learning further proceeds, first images are randomly selected from images belonging to subcategories included in the predetermined category. An image to a third image may be selected. That is, the information providing apparatus 10 may increase the similarity of images to be selected each time learning progresses.

[1-7. (About multi-stage learning)
Further, the information providing apparatus 10 may execute a combination of the conventional triplet loss learning method and the generation process described above. For example, the information providing apparatus 10 selects a fourth image whose similarity with the first image is greater than or equal to the first threshold and a fifth image whose similarity with the first image is less than or equal to the second threshold. In such a case, the information providing apparatus 10 ensures that the distributed representation of the first image and the distributed representation of the fourth image are similar, and the distributed representation of the first image and the distributed representation of the fifth image are not similar. Generate a distributed representation of each image.

  Then, the information providing apparatus 10 calculates the first similarity and the second similarity for all the sets including three pieces of information from the first image to the fifth image. That is, the information providing apparatus 10 generates all combinations of the reference image and the two images, and calculates the first similarity and the second similarity for the generated combination. Then, the information providing apparatus 10 sets the correct answer data and the incorrect answer data in each combination using the first similarity and the second similarity, the distributed expressions of the correct answer pairs are similar, and the distributed expressions of the incorrect answer pairs are similar. Learn distributed expressions so that they don't.

  For example, in the initial stage of generating the distributed representation, the information providing apparatus 10 has the similarity between the reference image and the correct answer data in which the similarity between the reference image exceeds a predetermined threshold and the reference image is below the predetermined threshold. Incorrect data is selected, and distributed expressions are learned so that the distributed expressions of correct pairs are similar and the distributed expressions of incorrect pairs are not similar.

  Then, the information providing apparatus 10 calculates the accuracy of the distributed representation at a predetermined timing. For example, the information providing apparatus 10 generates a ranking of an image similar to a predetermined image using the distributed representation, and ranks an image similar to the predetermined image based on the similarity of meta information of each image. Generate. Then, the information providing apparatus 10 calculates the accuracy of the distributed expression based on the degree of coincidence between the ranking using the distributed expression and the ranking using the meta information.

  Here, when the accuracy of the distributed representation exceeds a predetermined threshold or when the increase rate of the accuracy of the distributed representation does not increase for a predetermined period, the information providing apparatus 10 executes the generation process described above. To do. That is, the information providing apparatus 10 selects the first image to the third image, calculates the first similarity and the second similarity based on the similarity between the selected first image to the third image, and calculates Based on the comparison result of the first similarity and the second similarity, the second image and the third image are set as correct answer data and incorrect answer data. Then, the information providing apparatus 10 learns the distributed expression so that the distributed expressions of the correct answer pairs are similar and the distributed expressions of the incorrect answer pairs are not similar.

[1-8. About generation)
Here, the information providing apparatus 10 generates a distributed representation by an arbitrary method so long as the distributed representation is generated so that the distributed representation of the correct answer pair is similar and the distributed representation of the incorrect answer pair is not similar. Good. For example, the information providing apparatus 10 sets each difference such that the difference between the distributed representation of the first image and the distributed representation of the second image is smaller than the difference between the distributed representation of the first image and the distributed representation of the third image. A distributed representation may be generated.

  Hereinafter, an example of a mathematical expression used by the information providing apparatus 10 to generate a distributed expression will be described. For example, the information providing apparatus 10 generates a distributed representation of each image according to the triplet loss method using Expression (1).

Here, x ^a _i in Equation (1) indicates a reference image, x ^p _i indicates an image of correct data set based on the first similarity and the second similarity, and x ⁿ _i is An image of incorrect answer data set based on the first similarity and the second similarity is shown. F (x) in Expression (1) is a distributed representation of the image x, and indicates a distributed representation of a predetermined number of dimensions. In addition, α in Expression (1) is a predetermined coefficient. The information providing apparatus 10 sets the value of the distributed representation f (x) of each image so that the value of L in Expression (1) is maximized.

  Here, in the conventional triplet loss, absolute correct data and incorrect data are set according to the degree of similarity between the reference image and another image. Therefore, when the meta information of the image x is described as w (x), the relationship between the reference image, the correct data, and the incorrect data in the conventional triplet loss is expressed by the following formulas (2) and (3). .

  On the other hand, the information providing apparatus 10 does not select an image to be learned as absolute correct answer data and incorrect answer data, but instead of selecting correct answer data from the selected image based on the relative similarity of the selected image. Set incorrect data and generate a distributed representation of each image. For this reason, the relationship between the reference image selected by the information providing apparatus 10, the correct answer data, and the incorrect answer data is expressed by the following expressions (4) and (5).

  As a result of such processing, the information providing apparatus 10 not only has a relationship between images similar to and not similar to the reference image, but also a relationship between a plurality of images similar to the reference image, and is not similar to the reference image. The relationship between multiple images is also included in the distributed expression. As a result, the information providing apparatus 10 can reduce the relative similarity of all the images to be processed on the distributed representation section, and thus can improve the accuracy of the distributed representation.

[1-9. (Example of generation process)
Next, an example of a generation process executed by the information providing apparatus 10 will be described with reference to FIG. First, the information providing apparatus 10 randomly selects the first image P1, the second image P2, and the third image P3 from the images to be processed, and acquires meta information M1 to M3 of each image (step S1). . In such a case, the information providing apparatus 10 calculates the similarity between the meta information and compares the calculated similarities (step S2). For example, the information providing apparatus 10 uses the similarity S1 between the meta information M1 and the meta information M2 (that is, the first similarity) and the similarity S2 between the meta information M1 and the meta information M3 (that is, the second similarity). And the calculated similarity S1 and similarity S2 are compared.

  And the information provision apparatus 10 produces | generates the dispersion | distribution expression of each image according to the comparison result of similarity (step S3). For example, when the value of the similarity S1 is larger than the value of the similarity S2, the information providing apparatus 10 sets the second image as correct answer data and the third image as incorrect answer data. Then, the information providing apparatus 10 is configured so that the distributed representation P1 of the first image and the distributed representation P2 of the second image are similar, and the distributed representation P1 of the first image and the distributed representation P3 of the third image are not similar. Each distributed representation P1-P3 is generated. On the other hand, when the value of the similarity S2 is larger than the value of the similarity S1, the information providing apparatus 10 sets the third image as correct answer data and the second image as incorrect answer data. Then, the information providing apparatus 10 is configured such that the distributed representation P1 of the first image and the distributed representation P3 of the third image are similar, and the distributed representation P1 of the first image and the distributed representation P2 of the second image are not similar. Each distributed representation P1-P3 is generated.

  In addition, the information providing apparatus 10 performs the same processing between other images (step S4). More specifically, the information providing apparatus 10 generates triples of all combinations in which an image that is a reference image and images other than the reference image are the second image and the third image. And the information provision apparatus 10 produces | generates the dispersion | distribution expression of all the images by performing step S1-step S3 about all the produced | generated triples.

  Next, an example of search processing executed by the information providing apparatus 10 will be described. First, the information providing apparatus 10 receives a search query from the user terminal 100 (step S5). For example, the information providing apparatus 10 receives a search query q1 as a search query from the user terminal 100.

  In such a case, the information providing apparatus 10 specifies an image corresponding to the search query q1, and generates a ranking of images according to the search query based on the distance from the distributed representation of the specified image (step S6). . For example, the information providing apparatus 10 selects the first image P1 that is most relevant to the search query q1. In such a case, the information providing apparatus 10 calculates the Euclidean distance between the distributed representation P1 of the image P1 and the other distributed representations P2 to P4.

  For example, when the Euclidean distance from the distributed representation P1 is short in the order of the distributed representation P3, the distributed representation P2, and the distributed representation P4, the information providing apparatus 10 displays the image P3 and the distributed representation P2 corresponding to the distributed representation P3. A search result in a ranking format in which the images P2 to P4 are arranged in the order of the corresponding image P2 and the image P4 corresponding to the distributed representation P4 is generated. Then, the information providing apparatus 10 provides the generated search result in the ranking format to the user terminal 100 (step S7).

[2. Configuration of information providing device]
Subsequently, an example of a functional configuration of the information providing apparatus 10 described above will be described. FIG. 2 is a diagram illustrating a configuration example of the information providing apparatus according to the embodiment. As illustrated in FIG. 2, the information providing apparatus 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

  The communication unit 20 is realized by, for example, a NIC (Network Interface Card). The communication unit 20 is connected to the network N by wire or wireless, and transmits / receives information to / from the user terminal 100.

  The storage unit 30 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 30 also stores an image database 31 and a distributed representation database 32.

  In the image database 31, an image for which a distributed expression is to be generated is registered. For example, FIG. 3 is a diagram illustrating an example of information registered in the image database according to the embodiment. As shown in FIG. 3, information having items such as “image ID (Identifier)”, “image data”, and “meta information” is registered in the image database 31. In the “meta information”, information such as “search query”, “tag information”, and “feature information” with different priorities is registered.

  Here, the “image ID” is an image identifier. The “image data” is image data of an image indicated by the associated “image ID”. The “meta information” is meta information given to the image indicated by the associated “image ID”. For example, “search query” is meta information of “priority 1” that is considered most preferentially when calculating the similarity between images, and the image indicated by the associated “image ID” is the user The search query entered by the user U when U is selected. The “tag information” is meta information of “priority 2” that is considered second in priority when calculating the similarity between images, and is included in the image indicated by the associated “image ID”. This is tag information previously assigned to the image, such as the characteristics of the imaged object. The “feature information” is meta information of “priority 3” that is considered with the third highest priority when calculating the similarity between images, and the image information indicated by the associated “image ID”. This is feature information indicating structural (appearance) features.

  For example, in the example shown in FIG. 3, the image ID “image ID # 1”, the image data “image data # 1”, the search query “search query # 1”, the tag information “tag information # 1”, and the feature information “feature” "Information # 1" is registered in association with each other. In such information, the image data of the image indicated by the image ID “image ID # 1” is “image data # 1”, and the search query input by the user U when the image is selected is “search query”. # 1 ”, tag information“ tag information # 1 ”is assigned, and the appearance characteristic of the image is“ feature information # 1 ”. In the example shown in FIG. 3, conceptual values such as “image ID # 1”, “image data # 1”, “search query # 1”, “tag information # 1”, “feature information # 1”, etc. Although described, actually, a character string for identifying an image, image data in various formats, a character string input as a search query, a character string included in tag information, a multidimensional quantity indicating a feature, and the like are registered It becomes.

  Returning to FIG. 2, the description will be continued. In the distributed expression database 32, distributed expressions of images are registered. For example, FIG. 4 is a diagram illustrating an example of information registered in the distributed expression database according to the embodiment. In the example illustrated in FIG. 4, information having items such as “image ID” and “distributed expression” is registered in the distributed expression database 32. Here, the “distributed expression” is a distributed expression generated from the image indicated by the associated “image ID”.

  For example, in the example illustrated in FIG. 4, information such as an image ID “image ID # 1” and a distributed expression “distributed expression # 1” is registered in association with each other. Such information indicates that the distributed representation of the image indicated by the image ID “image ID # 1” is “distributed representation # 1”. In the example shown in FIG. 4, a conceptual value such as “dispersion expression # 1” is described, but in practice, a multidimensional quantity generated as a dispersion expression is registered.

  Returning to FIG. 2, the description will be continued. The control unit 40 is a controller. For example, various programs stored in a storage device inside the information providing apparatus 10 are stored in a RAM or the like by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Is implemented as a work area. The control unit 40 is a controller, and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  As illustrated in FIG. 2, the control unit 40 includes a selection unit 41, a calculation unit 42, a comparison unit 43, a generation unit 44, a reception unit 45, and a search unit 46. The selection unit 41 selects an image to be processed. For example, the selection unit 41 refers to the image database 31 and selects one image to be a reference image. The selection unit 41 selects all combinations of two other images that are correct data or incorrect data with respect to the reference image. Then, the selection unit 41 notifies the calculation unit 42 of the selected triple image as a triple. In the distributed expression learning, the selection unit 41 selects all images as reference images, and generates all triples using the selected images as reference images.

  Note that when the distributed representation is generated step by step, the selection unit 41 selects, for example, a fourth image having a similarity with the reference image equal to or higher than the first threshold as correct data using an arbitrary technique, A fifth image having a similarity to the image equal to or lower than the second threshold value may be selected as incorrect answer data. Further, the selection unit 41 may select an image whose similarity with the reference image exceeds a predetermined threshold as the second image and the third image. Further, the selection unit 41 may gradually narrow down the field that is the source of selection of the correct answer data and the incorrect answer data step by step. For example, the selection unit 41 may select triple images so that the similarity of the meta information of each image data is gradually reduced.

  The calculation unit 42 calculates a first similarity that is a similarity between the reference image and the second image, and a second similarity that is a similarity between the reference image and the third image. For example, the calculation unit 42 calculates the first similarity based on the similarity between the meta information of the reference image and the meta information of the second image, and the similarity between the meta information of the reference image and the meta information of the third image Based on the degree, the second similarity is calculated.

  For example, the calculation unit 42 reads the meta information of the reference image and the tag information of the second image included in the triple from the image database 31. Then, the calculation unit 42 calculates a similarity such as the degree of coincidence of the read meta information and IoU, and calculates the first similarity based on the calculated similarity. Similarly, the calculation unit 42 calculates the second similarity from the similarity between the tag information of the reference image and the meta information of the third image.

  Note that the calculation unit 42 includes a plurality of types of meta information (for example, search query, tag information, feature information, etc.) associated with the reference image and a plurality of types of meta information (for example, associated with the second image or the third image). , Search query, tag information, feature information, etc.), the similarity is calculated for each type of meta information, and the first similarity and the second similarity are calculated based on the calculated plurality of similarities. May be. Further, the calculation unit 42 may calculate the first similarity and the second similarity with a predetermined number of digits by combining the similarity for each type in descending order of priority.

  For example, FIG. 5 is a diagram illustrating an example of the similarity calculated by the information providing apparatus according to the embodiment. In the example illustrated in FIG. 5, an example in which a decimal priority is calculated is described. For example, the calculation unit 42 calculates the similarity “AAAAA” between the search queries of the reference image and the second image, calculates the similarity “BBB” between the tag information, and calculates the similarity “CCC” between the feature information. calculate. In such a case, the calculation unit 42 calculates a coefficient α “1000000” corresponding to the number of digits of similarity between tag information and similarity between feature information with respect to the similarity “AAAAAA” of the search query having the highest priority. Accumulate. In addition, the calculation unit 42 accumulates a coefficient β “1000” corresponding to the number of digits of similarity between feature information to the similarity “BBB” of the search query having the second highest priority. In addition, the calculation unit 42 adds the coefficient γ “1” to the similarity “CCC” of the search query having the third highest priority. Then, the calculation unit 42 sets the value “AAAAAABBBCCC” obtained by adding the coefficient to each similarity as the first similarity.

  For example, the calculation unit 42 calculates the similarity “aaaaaa” between the search queries of the reference image and the third image, calculates the similarity “bbb” between the tag information, and calculates the similarity “ccc” between the feature information. calculate. In such a case, the calculation unit 42 adds the coefficient α to the similarity “aaaaaa” of the search query with the highest priority, and adds the coefficient β “to the similarity“ bbb ”of the search query with the second highest priority. 1000 ”and the coefficient γ is added to the similarity“ ccc ”of the search query having the third highest priority. Then, the calculation unit 42 sets the value “aaaaaabbbccc” obtained by adding the coefficient to each similarity as the second similarity.

  When the first similarity and the second similarity calculated in this way are compared, the type of meta information having a higher priority is more preferentially reflected in the comparison result. For this reason, the calculation unit 42 can easily compare a plurality of types of meta information having different priorities.

  Note that the calculation unit 42 determines the similarity of each image using various meta information registered in the image database 31, thereby meaning or notation of the meta information of the reference image and the meta information of the second image. The first similarity is calculated based on the similarity, and the second similarity is calculated based on the similarity between the meta information of the reference image and the meta information of the third image or the notation similarity. For example, the calculation unit 42 calculates the search query input by the user U when the reference image is selected by the user U and the search query input by the user U when the second image is selected by the user U. The first similarity is calculated based on the similarity to the search query entered by the user U when the reference image is selected by the user U, and the third image is selected by the user U. The second similarity is calculated based on the similarity with the search query input by the user U.

  Note that the calculation unit 42 does not depend on the identity of the user U who selected each image, but based on the similarity of the search query input by each user when each image is selected, The second similarity may be calculated. The calculating unit 42 calculates the first similarity based on the structural similarity between the reference image and the second image and the semantic similarity between the reference image and the second image, The second similarity is calculated based on the structural similarity with the third image and the semantic similarity between the reference image and the third image.

  Note that the calculation unit 42 may calculate the first similarity and the second similarity based on the similarity of search queries in which the amount of tokens exceeds a predetermined threshold among the search queries. In addition, when performing stepwise learning, the calculation unit 42 calculates the first similarity and the second similarity between the fourth image and the fifth image selected by the selection unit 41 and the reference image. It will be.

  Returning to FIG. 2, the description will be continued. The comparison unit 43 compares the first similarity and the second similarity calculated for each triple by the calculation unit 42 and notifies the generation unit 44 of the comparison result. For example, the comparison unit 43 determines which of the first similarity value and the second similarity value is greater, and notifies the generation unit 44 of the determination result.

  When the first similarity is greater than the second similarity, the generation unit 44 resembles the distributed representation of the reference image and the distributed representation of the second image, and the distributed representation of the reference image and the distributed representation of the third image. Each distributed expression is generated so that and are not similar to each other. In addition, when the second similarity is greater than the first similarity, the generation unit 44 is similar to the distributed representation of the reference image and the distributed representation of the third image, and the distributed representation of the reference image and the second image Each distributed representation is generated so as not to be similar to the distributed representation. Then, the generation unit 44 registers the image ID of each image and the generated distributed expression in the distributed expression database 32.

  For example, when the first similarity is greater than the second similarity, the generation unit 44 sets the second image included in the triple as correct data and the third image as incorrect data. Further, when the second similarity is greater than the first similarity, the generation unit 44 sets the third image included in the triple as correct data and sets the second image as incorrect data. That is, the generation unit 44 sets correct answer data and incorrect answer data based on a comparison result between the first similarity and the second similarity. Then, the generation unit 44 generates a correct answer pair and an incorrect answer pair from each triple, and generates a distributed representation of each image so that the value of L in Expression (1) is maximized.

  When the generation unit 44 generates the distributed representation step by step, the distributed representation of the reference image is similar to the distributed representation of the fourth image, and the distributed representation of the reference image and the distributed representation of the fifth image are different. A distributed representation of the reference image, the fourth image, and the fifth image is generated so as not to be similar. Then, the generation unit 44 calculates the accuracy of the generated distributed representation, and when the calculated accuracy exceeds a predetermined threshold or when the calculated accuracy does not increase for a predetermined period, the first similarity and the second similarity The correct answer data and the incorrect answer data may be set based on the degree comparison result, and the distributed expression may be corrected based on the setting result. Further, the generation unit 44 sets each distributed representation so that the difference between the distributed representation of the reference image and the distributed representation of the second image is smaller than the difference between the distributed representation of the reference image and the distributed representation of the third image. It may be generated.

  The accepting unit 45 accepts a search query from the user terminal 100. In such a case, the reception unit 45 identifies an image corresponding to the search query. For example, when accepting text as a search query, the accepting unit 45 refers to the image database 31 and selects an image associated with meta information having the highest degree of matching with the text of the search query. For example, when receiving an image as a search query, the receiving unit 45 refers to the image database 31 and searches for an image having the highest degree of similarity with the image serving as the search query. Note that the reception unit 45 uses, for example, a predetermined learning model that has learned semantic and structural similarity between images or between an image and text, and an image corresponding to a search query. You may search for.

  The search unit 46 specifies images similar to the search query in a ranking format. For example, the search unit 46 acquires a distributed representation of the image (hereinafter referred to as “query image”) specified by the reception unit 45 as a result of the search from the distributed representation database 32. Then, the search unit 46 calculates the distance between the acquired distributed representation and the distributed representation registered in the distributed representation database 32, and specifies a predetermined number of distributed representations in order from the shortest distance. Further, the search unit 46 specifies an image associated with the specified distributed expression from the distributed expression database 32, and reads image data of the specified image from the image database 31. Then, the search unit 46 generates content in which the read image data is arranged in the ranking format in order from the closest distance to the distributed representation of the query image, and provides the generated content to the user terminal 100. .

[3. Example of flow of processing executed by information providing apparatus]
Subsequently, a flow of generation processing executed by the information providing apparatus 10 will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of a flow of generation processing executed by the information providing apparatus according to the embodiment. Note that the information providing apparatus 10 can execute the process illustrated in FIG. 6 in an arbitrary unit at an arbitrary timing.

  First, the information providing apparatus 10 selects a set of a first image, a second image, and a third image (step S101). The information providing apparatus 10 calculates the first similarity between the meta information of the first image and the meta information of the second image, and the second similarity of the meta information of the first image and the meta information of the third image. (Step S102).

  Subsequently, the information providing apparatus 10 determines whether or not the first similarity is larger than the second similarity (step S103). If the first similarity is larger (step S103: Yes), the second image is set as correct data, The third image is set as incorrect answer data (step S104). On the other hand, when the first similarity is not greater than the second similarity (step S103: No), the information providing apparatus 10 sets the second image as incorrect data and sets the third image as correct data (step S105). ). Then, the information providing apparatus 10 generates the distributed representation so that the distributed representation of the first image is similar to the distributed representation of the correct answer data, and the distributed representation of the first image is not similar to the distributed representation of the incorrect answer data. (Step S106), the process ends.

[4. (Modification)
In the above, an example of generation processing and search processing by the information providing apparatus 10 has been described. However, the embodiment is not limited to this. Hereinafter, the variation of the provision process and provision process which the information provision apparatus 10 performs are demonstrated.

[4-1. (Target information)
In the example described above, the information providing apparatus 10 generates a distributed expression that relatively reflects the semantic and structural similarity between images. However, the embodiment is not limited to this. For example, in addition to images such as still images and moving images, the information providing apparatus 10 makes relative semantic and structural similarities between arbitrary information such as music, movies, news articles, various posts, and web contents. Alternatively, a distributed expression reflecting the above may be generated. In such a case, the information providing apparatus 10 employs, for example, a music composer, lyrics content, performance time, performance date, movie director, performer, screening time, news article or summary of posted content as meta information. May be.

  Further, the information providing apparatus 10 may generate a distributed representation of each piece of information using triples including different types of information. For example, the information providing apparatus 10 calculates and calculates the first similarity between the image and the text and the second similarity between the image and the music for the triple including the image, the text, and the music. Depending on the comparison result between the first similarity and the second similarity, the correct answer data or the incorrect answer data may be used to generate a distributed representation of each information.

  That is, the information providing apparatus 10 performs the first similarity between the first information of the arbitrary type and the second information of the arbitrary type, and the second between the first information and the third information of the arbitrary type. When the similarity is calculated and the first similarity is greater than the second similarity, the distributed representation of the first information is similar to the distributed representation of the second information, and the distributed representation of the first information and the third information Each distributed representation may be generated so that the distributed representation of information is not similar. In addition, when the second similarity is greater than the first similarity, the information providing apparatus 10 has a similar distributed representation of the first information and the distributed representation of the third information, and the first and second distributed representations of the first information and the first information. Each distributed representation may be generated so that the two information distributed representations are not similar.

[4-2. Device configuration〕
The databases 31 and 32 registered in the storage unit 30 may be held in an external storage server. Further, the information providing apparatus 10 may be realized by a front-end server that realizes a search process and a back-end server that realizes a generation process. In such a case, the reception unit 45 and the search unit 46 shown in FIG. 2 are arranged in the front end server, and the selection unit 41, the calculation unit 42, the comparison unit 43, and the generation unit 44 are arranged in the back end server. Is done.

[4-3. Comparison results between first similarity and second similarity]
In the example described above, the information providing apparatus 10 sets the second information as correct data when the first similarity is greater than the second similarity, and sets the third information as correct data in other cases. However, the embodiment is not limited to this. For example, the information providing apparatus 10 may reselect the second information or the third information when the first similarity is equal to the second similarity.

  For example, when the first similarity is greater than the second similarity, the information providing apparatus 10 sets the second information as correct data, and when the second similarity is greater than the first similarity, Correct data. On the other hand, when the first similarity is the same as the second similarity, the information providing apparatus 10 reselects new second information and third information at random. At this time, for example, the information providing apparatus 10 may randomly select the second information and the third information in the same or similar field as the first information, and select either the second information or the third information, You may select again. In addition, when the first similarity is the same as the second similarity, the information providing apparatus 10 probabilistically determines whether or not to limit the field that is the selection source, You may reselect at least one of 2nd information or 3rd information from the information which belongs to the field | area which is the same as or similar to information, ie, the information whose similarity is higher than a predetermined threshold value.

[4-4. Others]
In addition, among the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, and conversely, the processes described as being performed manually. All or a part of the above can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  In addition, the above-described embodiments can be appropriately combined within a range in which processing contents do not contradict each other.

[4-5. program〕
In addition, the information providing apparatus 10 according to the above-described embodiment is realized by a computer 1000 configured as shown in FIG. 7, for example. FIG. 7 is a diagram illustrating an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected via a bus 1090. Have

  The arithmetic device 1030 operates based on a program stored in the primary storage device 1040 and the secondary storage device 1050, a program read from the input device 1020, and the like, and executes various processes. The primary storage device 1040 is a memory device such as a RAM that temporarily stores data used by the arithmetic device 1030 for various arithmetic operations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, such as ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, and the like. It is realized by.

  The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information such as a monitor and a printer. For example, USB (Universal Serial Bus), DVI (Digital Visual Interface), This is realized by a standard connector such as HDMI (registered trademark) (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, and a scanner, and is realized by, for example, a USB.

  The input device 1020 includes, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), and a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), and a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. The input device 1020 may be an external storage medium such as a USB memory.

  The network IF 1080 receives data from other devices via the network N and sends the data to the arithmetic device 1030, and transmits data generated by the arithmetic device 1030 to other devices via the network N.

  The arithmetic device 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

  For example, when the computer 1000 functions as the information providing device 10, the arithmetic device 1030 of the computer 1000 implements the function of the control unit 40 by executing a program loaded on the primary storage device 1040.

[5. effect〕
As described above, the information providing apparatus 10 includes the first similarity that is the similarity between the first information and the second information, and the second similarity that is the similarity between the first information and the third information. Calculate the degree. Then, the information providing apparatus 10 determines that one of the distributed representations of the second image and the third image is similar to the distributed representation of the first image based on the relationship between the first similarity and the second similarity. Then, a distributed representation of each image is generated so that the other is not similar. For example, when the first similarity is larger than the second similarity, the information providing apparatus 10 has a similar distributed representation of the first information and the distributed representation of the second information, and the first information and the distributed representation of the first information. Each distributed representation is generated so that the distributed representation of the three information is not similar. In addition, when the second similarity is greater than the first similarity, the information providing apparatus 10 has a similar distributed representation of the first information and the distributed representation of the third information, and the first and second distributed representations of the first information and the first information. Each distributed representation is generated so that the two information distributed representations are not similar.

  As a result of such processing, the information providing apparatus 10 can drop the relative similarity of each information on the distributed expression space, so that the distributed expression space that guarantees that the distance function satisfying the distance axiom can be used. Can be generated. That is, the information providing apparatus 10 can reflect the similarity of information on the distance in the distributed expression space.

  The information providing apparatus 10 calculates the first similarity based on the similarity between the meta information associated with the first information and the meta information associated with the second information, and associates the first information with the first information. The second similarity is calculated based on the similarity between the meta information and the meta information associated with the third information. For this reason, the information providing apparatus 10 can reflect the semantic similarity of each information in the distributed expression.

  Further, the information providing device 10 calculates the first similarity based on the similarity for each type of the plurality of types of information linked to the first information and the plurality of types of information linked to the second information, The second similarity is calculated based on the similarity for each type of the plurality of types of information associated with the first information and the plurality of types of information associated with the third information. For this reason, the information provision apparatus 10 can reflect the similarity in various viewpoints to a distributed expression integratively.

  Moreover, the information provision apparatus 10 calculates the 1st similarity and 2nd similarity of predetermined number of digits by combining the similarity for every classification in order with high priority. For this reason, the information provision apparatus 10 can determine the relative similarity of each information in consideration of various hierarchical priorities.

  Further, the information providing device 10 calculates the first similarity based on the meaning or notation similarity between the information associated with the first information and the information associated with the second information, and associates the information with the first information. The second similarity is calculated on the basis of the meaning or notation similarity between the information attached and the information associated with the third information. For this reason, the information providing apparatus 10 can reflect the semantic and structural similarity of each information in the distributed expression.

  In addition, the information providing apparatus 10 includes a search query input by the user U when the first information is selected by the user U and a user U input when the second information is selected by the user U. The first similarity is calculated based on the similarity with the search query. In addition, the information providing apparatus 10 includes a search query input by the user U when the first information is selected by the user U and a user U input when the third information is selected by the user U. Based on the similarity with the search query, the second similarity is calculated. For this reason, the information providing apparatus 10 can reflect the similarity of the search intention of the user U in the distributed expression.

  Further, the information providing apparatus 10 calculates the first similarity and the second similarity based on the similarity of the search query in which the amount of tokens exceeds a predetermined threshold among the search queries. For this reason, the information providing apparatus 10 can reflect the search intention of the user U in the distributed expression with higher accuracy.

  Further, the information providing apparatus 10 calculates the first similarity based on the structural similarity between the first information and the second information and the semantic similarity between the first information and the second information, The second similarity is calculated based on the structural similarity between the first information and the third information and the semantic similarity between the first information and the third information. Therefore, the information providing apparatus 10 can reflect the semantic similarity and structural similarity of each information in the distributed expression.

  Further, the information providing apparatus 10 selects, as the second information and the third information, information whose similarity with the first information exceeds a predetermined threshold, and the first information, the selected second information and the third information From the above, the first similarity and the second similarity are calculated. For this reason, the information providing apparatus 10 can reflect the relative similarity between pieces of information that are similar to each other in the distributed expression.

  In addition, the information providing apparatus 10 selects the fourth information whose similarity with the first information is equal to or higher than the first threshold and the fifth information whose similarity with the first information is equal to or lower than the second threshold. Further, the information providing apparatus 10 calculates the first similarity and the second similarity for all the sets including three pieces of information from the first information to the fifth information. Then, the information providing apparatus 10 includes the first information so that the distributed representation of the first information is similar to the distributed representation of the fourth information, and the distributed representation of the first information is not similar to the distributed representation of the fifth information. A distributed representation of the first information to the fifth information included in the set is generated based on the first similarity and the second similarity calculated for each set after generating the distributed representation of the information, the fourth information, and the fifth information. Is generated. For this reason, the information providing device 10 can realize efficient distributed representation learning.

  Further, the information providing apparatus 10 is configured so that the difference between the distributed representation of the first information and the distributed representation of the second information is smaller than the difference between the distributed representation of the first information and the distributed representation of the third information. Generate a distributed representation. For this reason, the information providing apparatus 10 can appropriately generate the distributed representation.

  The information providing apparatus 10 also includes a first similarity between the first information that is an image and the second information that is an image, and a second similarity between the first information and the third information that is an image. Is calculated. For this reason, the information providing apparatus 10 can reflect the relative similarity between images in the distributed representation of each image.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications, including the aspects described in the disclosure section of the invention, based on the knowledge of those skilled in the art, It is possible to implement the present invention in other forms with improvements.

  Moreover, the above-mentioned “section (module, unit)” can be read as “means”, “circuit”, and the like. For example, the generation unit can be read as generation means or a generation circuit.

DESCRIPTION OF SYMBOLS 10 Information providing apparatus 20 Communication part 30 Storage part 31 Image database 32 Distributed expression database 40 Control part 41 Selection part 42 Calculation part 43 Comparison part 44 Generation part 45 Reception part 46 Search part 100 User terminal

Claims (16)

A calculation unit that calculates a first similarity that is a similarity between the first information and the second information, and a second similarity that is a similarity between the first information and the third information;
Based on the relationship between the first similarity and the second similarity, one of the distributed representations of the second information and the third information is similar to the distributed representation of the first information, and the other is A generation device comprising: a generation unit configured to generate each distributed expression so as not to be similar. When the first similarity is greater than the second similarity, the generation unit is configured such that the distributed representation of the first information and the distributed representation of the second information are similar, and the distributed representation of the first information The generation apparatus according to claim 1, wherein each distributed expression is generated so that the distributed expression of the third information is not similar to the distributed expression. When the second similarity is greater than the first similarity, the distributed representation of the first information and the distributed representation of the third information are similar, and the distributed representation of the first information and the distributed of the second information The generation apparatus according to claim 1, wherein each distributed expression is generated so that the expression is not similar to the expression. The calculation unit calculates the first similarity based on the similarity between the information associated with the first information and the information associated with the second information, and information associated with the first information The generation device according to any one of claims 1 to 3, wherein the second similarity is calculated based on a similarity between the first information and information associated with the third information. The calculation unit calculates the first similarity based on a similarity for each type of information of a plurality of types associated with the first information and information of a plurality of types associated with the second information, The second similarity is calculated based on a similarity for each type of information of a plurality of types associated with the first information and information of a plurality of types associated with the third information. Item 5. The generating device according to Item 4. 6. The calculation unit according to claim 5, wherein the calculation unit calculates the first similarity and the second similarity of a predetermined number of digits by combining similarities of the types in descending order of priority. The generator described. The calculation unit calculates the first similarity based on the meaning or notation similarity between the information associated with the first information and the information associated with the second information, and includes the first information in the first information. The second similarity is calculated based on the meaning or notation similarity between the information associated with the information and the information associated with the third information. The generating device described in 1. The calculation unit includes a search query input by the user when the first information is selected by the user, and a search query input by the user when the second information is selected by the user. The first similarity is calculated based on the similarity, and the search query input by the user when the first information is selected by the user and the third information is selected by the user The generation apparatus according to any one of claims 4 to 7, wherein the second similarity is calculated based on a similarity with a search query input by the user. The said calculation part calculates the said 1st similarity and the said 2nd similarity based on the similarity of the search query from which the amount of tokens exceeds a predetermined threshold value among the said search queries. 9. The generating device according to 8. The calculation unit calculates the first similarity based on a structural similarity between the first information and the second information and a semantic similarity between the first information and the second information. And calculating the second similarity based on the structural similarity between the first information and the third information and the semantic similarity between the first information and the third information. The generation device according to claim 1, wherein the generation device is a feature. As the second information and the third information, a first selection unit that selects information whose similarity with the first information exceeds a predetermined threshold,
The calculation unit calculates the first similarity and the second similarity from the first information and the second information and the third information selected by the first selection unit. The generation device according to any one of claims 1 to 10. A second selection unit that selects the fourth information whose similarity to the first information is equal to or higher than a first threshold and the fifth information whose similarity to the first information is equal to or lower than a second threshold;
The calculation unit calculates the first similarity and the second similarity for all sets including three pieces of information from the first information to the fifth information,
The generation unit is configured so that the distributed representation of the first information is similar to the distributed representation of the fourth information, and the distributed representation of the first information is not similar to the distributed representation of the fifth information. , Generating a distributed representation of the fourth information and the fifth information, and then, based on the first similarity and the second similarity calculated by the calculation unit for each of the sets, the first information or the first information included in the set The generation apparatus according to claim 1, wherein a distributed representation of the fifth information is generated. The generation unit is configured such that a difference between the distributed representation of the first information and the distributed representation of the second information is smaller than the difference between the distributed representation of the first information and the distributed representation of the third information. Each distributed expression is produced | generated. The production | generation apparatus as described in any one of Claims 1-12 characterized by the above-mentioned. The calculation unit includes the first similarity between the first information that is an image and the second information that is an image, and the second information between the first information and the third information that is an image. The generation device according to any one of claims 1 to 13, wherein similarity is calculated. A generation method executed by a generation device,
A calculation step of calculating a first similarity that is a similarity between the first information and the second information, and a second similarity that is a similarity between the first information and the third information;
Based on the relationship between the first similarity and the second similarity, one of the distributed representations of the second information and the third information is similar to the distributed representation of the first information, and the other is A generation method characterized by including a generation step of generating each distributed expression so as not to be similar. A calculation procedure for calculating a first similarity that is a similarity between the first information and the second information, and a second similarity that is a similarity between the first information and the third information;
Based on the relationship between the first similarity and the second similarity, one of the distributed representations of the second information and the third information is similar to the distributed representation of the first information, and the other is A generation program characterized by causing a computer to execute a generation procedure for generating each distributed expression so as not to be similar.

Images