JP7058556B2 - Judgment device, judgment method, and judgment program - Google Patents

Description

The present invention relates to a determination device, a determination method, and a determination program .

Conventionally, there is known a technique of generating a distributed representation of input information and determining the similarity of information based on the comparison result between the generated distributed representations. As an example of such a technique, the DSSM is such that distributed representations of similar or related texts are close to each other in Euclidean space, and distributed representations of dissimilar or dissimilar texts are separated from each other in Euclidean space. Techniques for learning models such as (Deep Semantic Similarity Model) and RNN (Reccurent Neural Network) are known.

"Poincare Embeddings for Learning Hierarchical Representations", Maximilian Nickel, Douwe Kiela <Internet> https://arxiv.org/pdf/1705.08039.pdf (Searched May 1, 2018)

"Hyperbolic geometry MA 448", p32, Caroline Series <Internet> https://homepages.warwick.ac.uk/~masbb/Papers/MA448.pdf Searched May 1, 2018)

“DSSM”, Jianfeng Gao, Po-Sen Huang, Hamid Palangi, Yelong Shen <Internet> https://www.microsoft.com/en-us/research/project/dssm/ Searched May 1, 2018)

However, in the above-mentioned technique, there is room for improving the comparison accuracy of the distributed representation.

For example, in the technique described above, the model transforms the text into a distributed representation in Euclidean space. However, in such a technique, although the text has a different character from the text used as the learning data, when the text having a similar meaning to the text used as the learning data is input, the dispersion is not similar in the Euclidean space. May produce expressions.

The present application has been made in view of the above, and an object thereof is to improve the comparison accuracy of the distributed representation.

The determination device according to the present application is a distributed representation of an information group containing a plurality of information, and is a generator that generates a distributed representation showing the concept of the information group on a hyperbolic space, which is a space on hyperbolic geometry. It is characterized by having a determination unit for determining similarity between information groups based on the distance on the hyperbolic space.

According to one aspect of the embodiment, the comparative accuracy of the distributed representation can be improved.

FIG. 1 is a diagram showing an example of processing executed by the determination device according to the embodiment. FIG. 2 is a diagram showing a configuration example of the determination device according to the embodiment. FIG. 3 is a diagram showing an example of information registered in the distributed representation database according to the embodiment. FIG. 4 is a diagram illustrating an example of adopting a Poincare disc as a hyperbolic space. FIG. 5 is a diagram illustrating an example of adopting a hyperbolic space as a hyperbolic space. FIG. 6 is a flowchart illustrating an example of the flow of the determination process according to the embodiment. FIG. 7 is a diagram showing an example of a hardware configuration.

Hereinafter, a determination device according to the present application, a determination method, and an embodiment for implementing the determination program (hereinafter, referred to as “embodiment”) will be described in detail with reference to the drawings. It should be noted that this embodiment does not limit the determination device, determination method, and determination program according to the present application. Further, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicate description is omitted.

[Embodiment]
[1-1. Example of judgment device]
First, an example of the process executed by the determination device will be described with reference to FIG. In the following description, as an example of the process executed by the determination device, an example of the process of determining whether or not the concepts of sentences are similar is described, but the embodiment is not limited to this. .. For example, the determination device 10 described below is arbitrary information as long as it is information composed of a plurality of information (hereinafter, may be referred to as "information group") such as voice data in addition to sentences. It may be determined whether or not they are similar to each other.

FIG. 1 is a diagram showing an example of processing executed by the determination device according to the embodiment. In FIG. 1, the determination device 10 is an information processing device that executes the determination process described below, and is realized by, for example, a server device, a cloud system, or the like.

More specifically, the determination device 10 can communicate with any device such as the input / output device 100 (see, for example, FIG. 2) and the information management device 110 via a predetermined network N such as the Internet.

The input / output device 100 acquires a user's remark by using a voice acquisition device such as a microphone that acquires voice. Then, the input / output device 100 converts the remark into text data by using an arbitrary voice recognition technique, and transmits the converted text data to the determination device 10. Further, the input / output device 100 reads out the text data received from the determination device 10 by using a device such as a speaker that outputs voice. The input / output device 100 may display the text data received from the determination device 10 on a predetermined display device.

The input / output device 100 is realized by an information processing device such as a smart device such as a smartphone or tablet, a desktop PC (Personal Computer), a notebook PC, or a server device. The input / output device 100 may be realized by, for example, the same information processing device, or may be realized by, for example, a device such as a robot.

The information management device 110 is an information processing device that manages various types of information, and is realized by, for example, a server device, a cloud system, or the like. For example, the information management device 110 holds various text data that can be used as learning data. To give a more specific example, the information management device 110 manages learning data in which text data of sentences and labels indicating whether or not the concepts of each sentence are similar are associated with each other, and the determination device 10 manages the learning data. The learning data is provided to the determination device 10 in response to the request of.

[1-2. About the generation of conventional distributed representation]
Here, an example of a conventional technique for converting information to be compared into a distributed expression and determining the similarity of the information based on the comparison result between the distributed expressions will be described. Further, in the following description, an example of a conventional technique for generating a distributed expression of sentences will be described.

For example, in a conventional information processing device, each word included in a sentence is input in order to a model such as RNN or DSSM, and a distributed expression output by the model is obtained. Then, the information processing device compares the distributed expressions generated from a plurality of sentences so that the distributed expressions of the sentences having similar concepts are similar and the distributed expressions of sentences having dissimilar concepts are not similar. By advancing the learning of the model, we will realize a model that transforms the concept shown by the sentence into a distributed expression. In addition, the conventional information processing device converts the input sentence into a distributed expression by using the model obtained by the above-mentioned learning, and compares the converted distributed expression with the distributed expression generated from other sentences. So, I was searching for other sentences that have similar concepts to the entered sentences.

Here, the conventional information processing apparatus calculates the distance between the distributed expressions in the Euclidean space at the time of learning the model or executing the process using the model (hereinafter, referred to as “measurement time”). Based on the calculated distance, the similarity between the concepts shown by the distributed expression was determined. More specifically, the information processing device makes the distance between distributed expressions of similar sentences shorter in Euclidean space, and the distance between distributed expressions of dissimilar sentences becomes longer in Euclidean space. , We learned a model to generate distributed expressions from sentences. As a result, the information processing apparatus projected the concept shown by each sentence onto the Euclidean space, and determined the similarity of each concept on the Euclidean space.

However, in such a method, there is room for improving the accuracy of the distributed representation. For example, the above-mentioned model may generate dissimilar distributed expressions in Euclidean space when a sentence in which the word order of words included in the sentence included in the learning data is changed is input. Moreover, in order to project many sentences on the Euclidean space, it is necessary to set many dimensions in advance, which increases the calculation cost.

On the other hand, there is known a technique for improving the accuracy of distributed expression of words by projecting a distributed expression on a hyperbolic space, which is an example of a non-Euclidean space (for example, Non-Patent Document 1). For example, in Non-Patent Document 1, in the task of predicting words that come before and after a certain word, the distributed expression of the words is learned on the Poincare space, which is a hyperbolic space. In such a Poincare space, the closer to the edge of the space, the more the scale of distance increases exponentially, so it is possible to project infinite information (points in space) in a finite space. can.

For example, consider a process of embedding branching information in a Poincare disk, which is an example of such a Poincare space. In such a Poincare disc, the measure of distance increases exponentially from the center to the circumference. For this reason, when a tree that branches in the circumferential direction from the center of the Poincare disc is embedded, it has an arbitrary number of branches on the Poincare disc while keeping the angle and length of each branch constant. You can embed the tree naturally. Such characteristics of the Poincare disc are retained even in a higher-dimensional Poincare space.

On the other hand, information such as a knowledge database, various classification systems, and vocabulary is considered to have a hierarchical structure, that is, a tree structure. Therefore, when a word is converted into a distributed expression on the Poincare space, the hierarchical structure of each word can be naturally embedded. Further, in the Poincare space, the scale of the distance increases as it approaches the circumferential direction. Therefore, when learning the distributed expression of words in Poincare space, many hierarchical relationships can be reflected in the distributed expression, so the accuracy of clustering using the distributed expression can be improved and the number of dimensions of the distributed expression can be reduced. Is known to achieve.

[1-3. About the processing executed by the judgment device]
Here, considering the concept of a sentence, it can be considered that it has a system of branching in a tree shape according to the words contained in the sentence. Therefore, when the distributed expression of a sentence is learned on the hyperbolic space, it is considered that the concept of the sentence can be learned accurately. In addition, when the distributed expressions are compared on the hyperbolic space, it is considered that the concepts of sentences can be compared accurately.

Therefore, the determination device 10 executes the following determination process. For example, the determination device 10 generates a distributed representation of an information group including a plurality of information, and shows the concept of the information group on a hyperbolic space, which is a space on hyperbolic geometry. Then, the determination device 10 determines the similarity between the information groups based on the distance on the hyperbolic space.

For example, the determination device 10 generates a distributed expression of a sentence including a plurality of words as an information group, and determines the similarity between the sentences based on the distance in the hyperbolic space of the distributed expression. Then, the determination device 10 executes the learning process and the measurement process using the generated distributed representation. For example, the determination device 10 learns to learn a model that converts an information group into a distributed expression by using the distributed expression generated from the learning data. Further, the determination device 10 generates a distributed representation of the information group acquired from the user, and compares the generated distributed representation with the distributed representation of another information group on the hyperbolic space to obtain the information group. Identify other similar sources of information. Then, the determination device 10 provides the user with another specified group of information.

[1-4. About an example of the processing executed by the judgment device]
Hereinafter, an example of the process executed by the determination device 10 will be described with reference to FIG. First, the determination device 10 acquires a sentence to be learning data from the information management device 110 (step S1). For example, the determination device 10 acquires a plurality of sentences composed of a plurality of words and label information indicating the similarity of each sentence. In such a case, the determination device 10 converts the text into a distributed representation using a model such as RNN or DSSM (step S2).

For example, the determination device 10 uses techniques such as morphological analysis to change sentences such as "tomorrow will be fine" to "tomorrow", "ha", "sunny", "ru", and "may". Break down into multiple words that make up a sentence. Then, the determination device 10 inputs words into the model in the order of appearance to generate a distributed expression of sentences that are word groups (that is, information groups). More specifically, the determination device 10 generates a distributed representation of the information group by sequentially inputting a plurality of words included in the information group into the recurrent neural network.

In the example shown in FIG. 1, the determination device 10 converts a sentence into a distributed expression using a model trained to convert the sentence into a distributed expression in Euclidean space, that is, an existing model. do. In addition to the RNN, the determination device 10 may convert a sentence into a distributed expression by using, for example, a convolutional neural network. Further, for example, the determination device 10 is a model in which learning is performed so that when words included in the same sentence are input in a random order, a distributed expression similar to the distributed expression when the words are input in the order of appearance is output. May be used. That is, the determination device 10 generates a distributed representation that shows the concept of the input sentence on the Euclidean space. In the following description, the distributed expression showing the concept of the input sentence in the Euclidean space may be described as "distributed expression in the Euclidean space". Further, in the following description, a model for converting a sentence into a distributed representation on Euclidean space is described as a first model.

Subsequently, the determination device 10 projects the distributed expression onto the hyperbolic space (step S3). For example, the determination device 10 projects a distributed expression on the Minkowski space as a hyperbolic space. More specifically, the determination device 10 generates a distributed representation on a hyperplane whose coordinates on the focal axis are positive among the two hyperplanes of the Hyperboloid of two sheets in Minkowski space. ..

For example, the determination device 10 sets a three-dimensional space composed of the x-axis, the y-axis, and the z-axis, and sets a two-leaf hyperboloid with the z-axis as the focal axis in the three-dimensional space. Further, the determination device 10 uses the first model to generate a distributed representation on a two-dimensional Euclidean space composed of an x-axis component and a y-axis component. Then, the determination device 10 sets a perpendicular line with respect to the xy plane from the position indicated by the generated dispersion expression in the set three-dimensional space, and the curved surface and the perpendicular line whose z-axis direction component is positive among the two-leaf hyperboloids. The intersection is the projection destination of the distributed expression. That is, the determination device 10 generates a distributed representation on the hyperbolic space showing the concept of sentences.

Then, the determination device 10 determines the distance between the distributed expressions on the hyperbolic space (step S4). For example, the determination device 10 generates the distributed expression u on the Euclidean space from the sentence # 1 and the distributed expression v on the Euclidean space from the sentence # 2. Then, the determination device 10 calculates the distance in the hyperbolic space between the distributed expression u and the distributed expression v. The method of calculating the distance in the hyperbolic space will be described later.

Then, the determination device 10 learns the model based on the determination result (step S5). For example, when the sentence # 1 and the sentence # 2 are similar, the determination device 10 learns the model so that the distance between the distributed expression u and the distributed expression v on the hyperbolic space becomes smaller. When sentence # 1 and sentence # 2 are not similar, the model is learned so that the distance between the distributed expression u and the distributed expression v on the hyperbolic space becomes larger.

Even if the determination device 10 learns the model so that the distance between the distributed expression u and the distributed expression v on the hyperbolic space reflects the degree of similarity between the sentence # 1 and the sentence # 2. good. For example, when the similarity between the sentence # 1 and the sentence # 3 is higher than the similarity between the sentence # 1 and the sentence # 2, the determination device 10 is in the hyperbolic space of the distributed expression u and the sentence # 3. The model is trained so that the distance on the hyperbolic space with the distributed expression is shorter than the distance on the hyperbolic space between the distributed expression u and the distributed expression v. That is, the determination device 10 learns a model that generates a distributed expression of sentences so that the similarity of the concepts of each sentence is dropped on the hyperbolic space.

For example, the determination device 10 fixes a function f that projects a distributed expression on the Euclidean space generated by the first model onto the hyperbolic space, and uses a learning method such as backpropagation between the nodes of the first model. You may modify the connection factor to connect. Further, the determination device 10 may modify the function f that projects the distributed representation on the Euclidean space generated by the first model onto the hyperbolic space. That is, the determination device 10 is arbitrary as long as it learns a model (hereinafter, referred to as "second model") capable of projecting the similarity of concepts of sentences onto a distance in hyperbolic space. You may study. In other words, the determination device 10 may realize the learning of the second model including the first model and the function f by modifying the first model, or by modifying the function f. May be good. Further, the determination device 10 may modify both the first model and the function f. Further, the determination device 10 may learn the second model by preparing a new model and learning the prepared model.

Subsequently, the determination device 10 acquires a sentence to be determined from the user (step S6). In such a case, the determination device 10 projects the distributed expression of the acquired sentence on the hyperbolic space using the second model, and calculates the distance from the distributed expression of other sentences on the hyperbolic space. .. Then, the determination device 10 identifies a sentence in which the calculated distance satisfies a predetermined condition. For example, the determination device 10 identifies a sentence having the shortest calculated distance, that is, a sentence having a similar concept to the sentence acquired from the user. Then, the determination device 10 provides the specified sentence to the user (step S8).

In this way, the determination device 10 converts the sentence into a distributed representation on the hyperbolic space instead of the distributed representation on the Euclidean space. Then, the determination device 10 compares the concepts of sentences with each other based on the distance on the hyperbolic space. When such a process is executed, the determination device 10 can compare the tree-row system of the concept of the text, so that the comparison accuracy of the concept of the text can be improved. That is, the determination device 10 can improve the accuracy with which the distributed expression shows the concept. Further, the determination device 10 can reduce the number of dimensions of the distributed representation while maintaining the accuracy of the distributed representation, so that the calculation cost can be reduced.

[1-5. About the model]
Here, in the example shown in FIG. 1, the distributed representation on the Euclidean space is acquired by using the first model that transforms the sentence into the distributed representation on the Euclidean space, and the distributed representation on the Euclidean space is used by using the function f. Was projected onto the hyperbolic space. However, the embodiments are not limited to this.

For example, the determination device 10 prepares a new CNN (Convolutional Neural Network) or RNN as an integrated model, and generates a distributed representation of sentences using the prepared integrated model. Then, the determination device 10 calculates the distance of the distributed expression generated from each sentence, assuming that the generated distributed expression is a distributed expression on a predetermined hyperbolic space, and learns the integrated model based on the calculated distance. May be done. Further, the determination device 10 may realize the comparison between the concepts of the sentences by using the integrated model in which the learning is performed by such a learning process.

Further, when the determination device 10 uses RNN to generate a distributed expression of a sentence, the determination device 10 may generate a distributed expression of the sentence based on the concept of each word and the order of appearance of each word. For example, in the determination device 10, the distributed expression when each word included in the sentence is input in the order of appearance is different from the distributed expression when each word included in the sentence is input in an order different from the order of appearance. The second model or the integrated model may be trained.

[1-6. About hyperbolic space]
Here, the determination device 10 can adopt any space as the hyperbolic space as long as it generates a distributed expression on the hyperbolic space. For example, the determination device 10 may generate a distributed representation on a Minkowski space having an arbitrary number of dimensions. Further, the determination device 10 may generate a distributed representation on any surface of the bilobed hyperboloid.

Further, the determination device 10 may generate a distributed representation on a disk of any dimension in the Poincare disk model (so-called Poincare disk) or on a spherical surface of any dimension in the Poincare sphere model. That is, the determination device 10 generates a distributed representation in space to which hyperbolic geometry is applicable, and based on the distance using hyperbolic geometry in such a space, the similarity of the concepts of the information is obtained. If it is determined, any hyperbolic space may be adopted.

[1-7. Information to be judged]
In the above-mentioned example, the determination device 10 has converted the concept of the text into a distributed expression. However, the embodiments are not limited to this. For example, the determination device 10 may divide the voice into a plurality of partial voices, sequentially input each divided partial voice into the first model, and project the distributed expression output by the first model onto the hyperbolic space. .. Further, the determination device 10 may sequentially input the images of each frame of the moving image into the first model and project the distributed expression output by the first model onto the hyperbolic space.

That is, if the determination device 10 is a distributed representation showing the concept of the information group and is to be converted into a distributed representation on the hyperbolic space, the determination device 10 is an information group of any kind. By converting the concept possessed by the above into a distributed expression on the hyperbolic space and calculating the distance of the distributed expression on the hyperbolic space, the concepts possessed by the information groups may be compared with each other. Further, for example, the determination device 10 may compare the concepts of different types of information groups such as sentences and moving images.

[2. Judgment device configuration]
Hereinafter, an example of the functional configuration of the determination device 10 that realizes the above-mentioned provision processing will be described. FIG. 2 is a diagram showing a configuration example of the determination device according to the embodiment. As shown in FIG. 2, the determination device 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

The communication unit 20 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 20 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the input / output device 100.

The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. Further, the storage unit 30 stores the distributed representation database 31 and the model 32.

The distributed representation of sentences is registered in the distributed representation database 31. For example, FIG. 3 is a diagram showing an example of information registered in the distributed representation database according to the embodiment. As shown in FIG. 3, information such as "sentence ID", "sentence data", "Euclid distributed representation", and "hyperbolic space distributed representation" are registered in association with each other in the distributed representation database 31.

Here, the "sentence ID" is an identifier indicating a sentence. Further, the "text data" is a character string of a text, that is, text data. Further, the "Euclidean distributed expression" is a distributed expression on the Euclidean space of the associated sentences. Further, the "hyperbolic space distributed expression" is a distributed expression on the hyperbolic space of the associated sentences.

For example, in the example shown in FIG. 3, the sentence ID “sentence # 1”, the sentence data “data # 1”, the Euclidean distributed expression “distributed expression # 1-1”, and the hyperbolic space distributed expression “distributed expression # 1-2”. "Is registered in association with each other. For such information, the sentence data of the sentence indicated by the sentence ID "sentence # 1" is "data # 1", and the Euclidean distributed expression of "data # 1" is "distributed expression # 1-1". Indicates that the music space distributed expression is "distributed expression # 1-2".

In the example shown in FIG. 3, conceptual values such as "sentence # 1", "data # 1", and "distributed expression # 1-1" are described, but in reality, a character string for identifying a sentence is described. , Numerical values, character strings that make up sentences, multidimensional quantities that are distributed expressions, etc. will be registered. Further, the information shown in FIG. 3 is merely an example, and arbitrary information other than the information shown in FIG. 3 may be registered in the distributed representation database 31.

Returning to FIG. 2, the explanation will be continued. The model 32 is a model used by the determination device 10, and is, for example, data of a first model that generates a distributed representation on the Euclidean space from a sentence. More specifically, in the storage unit 30, the connection relationship of the nodes constituting the first model and the connection coefficient which is the weight between the nodes are registered. Here, the model 32 is assumed to be used as a program module which is a part of artificial intelligence software, for example.

For example, the model 32 is a distributed representation of an information group containing a plurality of information, and generates a distributed representation that shows the concept of the information group on the hyperbolic space, which is a space on hyperbolic geometry, and generates a distributed representation on the hyperbolic space. It is a program parameter including information of a model learned by a learning method of learning a model that determines the similarity between information groups based on the distance of and converts the information group into a distributed representation based on the determination result. For example, the model 32 is realized by a combination of the first model and the function f.

Further, it is assumed that the model 32 is realized by a neural network having one or a plurality of intermediate layers such as a DNN (Deep Neural Network). In this case, the first element included in the model 32 corresponds to either the node of the input layer or the intermediate layer. Further, the second element corresponds to a node in the next stage, which is a node to which a value is transmitted from a node corresponding to the first element. Further, the weight of the first element corresponds to a connection coefficient which is a weight considered for the value transmitted from the node corresponding to the first element to the node corresponding to the second element.

The determination device 10 calculates the distributed representation using a model having an arbitrary structure such as the regression model and the neural network described above. Specifically, in the model 32, when the information group is input, the coefficient is set so as to output a distributed representation indicating the concept indicated by the information group.

In the above example, the model 32 is a model (referred to as model X) that outputs a distributed representation indicating the concept indicated by the information group when the information group is input. However, the model 32 according to the embodiment may be a model generated based on the result obtained by repeatedly inputting / outputting data to the model X. For example, the model 32 may be a model (model Y) trained to input a group of information and output a distributed representation output by the model X. Alternatively, the model 32 may be a model trained to use the information group as an input and the output value of the model Y as an output. Further, when the determination device 10 performs processing using GAN (Generative Adversarial Networks), the model 32 may be a model constituting a part of GAN.

The control unit 40 is a controller, and for example, various programs stored in a storage device inside the determination device 10 by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) store a RAM or the like. It is realized by being executed as a work area. Further, the control unit 40 is a controller, and may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Further, for example, the control unit 40 will execute the determination process according to the instruction of the model 32 which is a program module or the program module including the model 32.

As shown in FIG. 2, the control unit 40 includes a generation unit 41, a determination unit 42, a learning unit 43, and a providing unit 44. The generation unit 41 generates a distributed representation of an information group including a plurality of information, and shows the concept of the information group in a hyperbolic space, which is a space on hyperbolic geometry.

For example, the generation unit 41 acquires sentences and labels to be learning data from the information management device 110 in the learning process of the model. In such a case, the generation unit 41 uses the first model included in the model 32 to generate a distributed representation that shows the concept of the sentence on the Euclidean space. For example, the generation unit 41 sequentially inputs each word included in the sentence to the first model which is the RNN, that is, a plurality of information included in the information group, thereby generating a distributed expression of the sentence which is the information group. .. In other words, the generation unit 41 generates a distributed representation of the information group based on the concept of each information included in the information group and the appearance order of each information.

Then, the generation unit 41 uses the function f to project the generated distributed expression onto the hyperbolic space to generate a distributed expression that shows the concept of the sentence on the hyperbolic space. For example, the generation unit 41 generates a distributed representation as a hyperbolic space on a disk of any dimension in the Poincare disk model or on a spherical surface of any dimension in the Poincare sphere model. Further, for example, the generation unit 41 generates a distributed representation on the Minkowski space as a hyperbolic space. To give a more specific example, as a hyperbolic space, the generation unit 41 expresses a dispersion on the hyperplane in which the coordinates on the focal axis are positive among the two hyperplanes of the bilobed hyperboloid in the Minkowski space. Generate. Then, the generation unit 41 registers the acquired sentence, the generated distributed expression on the Euclidean space, and the distributed expression on the hyperbolic space in the distributed expression database 31.

Further, the generation unit 41 receives the text input by the user from the input / output device 100 in the measurement process of the model. In such a case, the generation unit 41 inputs the words included in the received sentence into the first model in order, and uses the function f to express the distributed representation on the Euclidean space output by the first model on the hyperbolic space. Convert to a distributed representation. In the following description, the distributed expression on the hyperbolic space is described as the hyperbolic expression, and the distributed expression on the hyperbolic space of the sentence input by the user is described as the acquired hyperbolic expression.

The determination unit 42 determines the similarity between the information groups based on the distance on the hyperbolic space. For example, the determination unit 42 determines the similarity between sentences based on the distance on the hyperbolic space of the distributed expression. For example, at the time of learning, the determination unit 42 selects a pair of sentences from the learning data, reads the distributed expression on the hyperbolic space of each selected sentence from the distributed expression database 31, and reads out the distributed expressions. Calculate the distance on the music space. Then, the determination unit 42 determines that the selected pairs are not similar when the calculated distance exceeds a predetermined threshold value, and determines that the selected pairs are similar when the calculated distance is equal to or less than the predetermined threshold value. ..

Further, the determination unit 42 calculates the distance between the acquired hyperbolic expression and the hyperbolic expression registered in the distributed expression database 31 in the hyperbolic space at the time of measurement. Then, the determination unit 42 identifies a sentence corresponding to the hyperbolic expression having the shortest calculated distance, that is, a sentence most similar to the sentence acquired from the user.

The learning unit 43 learns a model for converting an information group into a distributed representation by using the generated distributed representation. That is, the learning unit 43 learns the model by using the distributed representation that shows the concept of the information group on the hyperbolic space.

For example, the learning unit 43 receives a pair of sentences selected from the determination unit 42 and a determination result based on the distance in the hyperbolic space, that is, a determination result of whether or not the pairs are similar. In such a case, the learning unit 43 determines whether or not the pair is a similar sentence based on the label of the learning data. Then, the learning unit 43 learns the model 32 so that the determination result based on the label of the learning data and the determination result based on the distance on the hyperbolic space match.

For example, when the determination result based on the label of the learning data and the determination result based on the distance on the hyperbolic space do not match, the learning unit 43 executes the following processing. For example, when the label of the learning data indicates that the pair of sentences is similar, the learning unit 43 makes the model so that the hyperbolic expressions of each pair are similar (so that the distance between the hyperbolic expressions becomes short). Modify 32. On the other hand, when the label of the learning data indicates that the pair of sentences is not similar, the learning unit 43 sets the model 32 so that the hyperbolic expressions of each pair are similar (so that the hyperbolic expressions are separated from each other). To fix.

That is, the learning unit 43 is a process of generating a distributed expression that is a distributed expression of an information group containing a plurality of information and that shows the concept of the information group in a hyperbolic space that is a hyperbolic geometric space. Model 32 learned by a determination process that determines the similarity between information groups based on the distance in the hyperbolic space, and a process that learns a model that converts the information groups into a distributed representation based on the determination result. To generate. Such a model 32 is used as information included in a program module that causes the control unit 40 to execute various processes at the time of measurement.

The learning unit 43 may learn the model 32 using any learning algorithm. For example, the learning unit 43 may learn the model 32 by using a learning algorithm such as a neural network, a support vector machine, clustering, or reinforcement learning.

Based on the determination result, the providing unit 44 identifies another information group similar to the information group acquired from the user, and provides the specified other information group to the user. For example, the providing unit 44 receives a notification of a sentence most similar to the sentence acquired from the user from the determination unit 42. In such a case, the providing unit 44 reads the text data of the notified text from the distributed expression database 31, and outputs the read text data from the input / output device 100.

[3. About an example of a formula]
Subsequently, an example of the process executed by the determination device 10 will be described using a mathematical formula. In the following description, the distributed representation on the Euclidean space is shown by v = (v ₁ , v ₂ ) and u = (u ₁ , u ₂ ), and the distributed representation projected on the hyperbolic space is shown with a tilde. Indicated by v or u.

For example, an example of generating a distributed representation on a two-dimensional Poincare disk will be described. For example, FIG. 4 is a diagram illustrating an example of adopting a Poincare disc as a hyperbolic space. For example, assuming that the distributed representation generated by using the first model is v, the distributed representation v projected on the Poincare disk can be expressed by the following equation (1).

In the equation (1), the distributed expression v projected on the Poincare disk is represented by v with a tilde. Further, a in the equation (1) is a scaling factor, and any numerical value can be adopted. When a is set to 1/2, the distance between the origin and the distributed representation will be the same on the Euclidean space before projection and on the Poincare disk (see, for example, Non-Patent Document 2, p32). ).

Here, the distance d between the distributed expression v and the distributed expression u projected onto the Poincare disk can be expressed by the following equation (2).

Here, from the equation (1), the equation (2) can be transformed as shown by the equation (3). That is, the distance d between the distributed expression v and the distributed expression u projected on the Poincare disk can be expressed by the following equation (3) using the distributed expression v and the distributed expression u on the Euclidean space.

By using such an equation (3), the determination device 10 can calculate the distance on the hyperbolic space by using the distributed representation on the Euclidean space generated by the first model.

Here, in theory, infinite information can be embedded in the Poincare disk, but when the actual calculation is performed by various information processing devices, an accuracy problem arises. For example, a distributed representation projected onto a Poincare disk is not allowed to be projected outside the circumference of the Poincare disk (ie, it is not allowed to extend beyond the unit circle), but it is a single-precision, distributed representation of the numerical value. When calculated, the distributed representation may be projected outside the circumference of the Poincare disk, depending on the number of distributed representations to be projected. In addition, information cannot be embedded in the origin of the Poincare disc.

On the other hand, of the two hyperplanes of the Futaba hyperboloid in the Minkowski space, on the hyperplane where the coordinates on the focal axis are positive, the confinement on the unit circle has the same properties as the Poincare disk. Is known to be unnecessary. Therefore, the determination device 10 may project the distributed expression on the hyperplane whose coordinates on the focal axis are positive among the two hyperplanes of the two-leaf hyperboloid in the Minkowski space.

For example, FIG. 5 is a diagram illustrating an example of adopting a hyperbolic bicurved surface as a hyperbolic space. For example, when the focal axis is the z-axis, as shown in FIG. 5, of the two hyperplanes of the Futaba hyperboloid, the hyperplane whose coordinates on the focal axis are positive, that is, the hyperplane where z> 0. (Hereinafter referred to as "Minkowski plane") can be represented by a set of points satisfying the following equation (4).

An example of the process of projecting a distributed representation on a two-dimensional Euclidean space on such a Minkowski plane will be described. For example, if a point on the Minkowski plane is represented by v with a tilde and the component in the focal axis direction is v ₀ with a tilde, it can be expressed by the following equation (5). Further, each component represented by the formula (5) satisfies the following formula (6).

By modifying such an equation (6), the following equation (7) can be obtained.

Using such an equation (7), the coordinates of points on the Minkowski plane can be expressed by the following equation (8).

Here, as shown in FIG. 1, the function f that projects the distributed representation on the two-dimensional Euclidean space onto the Minkowski plane can be shown by the following equation (9).

Further, the dispersion expression u and the dispersion expression v on the Minkowski plane can be represented by the following equations (10) and (11).

As a result, the pseudo-inner product of the distributed expression u and the distributed expression v can be expressed by the following equation (12). Here, the dot product symbol in the mathematical formula is implicitly used as an ordinary inner product for vectors in Euclidean space and a pseudo-inner product for vectors in hyperbolic space.

Therefore, if it is expressed as a distance using tilded u and v, which are vectors of the Minkowski plane, using the pseudo-inner product and the inverse hyperbolic function (arcosh), it can be expressed by the following equation (13). Further, the distance using the vectors u and v of the Euclidean plane can be shown by (14).

The determination device 10 may calculate the distance between the distributed expressions on the Minkowski plane using such an equation (14). The above equation (14) can be applied to a distance on the Minkowski hyperplane of any dimension.

[4. An example of the flow of processing executed by the judgment device]
Next, an example of the flow of the provision process executed by the determination device 10 will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of the flow of the determination process according to the embodiment. First, the determination device 10 acquires a sentence (step S101), inputs each word included in the sentence into the first model in order, and generates a distributed expression on the Euclidean space (step S102). Subsequently, the determination device 10 converts (projects) the distributed expression into the distributed expression on the hyperbolic space (step S103), and determines the distance between the distributed expressions on the hyperbolic space (step S104). Then, the determination device 10 learns the model or extracts similar sentences based on the determination result (step S105), and ends the process.

[5. Modification example]
In the above, an example of the provision process by the determination device 10 has been described. However, the embodiments are not limited to this. Hereinafter, variations of the determination process executed by the determination device 10 will be described.

[5-1. Device configuration〕
In the above-mentioned example, the determination device 10 executes the determination process in the determination device 10. However, the embodiments are not limited to this. For example, the determination device 10 may execute the determination process using the hyperbolic expression generated by the generation device that generates the hyperbolic expression of the information group. Further, the determination device 10 may realize each of the above-mentioned processes by interlocking with the learning device that executes the learning process and the providing process that executes the providing process. For example, the generation device has the generation unit 41 shown in FIG. 2, the determination device 10 has only the determination unit 42 shown in FIG. 2, and the learning device has only the learning unit 43 shown in FIG. The providing device may have only the providing unit 44 shown in FIG. Further, the determination device 10 may store the distributed representation database 31 in an external storage server.

[5-2. others〕
Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or part of it can be done automatically by a known method. In addition, the processing procedure, specific name, and information including various data and parameters shown in the above text and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown in the figure.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of them may be functionally or physically distributed / physically in any unit according to various loads and usage conditions. Can be integrated and configured.

In addition, the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other.

[5-3. program〕
Further, the determination device 10 according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration as shown in FIG. 7. FIG. 7 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to the output device 1010 and the input device 1020, and the arithmetic unit 1030, the primary storage device 1040, the secondary storage device 1050, the output IF (Interface) 1060, the input IF 1070, and the network IF 1080 are connected by the bus 1090. Have.

The arithmetic unit 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The primary storage device 1040 is a memory device that temporarily stores data used by the arithmetic unit 1030 for various operations such as RAM. Further, the secondary storage device 1050 is a storage device in which data used by the arithmetic unit 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD, a flash memory, or the like.

The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various information such as a monitor and a printer. For example, USB (Universal Serial Bus), DVI (Digital Visual Interface), and the like. It is realized by a connector of a standard such as HDMI (registered trademark) (High Definition Multimedia Interface). Further, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, a scanner, and the like, and is realized by, for example, USB.

The input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. Further, the input device 1020 may be an external storage medium such as a USB memory.

The network IF 1080 receives data from another device via the network N and sends it to the arithmetic unit 1030, and also transmits the data generated by the arithmetic unit 1030 to the other device via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the determination device 10, the arithmetic unit 1030 of the computer 1000 functions as a control unit 40 by executing a program or data (for example, a model 32) loaded on the primary storage device 1040. To realize. The CPU 1100 of the computer 1000 reads these programs or data (eg, model 32) from the recording medium 1800 and executes them, but as another example, even if these programs are acquired from another device via the network N. good.

[6. effect〕
As described above, the determination device 10 generates a distributed representation of an information group containing a plurality of information, and shows the concept of the information group on the hyperbolic space, which is a space on hyperbolic geometry. .. Then, the determination device 10 determines the similarity between the information groups based on the distance on the hyperbolic space. As described above, the determination device 10 transfers the concepts indicated by the information group consisting of a plurality of information, that is, the information group presumed to have a tree-type concept, based on the distance on the hyperbolic space instead of the Euclidean space. Compare. Therefore, the determination device 10 can improve the comparison accuracy of the distributed expression.

Further, the determination device 10 learns a model for converting the information group into the distributed representation by using the generated distributed representation. For example, the determination device 10 uses a model to generate a distributed expression that shows the concept of the information group on the Euclidean space, and projects the generated distributed expression onto the hyperbolic space, so that the concept of the information group has the concept. Is generated on the hyperbolic space, and the model is trained using the distributed expression that shows the concept of the information group on the hyperbolic space. Therefore, since the determination device 10 can appropriately generate the hyperbolic expression of the information group, the comparison accuracy of the distributed expression can be improved.

Further, the determination device 10 identifies another information group similar to the information group acquired from the user based on the determination result, and provides the specified other information group to the user. Therefore, the determination device 10 can provide the user with another information group having a similar concept to the information group acquired from the user.

Further, the determination device 10 sequentially inputs a plurality of information included in the information group to the recurrent neural network to generate a distributed representation of the information group. Therefore, the determination device 10 can improve the comparison accuracy of the distributed representation showing the concept of the information group.

Further, the determination device 10 generates a distributed representation as a hyperbolic space on a disk of any dimension in the Poincare disk model or on a spherical surface of any dimension in the Poincare sphere model. Further, the determination device 10 generates a distributed representation on the Minkowski space as a hyperbolic space. For example, the determination device 10 generates a distributed representation on the hyperplane in which the coordinates on the focal axis are positive among the two hyperplanes of the bilobed hyperboloid in the Minkowski space as the hyperbolic space. Therefore, the determination device 10 can appropriately generate a hyperbolic expression of the information group.

Further, the determination device 10 generates a distributed representation of the information group based on the concept of each information included in the information group and the appearance order of each information. Therefore, the determination device 10 can appropriately generate a hyperbolic expression showing the concept of an information group composed of a plurality of information, that is, the concept of a tree-type information group.

Further, the determination device 10 generates a distributed expression of a sentence including a plurality of words as an information group, and determines the similarity between the sentences based on the distance in the hyperbolic space of the distributed expression. Therefore, the determination device 10 can improve the comparison accuracy of the concept of sentences.

Although some of the embodiments of the present application have been described in detail with reference to the drawings, these are examples, and various modifications are made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure column of the invention. It is possible to carry out the present invention in other modified forms.

Further, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the generation unit can be read as a generation means or a generation circuit.

10 Judgment device 20 Communication unit 30 Storage unit 31 Distributed representation database 32 Model 40 Control unit 41 Generation unit 42 Judgment unit 43 Learning unit 44 Providing unit 100 Input / output device 110 Information management device

Claims (10)

A generator that generates a distributed representation of an information group containing a plurality of information and that shows the concept of the information group in a hyperbolic space, which is a hyperbolic space.
A determination unit that determines the similarity between information groups based on the distance on the hyperbolic space.
A learning unit that learns a model that converts the information group into a distributed representation using the distributed representation generated by the generation unit.
Have,
The generation unit uses the model to generate a distributed expression that shows the concept of the information group on the Euclidean space, and projects the generated distributed expression onto the hyperbolic space, so that the information group has the information group. Generate a distributed representation that shows the concept in hyperbolic space,
The learning unit learns the model by using a distributed representation that shows the concept of the information group on the hyperbolic space.
A judgment device characterized by the fact that. A specific unit that identifies another information group similar to the information group acquired from the user based on the determination result by the determination unit.
The determination device according to claim 1, further comprising a providing unit that provides the user with another information group specified by the specific unit. The generation unit according to claim 1 or 2 , wherein a plurality of information included in the information group is sequentially input to the recurrent neural network to generate a distributed representation of the information group. The determination device described. The generation unit is characterized in that, as the hyperbolic space, a distributed representation on a disk of any dimension in the Poincare disk model or on a spherical surface of any dimension in the Poincare sphere model is generated. The determination device according to any one of 3 to 3 . The determination device according to any one of claims 1 to 4 , wherein the generation unit generates a distributed representation on a Minkowski space as the hyperbolic space. The generation unit is characterized in that, as the hyperbolic space, it generates a distributed representation on a hyperplane in which the coordinates on the focal axis are positive among the two hyperplanes of the bilobed hyperboloid in the Minkowski space. Item 5. The determination device according to Item 5. Any one of claims 1 to 6 , wherein the generation unit generates a distributed representation of the information group based on the concept of each information included in the information group and the appearance order of each information. The determination device according to one. The generation unit generates a distributed expression of a sentence including a plurality of words as the information group.
The determination device according to any one of claims 1 to 7 , wherein the determination unit determines similarity between sentences based on the distance of the distributed expression on the hyperbolic space. It is a judgment method executed by the judgment device.
A generation process for generating a distributed representation of an information group containing a plurality of information and showing the concept of the information group on a hyperbolic space, which is a space on hyperbolic geometry.
A determination step for determining similarity between information groups based on the distance on the hyperbolic space.
A learning process for learning a model that converts the information group into a distributed representation using the distributed representation generated by the generation step.
Have,
The generation step uses the model to generate a distributed expression that shows the concept of the information group on the Euclidean space, and projects the generated distributed expression onto the hyperbolic space, so that the information group has the information group. Generate a distributed representation that shows the concept in hyperbolic space,
In the learning step, the model is learned using a distributed representation that shows the concept of the information group on the hyperbolic space.
Judgment method characterized by that. A generation procedure for generating a distributed representation of an information group containing a plurality of information and showing the concept of the information group on a hyperbolic space, which is a space on hyperbolic geometry.
Judgment procedure for determining similarity between information groups based on the distance on the hyperbolic space
A learning procedure for learning a model that converts the information group into a distributed representation using the distributed representation generated by the generation procedure.
Have,
The generation procedure uses the model to generate a distributed expression that shows the concept of the information group on the Euclidean space, and projects the generated distributed expression onto the hyperbolic space, so that the information group has the information group. Generate a distributed representation that shows the concept in hyperbolic space,
In the learning procedure, the model is learned using a distributed representation that shows the concept of the information group on the hyperbolic space.
A judgment program that lets a computer do things .

Images