JP2019164669A - Information processing device, information processing method, and program - Google Patents

Abstract

To provide an information processing device, an information processing method, and a program capable of more efficiently deriving an answer to a question with higher accuracy.SOLUTION: An information processing device is provided with a derivation unit learning part for learning a derivation unit for deriving a feature amount of a question with reference to a second database including a feature amount group which performs learning such that a feature amount obtained by adding a feature amount of a first relation showing a relation between a first entity and a second entity to a feature amount of the first entity selected from a plurality of entities approaches the feature amount of the second entity, and learns the derivation unit such that difference between a feature amount of a question and the feature amount of the second entity is made small, and that difference between the feature amount of the question and a feature amount of a sampled entity other than the second entity is made large.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  Conventionally, research and practical use have been conducted on a mechanism for automatically answering a question entered in a natural sentence. In this regard, receiving a query having one or more words; querying the database to find at least one candidate subject whose name or alias has the same surface format as the subject chunk; Querying a database to find one or more relationship vectors representing one or more relationships associated with the at least one candidate subject; and each of the one or more relationships of the query Determining a ranking score indicating semantic similarity with the correspondence, selecting a relationship having the highest ranking score from the one or more relationships as a prediction relationship, and selecting the at least one candidate subject as a prediction topic subject Step to select as Querying a database with the prediction relationship and the prediction topic subject to find an answer to the query, wherein the one or more words describe the topic subject of the query A computer-implemented method for providing an answer to a query characterized by including a subject chunk is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2017-76403

In the conventional technology, there are cases where the processing cannot proceed efficiently or the accuracy is insufficient.
The present invention has been made in view of such circumstances, and provides an information processing apparatus, an information processing method, and a program capable of deriving an answer to a question more efficiently and with high accuracy One of the purposes is to do.

  One aspect of the present invention is the first database selected from the plurality of entities based on a first database in which an acquisition unit for acquiring a question, a plurality of entities, and a relation indicating a relationship between the entities are registered. A feature quantity group learned so that a feature quantity obtained by adding the feature quantity of the first relation indicating the relationship between the first entity and the second entity to the feature quantity of the entity approaches the feature quantity of the second entity. A derivation learning unit that learns a derivation unit that derives the feature amount of the question with reference to a second database that includes the feature amount of the question, and a feature of the second entity The difference between the amount and the feature amount of the question is increased, and the difference between the feature amount of the sampled entity other than the second entity is increased. Learning the derivation device, an information processing apparatus.

  According to one embodiment of the present invention, an answer to a question can be derived more efficiently and with high accuracy.

2 is a diagram illustrating an example of a configuration of an information processing device 100. FIG. It is the figure which illustrated the relationship between the two entities which comprise a triplet, and a relationship. FIG. 3 is an image diagram schematically showing the contents of data registered in a knowledge base 50. It is a figure which shows typically the geometric relationship of the vector Vhr of a head entity, the vector Vr of a relation, and the vector Vgt of a tail entity. It is an image figure which shows typically the content of the data registered into entity relation vector DB60. It is an image figure which shows notionally the function of a deriver. FIG. 4 is an image diagram conceptually showing the content of processing by a second learning unit 30. It is an image figure which shows the content of the process by a comparative example typically.

[Overview]
Hereinafter, an information processing apparatus, an information processing method, and a program according to embodiments of the present invention will be described with reference to the drawings. The information processing apparatus is realized by one or more processors. The information processing device may be a device that provides a cloud service, or may be a device that is installed with a program such as a tool or firmware and that can execute processing alone. The information processing apparatus may or may not be connected to a network such as the Internet or WAN. That is, there are no particular restrictions on the computer device for realizing the information processing device, and the information processing device may be realized by any computer device as long as the processing described below can be executed.

  The information processing apparatus is used for an apparatus or a system that automatically responds to a question input in the form of text or voice by a person or a computer, for example. The device that automatically responds may be provided with a conversational interface, is included in the search device, interprets the query as a question, and returns the answer together with the search result to the terminal device of the query input person It may be.

  The information processing device derives the feature amount of the question, and selects an entity having a feature amount close to the feature amount obtained from the question as the answer to the question. In the following description, the feature quantity is assumed to be a vector in the Euclidean space, but the feature quantity may not be a vector as long as the distance can be defined and addition / subtraction can be performed. Assume that the vectors appearing below are normalized so that the L2 norm is between 0 and 1.

[Constitution]
FIG. 1 is a diagram illustrating an example of the configuration of the information processing apparatus 100. The information processing apparatus 100 includes, for example, a question acquisition unit 10, a first learning unit 20, a second learning unit 30 (an example of “derivative learning unit”), and an answer output unit 40. These components are realized, for example, when a hardware processor such as a CPU (Central Processing Unit) executes a program (software). Some or all of these components are hardware (circuit units) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit); (including circuitry), or may be realized by cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or a flash memory, or is stored in a removable storage medium such as a DVD or CD-ROM, and the storage medium is stored in the drive device. It may be installed by being attached.

  The information processing apparatus 100 stores information such as a knowledge base 50, an entity / relation vector DB (database) 60, and derivation information 70 in a storage device such as an HDD or a flash memory. Such information may be held by a database server or the like separate from the information processing apparatus 100. That is, the information processing apparatus 100 does not have to include a storage device for holding such information.

  For example, the question acquisition unit 10 acquires a question from another device via the network NW. The network NW includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a cellular network, and the like. The question may be acquired in the form of text or may be acquired in the form of speech. In the latter case, the information processing apparatus 1 may include a voice recognition unit. In any case, it is assumed that the question is composed of natural sentences and not an abstracted code.

  Based on the knowledge base 50 in which a plurality of entities and relations indicating relationships between entities are registered, the first learning unit 20 selects entities and relation vectors (hereinafter referred to as entity vectors and relation vectors, respectively). Learn and register in the entity relation vector DB 60.

[Knowledge Base]
Here, the knowledge base 50 will be described. The knowledge base 50 is a database in which information about things and information about semantic relationships between things are described as graphs. Things in the knowledge base 50 are, for example, abstract concepts such as “human”, “machine”, “building”, “organization”, “beauty”, “study”, “travel”, and specific humans, specific Such individuals, such as buildings, specific organizations, etc. (hereinafter “instances”). In the present embodiment, among the things, a thing for which information is described in the knowledge base 50 will be specifically referred to as an “entity”.

  An entity may represent, for example, an instance of an object of interest (for example, an object existing in the real world), or a concept of an object of interest (for example, a concept defined in the real world or virtual world). ). For example, there is an entity representing a concept such as “building”, and another entity representing an instance of the concept “building” such as “XX tower”.

  The knowledge base 50 is described using classes and relations defined by a vocabulary system called ontology in order to enable semantic processing by a computer. An ontology is a definition of classes and relations of things, and is a collection of constraints that hold between classes and relations.

  A class is a group of things that have the same properties in an ontology. The properties of classes and properties of things are described by relations described later.

  For example, an object that has a beak, is an ovarian vertebrate, and has a wing on the forelimb, is classified into a class of “bird” or a subordinate class thereof. In addition, in the class of “bird”, things that cannot fly are classified into lower classes such as “penguin” and “ostrich”, for example. In this way, the class system has a hierarchical structure having an upper and lower relationship, and the properties of the upper class are inherited by the lower class. In the above example, the nature of the class “bird”, “beaked, ovarian vertebrate, wings on the forelimbs” is the nature of the lower class of “penguins” and “ostriches”. Will also be included. The class name itself for identifying the class does not necessarily represent the meaning of the class, but in the following description, for simplicity, it is assumed that a class name representing the meaning of the class is given.

  Relations are attributes that describe the nature and characteristics of things and the relationships between classes. As a result, the relation becomes information representing the relationship between the entities. For example, a relation may be an attribute indicating the property of “having as a component of the body” or the property of “having inhabit”, or “a certain class is a higher class and a certain class is a lower class. It may be an attribute indicating the upper / lower relationship between classes, which is “class”. The relation name itself for identifying the relation does not necessarily indicate the meaning of the relation, but in the following description, for the sake of simplicity, it is assumed that the relation name indicating the meaning of the relation is given.

  The basic unit of the knowledge base 50 is a set of three information (hereinafter referred to as a triplet) in which entities are connected by relations. FIG. 2 is a diagram illustrating the relationship between two entities constituting a triplet and a relation. In the illustrated example, a triplet [entity “Japan”, relation “capital”, entity “Tokyo”] is cited. From such a triplet, it is possible to acquire semantic information that “the capital of Japan is Tokyo”. By using the knowledge base 50, information on entities and relationships between entities are clearly expressed, and various machine processes are possible. Hereinafter, “Japan” in FIG. 2, that is, the entity on the original side with respect to the semantic direction of the relation in the triplet is referred to as the head entity, and “Tokyo” in FIG. Is called a tail entity. The knowledge base 50 includes a plurality of triplets. One head entity can be a tail entity for another head entity and vice versa. FIG. 3 is an image diagram schematically showing the contents of data registered in the knowledge base 50. In the figure, arrows indicate relations, and broken ellipses indicate triplets. Entity 1 in the figure is a head entity for entity 2 and a tail entity for entity 3.

[First learning unit]
For each of the plurality of triplets (known information) sampled from the knowledge base 50, the first learning unit 20 adds the relation vector to the head entity vector so that the vector of the tail entity approaches the vector of the tail entity. 50 learns each entity included in 50 and the vector of each relation.

Here, the head entity vector is represented as Vh, the relation vector is represented as Vr, and the tail entity vector is represented as Vt. Since the entity space includes a larger amount of information than the relation space, for example, the entity vector is generated with a dimension number (k) higher than the dimension number (d) of the relation vector Vr. In the following description, vectors Vhr and Vtr mapped to the space to which the relation vector Vr belongs by multiplying the vector Vh of the head entity and the vector Vt of the tail entity by the k × d matrix M ^{k × d} , respectively, It is called a vector of tail entities (see equations (1) and (2); M is a matrix). The head entity vector Vhr is an example of “first entity feature quantity”, the tail entity vector Vtr is an example of “second entity feature quantity”, and the relation vector Vr connecting them is “first entity feature quantity”. It is an example of a “relation vector”. In the figure, O is the origin of the vector space.

Vhr = Vh × M ^{k × d} (1)
Vtr = Vt × M ^{k × d} (2)

  Ideally, the vector sum of the head entity vector Vhr and the relation vector Vr matches the tail entity vector Vgt. FIG. 4 is a diagram schematically illustrating a geometric relationship among the head entity vector Vhr, the relation vector Vr, and the tail entity vector Vgt.

The first learning unit 20 calculates the head entity vector Vhr, the relation vector Vr, and the tail entity vector Vgt so that the score C based on the objective function fr represented by the expression (3) is sufficiently small and converges. learn. In the equation, || A || ₂ represents the L2 norm of the vector A. The objective function fr is zero when the geometrical relationship in FIG. 3 is established.

fr (h, t) = {|| Vhr + Vr−Vtr || ₂ } ² (3)

The score C is represented by the formula (4). Where S is the combination of head entities, relations and tail entities that make up the correct triplet, and S * is the combination of head entities, relations, and tail entities that do not make up the correct triplet. γ is an arbitrarily determined margin parameter.

  The entity and relation vectors learned by the first learning unit 20 are registered in the entity / relation vector DB 60 and used by the second learning unit 30. FIG. 5 is an image diagram schematically showing the contents of data registered in the entity / relation vector DB 60. Focusing on a triplet including entity 1 and entity 2, the vector V1r of entity 1 corresponds to Vhr, the vector V2r of entity 2 corresponds to Vtr, and the relation vector V12r connecting them corresponds to Vr. Focusing on a triplet including entity 1 and entity 3, the vector V1r of entity 1 corresponds to Vtr, the vector V3r of entity 3 corresponds to Vhr, and the relation vector V31r connecting them corresponds to Vr.

[Second learning unit]
The second learning unit 30 learns parameters and the like of the derivation expressed by the derivation information 70. The derivation unit generates a vector of text composing the question (hereinafter, question vector Vq). When the processing of the second learning unit 30 is semantically decomposed, the learning of the question vector Vq itself according to the relationship with the head entity, the relation, and the tail entity, and the rule that makes the text constituting the question the question vector Vq And learning. The question vector Vq is a vector in the same vector space as the vectors Vhr, Vr, and Vtr, that is, a vector having the same dimensionality as these.

  In the following description, the second learning unit 30 includes information associating a question with a triplet including a head entity and a relation that is considered to represent the meaning of the question and a tail entity that is a correct answer of the question. It shall be given as data. For example, [question “where is the capital of Japan”, head entity “Japan”, relation “capital”, tail entity “Tokyo”] is one record, and data including a plurality of records is the teacher data. Given to.

  The second learning unit 30 reduces the difference (distance) between the question vector Vq and the tail entity vector Vtr based on the teacher data and the entity relation vector DB 60, The question vector Vq is learned so as to increase the difference (distance) from the sampled entity vector Vtr * other than the tail entity. That is, the second learning unit 30 learns the question vector Vq so as to reduce the objective function g1 of the equation (5).

g1 = {|| Vt−Vq || ₂ } ² (5)

  Further, the second learning unit 30 further reduces the difference between the question vector Vq and the sum of the head entity vector Vhr and the relation vector Vr, and samples the questions other than the head vector Vq and the head entity. The question vector Vq is learned so as to increase the difference between the vector Vhr * of the entity and the sum of the sampled relation vectors Vr * other than the relation. That is, the second learning unit 30 learns the question vector Vq so as to reduce the objective function g2 of the equation (6).

g2 = {|| Vhr + Vr−Vq || ₂ } ² (6)

  More specifically, the second learning unit 30 learns the question vector Vq so that the score L based on the sum of the objective function g1 and the objective function g2 becomes sufficiently small and converges. The score L is expressed by Expression (7). Where S is the combination of the head, relation, and tail entities that make up the correct triplet and the corresponding question, and S * does not make up the correct triplet, or the question and other information do not correspond, A combination of head entity, relation, tail entity, and question. γ is an arbitrarily determined margin parameter.

  The second learning unit 30 learns parameters of a derivation unit that derives a question vector Vq from a text that is a natural sentence. FIG. 6 is an image diagram conceptually showing the function of the deriver. As illustrated, the deriver is generated based on, for example, the RNN. For example, a code of a word obtained by dividing a question into characters is sequentially input to the deriver. The RNN outputs temporary question vectors Vq (1), Vq (2),... At each time point, and propagates at least a part of the calculation result to the next time point. The output of the RNN at the time when the codes of all the words of the question are input becomes the question vector Vq. The second learning unit 30 derives a vector Vq of a question learned so as to reduce the score L based on the vectors Vhr, Vr, and Vtr of the corresponding known head entity, relation, and tail entity for a certain question. Thus, the RNN parameters are learned. This parameter is held as derivator information 70.

  If the question is in English, words separated by spaces may be entered sequentially into the RNN at each point of time. If the question is Japanese or other, the words separated by morphological analysis are entered sequentially into the RNN at each point in time. May be. The second learning unit 30 performs learning according to these modes.

  FIG. 7 is an image diagram conceptually showing the contents of processing by the second learning unit 30. As shown in the figure, a question vector Vq is obtained by inputting the question to the derivation unit. The second learning unit 30 has sufficient distance d1 between the question vector Vq and the tail entity vector Vtr and the distance d2 between the question vector Vq, the head entity vector Vhr, and the relation vector Vr. The parameters of the derivation are learned so as to be smaller.

  Further, as described above, the knowledge base 50 is provided with information (for example, class) on the nature of the entity. The second learning unit 30 samples the entity to be negatively sampled regardless of the class in the first period until a certain amount of time elapses from the start of learning, and in the second period after the first period, The negative sampling entity is sampled from entities having the same class as the correct answer entity. Thus, efficient learning can be performed and convergence can be accelerated.

  The answer output unit 40 inputs, for example, a question acquired from another device via the network NW to a derivation specified by the derivation information 70, and outputs the vector of the entity closest to the outputted question vector Vq as an entity. Extracted from the relation vector DB 60. At this time, the answer output unit 40 extracts a vector of an entity closest to the question vector Vq by, for example, a 1NN (Nearest Neighbor) technique. The answer output unit 40 returns the content of the entity corresponding to the closest entity vector as an answer to the question to, for example, the other apparatus via the network NW. When the information processing apparatus processes alone, the information processing apparatus processes a question input to an input unit (not shown) and causes the output unit to output an answer.

  According to the information processing apparatus described above, the answer to the question can be derived more efficiently and with high accuracy.

  Here, as a comparison target, a case where learning is performed so that the question vector Vq is simply brought close to the vector Vhr + Vr (hereinafter, a comparative example) is considered. FIG. 8 is an image diagram schematically showing the contents of processing according to the comparative example. In the comparative example, the answer is derived through two-stage learning (inference): learning that brings Vq closer to Vhr + Vr (reducing the distance d2) and learning that brings Vhr + Vr closer to Vtr (reducing the distance d3). However, since the information processing apparatus according to the embodiment includes learning that directly brings Vq close to Vtr, the accuracy of deriving an answer can be improved.

  The same can be said from the viewpoint of compression of information amount. In learning to bring Vq closer to Vhr + Vr, a kind of dimensional compression is performed. This is because the natural sentence of the question includes information (hereinafter referred to as additional information) beyond the abstract semantic information represented by the head entity and the relation. In the comparative example, learning to approach Vhr + Vr is performed in a state where the additional information is deleted, whereas in the information processing apparatus of the embodiment, learning to approach Vtr is performed in a state including the additional information, so Vhr + Vr is originally set to Vhr + Vr. The result can be obtained with higher accuracy than the learning approaching Vtr.

  Further, according to the information processing apparatus of the embodiment, as shown in Expression (7), Vq is learned under the two constraints of the constraint that Vq is brought close to Vtr and the constraint that Vq is brought closer to Vhr + Vr. For this reason, the convergence of processing can be accelerated by reducing the degree of freedom in learning. As a result, processing can be performed faster than the comparative example.

  In addition, according to the information processing apparatus of the embodiment, as shown in Expression (7), when solving the problem of minimizing the distance to the positive sample and maximizing the distance to the negative sample, By making the sign minus, the entire problem is minimized. Since processing such as deep learning is suitable for solving the problem of minimizing the objective function, the processing of the embodiment can be suitably realized using a computer.

  In the above description, the information processing apparatus is assumed that the code of a word separating the questions is sequentially input to the RNN. However, the synonym dictionary or the like is used to make a regular expression to some extent and then input to the RNN. May be.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, In the range which does not deviate from the summary of this invention, various deformation | transformation and substitution Can be added.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 10 Question acquisition part 20 1st learning part 30 2nd learning part 40 Answer output part 50 Knowledge base 60 Entity relation vector DB
70 Deriver information

Claims (9)

The feature quantity of the first entity selected from the plurality of entities based on a first database in which a plurality of entities and relations indicating relationships between the entities are registered, the first entity and the second entity The feature quantity of the question is derived by referring to the second database including the feature quantity group learned so that the feature quantity obtained by adding the feature quantity of the first relation indicating the relation of the second entity is close to the feature quantity of the second entity A derivation learning unit for learning a derivation
The deriver learning unit includes:
Reducing the difference between the feature quantity of the question and the feature quantity of the second entity;
Learning the derivation unit so as to increase a difference between a feature amount of the question and a feature amount of a sampled entity other than the second entity;
Information processing device. The deriver learning unit further includes:
Reducing the difference between the feature quantity of the question and the sum of the feature quantity of the first entity and the feature quantity of the first relation;
The derivation unit is configured to increase a difference between a feature amount of the query and a feature amount of a sampled entity other than the first entity and a sum of feature amounts of sampled relationships other than the first relation. learn,
The information processing apparatus according to claim 1. The feature amount of the question is a feature amount in the same space as the feature amount of the first entity, the feature amount of the first relation, and the feature amount of the second entity.
The information processing apparatus according to claim 1 or 2. The first database is provided with information on the nature of the entity,
The derivation learning unit samples entities other than the second entity regardless of the property in the first period, and the property is the second entity in the second period after the first period. Sampling entities other than the second entity among entities in common with
The information processing apparatus according to any one of claims 1 to 3. The derivation unit is configured based on RNN (Recurrent Neural Networks) into which words obtained by dividing the question are sequentially input.
The deriving unit learning unit learns the parameters of the RNN;
The information processing apparatus according to any one of claims 1 to 4. The feature quantity of the first entity selected from the plurality of entities based on a first database in which a plurality of entities and relations indicating relationships between the entities are registered, the first entity and the second entity A feature amount learning unit that learns the feature amount so that the feature amount obtained by adding the feature amount of the first relation indicating the relationship is closer to the feature amount of the second entity;
The information processing apparatus according to any one of claims 1 to 5. An acquisition unit for acquiring a question;
An answer output unit that extracts an entity close to the feature amount of the question obtained by inputting the question acquired by the acquisition unit into the derivation unit from the first database and outputs the response as an answer to the question; Prepare
The information processing apparatus according to any one of claims 1 to 6. Computer
The feature quantity of the first entity selected from the plurality of entities based on a first database in which a plurality of entities and relations indicating relationships between the entities are registered, the first entity and the second entity The feature quantity of the question is derived by referring to the second database including the feature quantity group learned so that the feature quantity obtained by adding the feature quantity of the first relation indicating the relation of the second entity is close to the feature quantity of the second entity Learn the derivator to
During the learning,
Reducing the difference between the feature quantity of the question and the feature quantity of the second entity;
Learning the derivation unit so as to increase a difference between a feature amount of the question and a feature amount of a sampled entity other than the second entity;
Information processing method. On the computer,
The feature quantity of the first entity selected from the plurality of entities based on a first database in which a plurality of entities and relations indicating relationships between the entities are registered, the first entity and the second entity The feature quantity of the question is derived by referring to the second database including the feature quantity group learned so that the feature quantity obtained by adding the feature quantities of the first relation indicating the relationship between the second entity and the feature quantity of the second entity. To learn the derivation
When learning,
Reducing the difference between the feature quantity of the question and the feature quantity of the second entity;
Learning the derivation unit so as to increase a difference between a feature amount of the question and a feature amount of a sampled entity other than the second entity;
program.

Images