JP2013016151A - Language processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a reference expression processing apparatus using a probability model that interprets and generates reference expression including descriptive expression, anaphoric expression and deictic expression according to a progress status of a dialogue.SOLUTION: The language processing apparatus according to the present invention includes: a reference expression processing unit for performing at least one of the interpretation and generation of the reference expression by a probability model comprising a reference expression Bayesian network that represents a relationship among a reference domain (D) as a collection of conceivable instruction objects, the instruction objects (X) in the reference domain, a concept (C) relating to the instruction objets, and words (W) representing the concept; and a storage unit for storing data necessary for forming the reference expression Bayesian network.

Description

  The present invention relates to a reference expression processing apparatus that processes a reference expression, a language processing apparatus including the reference expression processing apparatus, and a reference expression processing method.

  For example, consider a case where a human and a robot communicate using a voice dialogue system. Suppose that there are multiple desks and chairs in a room, and that a person has designated one desk by the reference expression “a white table with red legs”. Here, the reference expression is a linguistic expression that indicates a specific thing that the speaker is interested in to the listener. The task of the robot language processing device to specify a desk designated by a person from the reference expression is understanding of the reference expression. In addition, in order for the language processing device of the robot to make the person specified by the robot a human, a reference expression is generated by generating a human-readable expression that represents the desk. Since human reference expressions naturally depend on human knowledge, the robot language processing apparatus needs to use information related to human knowledge in understanding and generating the reference expressions.

  It is conceivable that the language processing device uses a probabilistic model to use information about human knowledge in understanding and generating the reference expression.

  Patent Document 1 by the same applicant as the applicant of the present application discloses a language processing apparatus that handles a reference to an object part of a reference expression and a probability calculation method used in language processing by the language processing apparatus. .

  However, Patent Document 1 does not disclose a mechanism for understanding and generating a reference expression including a descriptive expression, an anaphoric expression, and a direct expression according to the progress of the dialogue.

JP 2010-224536 A

  Therefore, there is a need for a reference expression processing device, a language processing device, and a reference expression processing method using a probability model that understands and generates a reference expression including a description expression, an anaphoric expression, and a direct expression according to the progress of the dialogue. is there.

  The reference expression processing apparatus according to the first aspect of the present invention expresses a reference domain (D) that is a set of possible instruction objects, an instruction object (X) in the reference domain, a concept (C) and an object related to the instruction object. Necessary for forming the reference expression Bayesian network, and a reference expression processing unit for understanding and generating the reference expression by a probability model composed of the reference expression Bayesian network representing the relationship between the words (W) And a storage unit for storing various data.

  The reference expression processing apparatus according to this aspect uses a probability model composed of a reference expression Bayesian network that represents a relationship between an instruction object (X), a concept (C) related to the instruction object, and a word (W) expressing the concept. Therefore, it is possible to understand and generate reference expressions including descriptive expressions, anaphoric expressions, and direct expressions. Further, since the expression Bayesian network includes a reference domain (D) that is a set of possible instruction targets, the reference expression processing apparatus can process the reference expression according to the situation.

  A reference expression processing apparatus according to an embodiment of the present invention is configured such that the reference expression Bayesian network is formed for each processing of a reference expression while a dialog is in progress.

  According to the reference expression processing apparatus of the present embodiment, it is possible to process the reference expression in accordance with the progress of the dialogue.

  A reference expression processing device according to an embodiment of the present invention is configured to change the way of determining the reference domain according to the type of reference expression.

  According to the reference expression processing apparatus of this embodiment, the reference expression can be processed with higher accuracy by considering the type of the reference expression.

  A reference expression processing device according to an embodiment of the present invention is configured such that, when a reference expression includes an indicator, the reference domain includes all elements (objects that can be specified).

  According to the reference expression processing apparatus of the present embodiment, when the reference expression includes a directive, the reference expression is processed with higher accuracy by considering only a single reference domain including all elements as the reference domain. can do.

  A reference expression processing device according to an embodiment of the present invention forms a plurality of estimation models of a reference domain using the manifestation of the reference domain as a parameter, and determines whether the reference expression instruction target is a single object or a set. One of the plurality of estimation models is selected and used.

  According to the reference expression processing apparatus of the present embodiment, a reference expression can be obtained with higher accuracy by selecting and using one of a plurality of estimation models depending on whether the target object of the reference expression is a single object or a set. Can be processed.

  The language processing apparatus according to the second aspect of the present invention includes the reference expression processing apparatus according to the present invention.

  Since the language processing apparatus of this aspect includes the reference expression processing apparatus according to the present invention, the reference expression can be processed with high accuracy as described above.

  In the reference expression processing method according to the third aspect of the present invention, the reference expression processing unit of the language processing device uses the data stored in the storage unit to perform the reference domain (D) and the reference domain on the reference expression. Forming a reference expression Bayesian network representing a relationship between a pointing object (X), a concept (C) related to the pointing object, and a word (W) expressing the concept, and the reference expression processing unit includes the Bayesian network And obtaining the probability P (X | W), and the reference expression processing unit obtaining x ′ that maximizes the probability P (X | W), and setting the reference expression as an instruction target; ,including.

  The reference expression processing method according to this aspect uses a probability model composed of a reference expression Bayesian network that represents a relationship between an instruction object (X), a concept (C) related to the instruction object, and a word (W) expressing the concept. Therefore, it is possible to understand and generate reference expressions including descriptive expressions, anaphoric expressions, and direct expressions. Further, since the expression Bayesian network includes a reference domain (D) that is a set of possible instruction targets, according to the reference expression processing method, it is possible to process the reference expression according to the situation.

を求める方法を示す流れ図である。 It is a figure which shows the structure of the language processing apparatus by one Embodiment of this invention. It is a figure which shows the basic network structure of a reference Bayesian network. It is a figure which shows the reference Bayesian network of the reference expression which shows one instruction | indication object like "the table." It is a figure which shows the reference Bayesian network of the reference expression which shows two instruction | indication objects like "his table." It is a figure which shows the scene which is solving the tangram puzzle which consists of seven pieces. It is a flowchart which shows the process of the understanding of the reference expression by a reference expression process part. It is a flowchart for demonstrating the detailed process of step S1010 of FIG. It is a flowchart which shows the production | generation process of the reference expression by a reference expression process part. It is a figure which shows the excerpt of the concept dictionary defined with respect to the REX-J corpus. It is a figure which shows the excerpt of the static fitness table defined with respect to seven pieces shown in FIG. Using a list of reference domains

It is a flowchart which shows the method of calculating | requiring.

  FIG. 1 is a diagram showing a configuration of a language processing apparatus 100 according to an embodiment of the present invention. The language processing apparatus 100 may be configured by a single or a plurality of processors and a single or a plurality of memories.

  The language processing apparatus 100 includes a reference expression processing unit 101 and a storage unit 105 as characteristic components.

  The reference expression processing unit 101 includes a probability model for calculating a probability that a certain reference expression represents a candidate instruction target. The storage unit 101 stores data used by the probability model. The reference expression processing unit 101 updates the probability model according to the progress of the dialogue. The reference expression processing unit 101, when understanding the reference expression, that is, when specifying the instruction target represented by the reference expression, calculates the probability that the reference expression represents the candidate instruction object based on the probability model, and the probability is The largest candidate instruction target is specified as the instruction target represented by the reference expression. Further, the reference expression selection unit 107 calculates the probability that the candidate reference expression represents the instruction object when generating the reference expression, that is, when selecting the reference expression representing the instruction object, and the probability is the maximum. Is selected as a reference expression representing the instruction target. The reference expression processing unit 101 and the storage unit 105 constitute a reference expression processing device. Details of components of the reference expression processing device will be described later.

  For example, the speech recognition unit 115 recognizes human speech, divides it into morphemes that are the smallest units of language, and determines the part of speech of each morpheme using a dictionary. The structure analysis unit 111 analyzes the structure of the sentence from morpheme information obtained by the speech recognition unit 115.

  The surface layer realizing unit 113 generates a natural language expression including the reference expression generated by the reference expression processing unit 101. The speech synthesis / display unit 117 synthesizes or displays a natural language expression.

  The language processing apparatus 100 further includes a language understanding processing unit 103, a language generation processing unit 107, and a dialogue management unit 109. The language understanding processing unit 103 receives the result of the structural analysis from the structural analysis unit 111, performs language processing other than the reference expression processing performed by the reference expression processing unit 101, and sends the result to the dialog management unit 109. The language generation processing unit 107 performs language generation other than the generation of the reference expression performed by the reference expression processing unit 101, and sends the result to the surface layer realizing unit 113. The dialogue management unit 109 receives the instruction target represented by the reference expression from the reference expression processing unit 101, and receives and receives the result of language processing other than the reference expression processing performed by the reference expression processing unit 101 from the language understanding processing unit 121. Perform input processing. The dialogue management unit 109 creates an output based on the input or other conditions, and sends the output to the reference expression processing unit 101 and the language generation processing unit 107. The reference expression processing unit 101 receives the output of the dialogue management unit 109 and generates an appropriate reference expression. The language generation processing unit 107 receives the output of the dialogue management unit 109 and performs language generation other than the selection of the reference expression.

  Here, the probability model used by the reference expression processing unit 101 will be described. The probabilistic model uses a reference representation Bayesian network.

  FIG. 2 is a diagram illustrating a basic network structure of a reference Bayesian network. In FIG. 2, four nodes W, C, X, and D represent an observed word, a concept indicated by the word, an indication target of a reference expression, and an assumed reference domain, respectively. A word is an item of a concept dictionary described later.

Here, the reference domain (Susan Salmon-Art and Laurent Romary. 2000. Generating referring expressions in multimodal context. In Proceedings of the INLG 2000 workshop on Conference in Generated Multimedia, Mitzpe Ramon, Israel, June; Susan Salmon-Art and Laurent Romary 2001. Reference resolution within the framework of cognitive grammar.In Proceedings of the International Colloquium on Cognitive Science, San Sabastian, Spain, May; and Alexandre Dennis. 2010. Generating referring expressions with reference domain theory.In Proceedings of the 6 ^th International Natural Language Generating Conference (INLG), pages 27-35). The reference domain is a set including the instruction target. The elements included in the reference domain may be individual specific objects or other reference domains. Each reference domain d has a focus and a degree of manifestation (a non-negative real number). The degree of focus and manifestation is expressed as foc (d) and sal (d), respectively. Reference domains are sorted in descending order according to manifestation.

FIG. 5 is a diagram showing a scene in which a tangram puzzle consisting of seven pieces is being solved. The tangram puzzle will be explained later. The reference domain is introduced into the intellectual space of the dialog participant either by language by listening to the reference expression or visually by observing the physical situation. In the situation shown in FIG. 5, when “two large triangles” are referred to, a reference domain composed of pieces 1 and 2 is recognized. If piece 1 is moved and attached to piece 2, the reference domain composed of pieces 1, 2 and 6 is perceptually recognized from the proximity (Kristinn R. Thorisson. 1994. Simulated perceptual grouping: An application to human-computer interaction. In Proceedings of the 16 th annual Conference of the Cognitive Science Society, pages 876-881, Atlanta, GA, USA). Similarly, a reference domain composed of pieces 5 and 7 is also recognized. In the following, the reference domain is indicated by @ with an index, and its elements are enclosed in []. For example, @ ₁ = [1,2], @ ₂ = [1,2,6], @ ₃ = [5,7]. * Mark the focus element. For example, foc ([1 *, 2]) = 1.

The reference target of the reference domain differs depending on which reference domain is assumed. That is, if @ ₁ = [1,2] or @ ₂ = [1,2,6] is assumed, the instruction target of “right piece” is piece 1. If @ ₃ = [5, 7] is assumed, the instruction target of “right piece” is piece 5.

  The above literature on the reference domain adopts an approach based on logical symbolic operation rather than a probabilistic approach.

  FIG. 3 is a diagram illustrating a reference Bayesian network of a reference expression indicating one indication target such as “the table”. In this way, a reference expression to an instruction target that a human recognizes as one unit is called a unit reference expression. In addition to the above example, reference expressions such as “red ball”, “that couple”, “next room”, and “yesterday” are unit reference expressions.

  FIG. 4 is a diagram illustrating a reference Bayesian network of a reference expression indicating two target objects such as “his table”. As described above, a reference expression including two or more unit reference expressions is referred to as a composite reference expression. In addition to the above example, reference expressions such as “my ball”, “on the desk”, and “December last year” are compound reference expressions.

  Although the reference Bayesian network can process a composite reference expression, a case where a unit reference expression is processed will be described below for the sake of simplicity.

は、ランダム変数Ｖの領域を示す。

は、対応する観測される語ｗ_ｉ及び他の可能性を表すωを含み、

となる。各Ｗｉは対応するノードＣｉを有する。

は、ｗ_ｉによって表すことのできるＭ個の概念及び他の可能性を表す特別の概念Ωを含み、

となる。

は、後で説明する概念辞書から参照される。

は、その時点までに認識されたＬ＋１個の参照ドメインを含み、

となる。@₀は、対話において参照されるすべての個別の具体物を含む基底ドメインである。対話の初期において、

である。他のＬ個の参照ドメインは、対話中に徐々に追加される。

は、全ての考えうる指示対象を、すなわち、Ｋ個の個々の具体物及びＬ＋１個の参照ドメインを含む。したがって、

となる。 The reference Bayesian network for an N-word unit reference expression instance has 2N + 2 individual random variables W ₁ ,..., W _n , C ₁ ..., C _n , X and D. The area of each variable is determined by the current context and the corresponding reference domain. here,

Indicates the region of the random variable V.

Contains the corresponding observed word w _i and other possibilities ω,

It becomes. Each Wi has a corresponding node Ci.

Contains the M concepts that can be represented by w _i and the special concept Ω representing other possibilities,

It becomes.

Are referred to from a concept dictionary described later.

Contains L + 1 reference domains recognized so far,

It becomes. @ ₀ is the base domain that contains all the individual objects referenced in the dialog. Early in the conversation

It is. The other L reference domains are added gradually during the interaction.

Contains all possible referents, ie K individual specifics and L + 1 reference domains. Therefore,

It becomes.

と定式化される。

は、以下に説明する確率表によって計算される結合確率を周辺化することによって得られる。 How to find the target using the reference Bayesian network

Is formulated.

Is obtained by marginalizing the joint probabilities calculated by the probability table described below.

は、話し手が指示することを意図するｃ及びｘから聴き手がｗを観測する確率である。 Since all random variables used in the reference Bayesian network are discrete, the probability distribution is given as a probability table. The four probability tables used in the reference Bayesian network are described below.

Is the probability that the listener will observe w from c and x that the speaker intends to direct.

である。しかし、Ｘは、個別化された項目(名前)を扱うのに必要である。 In most cases, Wi does not depend on X. That is,

It is. However, X is necessary to handle individualized items (names).

に対して

とし、Ωに対して、

とする。ここで、Ｔは、概念

に対して考えられる語の数であり、εは、１０^−８のような予め定めた小さな数である。

は、ｄ内のｘを指示するために、

から概念ｃが選択される確率である。

は、コンテキストに依存するので、対話システムの開発者は、あらかじめ

を与えることはできない。したがって、

から

を構成するアプローチを採用する。

は、ｄに関して、概念

の指示対象ｘに対する適合度であり、

である。１は、完全な適合度を意味し、０は適合度がないことを意味する。０．５は中立を意味する。たとえば、ｘがスーツケースであるとき、概念「箱」は、０．８のような、高い適合度を有するが、概念「ボール」は、０．１のような、低い適合度を有する。ｘがｄの中になければ、

は、０である。

のいずれもが高い適合度を有さない場合には、概念Ωには高い確率が割り当てられる。

が静的であれば、

は、予め表の形で数値として与えられる。静的でなければ、対話システム開発者によって関数の形で組み込まれる。すなわち、

である。ここで、Ｉは、対話システムから得られるすべての情報である。 There are several methods for assigning probabilities. One simple method is as follows. Each

Against

And for Ω,

And Where T is the concept

Ε is a predetermined small number such as 10 ⁻⁸ .

To denote x in d

The probability that concept c is selected from

Depends on the context, so interactive system developers

Can not give. Therefore,

From

Adopt the approach that composes.

Is a concept with respect to d

The degree of conformity to the instruction target x of

It is. 1 means perfect fitness, 0 means no fitness. 0.5 means neutrality. For example, when x is a suitcase, the concept “box” has a high fitness, such as 0.8, whereas the concept “ball” has a low fitness, such as 0.1. If x is not in d,

Is 0.

If none of these have a high fitness, the concept Ω is assigned a high probability.

Is static,

Is given as a numerical value in the form of a table in advance. If it is not static, it is incorporated in the form of a function by the dialog system developer. That is,

It is. Here, I is all information obtained from the interactive system.

ここで、ｕ_ｘ，ｕ_ｌｕ_ｒは、それぞれ、ｘ、ｄにおける最も左のピース、及びｄにおける最も右のピースの水平座標である。これらは、Ｉから得られる。ｘが参照ドメインであれば、適合度は、該参照ドメインに含まれる要素の平均値として与えられる。

は、参照ドメインｄにおける要素ｘが参照される確率であり、参照ドメインにおける属性情報に関係なく、対応する参照表現が発せられた時点におけるコンテキスト情報にしたがって推定される。コンテキスト情報は、それまでの参照の歴史(談話)及び参照者の注視のような物理的な状態(状況)を含む。

を予測モデルと呼称する。予測モデルについては実験に関連して後で説明する。

は、参照表現が発せられた時点で、参照ドメインｄが予想される確率である。参照ドメインは、陰(implicit)であるので、この確率モデルを推定するためのデータを収集することはできない。したがって、顕現性ｄに基づくアプリオリな近似関数を試験する。顕現性は、最新性に比例する。顕現性については評価実験に関連して後で説明する。
一様モデル
このモデルは顕現性を無視する。このモデルは、顕現性の重要度を理解するために導入する。

線形モデル
このモデルは、顕現性に比例するように確率を分配する。

指数モデル
このモデルは、最近の参照ドメインを強調する。この関数は、いわゆるソフト・マックスである。

For example, in the situation shown in FIG. 5, the fitness function of the positional concept “left” is implemented as follows.

Here, u _x and u _l u _r are the horizontal coordinates of the leftmost piece in x and d and the rightmost piece in d, respectively. These are derived from I. If x is a reference domain, the goodness of fit is given as an average value of elements included in the reference domain.

Is the probability that the element x in the reference domain d is referred to, and is estimated according to the context information at the time when the corresponding reference expression is issued regardless of the attribute information in the reference domain. The context information includes a history of previous references (discourse) and a physical state (situation) such as a gaze of the reference.

Is called a prediction model. The prediction model will be described later in connection with the experiment.

Is the probability that the reference domain d is expected when the reference expression is issued. Since the reference domain is implicit, data for estimating this probability model cannot be collected. Therefore, an a priori approximation function based on the manifestation d is tested. The manifestation is proportional to the freshness. The manifestation will be described later in connection with the evaluation experiment.
Uniform model This model ignores the manifestation. This model is introduced to understand the importance of manifestation.

Linear model This model distributes probabilities in proportion to the manifestation.

Exponential model This model highlights recent reference domains. This function is a so-called soft max.

  FIG. 6 is a flowchart showing a reference expression understanding process performed by the reference expression processing unit 101.

を求める。 In step S1010 of FIG. 6, the reference expression processing unit 101 creates a reference expression Bayesian network (REBN) for all possible instruction targets x, and the probability

Ask for.

が最大となるｘ’を参照表現の指示対象として選択する。 In step S1020 of FIG. 6, the reference expression processing unit 101 determines the probability.

X ′ that maximizes the reference expression is selected as a reference expression instruction target.

  FIG. 7 is a flowchart for explaining detailed processing of step S1010 of FIG.

  In step S2010 in FIG. 7, the reference expression processing unit 101 obtains P (D). The method for obtaining P (D) is as described above.

を求める。

の予測モデルについては、上述のように後で説明する。 In step S2020 of FIG. 7, the reference expression processing unit 101

Ask for.

This prediction model will be described later as described above.

を求める。

の求め方は上述のとおりである。 In step S2030 of FIG. 7, the reference expression processing unit 101

Ask for.

The method of obtaining is as described above.

を求める。

の求め方は上述のとおりである。 In step S2040 of FIG. 7, the reference expression processing unit 101

Ask for.

The method of obtaining is as described above.

を求める。 In step S2050 of FIG. 7, the reference expression processing unit 101

Ask for.

を、既存の手法を用いて周辺化し、

を求める。 In step S2060 of FIG. 7, the reference expression processing unit 101

Is marginalized using existing methods,

Ask for.

  FIG. 8 is a flowchart showing a process of generating a reference expression by the reference expression processing unit 101.

  In step S3010 of FIG. 8, the reference expression processing unit 101 receives the instruction target x and determines W candidates.

を求める。 In step S3020 of FIG. 8, the reference expression processing unit 101 uses the reference expression Bayesian network and performs the procedure shown in the flowchart of FIG.

Ask for.

  In step S3030 of FIG. 8, the reference expression processing unit 101 determines whether or not processing has been performed for all Ws. If the process is performed for all Ws, the process proceeds to step S3040. If all the W have not been processed, the process returns to step S3020.

が最大となるＷを参照表現として選択する。 In step S3040 of FIG. 8, the reference expression processing unit 101 performs processing for the instruction target x.

Is selected as a reference expression.

  As described above, the reference expression unit according to the present embodiment can perform the process of understanding and generating the reference expression using a single probability model using the reference expression Bayesian network.

  An experiment for evaluating reference expression processing according to the present embodiment will be described.

  REX-J Corpus (Phlipp Spanger, Masaaki Yasuhara, Ryu Iida, Takenobu Tokunaga, Asuka Terai, and Naoko Kuriyama. 2010. REX-J: Japanese referring expression corpus of located dialog. Language Resources and Evaluation. Online First, DOI: 10.1007 / s10579-010-9134-8) was used. The REX-J corpus is composed of 24 human-human dialogues, in which two participants solve the seven piece tangram puzzle shown in FIG. The goal of the puzzle is to combine seven pieces to form a specified shape. One of the two subjects plays the role of the operator (OP) and the other one plays the role of the solver (SV). The OP can operate the virtual puzzle piece displayed on the monitor of the personal computer by operating the mouse, but does not know the shape of the goal. The SV knows the shape of the goal but cannot manipulate the pieces. The state of the mouse cursor operated by the piece and the OP is shared by the two subjects in real time. In this way, the two participants advance a collaborative dialogue involving multiple reference expressions to the piece. In addition to the reference expression, the position and direction of the piece, the position of the mouse cursor, and the operation by the OP were recorded together with the time stamp and the ID of the pointing target piece.

Table 1 is a table showing each reference expression to which the instruction target is added. The first reference expression “big triangle” in Table 1 is ambiguous and refers to piece 1 or 2. The seventh and eighth reference expressions refer to the set of pieces 1 and 2. Other reference expressions refer to individual pieces.

  In order to avoid problems due to errors in structural analysis of reference expressions, an intermediate structure (REX-graph) from which the reference Bayesian network is constructed is also added. The intermediate structure is a list of separated words in parentheses.

  BNJ (http://bnj.sourceforge.net/) is used for the probability calculation. In the following, more or less specific implementations specific to the task area of the REX-J corpus will be described.

  FIG. 9 is a diagram showing an excerpt of the concept dictionary defined for the REX-J corpus. Forty concepts were defined by observing the dialogue.

  FIG. 10 is a diagram showing an excerpt of the static fitness table defined for the seven pieces shown in FIG. Goodness-of-fit values were determined for 13 out of 40 concepts. The OBJ fits uniformly and perfectly for all pieces. FIG is uniform for all pieces, but not very well suited. TRI fits only for pieces 1-5. SQR only fits for pieces 6 and 7, but piece 7 is not "square" in the strict sense and therefore does not fit perfectly.

  The fitness function was implemented in the remaining 27 concepts. Some of these will be described below.

「残りの」（REST）

のように２個の群を要素に持つ参照ドメインの場合に限って、焦点の当っていない方の群の適合度を１とすし、それ以外の場合は０とする。

「両方」（BOTH）
ｘが群でその要素が２個であるとき１とする。

「図形」（FIG）
この表現は組み上がっているピースの群を参照する。そこで、ｘが単一ピースの場合(single(x)=true)は静的適合表から得た数値ｒとし、ｘが群であり互いに接続して形をなしている場合(shape(x)=true)は１とする。

「全部」（ALL）
すべての参照ドメインは、自分自身への特殊な参照(自己参照)をその要素に含むと考える。つまり、

である。その上で、

とする。 “Another” (ANOTHER)
The focus element among the elements of the reference domain d is represented by focus (d).

“Remaining” (REST)

Only in the case of a reference domain having two groups as elements, the matching degree of the unfocused group is set to 1, and 0 is set otherwise.

“Both” (BOTH)
Set to 1 when x is a group and there are two elements.

"Figure" (FIG)
This representation refers to a group of assembled pieces. Therefore, when x is a single piece (single (x) = true), the numerical value r obtained from the static fit table is used, and when x is a group and connected to each other (shape (x) = true) is 1.

"All" (ALL)
Every reference domain considers its element to contain a special reference to itself (self-reference). In other words,

It is. Moreover,

And

  Next, a list of reference domains will be described. As the reference resolution (understands the reference expression) progresses, the reference domain is added to the list and updated as follows. The reference domains in the list are sorted in descending order according to the manifestation.

  For each reference resolution, assume that all previous reference expressions are resolved correctly. Therefore, after each reference resolution time point, if the correct reference target of the last reference expression is a set, a new reference domain that is the same as the set is added to the list of reference domains unless it is included in the list of reference domains. . In any case, unless the same reference expression as the set is already at the top of the list of reference domains, the manifestation of the same reference expression as the set is set to σ + 1. Here, sigma is the maximum visibility value in the list of reference domains at that time, that is, the visibility value of the leading reference domain.

  Before each reference resolution point, whether the most recently manipulated piece after the preceding reference expression constitutes a perceptual group is determined at the start of the target reference expression by a perceptual grouping described later. Check. If the group is recognized, the same new reference domain as the recognized group is added to the list of reference domains unless it is included in the list of reference domains. In any case, unless the same reference domain as the set is already at the top of the list of reference domains, the manifestation of the same reference domain as the set is set to σ + 1, and the focus of the same reference domain as the set is operated most recently. Set to a piece.

及び

である。この操作は、「残りの」(REST)のような概念を扱うのに必要である。 When a new reference domain @ _m has been added to the list, see the domain @ _n and comprehensive reference domain @ _l of the complement is also inserted after the @ _m of the list. here,

as well as

It is. This operation is necessary to handle concepts like "REST" (REST).

  Describe perceptual grouping. Here, when the shortest distance between two pieces is less than or equal to a predetermined value, the two pieces are considered to be in contact with each other, and only a set of the pieces in contact is recognized as a group. Although this method is not universal, it works satisfactorily in the REX-J corpus domain due to the nature of the tangram puzzle.

を求める方法を示す流れ図である。この処理は、対話管理部１０９が行ってもよい。 Figure 11 uses a list of reference domains

It is a flowchart which shows the method of calculating | requiring. This process may be performed by the dialogue management unit 109.

  In step S4010 of FIG. 11, the dialog management unit 109 updates the reference domain list based on the perceptual grouping result immediately before the reference expression processing unit 101 performs reference resolution.

を求める。 In step S4020 of FIG. 11, the reference expression processing unit 101 inputs the manifestation obtained from the list of reference domains into the above-described manifestation model.

Ask for.

  In step S4030 of FIG. 11, the dialog management unit 109 updates the reference domain list according to the result immediately after the reference resolution by the reference expression processing unit 101.

を構築するために、SVMrank（Thorsten Joachism. 2006. Training linear SVMs in linear time. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD), pages 217-226, Philadelphia, PA, USA, August）を使用するランキングに基づく方法(Ryu IIDA, Shumpei Kobayashi, and Takenobu Tokunaga. 2010. Incorporating extra-linguistic information into reference resolution in collaboration task dialogue. In Proceedings of the 48^th Annual Meeting of the Association for Computational Linguistics, pages 1259-1267, Uppsala, Sweden, July)を採用した。このモデルは、目標の要素は以前に参照されたか(談話素性)、目標はマウスカーソルの下にあるか(マウスカーソル素性)などの１６個の２値素性にしたがって要素をランク付けする。 As mentioned above, the prediction model

SVMrank (Thorsten Joachism. 2006. Training linear SVMs in linear time. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD), pages 217-226, Philadelphia, PA, USA, August) Ryu IIDA, Shumpei Kobayashi, and Takenobu Tokunaga. 2010. Incorporating extra-linguistic information into reference resolution in collaboration task dialogue.In Proceedings of the 48 ^th Annual Meeting of the Association for Computational Linguistics, pages 1259- 1267, Uppsala, Sweden, July). This model ranks the elements according to 16 binary features such as whether the target element has been previously referenced (discourse feature) or whether the target is under the mouse cursor (mouse cursor feature).

  If the goal is a set, ie a reference domain, the discourse features for the set are calculated as in the case of pieces. On the other hand, mouse cursor features are handled differently. That is, if one of the group members satisfies the mouse cursor feature criterion, the group is determined to satisfy the criterion.

で表せる。θはコンテキスト情報である。

は、ランクを使用して以下の式で求められる。

ここで、

は

となるための正規化係数である。 The rank is

It can be expressed as θ is context information.

Is obtained by the following equation using the rank.

here,

Is

Is a normalization coefficient for

は０である。 If there is no x in d,

Is 0.

  24 dialogues were used for evaluation. The 24 interactions include 1474 unit reference expressions and 28 compound reference expressions. In addition to compound reference expressions, we excluded reference expressions that refer to complex concepts that are difficult to instantiate the fitness function in a short time. After excluding these reference expressions, 1310 reference expressions were available. Of the 1310 reference expressions, 182 (13.9%) refer to the set, and 612 (46.7%) are pronouns such as “it”.

である。ｘがピースであれば、

及び

である。反対にｘが集合であれば、

及び

である。 The following conditions are assumed in the experiment.
Suppose that the speaker role independence reference expression is independent of the speaker role, ie, the "solver" and the "manipulator". All reference expressions are mixed and processed sequentially.
It is assumed that no errors arise from the complete processing and preprocessing including past information speech recognition, morphological analysis and syntax analysis. Furthermore, it is assumed that all correct instruction targets of past reference expressions are known.
No future information In human-to-human dialogue, information that is useful for resolving reference expressions is often given after the reference expression is issued. Such future information is not considered.
Numerous information Many languages, including English, require the use of articles, noun singular / plural forms and collocations to indicate the distinction of numbers. Japanese does not have such a grammatical mechanism, but such distinctions can be predicted using machine learning techniques using linguistic information and gesture information. Therefore, the effect of giving such a number of information was observed. In the following experiment, the reference Bayesian network is provided with information for distinguishing the singular and plural by looking at the annotation of the correct pointing object in advance. This is achieved by adding a special evidence node C0. here,

It is. If x is a piece,

as well as

It is. Conversely, if x is a set,

as well as

It is.

モデルを準備した。単一ドメインにおいて、

は、その時点までに認識された個々のピース及び参照ドメインを含む単一の参照ドメイン

のみから構成される。すなわち、

である。 As a baseline for the experiment, called a single domain

A model was prepared. In a single domain,

Is a single reference domain containing individual pieces and reference domains recognized up to that point

Consists of only. That is,

It is.

  In this experiment, when the reference expression includes a directive, performance is improved by using a single domain rather than using a reference domain according to the above model. In the following results, a single domain was always used when the reference expression included a directive.

Table 2 is a table showing experimental results. The performance of the reference solution is expressed with accuracy by category and condition. Here, the accuracy is a value obtained by dividing the number of correctly resolved reference expressions by the number of reference expressions.

  For the evaluation, three categories of “single”, “plural” and “whole” were set. The “single” category is a collection of reference expressions that point to a single piece. “Plural” is a collection of reference expressions indicating a collection of pieces. “Whole” is the sum of them. The ambiguous reference expression such as the first in Table 1 is counted as “single”, and if the resolved result is one of the possible target objects, the resolution of such a reference expression is considered correct. .

  “No single / multiple information” indicates an experimental result without single / multiple distinction information, and “with single / multiple information” indicates an experimental result with single / multiple distinction information. The singular / plural information clearly has a strong influence.

モデルを選択することができる。したがって、モデルを切り替えることによって、単数・複数の情報を使用した「全体」の裁量の性能は、８３．２%に達し、ベースラインに対して２．０ポイントの増加が達成された（符号検定、p<0.0001）。 The best performance for the “single” category was achieved by the linear model, while the best performance for the “plurality” was achieved by the “exponential model”. If you can know whether the reference expression is “single” or “plural”, that is, if singular / plural information is available,

A model can be selected. Thus, by switching models, the discretionary performance of “overall” using single or multiple information reached 83.2%, and an increase of 2.0 points over the baseline was achieved (sign test) , P <0.0001).

  By introducing a reference domain, the resolution in the “multiple” category was significantly improved. The maximum performance increase is 9.3 points (sign test, p <0.005).

  In addition, more reference representations, including location concepts such as LEFT and RIGHT, were solved correctly for uniform, linear and exponential models.

Table 3 is a table summarizing the solution results of the four position concepts using single and plural pieces of information. The numerical values in Table 3 are the total number or the number of correct answers. The baseline single model solved 65%, while the linear model solved 75% correctly (sign test, p <0.05).

  For each solution, a dedicated Bayesian network is built for the reference representation of the problem. The constructed reference expression Bayesian network handles description, direct or anaphoric reference expressions in a unified way. Reference expression Bayesian networks incorporate the idea of a reference domain that allows contextual attributes to be used to resolve a reference expression and to process the reference expression according to a set. Reference representation Bayesian networks can be a standard approach that can be used for any and all task-oriented applications such as personal agents such as smartphones, automotive systems, service robots, and the like.

DESCRIPTION OF SYMBOLS 100 ... Language processing apparatus, 101 ... Reference expression process part, 103 ... Language understanding process part, 105 ... Memory | storage part, 107 ... Language generation process selection part, 109 ... Dialog management part

Claims (7)

A reference expression Bayesian network representing a relationship between a reference domain (D) that is a set of possible instruction objects, an instruction object (X) in the reference domain, a concept (C) about the instruction object, and a word (W) that expresses the concept A reference expression processing unit that performs at least one of understanding and generation of the reference expression using a probability model composed of
A reference expression processing apparatus comprising: a storage unit that stores data necessary to form the reference expression Bayesian network.   The reference expression processing device according to claim 1, wherein the reference expression Bayesian network is formed for each processing of a reference expression while a dialogue is in progress.   The reference expression processing device according to claim 2, wherein the reference expression processing apparatus is configured to change a method of determining the reference domain according to a type of reference expression.   The reference expression processing device according to claim 3, wherein the reference domain includes all elements when the reference expression includes a directive.   A plurality of estimation models of the reference domain is formed with the reference domain's manifestation as a parameter, and one of the plurality of estimation models is selected and used depending on whether the target object of the reference expression is a single object or a set. The reference expression processing device according to claim 3, which is configured to do so.   A language processing apparatus comprising the reference expression processing apparatus according to claim 1. The reference expression processing unit of the language processing device uses the data stored in the storage unit to perform a reference domain (D), a target object (X) in the reference domain, and a concept (C) And a reference expression Bayesian network representing the relationship between the words (W) representing the concept and
The reference expression processing unit peripheralizing the Bayesian network to obtain a probability P (X | W);
A reference expression processing method including: a step of obtaining x ′ that maximizes the probability P (X | W) and making the reference expression instruction target.

Images