JP5783979B2 - Generation model creation device and estimation device - Google Patents

Description

  The present invention relates to a technique for creating a generation model using action information and acoustic event information, and a technique for estimating an action and a situation using the generated generation model.

  In the prior art disclosed in Non-Patent Document 1, an acoustic signal representing an action (for example, an action where a person is cooking) is transmitted for each short time (about 20 to 100 msec). An acoustic signal sequence with an acoustic event label or an acoustic event label sequence with a label indicating what sound (footstep, water flowing sound; hereinafter referred to as an acoustic event) is input, and a finite number of consecutive A histogram for each acoustic event label is created using the acoustic event labels for the frames. In addition, a behavior model is generated using a modeling technique such as GMM (Gaussian Mixture Model), HMM (Hidden Markov Model), or SVM (Support Vector Machine) for the generated histogram for each acoustic event label.

  Furthermore, the behavior model and the histogram of the acoustic event calculated from the newly input labeled acoustic signal sequence are respectively compared (for example, comparison is performed using Euclidean distance, cosine distance, etc.), and among the plurality of behavior models Then, it is determined that the one that is most suitable for the judgment criterion represents the action corresponding to the acoustic signal sequence. As described above, in the prior art, the behavior can be estimated from the acoustic signal sequence.

Imoto et al., “User Action Identification Based on Frequency of Multiple Living Sounds and its Application to Communication”, The 32nd VMA Research Meeting of the Institute of Image Electronics Engineers of Japan

  In the prior art, the occurrence frequency itself of the acoustic event represented by the data used for model calculation is directly modeled. Such a method cannot appropriately model the frequency of occurrence of other acoustic events. The behavior model generated by such a method is excessively fitted to the data used for model calculation. Therefore, when performing behavior identification using a behavior model calculated by the conventional technology, even if slightly different data is input from data used for model calculation, it is determined that the behavior is very low in similarity. . However, it is rare that the data used when calculating the behavior model and the data input at the time of estimation are almost the same. Therefore, the use of the behavior model of the prior art leads to degradation of behavior estimation accuracy.

  The present invention provides a technique that makes it possible to accurately estimate new data without excessively fitting a generated model to data used for model calculation.

  In the present invention, learning information including behavior information representing the content of the behavior and acoustic event information representing the acoustic event associated with the behavior is used, and the situation of the potential field defined by the acoustic event is the acoustic event. The generation model is generated by obtaining the probability P (acoustic event | situation) to be generated and the probability P (situation | action) that the action generates the situation.

  In the present invention, the generation frequency of the acoustic event itself is not directly modeled, but the generation process of the generation model is stochastically handled by the probability P (acoustic event | situation) and the probability P (situation | action). Thereby, it is possible to prevent the generated model from being excessively fitted to the data used for the model calculation, and it is possible to perform accurate estimation for new data.

The block diagram of the production | generation model creation apparatus of 1st Embodiment. The figure which illustrated the acoustic signal sequence with an acoustic event label and an action label. The block diagram of the production | generation model creation apparatus of the modification 1 of 1st Embodiment. The block diagram of the production | generation model creation apparatus of the modification 2 of 1st Embodiment. The block diagram of the estimation apparatus of 2nd Embodiment. The block diagram of the estimation apparatus of the modification 1 of 2nd Embodiment. The block diagram of the estimation apparatus of the modification 2 of 2nd Embodiment. The block diagram of the estimation apparatus of 3rd Embodiment. The block diagram of the estimation apparatus of the modification 1 of 3rd, 4th embodiment. The block diagram of the estimation apparatus of the modification 2 of 3rd, 4th embodiment. The block diagram of the estimation apparatus of 4th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<Definition of terms>
Terms used in the embodiments are defined.
“Action” means any action performed by a subject such as a human being, an animal, or a device. Specific examples of “action” include “cooking” and “cleaning”.
An “acoustic event” means a sound event that occurs with an action. Specific examples of the “acoustic event” include “knife sound”, “water flowing sound”, “water sound”, “ignition sound”, “fire sound”, “foot sound”, and “vacuum exhaust sound”. For example, “knife sound”, “water flowing sound”, and the like are acoustic events that occur with the action of “cooking”.
“Situation” means a potential field situation defined by an acoustic event. The probability of situation generation is defined by the action in the time interval in which the situation occurs, and the situation defines the probability of acoustic event generation in the time interval in which the situation occurs. That is, the situation can be expressed by an action and a generation probability defined by the action. An acoustic event can be expressed by a situation and a generation probability defined by the situation.
“Probability that X generates Y” refers to the probability that event Y will occur under the condition that event X occurs. The “probability that X generates Y” can also be expressed as “the conditional probability of Y under X” or “the conditional probability of Y in X”.

<First Embodiment>
In the first embodiment, an acoustic event label and an acoustic signal with an action label are input as learning information, and an action-situation generation model when the probability that an action generates a situation by learning is P (situation | action), Situation when the probability that the situation generates an acoustic event is P (acoustic event | situation) -Sound event generation model, and the action when the probability that the action generates an acoustic event is P (acoustic event | action)- An acoustic event generation model is calculated.

  As illustrated in FIG. 1, the generation model creation apparatus 100 of this embodiment includes an acoustic signal sequence synthesis unit 101, a behavior / situation modeling unit 102, and a storage unit 103. The generation model creation apparatus 100 is configured by, for example, reading a predetermined program into a known or dedicated computer.

  First, acoustic signal strings 11-1,..., 11-D (where D is an integer equal to or greater than 1) are input to the acoustic signal string synthesis unit 101. As illustrated in FIG. 2, the acoustic event label and the acoustic signal sequence with the action label are a time-series acoustic signal sequence divided every short time (several tens of milliseconds to several seconds), and an acoustic signal divided every short time. Element number corresponding to each element in the column, acoustic event label determined and assigned every short time (corresponding to "acoustic event information"), and determined every long time (several seconds or more) and every short time The given action label (corresponding to “behavior information”) is included. The acoustic signal sequence is a digital signal sequence (for example, a digital signal sequence of about 5 to 20 seconds) representing a sound accompanying the action. The acoustic signal sequence is obtained, for example, by recording in an environment where a single action or a plurality of actions are performed. The acoustic event label is a label that represents an acoustic event associated with the action. The action label is a label representing the content of the action. The element number, the acoustic event label, and the action label are given for each element of the acoustic signal sequence. A plurality of types of action labels may correspond to one acoustic signal sequence. A plurality of types of action labels may correspond to one element number.

  A plurality of acoustic event labels and action-labeled acoustic signal sequences 11-1,..., 11-D (hereinafter simply referred to as “labeled acoustic signal sequences 11-1,..., 11-D”) are acoustic. When input to the signal sequence synthesizing unit 101, the acoustic signal sequence synthesizing unit 101 joins them in a time-series direction, whereby one acoustic event label and an action-labeled acoustic signal sequence 11 (hereinafter simply referred to as “labeled”). And obtain and output (synthetic processing). When only one acoustic signal sequence 11-1 is input to the acoustic signal sequence synthesis unit 101, the acoustic signal sequence synthesis unit 101 outputs it as a labeled acoustic signal sequence 11. The acoustic event label and the behavior-labeled acoustic signal sequence output from the acoustic signal sequence synthesis unit 101 are input to the behavior / situation modeling unit 102. Note that one labeled acoustic signal sequence 11 may be directly input to the behavior / situation modeling unit 102 without going through the acoustic signal sequence synthesis unit 101.

  The behavior / situation modeling unit 102 generates a behavior-situation generation model 12, a situation-acoustic event generation model 13, and a behavior-acoustic event generation model 14 from the input labeled acoustic signal sequence 11 according to the following procedure. These are stored in the storage unit 103.

[Logical explanation of the process by which acoustic events are generated from actions]
The behavior / situation modeling unit 102, from the labeled acoustic signal sequence 11, the probability P (situation | action) that the behavior generates a situation, the probability P (acoustic event | situation) that the situation generates an acoustic event, and the behavior Calculates a probability P (acoustic event | action) of generating an acoustic event. The behavior / situation modeling unit 102 includes a behavior-situation generation model 12 when the probability P (situation | action) is assumed, a situation-acoustic event generation model 13 when the probability P (acoustic event | situation) is assumed, and a probability. A behavior-acoustic event generation model 14 when P (acoustic event | action) is set is generated. In other words, in this form, it is considered that the behavior prescribes the probability of generating a situation, and the situation prescribes the probability of generating an acoustic event (that is, the action indirectly prescribes the probability of generating an acoustic event). Is described as each generation model.

ただし、Ａは行動の種類の個数、Ｔは状況の種類の個数、Ｅは音響イベントの種類の個数、ｅはラベル付き音響信号列１１に与えられた音響イベントの列（ベクトル）、Θは行動ａ（ただし、ａ∈｛１，・・・，Ａ｝）が状況ｔ（ただし、ｔ∈｛１，・・・，Ｔ｝）を生成する確率Ｐ（ｔ｜ａ）を（ａ，ｔ）要素とするＡ×Ｔ行列、Φは状況ｔ（ただし、ｔ∈｛１，・・・，Ｔ｝）が音響イベントε（ただし、ε∈｛１，・・・，Ｅ｝）を生成する確率Ｐ（ε｜ｔ）を（ｔ，ε）要素とするＴ×Ｅ行列、Ωはラベル付き音響信号列１１を構成する、ラベル付き音響信号列１１−ｄに付与される行動の集合の列（ただし、各ラベル付き音響信号列１１−ｄには複数個の行動が付与されてもよい）、ｅ’_ｄは各ラベル付き音響信号列１１−ｄに与えられた音響イベントの列（Ｎ_ｄ次元ベクトル：Ｎ_ｄはラベル付き音響信号列１１−ｄに対応する音響イベントの個数）、ａ_ｄはラベル付き音響信号列１１−ｄに付与され得る行動の集合、Ｄはラベル付き音響信号列１１を構成するラベル付き音響信号列１１−ｄの個数を表す。なお、Ωとａ_ｄの関係は以下のように表される。
Ω＝[ａ_１,ａ_２,…,ａ_ｄ,…,ａ_Ｄ］ When the generation probability Θ, the generation probability Φ, and the action sequence Ω are given, the generation probability P (e | Θ, Φ, Ω) of the acoustic event sequence e is as follows.

Where A is the number of types of actions, T is the number of types of situations, E is the number of types of acoustic events, e is the sequence (vector) of acoustic events given to the labeled acoustic signal sequence 11, and Θ is the behavior The probability P (t | a) that a (where a∈ {1,..., A}) generates the situation t (where t∈ {1,..., T}) is (a, t). A × T matrix as an element, Φ is the probability that situation t (where t∈ {1,..., T}) generates an acoustic event ε (where ∈∈ {1,..., E}) A T × E matrix having P (ε | t) as an element (t, ε), Ω is a sequence of action sets that are included in the labeled acoustic signal sequence 11-d and that constitutes the labeled acoustic signal sequence 11 ( However, a plurality of actions may be applied to each labeled acoustic signal sequence _11-d), e 'd is applied to each labeled acoustic signal sequence 11-d Column of the acoustic event (N _d dimensional vector: N _d is the number of acoustic event corresponding to the labeled acoustic signal sequence _11-d), a d is a set of actions that may be applied to the labeled acoustic signal sequence 11-d, D Represents the number of labeled acoustic signal sequences 11-d constituting the labeled acoustic signal sequence 11. The relationship between Ω and a _d is expressed as follows.
Ω = [a ₁ , a ₂ , ..., a _d , ..., a _D ]

ただし、ｅ_ｉはラベル付き音響信号列１１−ｄの要素番号ｉに対応する音響イベントラベルが表す音響イベント、Ｎ_ｄはラベル付き音響信号列１１−ｄに対応する要素数（要素番号ｉの最大値）、ｚ_ｉはラベル付き音響信号列１１−ｄの要素番号ｉに対応する状況、ｘ_ｉはラベル付き音響信号列１１−ｄの要素番号ｉに対応する行動ラベルが表す行動、Ａ_ｄはラベル付き音響信号列１１−ｄの行動ラベルが表す行動の総数（行動の種類数）、φ_ｔは状況ｔが音響イベントε（ただし、ε∈｛１，・・・，Ｅ｝）を生成する確率Ｐ（ε｜ｔ）をε番目の要素とするＥ次元ベクトル、θ_ａは行動ａが状況ｔ（ただし、ｔ∈｛１，・・・，Ｔ｝）を生成する確率Ｐ（ｔ｜ａ）をｔ番目の要素とするＴ次元ベクトル、φ_ｅｉ，ｔ（下付き添え字の「ｅｉ，ｔ」は「ｅ_ｉ，ｔ」）は状況ｔが音響イベントｅ_ｉを生成する確率Ｐ（ｅ_ｉ｜ｔ）、θ_ｔａは行動ａが状況ｔを生成する確率Ｐ（ｔ｜ａ）、αはθ_ａ及びθ_ｔａが従うＤｉｒｉｃｈｌｅｔ分布の性質を決める超パラメータ（例えば０．０１などの非負値をとる）、βはφ_ｔ及びφ_ｅｉ，ｔが従うＤｉｒｉｃｈｌｅｔ分布の性質を決める超パラメータ（例えば０．０１などの非負値をとる）、Ｕｎｉ（・｜・）は一様分布を表す。ここで、Ｐ（ｘ_ｉ＝ａ｜ａ_ｄ）は一様分布に従い、Ｐ（φ_ｔ｜β）及びＰ（θ_ａ｜α）はそれぞれβ，αをパラメータとするＤｉｒｉｃｈｌｅｔ分布に従うと仮定する。Ｗ−１次（Ｗは２以上の整数）のＤｉｒｉｃｈｌｅｔ分布の確率密度関数は以下の通りである。

ただし、Γはガンマ関数を表す。 Given the generation probability Θ, the generation probability Φ, and the sequence Ω of action sets, the generation probability P (e ′ _d | Θ, Φ, Ω) of the acoustic event sequence e ′ _d is as follows: .

However, e _i is the acoustic event represented by the acoustic event label corresponding to the element number i of the labeled acoustic signal sequence 11-d, and N _d is the number of elements corresponding to the labeled acoustic signal sequence 11-d (the maximum of the element number i) Value), z _i is the situation corresponding to the element number i of the labeled acoustic signal sequence 11-d, x _i is the behavior represented by the action label corresponding to the element number i of the labeled acoustic signal sequence 11-d, and _Ad is The total number of actions represented by action labels in the labeled acoustic signal sequence 11-d (the number of kinds of actions), φ _t is the situation t that generates an acoustic event ε (where εε {1,..., E}). An E-dimensional vector having probability P (ε | t) as the ε-th element, θ _a is a probability P (t | a) that action a generates a situation t (where t∈ {1,..., T}). ) a T-dimensional vector having t-th element, phi _{ei, t (the} subscript " i, t "is" _e i, t ") probability _{P (e} i the situation t is to generate an acoustic event _{e i} is | t), θ _ta the probability behavior a to generate a situation t P (t | a) , Α is _a super parameter that determines the nature of the Dirichlet distribution according to θ _a and θ _ta (for example, takes a non-negative value such as 0.01), β is a super parameter that determines the nature of the Dirichlet distribution according to φ _t and φ _{ei, t} Uni (. |.) Represents a uniform distribution (for example, takes a non-negative value such as 0.01). Here, it is assumed that P (x _i = a | a _d ) follows a uniform distribution, and P (φ _t | β) and P (θ _a | α) follow a Dirichlet distribution with β and α as parameters, respectively. The probability density function of the W-1 order (W is an integer of 2 or more) Dirichlet distribution is as follows.

Where Γ represents a gamma function.

[Description of generation model calculation process]
The behavior / situation modeling unit 102 includes a probability P (situation | action) that the behavior generates a situation, a probability P (acoustic event | situation) that the situation creates an acoustic event, and a probability P ( Sound event | action is calculated, and the action-situation generation model 12, the situation-acoustic event generation model 13, and the action-acoustic event generation model 14 corresponding to each are calculated.

  The behavior / situation modeling unit 102 calculates only the probability that the behavior generates a situation and the probability that the situation generates an acoustic event, and obtains only the behavior-situation generation model 12 and the situation-acoustic event generation model 13 corresponding to each. Only the probability that an action generates an acoustic event is calculated using the probability that the action generates a situation and the probability that the situation generates an acoustic event, and only the corresponding action-acoustic event generation model 14 is calculated. You may get

  For the calculation of the behavior-situation generation model 12 and the situation-acoustic event generation model 13, a Markov chain Monte Carlo method (MCMC method) or a variational Bayes method (VB method) is used. Can be used. The MCMC method includes methods such as an MH algorithm and Gibbs sampling. Here, a generation model calculation method based on Gibbs sampling will be described.

[Example of generation model calculation method]
Below, the calculation method of the action-situation production | generation model 12 and the situation-acoustic event production | generation model 13 which used Gibbs sampling is illustrated.

(I) The behavior / situation modeling unit 102 determines the hyper parameters α and β. For the hyper parameters α and β, values stored in advance in the storage unit of the behavior / situation modeling unit 102 may be used, or may be determined at random. The hyperparameters α and β may be calculated and updated by the following procedure using values stored in advance in the storage unit of the behavior / situation modeling unit 102 or randomly determined values as initial values.

(II) The behavior / situation modeling unit 102 assigns behavior a and situation t to each element of the acoustic signal sequence included in the input labeled acoustic signal sequence 11 according to a uniform distribution. However, the allocation of actions a, performed in accordance with uniform distribution from the action a _d given to each labeled acoustic signal sequence 11-d. Alternatively, the behavior / situation modeling unit 102 may assign the behavior a and the situation t to each element of the acoustic signal sequence according to the method set in the behavior / situation modeling unit 102 in advance.

Further, when D ≧ 2, the behavior / situation modeling unit 102 reassigns the element number i to each element of the acoustic signal included in the labeled acoustic signal sequence 11. That is, different element numbers i (where i∈ {1,..., U}) are assigned to all elements included in the labeled acoustic signal sequence 11. However, U is the total number of elements corresponding to the labeled acoustic signal 11 _{satisfies U = N 1 + ··· + N} D.

  The behaviour / situation modeling unit 102 assigns an action signal a and a condition t to each element and reassigns the element number i (when D ≧ 2), and obtains an “acoustic signal with a label to be updated”. The initial value of “column”.

(III) The behavior / situation modeling unit 102 performs the following (III-1) and (III) for all elements (element numbers iε {1,..., N}) of the labeled acoustic signal sequence to be updated. III-2) for a specified number of times (positive value, about 1 to 1000 times), or until a desired result is obtained (for example, before and after the assignment, the change in the assignment destination of the action and the situation is a certain threshold (for example, Repeat until 30%) or less.

ただし、Ｃ_εｔ ^ＥＴは更新対象のラベル付き音響信号列で音響イベントεに状況ｔが割り当てられた回数を表し、Ｃ_ｔａ ^ＴＡは更新対象のラベル付き音響信号列で状況ｔに行動ａが割り当てられた回数を表す。なお、表記制約上の都合から「Ｃ_εｔ ^ＥＴ」「Ｃ_ｔａ ^ＴＡ」と表記するが、本来は式（３）に示すように「Ｃ_εｔ ^ＥＴ」の「ＥＴ」は「εｔ」の上に表記され、「Ｃ_ｔａ ^ＴＡ」の「ＴＡ」は「ｔａ」の上に表記される。また、ｚ_−ｉは要素番号ｉ以外の要素番号に対応する状況からなる列、ｘ_−ｉは要素番号ｉ以外の要素番号に対応する行動からなる列、ｅ_−ｉは要素番号ｉ以外の要素番号に対応する音響イベントからなる列を表す。 (III-1) The behavior / situation modeling unit 102 calculates a probability distribution in which the behavior a and the situation t are assigned to the acoustic event ε of the element number i for all the behavior a and the situation for the acoustic signal sequence to be updated. Update for t. The updated probability distribution P (x _i = a, z _i = t | e _i = ε, z _−i , x _−i , e _−i , Ω, α, β) is shown below.

However, C _εt ^ET represents the number of times the situation t is assigned to the acoustic event ε in the labeled acoustic signal sequence to be updated, and C _ta ^TA is the action a assigned to the situation t in the labeled acoustic signal sequence to be updated. Represents the number of times In addition, “C _εt ^ET ” and “C _ta ^TA ” are written for convenience of notation, but originally “ ^ET ” of “C _εt ^ET ” is written above “εt” as shown in Equation (3). “ ^TA ” of “C _ta ^TA ” is written on “ta”. Also, z- _i is a column composed of situations corresponding to element numbers other than element number i, x- _i is a column composed of actions corresponding to element numbers other than element number i, and e- _i is an element other than element number i. Represents a sequence of acoustic events corresponding to a number.

(III-2) The behavior / situation modeling unit 102 obtains the probability distribution P (x _i = a, z _i = t | e _i = ε, z _−i , x ₋ obtained by the above update formula (3). _i , e _−i , Ω, α, β), the behavior and the situation assigned to the element of each element number i are randomly sampled. The behavior / situation modeling unit 102 assigns the behavior and situation sampled in this way to the element of each element number i, and updates the labeled acoustic signal sequence to be updated.

(IV) The behavior / situation modeling unit 102 calculates the following using C _εt ^ET and C _ta ^TA finally obtained by repeating (III-1) and (III-2) above.

As a result, the behavior / situation modeling unit 102 sets a set of probabilities θ _ta (where a∈ {1,..., A}, t∈ {1,..., T}) that the action generates a situation and A set of probabilities φ _εt (where tε {1,..., T}, εε {1,..., E}) that the situation generates an acoustic event is obtained, and each is set as an action-situation generation model 12. And the situation-acoustic event generation model 13. For example, the behavior / situation modeling unit 102 uses the A × T matrix having the probability θ _ta as the (a, t) element as the behavior-situation generation model 12 and T × E having the probability φ _εt as the (t, ε) element. The matrix is a situation-acoustic event generation model 13.

Alternatively, the behavior / situation modeling unit 102 samples one or more of the probability θ _ta and the probability φ _εt obtained in the calculation process of the equation (3) during the repetition process of (III-1), and the equation (4) Instead of (5), the behavior-situation generation model 12 and the situation-acoustic event generation model 13 may be obtained by using the average value of the sampled probabilities θ _{ta and} the average value of the probability φ _εt .

例えば行動／状況モデル化部１０２は、確率Ｐ（λ_εａ｜Θ，Φ，Ω）を（ａ，ε）要素とする、以下のようなＡ×Ｅ行列を行動−音響イベント生成モデル１４とする。

Further, the behavior / situation modeling unit 102 performs peripheral processing as follows, so that the behavior a (where a∈ {1,..., A}) is an acoustic event ε (where ∈∈ {1,. _.. , E}) to obtain a set of probabilities P (λ _εa | Θ, Φ, Ω), and obtain a behavior-acoustic event generation model 14 corresponding thereto.

For example, the behavior / situation modeling unit 102 sets the following A × E matrix having the probability P (λ _εa | Θ, Φ, Ω) as the (a, ε) element as the behavior-acoustic event generation model 14. .

ただし、α_ｎｅｘｔ，β_ｎｅｘｔは更新後のハイパパラメータα，βであり、ψ（ｚ）はディガンマ関数を表す。また、ディガンマ関数は以下の式で表わされる、ガンマ関数Γ（ｚ）の対数微分である。

ただし、Γ’（ｚ）はガンマ関数Γ（ｚ）の導関数である。 The behavior / situation modeling unit 102 can also update the hyperparameters α and β using the following update formula each time the update process shown in (III) is performed once.

Here, α _next and β _next are the updated hyperparameters α and β, and ψ (z) represents a digamma function. The digamma function is a logarithmic derivative of the gamma function Γ (z) expressed by the following equation.

Where Γ ′ (z) is a derivative of the gamma function Γ (z).

In addition, as a result of updating α and β, when α and β values exceed or fall below a certain threshold δ ₁ (> 0), δ ₂ (> 0),
if α <δ ₁ then α = δ ₁
if β <δ ₂ then β = δ ₂
Such processing may be added.

<Variation 1 of the first embodiment>
In the first modification of the first embodiment, an action-labeled acoustic signal sequence (corresponding to “acoustic signal sequence with behavior information”) is used as an input, and the behavior-situation generation model 12, the situation-acoustic event generation model 13, And the behavior-acoustic event generation model 14 is calculated. Similarly to the first embodiment, only the probability that an action generates a situation and the probability that the situation generates an acoustic event are calculated, and only the action-situation generation model 12 and the situation-acoustic event generation model 13 corresponding to each of them are calculated. The probability that an action generates an acoustic event is calculated using the probability that the action generates a situation and the probability that the situation generates an acoustic event, and only the corresponding action-acoustic event generation model 14 is calculated. You may get Hereinafter, the same reference numerals are given to the same components, and description thereof will not be repeated.

  As illustrated in FIG. 3, the generation model creation device 110 of this embodiment includes a feature amount calculation unit 111, an acoustic event determination unit 112, an acoustic event model database (DB) 113, an acoustic signal sequence synthesis unit 101, an action / situation model. And a storage unit 103. The generation model creation device 110 is configured by, for example, reading a predetermined program into a known or dedicated computer.

  First, acoustic signal sequences 15-1,..., 15-D with action labels are input to the feature amount calculation unit 111. Each action-labeled acoustic signal sequence 15-d (where dε {1,..., D}) is a time-series acoustic signal sequence divided for each short time (several tens of milliseconds to several seconds). The element number corresponding to each element of the acoustic signal sequence divided every time, and the action label (corresponding to “behavior information”) determined every long time (several seconds or more) and given every short time are included. The element number and the action label are given for each element of the acoustic signal sequence. A plurality of types of action labels may correspond to one acoustic signal sequence. A plurality of types of action labels may correspond to one element number.

  The feature amount calculation unit 111 extracts each action label from the sound signal string included in each action label-attached sound signal string 15-d (corresponding to “a time-series sound signal string representing a sound associated with the action represented by the action information”). An acoustic feature quantity sequence (vector) corresponding to the attached acoustic signal sequence 15-d is calculated and output. For example, the feature amount calculation unit 111 performs the sound pressure level and the sound power for each frame of the above-described short time (several tens of milliseconds to several seconds) with respect to the acoustic signal sequence included in the input acoustic signal with action label 15-d. MFCC (Mel-Frequency Cepstrum Coefficient) feature quantity, LPC (Linear Predictive Coding) feature quantity, and the like are calculated and output as an acoustic feature quantity sequence. Furthermore, acoustic feature quantities such as rising characteristics, harmonicity, and time periodicity (see, for example, Non-Patent Document 1) may be added to the acoustic feature quantity sequence.

ただし、ｋはフレームをＫ個の微小な時間区間（例えば１ｍｓｅｃ程度）に区分した場合の各時間区間に対応し、ｐ￣_ｋはｋ番目の時間区間でのサンプルの大きさを表す指標の代表値又は平均値を表す。なお、「サンプルの大きさを表す指標」の例は、サンプルの振幅、サンプルの振幅の絶対値、サンプルの振幅の対数値、サンプルのエネルギー、サンプルのパワー、又はサンプルのパワーの対数値などである。「サンプル」は音響信号列の各音響信号を表す。また、Δｐ￣_ｋはｐ￣_ｋの変化率を表す。例えば、Δｐ⁻ _ｋ＝ｐ⁻ _ｋ−ｐ⁻ _ｋ−１である。Δｐ⁻ _ｋ＝ｐ⁻ _ｋ＋１−ｐ⁻ _ｋとしてもよい。また、最小二乗法等の近似手法を用いてｋ番目の時間区間におけるｐ⁻ _ｋを近似した直線を求め、その時間区間におけるその直線の傾きをΔｐ⁻ _ｋとしてもよい。また、ｋ番目の時間区間を含む複数の時間区間におけるｐ￣_ｋ-κ，・・・，ｐ￣_ｋ-1，ｐ⁻ _ｋ，ｐ￣_ｋ+1,...ｐ￣_ｋ-κ’の近時曲線を求め、そのｋ番目の時間区間に対応する点での傾き（微分値）をΔｐ⁻ _ｋとしてもよい。またχを任意の文字として、χの右肩の「−」は、χの上付きバーを意味する。また式（１０）の分子における（ｐ￣_ｎ）^２を（ｐ￣_ｎ）^ｍとし、ｍを任意の値としても良い。 The rising characteristic is an index representing the degree of increase in the index representing the magnitude of the acoustic signal every several tens to several hundreds of milliseconds. Here, the index representing the magnitude of the acoustic signal is, for example, an absolute value of the amplitude of the acoustic signal, a logarithmic value of the absolute value of the amplitude of the acoustic signal, a power of the acoustic signal, or a logarithmic value of the power of the acoustic signal. For example, if the value obtained by the following expression (10) is 0 or more, the value is the rising characteristic, and if the value obtained by the expression (10) is less than 0, 0 is the rising characteristic.

Here, k corresponds to each time interval when the frame is divided into K minute time intervals (for example, about 1 msec), and p￣ _k is a representative index indicating the size of the sample in the kth time interval. Represents a value or average value. Examples of “index indicating sample size” are sample amplitude, absolute value of sample amplitude, logarithm of sample amplitude, sample energy, sample power, logarithm of sample power, etc. is there. “Sample” represents each acoustic signal in the acoustic signal sequence. In addition, Δp¯ _k represents the rate of change of the p¯ _k. For example, Δp _{^- k} = ^{p -} a _{k-1 - k} ^{-p. _{Δp - k = p - k +}} 1 -p - may be as _k. Alternatively, an approximation method such as a least square method may be used to obtain a straight line that approximates p ⁻ _k in the k-th time interval, and the slope of the straight line in that time interval may be Δp ⁻ _k . Also, p の_k-κ ,..., P￣ _k−1 , p ⁻ _k , p￣ _{k + 1} ,... P￣ _{k-κ ′ in} a plurality of time intervals including the k-th time interval. A recent curve may be obtained, and a slope (differential value) at a point corresponding to the k-th time interval may be Δp ⁻ _k . Further, with χ as an arbitrary character, “−” on the right shoulder of χ means a superscript bar of χ. Further, (p￣ _n ) ² in the numerator of formula (10) may be (p￣ _n ) ^m, and m may be an arbitrary value.

また、Ｎはフレームに含まれるサンプル数を表す１以上の整数、ｎはフレーム内の各サンプル点を表す１以上のＮ以下の整数、ｘ（ｎ）はサンプル点ｎでのサンプルの大きさを表す指標である。Ｒ_ｆｆ（τ）はｆ（ｎ）のラグτでの自己相関係数、ｍａｘ｛・｝は「・」の最大値を表す。ラグτは１以上Ｎ以下の整数である。Ｒ_ｆｆ（τ）は、例えば以下のように定義される。

The harmonic characteristics are exemplified below.

N is an integer of 1 or more representing the number of samples included in the frame, n is an integer of 1 or more and N or less representing each sample point in the frame, and x (n) is the size of the sample at the sample point n. It is an index to represent. R _ff (τ) represents the autocorrelation coefficient at the lag τ of f (n), and max {·} represents the maximum value of “·”. The lag τ is an integer from 1 to N. R _ff (τ) is defined as follows, for example.

ただし、Ｌは一周期とみなすサンプル数、Ｍは時間周期性の度合を計算するための周期数を表す１以上の整数、ｐ（・）はサンプルの大きさを表す指標を時間平滑化した値、ｐ￣はフレーム内でのサンプルの大きさを表す指標の平均値を表す。 The time periodicity is exemplified below.

Where L is the number of samples regarded as one period, M is an integer of 1 or more representing the number of periods for calculating the degree of time periodicity, and p (·) is a value obtained by time-smoothing an index representing the sample size. , P￣ represents the average value of the index indicating the size of the sample in the frame.

  The acoustic event model DB 113 stores a plurality of acoustic event models calculated in advance. Each acoustic event model calculates an acoustic feature amount sequence from a learning acoustic signal sequence to which an acoustic event label is attached, and converts the acoustic feature amount sequence corresponding to each acoustic event into a well-known model such as GMM, HMM, or SVM. It is obtained by modeling using a technique (for example, reference: Manabu Okumura, Daiya Takamura, “Introduction to Machine Learning for Language Processing” Corona).

ただし、ｙ_ιは音響特徴量列（ベクトル）の要素、Ｊは正規分布の混合数、π_ｊは混合係数、Ｎ（・）は正規分布の確率密度関数、μ_ｊは分布の平均、Σ_ｊは分布の分散である。 For example, in the case of GMM, an acoustic event model corresponding to each type of acoustic feature is obtained for each acoustic event. For example, the sequence y ₁ ,..., Y _F of acoustic feature amounts y _ι (where ι∈ {1,..., F}) of F types (F is an integer of 1 or more) is included. , The acoustic event model corresponding to each acoustic event is a sequence p (y ₁ ),..., P (y _F ) whose elements are the following probability models p (y _ι ), respectively. .

Where y _ι is an element of the acoustic feature string (vector), J is the number of normal distributions, π _j is the mixing coefficient, N (•) is the probability density function of the normal distribution, μ _j is the average of the distribution, and Σ _j Is the distribution of the distribution.

  Alternatively, an acoustic event model may be obtained by associating an acoustic feature quantity sequence with each acoustic event.

The acoustic feature amount sequence output from the feature amount calculation unit 111 is input to the acoustic event determination unit 112. The acoustic event determination unit 112 compares the input acoustic feature quantity sequence with a plurality of acoustic event models stored in the acoustic event model DB 113, and the acoustic feature quantity of each frame (corresponding to each element number i). Determine the acoustic event corresponding to the column. For example, when the GMM is used as an acoustic event model, the acoustic event determination unit 112, for each frame (element number i), each element ρ _ι (where ι∈ {1,... , F}) is substituted into each probability model of the equation (13) corresponding to each acoustic event, and the sound that maximizes the probability p (ρ ₁ ) ×... × p (ρ _F ) corresponding to each acoustic event. Determine the event. Alternatively, for example, in the case of an acoustic event model in which an acoustic feature string is associated with each acoustic event, the acoustic event determination unit 112 determines the distance (Euclidean) from the input acoustic feature string for each frame (element number i). The acoustic event corresponding to the acoustic event model with the closest distance or cosine distance) is selected.

  The acoustic event determination unit 112 assigns an acoustic event label representing the acoustic event determined for each element number i to the element of each element number i of the action-labeled acoustic signal sequence 15-d. The acoustic event determination unit 112 performs this process on all elements (all element numbers i) of the acoustic signal sequences 15-1,. The acoustic signal trains 11-1,..., 11-D with labels and action labels are output.

  The acoustic signal labels 11-1,..., 11 -D with the acoustic event label and the action label are input to the acoustic signal string synthesis unit 101. The subsequent processing is the same as in the first embodiment.

  In addition, it replaces with performing the synthetic | combination process of the acoustic signal sequence 11-1, ..., 11-D with an acoustic event label and the action label in the acoustic signal sequence synthetic | combination part 101, and an action label in the front | former stage of the feature-value calculation part 111 is carried out. The attached acoustic signal sequences 15-1,..., 15-D may be combined, or the acoustic feature amount sequence may be combined before the acoustic event determination unit 112.

<Modification 2 of the first embodiment>
In the second modification of the first embodiment, the behavior-situation generation model 12 and the situation-acoustic event generation model are learned by learning using an acoustic feature sequence with action labels (corresponding to “acoustic feature sequence with behavior information”) as an input. 13. The behavior-acoustic event generation model 14 is calculated. Similarly to the first embodiment, only the probability that an action generates a situation and the probability that the situation generates an acoustic event are calculated, and only the action-situation generation model 12 and the situation-acoustic event generation model 13 corresponding to each of them are calculated. The probability that an action generates an acoustic event is calculated using the probability that the action generates a situation and the probability that the situation generates an acoustic event, and only the corresponding action-acoustic event generation model 14 is calculated. May be calculated.

  As illustrated in FIG. 4, the generation model creation device 120 of this embodiment includes an acoustic event determination unit 112, an acoustic event model database (DB) 113, an acoustic signal sequence synthesis unit 101, a behavior / situation modeling unit 102, and a storage. Part 103. The generation model creation device 120 is configured by, for example, reading a predetermined program into a known or dedicated computer.

  First, acoustic feature amount sequences 16-1,..., 16-D with action labels are input to the acoustic event determination unit 112. Each action-labeled acoustic feature sequence 16-d (where dε {1,..., D}) is a time-series acoustic signal sequence, short, divided every short time (several tens of milliseconds to several seconds). Element number corresponding to each element of the acoustic signal sequence divided by time, action label (corresponding to “behavior information”) determined every long time (several seconds or more) and given every short time, and sound An acoustic feature quantity sequence for each short time of the signal sequence is included. A specific example of the acoustic feature amount sequence is as described in the first embodiment.

  The acoustic event determination unit 112 uses the input acoustic feature sequence 16-d acoustic feature sequence 16-d and a plurality of acoustic event models stored in the acoustic event model DB 113 as a modification of the first embodiment. As described in 1, each is compared, and an acoustic event corresponding to the acoustic feature amount sequence of each frame (corresponding to each element number i) is determined. The acoustic event determination unit 112 assigns an acoustic event label representing the acoustic event determined for each element number i to the element of each element number i in the acoustic feature quantity column 16-d with action label. The acoustic event determination unit 112 performs this process for all the elements (all element numbers i) of the acoustic feature quantity columns 16-1,..., 16-D with action labels, Acoustic signal sequences 11-1,..., 11-D with action labels are output.

  The acoustic signal labels 11-1,..., 11 -D with the acoustic event label and the action label are input to the acoustic signal string synthesis unit 101. The subsequent processing is the same as in the first embodiment.

  In addition, instead of performing the synthesis process of the acoustic signal sequence 11-1,..., 11-D with the acoustic event label and the action label in the acoustic signal sequence synthesizing unit 101, an acoustic feature is provided in the preceding stage of the acoustic event determining unit 112. You may perform the synthetic | combination process of a quantity sequence.

Second Embodiment
In the second embodiment, an acoustic event newly input using the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model 14 obtained as described in the first embodiment is used. The behavior and situation are estimated from the labeled acoustic signal sequence.

  As illustrated in FIG. 5, the estimation apparatus 200 according to the present embodiment includes a storage unit 103 and a generated model comparison unit 201. The estimation device 200 is configured, for example, by reading a predetermined program into a known or dedicated computer.

  First, an acoustic signal string 21 with an acoustic event label (corresponding to “input information including acoustic event information representing an acoustic event”) is input to the generation model comparison unit 201. The acoustic signal label with acoustic event label 21 is a time-series acoustic signal sequence divided every short time (several tens of milliseconds to several seconds), an element number corresponding to each element of the acoustic signal sequence divided every short time, And an acoustic event label determined and given every short time (corresponding to “acoustic event information”). The element number and the acoustic event label are given for each element of the acoustic signal string.

  The generation model comparison unit 201 receives the input acoustic signal label-attached acoustic signal sequence 21, the action-situation generation model 12, the situation-acoustic event generation model 13, and the action-acoustic event generation model 14 stored in the storage unit 103. And the action and situation determined to be the most appropriate for the acoustic signal string 21 with the acoustic event label, or a plurality of actions and situations are determined in order from the most appropriate, and these are used as the determination results. Output.

ただし、γは事前に設定された緩和パラメータ（例えば０．０１などの非負値）を表し、Ｃ_εは、音響イベントラベル付き音響信号列２１で音響イベントεを表す音響イベントラベルが付された要素の個数を表し、Ｎ_ｓは音響イベントラベル付き音響信号列２１が含む音響信号列の要素数を表す。Ｎ_ｓ＝Ｎ_ｄであってもよいし、Ｎ_ｓ≠Ｎ_ｄであってもよい。 [Comparative Method Example 1]
The comparison method with the acoustic signal sequence 21 with an acoustic event label and each production | generation model is illustrated. In this example, the generation model comparison unit 201 first calculates p (ε) (where εε {1,..., E}) from the input acoustic signal sequence with acoustic event label 21 as follows. To do.

However, (gamma) represents the relaxation parameter set beforehand (for example, nonnegative values, such as 0.01), and C ( _epsilon ) is the element to which the acoustic event label showing the acoustic event (epsilon) was attached | subjected in the acoustic signal sequence 21 with an acoustic event label. N _s represents the number of elements of the acoustic signal sequence included in the acoustic signal sequence with acoustic event label 21. N _s = N _d may be satisfied, or N _s ≠ N _d may be satisfied.

Next, the generation model comparison unit 201 uses p (ε) and the action-acoustic event generation model 14 (or the action-situation generation model 12 and the situation-acoustic event generation model 13 in the vicinity using the method described in the first embodiment. The behavior-acoustic event generation model 14) obtained by converting the behavior into the following information such as the Kullback-Leibler divergence (KL divergence) and Jensen-Shannon divergence (JS divergence) By performing the comparison based on the information amount criterion, the behavior corresponding to the input acoustic signal label-attached acoustic signal sequence 21 is estimated.

In the case of the example of Expression (15) or (16), the generation model comparison unit 201 substitutes p (ε) (where εε {1,..., E}) for P (ε), and Q ( P ( _λεa | Θ, Φ, Ω) (where ε∈ {1,..., E), a∈ {1,..., A}) in Expression (6) is substituted into ε). As a result, the generation model comparison unit 201 obtains an information amount (a total of A information amount) corresponding to each action aε {1,..., A}.

  The generation model comparison unit 201 selects the behavior corresponding to the smallest information amount among the information amounts calculated for each behavior aε {1,..., A}, or a plurality selected in order from the smallest information amount. A plurality of actions corresponding to each information amount are determined and output as actions corresponding to the acoustic signal label-attached acoustic signal sequence 21.

Further, the generation model comparison unit 201 compares p (ε) of the equation (14) with the situation-acoustic event generation model 13 by using the above method, so that the input acoustic signal label-attached acoustic signal sequence 21 is obtained. The situation can also be estimated. For example, when using equation (15) or (16), the generation model comparison unit 201 substitutes p (ε) (where εε {1,..., E}) for P (ε), and Q (epsilon) to phi _.epsilon.t of formula (5) (wherein, ε∈ {1, ···, E ), t∈ {1, ···, substituting T}). Thereby, the generation model comparison unit 201 obtains the information amount (total T information amount) corresponding to each situation tε {1,..., T}. The generation model comparison unit 201 selects the situation corresponding to the smallest information amount among the information amounts calculated for each situation tε {1,... A plurality of situations corresponding to each information amount is determined and output as a situation corresponding to the acoustic signal label-attached acoustic signal sequence 21.

[Example 2 of comparison method]
You may compare with each production | generation model and the acoustic signal sequence 21 with an acoustic event label as follows. In this method, the generation model comparison unit 201 performs the sum or product of the likelihood of action under the action-acoustic event generation model 14 or the situation-acoustic event generation model for the input labeled acoustic signal sequence 21. The sum or product of the likelihood of the situation under 13 is obtained. Specific examples are shown below.

≪Example of sum of likelihood of action under action-acoustic event generation model 14≫

≪Example of action likelihood product under action-acoustic event generation model 14≫

<< Situation-Example of sum of likelihood of situation under acoustic event generation model 13 >>

<< Situation-Example of the product of the likelihood of a situation under the acoustic event generation model 13 >>

However, e _i in the equations (17) to (20) represents an acoustic event represented by the acoustic event label corresponding to the element number i of the inputted acoustic signal label-attached acoustic signal sequence 21. Equations (17) to (20) are the probabilities θ _ta , φ _εt , and P (λ _εa | Θ, Φ, Ω) of equations (4), (5), and (6), and the input acoustic signal with an acoustic event label. It can be calculated from e _{i in} column 21.

  The generation model comparison unit 201 inputs a plurality of actions and situations selected in order from the highest likelihood action or situation or the highest likelihood among the likelihoods calculated for each action and situation. The action / situation corresponding to the acoustic signal label 21 with the acoustic event label is determined and output.

  Note that the processing for determining the above-described action and the processing for determining the situation may be executed by different processing units.

<Modification Example 1 of Second Embodiment>
In the first modification of the second embodiment, the action-situation generation model 12, the situation-acoustic event generation model 13, and the action-acoustic event generation model 14 obtained as described in the first embodiment are used to newly input. The behavior and situation are estimated from the obtained acoustic signal sequence.

  As illustrated in FIG. 6, the estimation apparatus 210 according to the present embodiment includes a feature amount calculation unit 211, an acoustic event determination unit 212, an acoustic event model DB 113, a storage unit 103, and a generation model comparison unit 201. The estimation device 210 is configured, for example, by reading a predetermined program into a known or dedicated computer.

  First, the unlabeled acoustic signal sequence 22 is input to the feature amount calculation unit 211. The unlabeled acoustic signal sequence 22 includes a time-series acoustic signal sequence divided every short time (several tens of milliseconds to several seconds) and an element number corresponding to each element of the acoustic signal sequence divided every short time. .

  The feature amount calculation unit 211 calculates and outputs an acoustic feature amount sequence (vector) from the unlabeled acoustic signal sequence 22. For example, the feature amount calculation unit 211 calculates an acoustic feature amount sequence by the same method as the feature amount calculation unit 111 described in the first embodiment.

  The acoustic event determination unit 212 is the same method as the acoustic event determination unit 112 of the first embodiment, and a plurality of acoustic events stored in the acoustic event model DB 113 and the acoustic feature amount sequence output from the feature amount calculation unit 211. Each model is compared, and acoustic events are determined for all elements of the unlabeled acoustic signal sequence 22. The acoustic event determination unit 212 generates and outputs an acoustic event label-attached acoustic signal sequence 21 by attaching an acoustic event label representing the determined acoustic event to each element of the unlabeled acoustic signal sequence 22.

  The acoustic signal sequence with acoustic event label 21 is input to the generation model comparison unit 201. The subsequent processing is the same as in the second embodiment.

<Modification 2 of the second embodiment>
In Modification 2 of the second embodiment, a new input is made using the action-situation generation model 12, the situation-acoustic event generation model 13, and the action-acoustic event generation model 14 obtained as described in the first embodiment. The behavior and situation are estimated from the obtained acoustic feature quantity sequence.

  As illustrated in FIG. 7, the estimation apparatus 220 according to the present embodiment includes an acoustic event determination unit 212, an acoustic event model DB 113, a storage unit 103, and a generation model comparison unit 201. The estimation device 220 is configured, for example, by reading a predetermined program into a known or dedicated computer.

  First, the unlabeled acoustic feature string 23 is input to the acoustic event determination unit 212. The unlabeled acoustic feature column 23 includes time-series acoustic signal sequences divided every short time (several tens of milliseconds to several seconds), element numbers corresponding to the elements of the acoustic signal sequence divided every short time, and The acoustic feature amount sequence of the acoustic signal sequence is included. A specific example of the acoustic feature amount sequence is as described in the first embodiment.

  The acoustic event determination unit 212 is the same method as the acoustic event determination unit 112 of the first embodiment, and a plurality of acoustic event models stored in the acoustic feature amount sequence 23 of the unlabeled acoustic feature amount sequence 23 and the acoustic event model DB 113. Are respectively determined, and an acoustic event is determined for all elements of the acoustic signal sequence included in the unlabeled acoustic feature amount sequence 23. The acoustic event determination unit 212 generates the acoustic signal sequence 21 with the acoustic event label by attaching the acoustic event label representing the acoustic event of each determined element to the acoustic signal sequence included in the unlabeled acoustic feature amount sequence 23. Output.

  The acoustic signal sequence with acoustic event label 21 is input to the generation model comparison unit 201. The subsequent processing is the same as in the second embodiment.

<Third Embodiment>
This embodiment is a combination of the first embodiment and the second embodiment.
In this embodiment, in addition to estimating the behavior and situation using the acoustic event label-attached acoustic signal sequence 21 as an input, the acoustic event label and the behavior-labeled acoustic signal sequence 11-1,. The behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model 14 are also calculated. Furthermore, by assigning a behavior label representing the estimated behavior to each element of the acoustic signal sequence included in the acoustic signal sequence with acoustic event label 21, an acoustic event label and an acoustic signal sequence with behavior label are obtained and generated. It can also be used to update the model.

  As illustrated in FIG. 8, the estimation apparatus 300 according to the present embodiment includes storage units 103 and 303, an acoustic signal sequence synthesis unit 101, a behavior / situation modeling unit 102, and a generation model comparison unit 201. The estimation device 300 is configured, for example, by reading a predetermined program into a known or dedicated computer.

  The storage unit 303 includes the acoustic event labels and action-labeled acoustic signal sequences 11-1, ..., 11-D described in the first embodiment, and the acoustic event-labeled acoustic signal sequence described in the second embodiment. 21 is stored.

  The acoustic signal labels 11-1,..., 11 -D with the acoustic event label and the action label are input to the acoustic signal string synthesis unit 101. As described in the first embodiment, the acoustic signal sequence synthesizing unit 101 and the behavior / situation modeling unit 102 perform the behavior from the acoustic signal sequence 11-1, ..., 11-D with acoustic event labels and behavior labels. A situation generation model 12, a situation-acoustic event generation model 13, and an action-acoustic event generation model 14 are generated and stored in the storage unit 103.

  The acoustic signal sequence with acoustic event label 21 is input to the generation model comparison unit 201. As described in the second embodiment, the acoustic signal label-attached acoustic signal sequence 21 includes the input acoustic event-labeled acoustic signal sequence 21, the action-situation generation model 12 stored in the storage unit 103, and the situation-acoustics. The event generation model 13 and the behavior-acoustic event generation model 14 are compared, and the acoustic event sequence with the acoustic event label 21 is determined to be the most appropriate action or situation, or a plurality of items in order from the most appropriate one. Are selected and output as judgment results.

  Furthermore, the generation model comparison unit 201 assigns a behavior label representing the determined behavior to each corresponding element of the acoustic signal label-attached acoustic signal sequence 21, thereby creating a new acoustic event label and acoustic label with the behavior label 21 ′. May be generated and output. In this case, even if the acoustic signal label and action label-attached acoustic signal sequence 21 ′ is input to the acoustic signal sequence combining unit 101 together with the acoustic event labels and behavior label-attached acoustic signal sequences 11-1,. Good. In this case, the acoustic signal sequence synthesizing unit 101 connects all input acoustic event labels and acoustic signal sequences with action labels 11-1,..., 11-D, 21 ′ in a time-series direction, thereby One acoustic event label and action label-attached acoustic signal sequence 11 ′ is obtained and output. As described in the first embodiment, the behavior / situation modeling unit 102 again performs the behavior-situation generation model 12 and the situation-acoustic event generation from the input acoustic event label and the acoustic signal sequence with action label 11 ′. The model 13 and the behavior-acoustic event generation model 14 may be generated and stored in the storage unit 103.

  Further, either the processing of the generation model comparison unit 201 or the processing of the behavior / situation modeling unit 102 may be performed first. However, when the processing of the model comparison unit 201 is performed before the processing of the behavior / situation modeling unit 102 is performed, it is necessary that each generated model obtained in advance is stored in the storage unit 103.

Further, the acoustic signal label 21 and the acoustic signal sequence with action label 21 ′ may be input to the acoustic signal sequence synthesis unit 101 together with the newly input acoustic event label and acoustic signal sequence with the action label. The acoustic signal sequence synthesizing unit 101 may connect these in the time series direction and send them to the behavior / situation modeling unit 102.
Other processes are the same as those in the first embodiment and the second embodiment.

<Modification 1 of 3rd Embodiment>
This embodiment is a combination of the first modification of the first embodiment and the first modification of the second embodiment.
In this embodiment, the action-labeled acoustic signal sequences 15-1,..., 15-D are used as inputs, and the action-situation generation model 12, the situation-acoustic event generation model 13, and the action-acoustic event generation model 14 are learned by learning. Is calculated. Furthermore, in this embodiment, the behavior and situation are estimated from the newly input unlabeled acoustic signal sequence 22 using the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model 14.

  As illustrated in FIG. 9, the estimation apparatus 310 according to the present embodiment includes a feature amount calculation unit 111-1,..., 111-D, 211, and an acoustic event determination unit 112-1,. 212, acoustic event model DB113, and the estimation apparatus 300 (refer FIG. 8) of 3rd Embodiment.

  The acoustic signal sequences 15-1,..., 15-D with action labels are input to the feature amount calculation units 111-1,. As described in the first modification of the first embodiment, the feature amount calculation units 111-1,..., 111-D are based on acoustic signal sequences 15-1,. Obtain and output a sequence of acoustic features. The acoustic event determination units 112-1,..., 112-D are respectively input to the input acoustic feature quantity sequence and the acoustic event model DB 113 in the same manner as the acoustic event determination unit 112 of the first modification of the first embodiment. From the plurality of stored acoustic event models, acoustic event labels and acoustic signal sequences with action labels 11-1,..., 11-D (corresponding to “learning information”) are generated and output.

  The unlabeled acoustic signal sequence 22 is input to the feature amount calculation unit 211. As described in the first modification of the second embodiment, the feature amount calculation unit 211 calculates and outputs an acoustic feature amount sequence (vector) from the unlabeled acoustic signal sequence 22. As described in the first modification of the second embodiment, the acoustic event determination unit 212 includes an acoustic event label from the input acoustic feature quantity sequence and the plurality of acoustic event models stored in the acoustic event model DB 113. An acoustic signal sequence 21 (corresponding to “input information”) is generated and output.

  The acoustic signal labels 11-1,..., 11-D with acoustic event labels and action labels, and the acoustic signal string 21 with acoustic event labels are stored in the storage unit 303 (FIG. 8). The subsequent processing is the same as in the third embodiment.

<Modification 2 of 3rd Embodiment>
This embodiment is a combination of the second modification of the first embodiment and the second modification of the second embodiment.
In this embodiment, the behavior-labeled acoustic feature sequence 16-1,..., 16-D is used as an input, and the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model are learned by learning. 14 is calculated. Furthermore, in this embodiment, a behavior and a situation are estimated from a newly input unlabeled acoustic feature quantity sequence 23 using a behavior-situation generation model 12, a situation-acoustic event generation model 13, and a behavior-acoustic event generation model 14.

  As illustrated in FIG. 10, the estimation apparatus 320 according to the present embodiment includes acoustic event determination units 112-1,..., 112-D, 212, an acoustic event model DB 113, and an estimation apparatus 300 according to the third embodiment (see FIG. 10). 8).

  The acoustic feature quantity columns 16-1,..., 16-D with action labels are input to the acoustic event determination units 112-1,. The acoustic event determination units 112-1,..., 112-D are respectively input to the input acoustic feature quantity sequence and the acoustic event model DB 113 in the same manner as the acoustic event determination unit 112 of the second modification of the first embodiment. From the plurality of stored acoustic event models, acoustic event labels and acoustic signal sequences with action labels 11-1,..., 11-D (corresponding to “learning information”) are generated and output.

  The unlabeled acoustic feature string 23 is input to the acoustic event determination unit 212. Similarly to the second modification of the second embodiment, the acoustic event determination unit 212 includes the input acoustic feature amount sequence of the unlabeled acoustic feature amount sequence 23 and a plurality of acoustic event models stored in the acoustic event model DB 113. Then, an acoustic event label-attached acoustic signal sequence 21 (corresponding to “input information”) is generated and output.

  The acoustic signal labels 11-1,..., 11-D with acoustic event labels and action labels, and the acoustic signal string 21 with acoustic event labels are stored in the storage unit 303 (FIG. 8). The subsequent processing is the same as in the third embodiment.

<Fourth embodiment>
This embodiment is a modification of the third embodiment.
In this embodiment, acoustic event labels and action-labeled acoustic signal sequences 11-1,..., 11-D (corresponding to “first learning information”) and acoustic event-labeled acoustic signal sequence 21 (“second” In addition to calculating the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model 14, it corresponds to the acoustic signal sequence 21 with an acoustic event label. Estimate the behavior and situation to be performed. Furthermore, by assigning a behavior label representing the estimated behavior to each element of the acoustic signal sequence included in the acoustic signal sequence with acoustic event label 21, an acoustic event label and an acoustic signal sequence with behavior label are obtained and generated. It can also be used to update the model.

  As illustrated in FIG. 11, the estimation apparatus 400 according to the present embodiment includes storage units 103 and 303, an acoustic signal sequence synthesis unit 401, a behavior / situation modeling unit 402, and a generation model comparison unit 403. The estimation device 400 is configured, for example, by reading a predetermined program into a known or dedicated computer.

  The acoustic signal sequence synthesizing unit 401 includes acoustic signal labels and behavior-labeled acoustic signal sequences 11-1,..., 11-D (“acoustic events representing behavior information indicating behavior details and acoustic events associated with the behavior”). And an acoustic signal string 21 with an acoustic event label (corresponding to “second learning information including acoustic event information representing an acoustic event associated with an unknown action”). Entered. The acoustic signal sequence synthesizer 401 connects the acoustic event labels and acoustic signal sequences with action labels 11-1,..., 11-D, and the acoustic signal sequence with acoustic event labels 21 in the time series direction, thereby An acoustic event label and an action-labeled acoustic signal sequence 41 (hereinafter simply referred to as “labeled acoustic signal sequence 41”) are obtained and output. The labeled acoustic signal sequence 41 is input to the behavior / situation modeling unit 402. In addition, when the acoustic signal sequence 41 with the sound event label and the acoustic signal sequence 11 with the action label is obtained in advance from the acoustic signal sequence 21 with the acoustic event label, the labeled acoustic signal sequence 41 is obtained. The labeled acoustic signal sequence 41 may be directly input to the behavior / situation modeling unit 402 without going through the acoustic signal sequence synthesis unit 401.

  The behaviour / situation modeling unit 402 uses the input labeled acoustic signal sequence 41 in the same manner as the behaviour / situation modeling unit 102 of the first embodiment, and the behavior-situation generation model 12 and the situation-acoustic event generation. A model 13 and a behavior-acoustic event generation model 14 are calculated. However, the behavior / situation modeling unit 402 is given a special behavior label indicating that “behavior is unknown” to each element of the acoustic signal sequence with acoustic event label 21 included in the acoustic signal sequence with label 41. Calculation processing is performed assuming that That is, the behavior / situation modeling unit 402 regards that “behavior is unknown” as one independent “behavior” (referred to as “unknown behavior”). This special action label does not represent any general action (cooking, cleaning, etc.).

  That is, the behavior / situation modeling unit 402 has a probability P (acoustic event | situation) that a situation generates an acoustic event, a probability P (situation | action) that an action creates a situation, and a probability that an unknown action creates a situation. Get P (Situation | Unknown Behavior). Further, the behavior / situation modeling unit 402 obtains a probability P (acoustic event | action) that the behavior generates an acoustic event from the probability P (acoustic event | situation) and the probability P (situation | action), and the probability P ( The probability P (acoustic event | unknown action) that the unknown action generates an acoustic event is obtained from the acoustic event | situation and the probability P (situation | unknown action). The action-situation generation model 12 corresponds to the probabilities P (situation | action) and P (situation | unknown action), and the situation-acoustic event generation model 13 corresponds to the probability P (acoustic event | situation), and the action The acoustic event generation model 14 corresponds to the probability P (acoustic event | action), P (acoustic event | unknown behavior). The behavior / situation generation model 12, the situation / acoustic event generation model 13, and the behavior / acoustic event generation model 14 calculated by the behavior / situation modeling unit 402 are stored in the storage unit 103.

  The generation model comparison unit 403 receives the behavior-acoustic event generation model 14. The generation model comparison unit 403 compares the probability P (acoustic event | action) and the probability P (acoustic event | unknown action) corresponding to the input action-acoustic event generation model 14 to determine an unknown action (acoustic sound). The content of the action corresponding to the event-labeled acoustic signal sequence 21 is estimated, and the estimation results are output. For example, the generation model comparison unit 403 selects a plurality of actions in order from the action that is determined to be the most appropriate or the most appropriate action for the acoustic signal sequence 21 with the sound event label, and uses these as determination results. Output.

[Example of comparison method]
The comparison by the generation model comparison unit 403 can be performed based on the information amount criterion described in the second embodiment, for example. For example, when using the information criterion of the equation (15) or (16), the generation model comparison unit 403 makes P (ε) unknown to P (λ _εa | Θ, Φ, Ω) of the equation (6). Substitute P ( _{λεa ′} | Θ, Φ, Ω) (where εε {1,..., E)} corresponding to the action a′ε {1,..., A} and Q (ε ) To P (λ _{εa ″} | Θ, Φ, Ω) corresponding to another action a ″ ≠ a ′ (where a ″ ε {1,..., A}) (where εε { 1, ..., E)) is substituted. Thereby, the generation model comparison unit 403 has an information amount corresponding to another action a ″ ≠ a ′ (where a ″ ε {1,..., A}) (a total of A−1 information amounts). )

  The generation model comparison unit 403 includes an action corresponding to the smallest information amount among the information amounts calculated for each action a ″, or a plurality of information corresponding to a plurality of information amounts selected in order from the smallest information amount. Each action is determined and output as an action corresponding to the acoustic signal sequence 21 with the acoustic event label.

  The generation model comparison unit 403 assigns an action label representing the action determined as described above to each corresponding element of the acoustic signal label-attached acoustic signal sequence 21, thereby creating a new acoustic event label and an action-labeled acoustic signal. The column 41 ′ may be generated and output. In this case, even if the acoustic signal sequence 41 ′ with the acoustic event label and the behavior label is input to the acoustic signal sequence synthesizing unit 401 together with the acoustic signal sequences 11-1,. Good. In this case, the acoustic signal sequence synthesizing unit 401 connects all input acoustic event labels and acoustic signal sequences with action labels 11-1,..., 11-D, 41 ′ in the time series direction, thereby One acoustic event label and acoustic signal sequence 41 ″ with action label is obtained and output. The behavior / situation modeling unit 402 again performs the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation from the input acoustic event label and the acoustic signal sequence 41 ″ with the behavior label. The model 14 may be generated and stored in the storage unit 103.

  Moreover, the acoustic signal label and action label-attached acoustic signal string 21 ′ may be input to the acoustic signal string synthesizing unit 401 together with the newly input acoustic event label and action label-attached acoustic signal string. The acoustic signal sequence synthesizing unit 401 may connect these in the time-series direction and send them to the behavior / situation modeling unit 402.

Further, the generation model comparison unit 403 uses the input acoustic signal label-attached acoustic signal sequence 21 and the situation-acoustic event generation model 13 stored in the storage unit 103 to perform the acoustic event by the method described in the second embodiment. Each situation corresponding to the acoustic event represented by the acoustic event label included in the labeled acoustic signal sequence 21 (corresponding to “acoustic event information representing an acoustic event associated with an unknown behavior”) may be estimated and output.
Other processes are the same as those in the first embodiment, the second embodiment, and the third embodiment.

<Modification 1 of 4th Embodiment>
This embodiment is a modification of the first modification of the third embodiment.
In the present embodiment, the behavior-situation generation model 12, the situation-acoustic event generation model 13, and learning are performed with the action-labeled acoustic signal sequence 15-1,. The behavior-acoustic event generation model 14 is calculated. Furthermore, in this embodiment, the behavior and situation corresponding to the unlabeled acoustic signal sequence 22 are estimated using the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model 14.

  As illustrated in FIG. 9, the estimation apparatus 410 according to the present embodiment includes a feature amount calculation unit 111-1,..., 111-D, 211, and an acoustic event determination unit 112-1,. 212, acoustic event model DB113, and the estimation apparatus 400 (refer FIG. 11) of 4th Embodiment.

  The acoustic signal sequences 15-1,..., 15-D with action labels are input to the feature amount calculation units 111-1,. As described in the first modification of the first embodiment, the feature amount calculation units 111-1,..., 111-D are based on acoustic signal sequences 15-1,. Obtain and output a sequence of acoustic features. The acoustic event determination units 112-1,..., 112-D are respectively input to the input acoustic feature quantity sequence and the acoustic event model DB 113 in the same manner as the acoustic event determination unit 112 of the first modification of the first embodiment. Generate and output acoustic signal strings 11-1,..., 11-D (corresponding to “first learning information”) with acoustic event labels and action labels from a plurality of stored acoustic event models. .

  The unlabeled acoustic signal sequence 22 is input to the feature amount calculation unit 211. As described in the first modification of the second embodiment, the feature amount calculation unit 211 and the acoustic event determination unit 212 calculate and output an acoustic feature amount sequence (vector) from the unlabeled acoustic signal sequence 22. As described in the first modification of the second embodiment, the acoustic event determination unit 212 includes an acoustic event label from the input acoustic feature quantity sequence and the plurality of acoustic event models stored in the acoustic event model DB 113. An acoustic signal sequence 21 (corresponding to “second learning information”) is generated and output.

  The acoustic signal labels 11-1,..., 11-D with acoustic event labels and behavior labels, and the acoustic signal string 21 with acoustic event labels are stored in the storage unit 303 (FIG. 11). The subsequent processing is the same as in the fourth embodiment.

<Modification 2 of 4th Embodiment>
This embodiment is a modification of the second modification of the third embodiment.
In the present embodiment, the behavior-situation generation model 12 and the situation-acoustic event generation model are obtained by learning with the action-labeled acoustic feature quantity sequence 16-1,..., 16-D and the unlabeled acoustic feature quantity sequence 23 as inputs. 13. The behavior-acoustic event generation model 14 is calculated. Furthermore, in this embodiment, the behavior and situation corresponding to the unlabeled acoustic feature quantity sequence 23 are estimated using the behavior-situation generation model 12, the situation-acoustic event generation model 13, and the behavior-acoustic event generation model 14.

  As illustrated in FIG. 10, the estimation apparatus 420 of the present embodiment includes acoustic event determination units 112-1,..., 112 -D, 212, an acoustic event model DB 113, and an estimation apparatus 400 (FIG. 11).

  The acoustic feature quantity columns 16-1,..., 16-D with action labels are input to the acoustic event determination units 112-1,. The acoustic event determination units 112-1,..., 112-D are respectively input to the input acoustic feature quantity sequence and the acoustic event model DB 113 in the same manner as the acoustic event determination unit 112 of the second modification of the first embodiment. Generate and output acoustic signal strings 11-1,..., 11-D (corresponding to “first learning information”) with acoustic event labels and action labels from a plurality of stored acoustic event models. .

  The unlabeled acoustic feature string 23 is input to the acoustic event determination unit 212. Similarly to the second modification of the second embodiment, the acoustic event determination unit 212 includes the input acoustic feature amount sequence of the unlabeled acoustic feature amount sequence 23 and a plurality of acoustic event models stored in the acoustic event model DB 113. Then, an acoustic signal sequence 21 with acoustic event labels (corresponding to “second learning information”) is generated and output.

  The acoustic signal labels 11-1,..., 11-D with acoustic event labels and behavior labels, and the acoustic signal string 21 with acoustic event labels are stored in the storage unit 303 (FIG. 11). The subsequent processing is the same as in the third embodiment.

<Modifications>
The present invention is not limited to the above-described embodiments. For example, the processing of the generation model creation device or the estimation device may be distributed by a plurality of devices, or each data stored in the storage unit or DB in each of the above embodiments is distributed to a plurality of storage units or DBs. May be stored. For example, the action-situation generation model 12, the situation-acoustic event generation model 13, and the action-acoustic event generation model 14 may be stored in different storage units.

  An acoustic event label and an action label sequence may be used instead of the acoustic event label and the acoustic signal sequence with an action label. Further, an acoustic event label string may be used instead of the acoustic signal string with the acoustic event label. For example, only the acoustic event label and the action label sequence may be input to the behavior / situation modeling unit 102, or only the acoustic event label sequence may be input to the generation model comparison unit 201, or the acoustic event label and the behavior label Only the sequence and the acoustic event label sequence may be input to the behavior / situation modeling unit 402.

  An example of the acoustic event label and the action label sequence is a sequence obtained by removing the acoustic signal sequence from the acoustic event label and the acoustic signal sequence with the action label. Moreover, the example of an acoustic event label row | line | column is a row | line | column remove | excluding the acoustic signal row | line | column from the acoustic signal row | line | column with an acoustic event label. However, if the label sequence is input in the time series and sequentially processed, examples of the acoustic event label, the behavior label sequence, and the acoustic event label sequence may not include the element number.

  Also, if the acoustic signal sequence is input and processed sequentially in time series, the element number corresponding to each element of the acoustic signal sequence divided every short time is the acoustic signal sequence with the acoustic event label and the acoustic signal sequence. The event label and the action label-attached acoustic signal sequence may not be included.

  The various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  When the above configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. Data obtained as a result of processing in each unit is stored in the memory one by one, and is read and used as necessary.

  The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

  This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, this computer reads a program stored in its own recording device and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer. The processing according to the received program may be executed sequentially.

  In the above embodiment, the processing functions of the apparatus are realized by executing a predetermined program on a computer. However, at least a part of these processing functions may be realized by hardware.

100, 110, 120 Generation model creation device 200, 210, 220, 300, 310, 320, 400, 410, 420 estimation device

Claims (12)

A situation of a potential field that indirectly associates the behavior with the acoustic event using learning information including behavior information representing the content of the behavior and acoustic event information representing an acoustic event accompanying the behavior. and | (status acoustic event), the probability behavior generates the status P probability P that generates the acoustic event | and the obtained Ru means (status action),
Means for obtaining an action-acoustic event generation model corresponding to the probability P (acoustic event | action) that the action generates the acoustic event from the probability P (acoustic event | situation) and the probability P (situation | action) ; A generation model creation device having a behavior / situation modeling unit including . A generation model creation device according to claim 1,
A feature amount calculation unit that calculates an acoustic feature amount from a time-series acoustic signal sequence representing a sound accompanying the behavior represented by the behavior information;
An acoustic event determination unit that determines an acoustic event using the acoustic feature amount; and
The acoustic event information included in the learning information represents the acoustic event determined by the acoustic event determination unit.
A generation model creation device characterized by that. A generation model creation device according to claim 1,
Using an acoustic feature amount corresponding to an acoustic signal sequence representing a sound accompanying the behavior represented by the behavior information, and having an acoustic event determination unit that determines an acoustic event;
The acoustic event information included in the learning information represents an acoustic event determined by the acoustic event determination unit.
A generation model creation device characterized by that. And | (Status acoustic event), the probability P that the action to generate the status probability P status of potential fields that indirectly associate the behavior and the acoustic event generates the acoustic event | from the (status behavior) a storage unit for storing the acoustic events generated model, - action corresponding to | (action acoustic events) obtained, the action probability P that generates the acoustic event
Using the behavior-acoustic event generation model, the behavior corresponding to the input information including the acoustic event information representing the acoustic event is estimated, or the situation-acoustic event generation model corresponding to the probability P (acoustic event | situation) A generation model comparison unit for estimating a situation corresponding to the input information ,
An estimation device. An estimation apparatus according to claim 4, wherein
A feature amount calculation unit for calculating an acoustic feature amount from a time-series acoustic signal sequence;
An acoustic event determination unit that determines an acoustic event using the acoustic feature amount; and
The acoustic event information included in the input information represents the acoustic event determined by the acoustic event determination unit.
An estimation apparatus characterized by that. An estimation apparatus according to claim 4, wherein
An acoustic event determination unit that determines an acoustic event using an acoustic feature amount,
The acoustic event information included in the input information represents the acoustic event determined by the acoustic event determination unit.
An estimation apparatus characterized by that. A situation of a potential field that indirectly associates the behavior with the acoustic event using learning information including behavior information representing the content of the behavior and acoustic event information representing an acoustic event accompanying the behavior. and | (status acoustic event), the action probability P that generates the status probability P that generates the acoustic event | obtain the (status behavior), a behavior / situation modeling unit,
Using an action-acoustic event generation model corresponding to the probability P (acoustic event | action) that the action generates an acoustic event, obtained from the probability P (acoustic event | situation) and the probability P (situation | action), A situation corresponding to the input information is estimated using a situation-acoustic event generation model corresponding to the probability P (acoustic event | situation), or an action corresponding to the input information including the acoustic event information representing the acoustic event is estimated. A generation model comparison unit to be estimated;
An estimation device. The estimation apparatus according to claim 7, comprising:
A first feature amount calculation unit that calculates a first acoustic feature amount from a first time-series acoustic signal sequence that represents a sound accompanying the behavior represented by the behavior information;
A first acoustic event determination unit that determines an acoustic event using the first acoustic feature amount;
A second feature amount calculation unit for calculating a second acoustic feature amount from a second time-series acoustic signal sequence;
A second acoustic event determination unit that determines an acoustic event using the second acoustic feature amount;
The acoustic event information included in the learning information represents the acoustic event determined by the first acoustic event determination unit,
The acoustic event information included in the input information represents the acoustic event determined by the second acoustic event determination unit.
An estimation apparatus characterized by that. The estimation apparatus according to claim 7, comprising:
A first acoustic event determination unit that determines an acoustic event using a first acoustic feature amount corresponding to an acoustic signal sequence that represents a sound accompanying the behavior represented by the behavior information;
A second acoustic event determination unit that determines an acoustic event using the second acoustic feature amount;
The acoustic event information included in the learning information represents the acoustic event determined by the first acoustic event determination unit,
The acoustic event information included in the input information represents the acoustic event determined by the second acoustic event determination unit.
An estimation apparatus characterized by that. 2nd learning including the 1st information for learning containing the action information showing the contents of action, the sound event information showing the sound event accompanying the action, and the sound event information showing the sound event accompanying the unknown action using the use information, the probability potential field situations and actions and the acoustic events indirectly relate to generating the acoustic event P | and (acoustic event status), the probability that the action produces the situation and | (behavioral state), the unknown action probability P that generates the status P | obtain the (status unknown behavior), a behavior / situation modeling unit,
The probability P (acoustic event | action) that the action generates an acoustic event, obtained from the probability P (acoustic event | situation) and the probability P (situation | action), and the probability P (acoustic event | situation) The probability P (acoustic event | unknown action) obtained from the probability P (situation | unknown action) is compared with the probability P (acoustic event | unknown action) that the unknown action generates an acoustic event, and the content of the unknown action is estimated. Or a generation model comparison unit that estimates a situation corresponding to acoustic event information representing an acoustic event associated with the unknown behavior using a situation-acoustic event generation model corresponding to the probability P (acoustic event | situation); ,
An estimation device. The estimation apparatus according to claim 10, comprising:
A first feature amount calculation unit that calculates a first acoustic feature amount from a first time-series acoustic signal sequence that represents a sound accompanying the behavior represented by the behavior information;
A first acoustic event determination unit that determines an acoustic event using the first acoustic feature amount;
A second feature amount calculation unit for calculating a second acoustic feature amount from a second time-series acoustic signal sequence;
A second acoustic event determination unit that determines an acoustic event using the second acoustic feature amount;
The acoustic event information included in the first learning information represents the acoustic event determined by the first acoustic event determination unit,
The acoustic event information included in the second learning information represents the acoustic event determined by the second acoustic event determination unit.
An estimation apparatus characterized by that. The estimation apparatus according to claim 10, comprising:
A first acoustic event determination unit that determines an acoustic event using a first acoustic feature amount corresponding to a time-series acoustic signal sequence representing a sound accompanying the behavior represented by the behavior information;
A second acoustic event determination unit that uses the second acoustic feature amount to determine an acoustic event,
The acoustic event information included in the first learning information represents the acoustic event determined by the first acoustic event determination unit,
The acoustic event information included in the second learning information represents the acoustic event determined by the second acoustic event determination unit.
An estimation apparatus characterized by that.

Images