JP5777567B2 - Acoustic feature quantity calculation device and method, specific situation model database creation device, specific element sound model database creation device, situation estimation device, calling suitability notification device, and program - Google Patents

Description

  The present invention relates to a technique for extracting a feature quantity of an acoustic signal, a technique for estimating a situation using the extracted feature quantity, and a technique for notifying whether or not a call is appropriate.

  A technique described in Non-Patent Document 1 is known as a technique for calculating a rising characteristic that is one of the feature quantities of an acoustic signal. The rising characteristic is an index representing the degree of increase in the magnitude of the acoustic signal every tens to hundreds of milliseconds.

In the technique described in Non-Patent Document 1, a value of LAT (Log Attack Time) defined by the following equation is calculated as the rising characteristic (for example, see Non-Patent Document 1). T _start is a time when the amplitude of the acoustic signal starts to increase, and T _stop is a time when the amplitude of the acoustic signal becomes maximum.

LAT = log ₁₀ (T _stop -T _start )

Hyoung-Gook Kim, Nicolas Moreau, Thomas Sikora, “MPEG-7 AUDIO AND BEYOND: audio content indexing and retrieval”, WILEY

However, it may be difficult to specify the time T _{start at} which the amplitude of the acoustic signal starts increasing and the time T _{stop at} which the amplitude of the acoustic signal becomes maximum. In this case, with the technique described in Non-Patent Document 1, it is difficult to calculate LAT.

  The present invention relates to an acoustic feature quantity calculation device and method, a specific situation model database creation device, a specific element sound model database creation device, a situation estimation device, a call suitability notification device, and a device that can be applied to various acoustic signals than before. The purpose is to provide a program.

An acoustic feature amount calculation apparatus according to an aspect of the present invention includes a frame dividing unit that divides an input acoustic signal into frames of a predetermined time length, an acoustic signal of each frame is divided into K sections, and pk The average value of the index representing the magnitude of the acoustic signal in the kth section of the frame, Δp-k as the rate of change of pk in the kth section of each frame, and m as a predetermined integer of 2 or more, in calculates defined values, rising characteristic calculation if its value is 0 or more and the value is the rising characteristics of each frame, if the value is less than 0 to 0 rising characteristics of each frame A feature amount extraction unit including a unit.

  A specific situation model database creation device according to an aspect of the present invention includes an acoustic feature amount calculation device, a specific element sound model database that stores specific element sound models of a plurality of specific element sounds, and a feature calculated by the acoustic feature amount calculation device. The labeled sound of the specific element sound model of the most similar model by comparing the quantity and the specific element sound model stored in the specific element sound model database, or the label of the specific element sound model assigned to the acoustic signal sequence A specific element sound model in a histogram frame in which a predetermined number of frames are grouped with an element sound model comparison unit that outputs a signal sequence and a label of the specific element sound model output from the element sound model comparison unit or a labeled acoustic signal sequence as an input An element sound histogram generator for creating an element sound histogram which is the frequency of appearance of each label, and an element sound hiss As input grams, and a certain situation modeling unit for generating a specific situation models corresponding to a particular field by using a modeling approach to the element tone histogram.

  A specific element sound model database creation device according to an aspect of the present invention includes an acoustic feature amount calculation device and a feature amount calculated by the acoustic feature amount calculation device as input, and a specific element using a modeling method for the feature amount. A specific element sound modeling unit that generates a sound model.

  A situation estimation apparatus according to an aspect of the present invention includes an acoustic feature quantity calculation device, a specific element sound model database storing a specific element sound model generated by a specific element sound model database creation device, a specific element sound model, and an acoustic feature. An element sound model comparison unit that compares the feature amounts calculated by the quantity calculation device, determines the closest one as the element sound of each short-time acoustic signal, and assigns an element sound label for each frame, and a labeled acoustic signal The element sound histogram generation unit that creates a component sound histogram of a specific element sound model label and its frequency with a column as an input, and a plurality of specific situation models and situation classification models generated by the specific situation model database creation device, Compare the stored specific situation model database, element sound histogram, specific situation model or situation classification model, Also and a situation determining model comparison unit for outputting a state estimation result to estimate the situation represented by the specific situation model or situation classification model what similar.

  A call suitability notification device according to an aspect of the present invention includes an acoustic feature amount calculation device, a call recommendation model storage unit that stores a call recommendation model that associates the degree of ease of occurrence of a call, and an acoustic feature. The feature quantity calculated by the quantity calculation device is used as an input, and the call recommendation model is determined by referring to the call recommendation model that matches the feature quantity to determine whether the call is often made or not caused by the caller. A call recommendation status determination unit that transmits the notification information to the caller side.

  It can be applied to a variety of acoustic signals than in the past.

The functional block diagram of the acoustic feature-value calculation apparatus 1 of 1st embodiment. The figure which shows the operation | movement flow of the acoustic feature-value calculation apparatus 1. The figure for demonstrating the example of a process of the starting characteristic calculation part. The functional block diagram of the acoustic feature-value calculation apparatus 1 of 2nd embodiment. The functional block diagram of the acoustic feature-value calculation apparatus 1 of 3rd embodiment. The functional block diagram of the specific situation model database creation apparatus 100. FIG. The figure which shows the operation | movement flow of the specific condition model database creation apparatus 100. The figure which shows the relationship between a frame and a histogram frame. The figure which shows the example of an element sound histogram. The functional block diagram of the specific situation model database creation apparatus 200. FIG. The figure which shows the operation | movement flow of the specific condition model database creation apparatus 200. The functional block diagram of the specific element sound model database creation apparatus 300. The functional block diagram of the specific element sound model database creation apparatus 400. The functional block diagram of the situation estimation apparatus 500. FIG. The figure which shows the operation | movement flow of the condition estimation apparatus 500. The functional block diagram of the communication system 2000 incorporating the call recommendation model production | generation apparatus 600. FIG. The figure which shows the example of a communication history table. The functional block diagram of the communication system 2000 incorporating the call recommendation model production | generation apparatus 610. FIG. The functional block diagram of the communication system 2000 incorporating the call recommendation model production | generation apparatus 620. FIG. The functional block diagram of the call recommendation model production | generation apparatus 630. FIG. The functional block diagram of the call recommendation model production | generation apparatus 640. FIG. The functional block diagram of the call recommendation model production | generation apparatus 650. FIG. The functional block diagram of the call recommendation model production | generation apparatus 660. FIG. FIG. 3 is a functional block diagram of a communication system 2000 including a call terminal 2010 to which a call suitability notification device 700 is connected. 1 is a functional block diagram of a communication system 2000 including a call terminal 2010 to which a call suitability notification device 710 is connected. 1 is a functional block diagram of a communication system 2000 including a call terminal 2010 to which a call suitability notification device 720 is connected. The functional block diagram of the calling suitability notification apparatus 730. The functional block diagram of the calling suitability notification apparatus 740. The functional block diagram of the calling suitability notification apparatus 750. The functional block diagram of the calling suitability notification apparatus 760.

  Embodiments of the present invention will be described below with reference to the drawings.

  The first embodiment to the third embodiment are embodiments of the acoustic feature quantity calculation device and method, the fourth embodiment to the fifth embodiment are embodiments of the specific situation model database creation device, and from the sixth embodiment. The seventh embodiment is an embodiment of a specific element sound model database creation device, the eighth embodiment is an embodiment of a situation estimation device, and the ninth to thirteenth embodiments are call recommendation model generation devices. The fourteenth embodiment to the eighteenth embodiment are embodiments of the call suitability notification device.

[First embodiment]
As shown in FIG. 1, the acoustic feature quantity calculation device 1 of the first embodiment includes, for example, a frame division unit 11, a quantization unit 12, and a feature quantity extraction unit 13. FIG. 2 shows an operation flow of the acoustic feature quantity calculation apparatus 1 of the first embodiment.

  The frame dividing unit 11 divides the input acoustic signal into frames having a predetermined time length (step A1). The predetermined time length is, for example, about 50 milliseconds. Two consecutive frames may or may not overlap. The acoustic signal divided into frames is output to the feature amount extraction unit 13.

  In the example of FIG. 1, the input acoustic signal is a discrete signal quantized by the quantization unit 12 at regular time intervals and at regular sound pressures. Of course, the processing of the quantizing unit 12 may be performed after the processing of the frame dividing unit 11 or after the processing of the feature amount extracting unit 13.

  The feature amount extraction unit 13 includes a rising characteristic calculation unit 131. The rising characteristic calculation unit 131 calculates the rising characteristic of each frame (step A2).

  The rising characteristic is an index representing the degree of increase in the index representing the magnitude of the acoustic signal every several tens to several hundreds of milliseconds. Here, the index representing the magnitude of the acoustic signal is, for example, an absolute value of the amplitude of the acoustic signal, a logarithmic value of the absolute value of the amplitude of the acoustic signal, a power of the acoustic signal, or a logarithmic value of the power of the acoustic signal.

  In order to calculate the rising characteristic, the rising characteristic calculation unit 131 first divides the acoustic signal of each frame into K sections. K is a predetermined positive integer. The value of K is set so that each divided section is about 1 millisecond.

  Next, the rising characteristic calculation unit 131 calculates a value defined by the following equation. If the value is 0 or more, the value is specified as the rising edge of each frame, and if the value is less than 0, each value The rising characteristic of the frame is set to zero. The calculated rise characteristic is output from the feature amount extraction unit 13 as a feature amount.

p ^- _k is an average value of an index indicating the magnitude of the acoustic signal in the k-th section of the frame, and Δp ^- _k is a rate of change of p ^- _k in the k-th section of the frame. “x” on the right shoulder of x, where x is an arbitrary character, means a superscript bar of x. m is a predetermined integer of 2 or more. For example, m = 2.

For example, Δp ^- _k = p ^- a _k-1 ^- _k -p. _{^{Δp - k = p - k +}} 1 -p - may be as _k. Alternatively, a straight line approximating p ⁻ _k may be _obtained using an approximation method such as a least square method, and the slope of the straight line in the kth section may be Δp ⁻ _k .

p ^- a _k and power of the acoustic _{^{signal, Δp - k = p - k}} + 1 -p - If set to _k, as shown in FIG. _{^{3, Δp - 2 = p -}} 3 - p - 2 become.

Thus, by calculating the rising characteristics, there is no need to amplitude of the time T _start and acoustic signal amplitude of the acoustic signal starts increasing to identify the time T _stop having the maximum these times T _start, Even when it is difficult to specify T _stop , the rise characteristic can be calculated.

  In addition, since it is possible to extract the enhancement characteristic with emphasis, it is possible to effectively extract the rising characteristic even for an acoustic signal in which it is difficult to extract only the rising characteristic in the conventional technique.

[Second Embodiment]
The acoustic feature quantity calculation apparatus 1 of the second embodiment is different from the acoustic feature quantity calculation apparatus 1 of the first embodiment in that an acoustic feature quantity other than the rising characteristic is calculated. Hereinafter, the description will focus on the parts different from the first embodiment, and the description of the same parts as the first embodiment will be omitted.

  The feature quantity extraction unit 13 of the acoustic feature quantity calculation apparatus 1 according to the second embodiment includes an abruptness calculation unit 132, a time diffusivity calculation unit 133, and a narrow band calculation unit 134 in addition to the rising characteristic calculation unit 131. Further, at least one of a band spread calculation unit 135, a pitch characteristic calculation unit 136, and an amplitude unevenness calculation unit 137 is further provided. FIG. 4 shows a functional block diagram of the acoustic feature quantity calculation apparatus 1 of the second embodiment when the feature quantity extraction unit 13 includes all of these parts.

  The acoustic feature quantity calculation apparatus 1 of the second embodiment further includes a vector generation section 14 in addition to the quantization section 11, the frame division section 12, and the feature quantity extraction section 13.

The suddenness calculation unit 132 calculates the suddenness indicating the degree of concentration of the acoustic signal of each frame in the time domain. The suddenness is a value defined by the following equation, for example. μ ⁻ _n is the average value of the acoustic energy envelope in the n th section, σ ⁻ _n is the variance of the acoustic energy envelope in the n th section,

The time diffusivity calculation unit 133 calculates time diffusivity indicating the degree of diffusion in the time domain of the acoustic signal of each frame. The time diffusivity is a value defined by the following equation, for example. x _n is a distance from the calculation frame start position in the time domain, and x ⁻ _n is a position that is an average value of the acoustic energy envelope in the time domain.

The narrow band calculation unit 134 calculates the narrow band indicating the degree of concentration of the acoustic signal of each frame in the frequency domain. The narrow bandwidth is a value defined by the following equation, for example. f is the frequency, F is the number of frequency bins, p ^- (f) is the mean value of the acoustic energy at a frequency f, mu ^- _f is the frequency at which the mean value of the distribution of acoustic energy envelope, σ ⁻ _f is a variance value of the distribution of the acoustic energy envelope.

  The band spread calculation unit 135 calculates band spread indicating the degree of spread in the frequency domain of the acoustic signal of each frame. The band spreading property is a value defined by the following equation, for example.

  The pitch characteristic calculation unit 136 calculates a pitch characteristic indicating the degree of uneven distribution of energy in the frequency domain of the acoustic signal of each frame. The pitch characteristic is a value defined by the following equation, for example. p (f) is the acoustic energy of frequency f.

The amplitude unevenness calculation unit 137 calculates the amplitude unevenness indicating the degree of uneven distribution of the amplitude value distribution of the acoustic signal of each frame. The amplitude uneven distribution is a value defined by the following equation, for example. p _n is the amplitude value of the nth sample.

  The feature amount calculated by the feature amount extraction unit 13 is vectorized by the vector generation unit 14. The feature amount calculated by the feature amount extraction unit 13 is a rising characteristic and at least one of suddenness, time diffusibility, narrow band property, band spread property, pitch property, and amplitude unevenness.

[Third embodiment]
The acoustic feature quantity calculation device 1 of the third embodiment is the first embodiment or the point that calculates the acoustic feature quantity other than the feature quantity calculated by the feature quantity extraction unit 13 of the first embodiment or the second embodiment. It differs from the acoustic feature quantity calculation apparatus 1 of the second embodiment. The following description will focus on the parts that are different from the first embodiment or the second embodiment, and the description of the same parts as the first embodiment or the second embodiment will be omitted.

  The feature quantity extraction unit 13 of the acoustic feature quantity calculation device 1 according to the third embodiment further includes an acoustic feature quantity calculation unit 138. In FIG. 5, the feature amount extraction unit 13 has the suddenness calculation unit 132, the time diffusivity calculation unit 133, the narrow band calculation unit 134, the band diffusivity calculation unit 135, and the pitch characteristic calculation described in the second embodiment. It is a functional block diagram of the acoustic feature quantity calculation apparatus 1 of the third embodiment when all of the unit 136 and the amplitude unevenness calculation unit 137 are provided.

  The acoustic feature amount calculation unit 138 calculates acoustic feature amounts such as MFCC (Mel-Frequency Cepstrum Coefficient) and power spectrum. Of course, the acoustic feature quantity calculation unit 138 may calculate an acoustic feature quantity according to another existing technology as the acoustic feature quantity.

  The acoustic feature amount calculated by the acoustic feature amount calculation unit 138 is output to the vector generation unit 14 as the feature amount calculated by the feature amount extraction unit 13.

[Fourth embodiment]
The specific situation model database creation device 100 of the fourth embodiment creates a specific situation model database using the acoustic feature quantity calculation device 1 of the first embodiment to the third embodiment.

  In FIG. 6, the example of the functional block diagram of the specific condition model database creation apparatus 100 of 4th embodiment is shown. An example of the operation flow is shown in FIG. The specific situation model database creation device 100 includes the acoustic feature quantity calculation device 1, a specific element sound model database 20, an element sound model comparison unit 30, an element sound histogram conversion unit 40, and a specific condition model generation unit 50. It has. The specific situation model database creation apparatus 100 is realized by a predetermined program being read into a computer constituted by, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The acoustic feature quantity calculation device 1 is the acoustic feature quantity calculation device 1 according to any one of the first to third embodiments. The acoustic feature quantity calculation device 1 divides an acoustic signal sequence including a plurality of element sounds in a specific field into short-time frames by the method described in the first embodiment to the third embodiment, and features quantities for each frame. Is extracted (step S10). Step S10 corresponds to steps A1 and A2 in FIG. The feature amount calculated by the acoustic feature amount calculation device 1 is output to the element sound model comparison unit 30. Here, the acoustic signal sequence including a plurality of element sounds in a specific place is an acoustic signal sequence representing the situation in a specific place such as a situation where a person is cooking or a person is reading. That's it. That is, it is an acoustic signal whose time length recorded in a specific place is about 5 to 20 seconds, for example. The acoustic signal is divided into frames of 20 msec to 100 msec, the feature amount is calculated for each frame, and the calculated feature amount is set as the feature amount of the element sound.

  The feature quantity calculated by the acoustic feature quantity calculation device 1, in other words, the feature quantity extracted by the feature quantity extraction unit 13 is, for example, rise characteristics, suddenness, time diffusivity, narrow band characteristics, band spread characteristics, pitch characteristics, This is an acoustic feature such as amplitude unevenness, MFCC (Mel-Frequency Cepstrum Coefficient), power spectrum, and the like.

  The element sound model comparison unit 30 compares the feature amount output from the feature amount extraction unit 13 of the acoustic feature amount calculation device 1 with a plurality of specific element sound models stored in the specific element sound model database 20. The label of the specific element sound model with the shortest distance (Euclidean distance or cosine distance), or the labeled acoustic signal string obtained by adding the label of the specific element sound model to the acoustic signal string in units of frames is output (step S30). The feature quantity with a label of the specific element sound model is created by the specific element sound model database creation apparatus 300 described later.

  The element sound histogram generation unit 40 receives the label of the specific element sound model or the labeled acoustic signal sequence output from the element sound model comparison unit 30 as an input, and outputs the specific element sound model in the histogram frame in which a predetermined number of the frames are collected. An element sound histogram which is the appearance frequency for each label is created (step S40). FIG. 8 shows the relationship between frames and histogram frames.

  FIG. 8 is an example in which the specific place is set to, for example, “a situation where a person is cooking”. The labeled acoustic signal sequence is, for example, a signal sequence to which a specific element sound model label is assigned for each frame having a time width of 20 msec to 100 msec.

In the example of FIG. 8, the first frame f ₁ is a person's footsteps, the second frame f ₂ is a sound of cutting food with a kitchen knife, the third frame f ₃ is a person's footsteps, etc. The element sound model label is assigned. The histogram frame is a collection of P frames, and the P frames are, for example, 100 to 1000. From the first frame _{f 1} to _{f P} frame is first histogram frame _{H 1.} The second histogram frame H ₂ consists of frames f ₂ to f _{P + 1} frames. When the frame length of the labeled acoustic signal sequence is M, M−P + 1 histogram frames are created.

  The element sound histogram generator 40 creates an element sound histogram that is the appearance frequency for each label of the specific element sound model in the histogram frame. FIG. 9 illustrates an element sound histogram. The horizontal axis is the label of the specific element sound model, and the vertical axis is, for example, the number of times each specific element sound appears in one histogram frame and the likelihood of each specific element sound in each frame. This is the total sum for each specific element sound.

  The specific situation modeling unit 50 receives the element sound histogram output from the element sound histogram generation unit 40 as an input, and generates a specific situation model corresponding to a specific field using the modeling method for the element sound histogram ( Step S50). For example, when a GMM (Gaussian Mixture Model) is used, the generated modeling method uses a mixed normal distribution (Mixture of Gaussian as shown in, for example, Expression (1) using an EM (Expectation Maximization) algorithm or the like. ) To model the probability model p (x) fitted.

Here, x is a feature quantity (vector), k is the number of normal distributions, π _k is a mixing coefficient, N is a probability density function of normal distribution, μ _k is an average of distribution, and Σ _k is a distribution variance. For modeling the feature quantity, an HMM (Hidden Markov Model) that applies the feature quantity calculated to the probability distribution using the condition that the component at the next time is selected depending on the signal component observed in the past. Alternatively, an SVM (Support Vector Machine) that performs modeling by maximizing the margin between the clusters and determining the separation boundary with respect to the feature amount can be used. GMM, HMM, and SVM are well known (for example, references: Manabu Okumura, Daiya Takamura, “Introduction to Machine Learning for Language Processing” Corona).

  For example, when a specific situation model is generated using GMM, each of the M−P + 1 histogram frames created by the element sound histogram generator 40 has labels of N specific element sounds models. The specific situation model may be output as it is or may be stored in the specific situation model database 60.

  Under this assumption, the specific situation modeling unit 50 obtains an average and a variance from a plurality of histogram frames obtained from one or a plurality of long-time acoustic signals representing a specific situation such as cooking. At this time, if R types of specific situation models are calculated, R averages and variances are calculated, respectively, and the values become the specific situation models.

  As described above, according to the specific situation model database creation device 100 of the present invention, specific element sounds are identified from an acoustic signal sequence including a plurality of element sounds in a specific field, and the identification results are histogrammed. A specific situation model that identifies the place is generated from the distribution. Since this specific situation model is obtained from a plurality of specific element sounds, unlike the one piecewise feature amount of the prior art, the situation of a place (for example, a place being cooked) characterized by a plurality of different sounds for the first time is obtained. This is an effective model for estimation.

[Fifth embodiment]
In FIG. 10, the example of the functional block diagram of the specific condition model database creation apparatus 200 of 5th embodiment is shown. An example of the operation flow is shown in FIG. The specific situation model database creation device 200 records the sound signal in an unspecified place, that is, the input acoustic signal sequence may not be an acoustic signal representing a specific place with respect to the specific situation model database creation apparatus 100 described above. The distribution clustering processing unit 210 for classifying the element sound histogram created by the element sound histogram generating unit 40 according to the distribution shape, and the situation classification modeling unit 220 for generating a situation classification model from the output. And is different in that it comprises. Similarly to the specific situation model database creation apparatus 100, the specific situation model database creation apparatus 200 reads a predetermined program into a computer composed of, for example, a ROM, a RAM, and a CPU, and the CPU executes the program. It is realized.

  The specific situation model database creation device 200 includes an acoustic feature quantity calculation device 1, a specific element sound model database 20, an element sound model comparison unit 30, an element sound histogram conversion unit 40, a distribution clustering processing unit 210, and a situation classification. A modeling unit 220. The acoustic feature quantity calculation device 1, the specific element sound model database 20, the element sound model comparison unit 30, and the element sound histogram generation unit 40 are the same as the specific situation model database creation device 100 as is apparent from the reference numerals.

  The distribution clustering processing unit 210 receives a plurality of element sound histograms created by the element sound histogram generating unit 40, and classifies each element sound histogram according to its distribution shape (step S210). In other words, a group of B histograms is created by classifying M−P + 1 histograms with similar distribution shapes. B is a preset “number of element sounds to be classified”. As a method of classifying by the shape of the distribution, the same method as that for generating the specific situation model described above can be used. By using a classification method such as GMM or SVM, the M−P + 1 histograms are classified into groups (sets) of B histograms. Each group of histograms having similar distribution shapes corresponds to a specific field. The situation classification modeling unit 220 generates B types of situation classification models using modeling techniques such as GMM, HMM, and SVM for the B sets that are a group of histograms (step S220). The method for generating the situation classification model is the same as the method for generating the specific situation model described above.

[Sixth embodiment]
As a sixth embodiment, a specific element sound model database creation apparatus 300 that creates a specific element sound model database 20 constituting the specific situation model database creation apparatuses 100 and 200 will be described.

  FIG. 12 shows a functional block diagram of the specific element sound model database creation device 300. The specific element sound model database creation device 300 includes the acoustic feature quantity calculation device 1 and a specific element sound modeling unit 320.

  The acoustic feature quantity calculation apparatus 1 receives an acoustic signal string of a specific sound as an input, divides the acoustic signal string into short-time frames, and extracts feature quantities for each frame. For example, in the case where there are a plurality of footstep acoustic signals of a specific sound, feature quantities (vectors) of all the acoustic signals are calculated. If there are n footstep sound signals and each sound signal can be divided into m short time periods, n × m feature values (vectors) are calculated.

  The specific element sound modeling unit 320 generates one specific element sound model by using a modeling method for n × m feature quantities (vectors). As the modeling method, the same method as the method for generating the specific situation model by the specific situation modeling unit 50 described above is used. The generated specific element sound model is stored in the specific element sound model database 20. The specific element sound model database 20 constitutes the specific situation model database creation apparatus 100 as described above. Next, another embodiment of the specific element sound model database creation apparatus 300 will be described.

[Seventh embodiment]
FIG. 13 shows a functional block diagram of a specific element sound model database creation device 400 according to the seventh embodiment. The specific element sound model database creation apparatus 400 is different from the specific element sound model database creation apparatus 300 in that a feature amount clustering unit 410 and an element sound classification modeling unit 420 are provided, and a plurality of elements are included in the input acoustic signal sequence. It differs in that it includes sound.

  The acoustic feature quantity calculation apparatus 1 is the same as that of the specific element sound model database creation apparatus 300 except that the acoustic feature quantity calculation apparatus 1 is different only in that an acoustic signal sequence including a plurality of element sounds is input. The feature quantity clustering unit 410 classifies the feature quantities output by the acoustic feature quantity calculation device 1 and creates a set of feature quantities. A method such as GMM or SVM is used as a feature amount classification method, and the acoustic signal sequence is classified into C groups (groups). C is a preset “number of feature quantities to be classified”.

  The element sound classification modeling unit 420 receives a set of C feature amounts output from the feature amount clustering unit 410 and generates a component sound classification model using a modeling method for the set. As the modeling method, the same method as the method for generating the specific situation model from the element sound histogram by the specific situation modeling unit 50 described above is used.

  The specific element sound model database creation device 400 classifies an acoustic signal sequence including a plurality of element sounds based on the feature amount, and generates an element sound classification model from the classified group (set).

[Eighth embodiment]
In FIG. 14, the functional block diagram of the condition estimation apparatus 500 of 8th embodiment is shown. The operation flow is shown in FIG. The situation estimation apparatus 500 is generated by the specific element sound model database 20 that stores the specific element sound model generated by the specific element sound model database creation apparatus 300 described above, and by the specific situation model database creation apparatuses 100 and 200 described above. The specific situation model database 60 storing the specific situation model and the situation classification model is used to estimate the situation represented by the acoustic signal sequence. The situation estimation apparatus 500 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The situation estimation device 500 includes an acoustic feature quantity calculation device 1, a specific element sound model database 20, an element sound model comparison unit 30, an element sound histogram conversion unit 40, a situation determination model comparison unit 510, and a specific situation model database. 60. The acoustic feature quantity calculation device 1 divides the input acoustic signal sequence into short-time frames, and extracts feature quantities for each frame (step S10). The element sound model comparison unit 30 compares the feature amount output from the acoustic feature amount calculation device 1 with the specific element sound model or the element sound classification model stored in the specific element sound model database 20, and determines the closest ones. Is determined as an element sound of the short-time sound signal, and an element sound label is assigned to the sound signal string for each frame (step S30). The element sound histogram generation unit 40 receives the label of the specific element sound model or the labeled acoustic signal sequence output from the element sound model comparison unit 30 as an input, and outputs the specific element sound model in the histogram frame in which a predetermined number of the frames are collected. An element sound histogram which is the appearance frequency for each label is created (step S40). The operation so far is the same as that of the specific situation model database creation apparatus 100 or 200 described above.

  The situation determination model comparison unit 510 compares the element sound histogram and the specific situation model or the situation classification model stored in the specific situation model database 60, estimates the closest one as the situation represented by the specific situation model, and estimates the estimation. Output the result. Here, the comparison is performed using a plurality of specific situation models and the Euclidean distance and cosine distance of the element sound histogram.

  For the estimation of the situation of the field, for example, the model having the closest distance is estimated as the situation of the scene. If the distance is closer than the predetermined threshold, the model with the closest distance is estimated as the current situation, and if there is no model with the distance closer than the threshold, it may be estimated as “other situations”. It is done.

  According to the situation estimation apparatus 500 described above, it is possible to estimate the situation of a field characterized for the first time by a plurality of different sounds using an acoustic signal. In addition, by introducing a clustering process into the generation of the element sound determination model, it is possible to estimate the situation of the field without preparing an acoustic signal labeled with a specific sound and a specific situation in advance.

  The specific element sound model database creation device 300 that creates the specific element sound model database 20 constituting the situation estimation device 500 uses a modeling method for the feature amount (vector) of an acoustic signal of a specific sound. A sound model is generated. In the following, some devices when the technical idea is applied to a communication scene will be described. First, a call recommendation model generation apparatus 600 that generates a call recommendation model used for the purpose of knowing whether a communication partner in a remote place can communicate is described.

[Ninth embodiment]
The call recommendation model generation apparatus 600 according to the ninth embodiment is used in a communication system, and performs modeling of an acoustic signal when a telephone call frequently occurs and modeling of an acoustic signal when a telephone conversation does not occur so much. Is. FIG. 16 shows a functional block diagram of the call recommendation model generation device 600 and a functional block diagram of a communication system 2000 in which the call recommendation model generation device 600 is connected to one communication terminal.

  The communication system 2000 includes a network 2020 such as a telephone line network or the Internet, and communication terminals 2010 and 2030 arranged on one side and the other side of the network 2020. The communication terminal 2010 is, for example, a receiving side, and the communication terminal 2030 is, for example, a transmitting side. The communication terminal 2010 is connected with a call recommendation model generation device 600. The communication terminal 2010 includes an audio / video signal presentation unit 2011 and an audio / video signal acquisition unit 2012. The notation of the audio / video signal presentation unit and the acquisition unit on the communication terminal 2030 side is omitted.

  The call recommendation model generation device 600 includes an acoustic feature quantity calculation device 1, a call history extraction unit 602, a call recommendation model generation unit 603, and a call recommendation model storage unit 604. The acoustic feature quantity calculation device 1 divides the acoustic signal sequence acquired from the audio / video signal acquisition unit 2012 of one communication terminal 2010 into short frames, and extracts feature quantities for each frame. The extracted feature amount is output to the call recommendation model generation unit 603.

  The call history extraction unit 602 receives the call history from the communication terminal 2010 at any time and transmits an incoming / outgoing call signal indicating that there is a new call / incoming call to the call recommendation model generation unit 603 and also stores the call history table. create. FIG. 17 shows an example of a call history table. The call history table includes, for example, items of outgoing / incoming time, call end time, call time, outgoing / incoming call, destination number, and historical address. The null of the call end time of the history address 0002 in FIG. 17 indicates that “a call that has received an incoming call from the other communication terminal 2030 but one of the callers did not off-hook the receiver”. Further, null of the history address 0004 indicates that “the call originated from one communication terminal 2010 but the other party did not go off-hook”.

  In response to the incoming / outgoing call signal output from the call history extraction part 602, the outgoing call recommendation model generation unit 603 identifies the feature quantity classification of the feature quantity of the acoustic signal immediately before the incoming / outgoing call signal. The identification is performed within a range of the magnitude of the distance using, for example, a Euclidean distance or a cosine distance. Then, the call recommendation model generation unit 603 creates a call history model table. Table 1 shows an example of a call history model table.

  The history addresses 0003 and 0005 shown in FIG. 17 are classified, for example, in the feature amount classification a, and the history addresses 0001 and 0006 are classified in the feature amount classification d.

  The call recommendation model generation unit 603 classifies the feature amount of the acoustic signal immediately before the incoming / outgoing call signal, and then grades the feature amount classification from the call history table corresponding to the history address. The leveling is performed so that when the number of calls often occurs, the value of the degree of ease of making the call becomes large, and when the number of calls does not occur so much, the value of the degree of ease of making the call becomes small. . For example, the following leveling is performed.

  The side that made the call at the time when the call was made considers that the call often occurs, and adds 1 to the degree T of the ease of the call. If there is an incoming call but does not go off-hook, it is considered that the call does not occur so much, and 1 is subtracted from the degree T of the ease of the call. When a call occurs, a value of 0.0 to 2.0 is added to T according to the call time. Further, even when a call occurs, if the call time is within a predetermined time (for example, 60 seconds), it is considered that the call does not occur so much, and 0.5 is subtracted from T. Thus, by adjusting the value of the degree T of the likelihood of a call, the feature amount classification can be divided into those that are likely to cause a call and those that are unlikely to cause a call. For example, if the value of T is 10 or more, it can be determined that calls often occur, and if it is -10 or less, it is determined that calls do not occur much. Then, the feature quantity classification thus generated and the occurrence degree T are associated with each other to obtain a call recommendation model. The call recommendation model is stored in the call recommendation model storage unit 604. That is, the call recommendation model is a table in which the feature amount (vector) and the occurrence degree T are associated with each other. Therefore, by evaluating the feature quantity of the acoustic signal on the receiving side using the call recommendation model, it is possible to know whether or not the call on the receiving side is likely to occur.

  Note that in order to generate the call recommendation model, only the audio signal may be used as described above, or the video signal acquired by the audio / video signal acquisition unit 2012 may be used. Further, the call recommendation model may be generated using, for example, illuminance information, temperature information, acceleration information, and the like acquired by other sensors (605 indicated by a broken line in FIG. 16). Further, as the call recommendation model, only one of a model in which calls frequently occur and a model in which calls are not frequently generated may be generated.

[Tenth embodiment]
FIG. 18 shows a functional block diagram of the call recommendation model generation device 610 of the tenth embodiment. The call terminal 2010 to which the call recommendation model generation device 610 is connected constitutes a communication system 2000. For example, the call recommendation model generation device 610 prepares a model in which specific element sounds such as footsteps, sound of glass breaking, and the like are associated with feature amounts in advance, and the call recommendation model generation device 600 described above. A specific element sound is specified from the feature amount extracted by the acoustic feature amount calculation apparatus 1, and a call recommendation model is generated using the specific element sound.

  The call recommendation model generation device 610 differs from the call recommendation model generation device 600 in that it further includes a specific element sound model database 611 and an element sound specification unit 612. In the specific element sound model database 611, an element in which a feature amount (vector) obtained by the acoustic feature amount calculation device 1 and an element sound represented by the feature amount (for example, footsteps, sound that breaks glass, speech, etc.) Sound specific model is stored. The method for creating the specific element sound model database 611 is the same as in the seventh embodiment.

  The element sound specifying unit 612 receives the feature amount extracted by the acoustic feature amount calculation apparatus 1 as an input and compares it with each of the specific element sound models stored in the specific element sound model database 611 to determine the distance (Euclidean distance or The one having the closest cosine distance or the like is identified as the element sound of the frame, and the identification result is output to the call recommendation model generation unit 603.

  The outgoing call recommendation model generation unit 603 classifies the history addresses by the element sounds immediately before the incoming / outgoing call signal, and grades the element sounds from the call history table corresponding to the history addresses. The leveling for element sounds is the same as the method described above.

  The call recommendation model generation unit 603 associates the identification result output from the element sound identification unit 612 with the history address of the call history table, and makes a call recommendation model when a call often occurs and when a call hardly occurs Is generated. The call recommendation model in which the element sound of the specific result is associated with the occurrence degree T is stored in the call recommendation model storage unit 613. Modeling is done in the same way as described above.

  According to the call recommendation model generation device 610, it is possible to generate a call recommendation model in which element sounds of individual specific results are associated with the degree of occurrence T. An example is shown in Table 2.

表２は、例えば受話側で人の声がしている時は通話が発生し易く、ドアの開閉音が発生した時は通話が発生し難いことを示している。

Table 2 shows that, for example, a call is likely to occur when a voice is heard on the receiving side, and a call is unlikely to occur when a door opening / closing sound is generated.

[Eleventh embodiment]
FIG. 19 shows a functional block diagram of the call recommendation model generation device 620 of the eleventh embodiment. The call recommendation model generation device 620 clusters feature amounts for a certain time, determines a class to which the feature amount extracted by the acoustic feature amount calculation device 1 belongs from a model generated for each cluster, and determines the element sound thus determined. A call recommendation model is generated using a class.

  The call recommendation model generation device 620 includes a specific element sound model database 621 and an element sound cluster determination unit 622 instead of the specific element sound model database 611 and the element sound determination unit 612 of the call recommendation model generation device 610. Different. In the call recommendation model generation device 610, in order to generate a model corresponding to an element sound, for example, it is necessary to prepare in advance a feature amount corresponding to a sound such as a footstep or a sound of breaking glass. However, it is difficult to prepare in advance feature amounts corresponding to all the element sounds that can be generated.

  Therefore, the call recommendation model generation device 620 generates a model without preparing an association between element sounds and feature amounts in advance. In the specific element sound model database 621, element sound classifications created by classifying feature quantities (vectors) for a certain period of time that would be obtained by the acoustic feature quantity calculation apparatus 1 using a technique such as GMM, HMM, or SVM. The model is saved. The method for creating the specific element sound model database 621 is the same as in the seventh embodiment.

  The element sound cluster determination unit 622 compares the feature amount (vector) acquired from the acoustic feature amount calculation device 1 with the element sound classification model stored in the specific element sound model database 621, and determines the classification class to which the feature amount belongs. The classification result is output to the call recommendation model generation unit 624.

  The call recommendation model generation unit 624 associates the classification class output by the element sound cluster determination unit 622 with the history address of the call history table, and recommends calls when the call often occurs and when the call does not occur much. Generate a model. Modeling is done in the same way as described above.

  According to the call recommendation model generation device 620, it is possible to generate a call recommendation model in which a classification class of element sounds and an occurrence degree T are associated with each other.

[Twelfth embodiment]
The call recommendation model may be a model in which the degree T of occurrence of a call is associated with other information. For example, instead of the above-described feature quantity itself, the call / action information that can be estimated from the feature quantity, the action / behavior classification information that classifies the action / behavior information, and the call occurrence degree T are associated with each other. It may be a recommended model.

  FIG. 20 is a functional block diagram of a call recommendation model generation device 630 according to the twelfth embodiment that generates a call recommendation model in which operation / behavior information is associated with a call occurrence degree T. The call recommendation model generating device 630 is obtained by adding an operation / behavior specifying model storage unit 631 and an operation / behavior specifying unit 632 to the configuration of the call recommendation model generating device 610 described above.

  The call recommendation model generation device 630 prepares a model in which element sounds such as element sounds, footsteps, and glass breaking sounds specified by the element sound specifying unit 612 are associated with actions and actions that generate element sounds. Thus, the action or action that generates the element sound is specified, and a call recommendation model is generated using the specified action or action.

  The action / behavior specifying model storage unit 631 is an action / action in which a specific result output by the element sound specifying unit 612 and an action / behavior (for example, cooking, reading, sleeping) are associated with each other. An action specific model is stored. The creation method of the action / behavior identification model storage unit 631 is the same as the creation method of the specific situation model database of the fourth embodiment. For example, the action / behavior specifying model receives, as an input, the output of the element sound specifying unit 612 for each frame having a time width of 20 msec to 100 msec, for each histogram frame in which P frames are collected, for each label of the specific element sound model. An element sound histogram representing the appearance frequency is created, and the shape of the histogram is modeled using a modeling technique.

  The action / behavior identification unit 632 creates a histogram of the identification results output from the element sound identification unit 612, compares it with the action / behavior identification model stored in the action / behavior identification model storage unit 631, and most similar operation / behavior. The action / behavior is specified by specifying the classification model, and the action / behavior information is output to the call recommendation model generation unit 603.

  The call recommendation model generation unit 634 associates the operation / behavior information output by the operation / behavior identification unit 632 with the history address of the call history table, and generates a call when a call frequently occurs and when a call hardly occurs. Create a call recommendation model. Modeling is performed in the same manner as described above, and is stored in the call recommendation model storage unit 633.

  The call recommendation model generation device 630 can generate a call recommendation model in which the action / behavior information and the occurrence degree T are associated with each other. The action / behavior information is, for example, information such as “cooking” or “reading”, and is a call recommendation model in which the value of the degree of occurrence T is associated with each.

  Further, a functional configuration example of the call recommendation model generation device 640 having a configuration in which an operation / behavior specification model storage unit 631 and an operation / behavior specification unit 632 are further added to the configuration of the call recommendation model generation device 620 described above is also considered. It is done. FIG. 21 shows a functional block diagram of the call recommendation model generation device 640.

  The call recommendation model generation device 640 prepares a model in which the element sound class specified by the element sound cluster determination unit 622 is associated with the action or action that generates the element sound, thereby generating the element sound. And an action are specified, and a call recommendation model is generated using the specified action and action.

  The call recommendation model generation unit 634 associates the operation / behavior information output by the operation / behavior identification unit 632 with the history address of the call history table, and generates a call when a call frequently occurs and when a call hardly occurs. Create a call recommendation model. Modeling is done in the same way as described above. The call recommendation model generation device 640 can also generate a call recommendation model in which the action / behavior information and the occurrence degree T are associated with each other.

[Thirteenth embodiment]
FIG. 22 shows a call recommendation model generation device 650 according to the thirteenth embodiment in which an operation / behavior classification model storage unit 651 and an operation / behavior cluster determination unit 652 are further added to the configuration of the call recommendation model generation device 610 described above. A functional block diagram is shown.

  The call recommendation model generation device 650 determines the action or action class represented by the element sound specified by the element sound specifying unit 612, and generates a call recommendation model using the determined action or action class. .

  In the action / behavior classification model storage unit 651, the element sound histogram generated by the histogram processing from the element sound identification results over a plurality of frames acquired from the element sound specifying unit 612, and the action / behavior (for example, cooking) , Reading, sleeping, etc.) are stored. The creation method of the action / behavior identification model storage unit 651 is the same as the creation method of the specific situation model database of the fifth embodiment. For example, an output of the element sound specifying unit 612 for each frame having a time width of 20 msec to 100 msec is input, and an element sound histogram which is an appearance frequency for each label of the specific element sound model for each histogram frame in which P frames are collected. Create By classifying the histograms that are similar in shape into a set (group) of B histograms, and using these B sets using a modeling method such as GMM, HMM, or SVM, Generate an action / behavior specific model.

  The action / behavior cluster determination unit 652 calculates a frequency feature amount from the element sound identification result acquired from the element sound identification unit 612 and compares it with the action / behavior classification model stored in the action / behavior classification model storage unit 651. Then, the most similar operation / behavior classification model is identified as the operation / behavior classification represented by the identification result, and the operation / behavior classification information is output to the call recommendation model generation unit 603.

  The call recommendation model generation unit 654 associates the action / behavior classification information output from the action / behavior identification unit 652 with the history address of the call history table, and the case where the call often occurs and the case where the call does not occur much A call recommendation model is generated. The call recommendation model in which the action / behavior classification information and the occurrence degree T are associated with each other is stored in the call recommendation model storage unit 653.

  The call recommendation model generation device 650 can generate a call recommendation model in which the action / behavior classification information and the occurrence degree T are associated with each other.

  Also, a functional configuration example of the call recommendation model generation device 660 in which an operation / behavior classification model storage unit 651 and an operation / behavior cluster determination unit 652 are further added to the configuration of the call recommendation model generation device 620 described above is also conceivable. FIG. 23 shows a functional block diagram of the call recommendation model generation device 660.

  The call recommendation model generation device 660 determines an action or action class represented by the element sound class determined by the element sound cluster determination unit 622, and generates a call recommendation model using the determined action or action class. Is.

  The call recommendation model generation unit 654 associates the action / behavior classification information output by the action / behavior cluster determination unit 652 with the history address of the call history table, and the case where the call often occurs and the case where the call hardly occurs Generate a call recommendation model. Modeling is done in the same way as described above. The call recommendation model generation device 660 can also generate a call recommendation model in which the action / behavior classification information and the occurrence degree T are associated with each other.

  In addition, although the example using only an audio signal was demonstrated for the generation of the call recommendation model of the call recommendation model generation apparatus described in the tenth embodiment to the thirteenth embodiment, the audio / video signal acquisition unit 2012 has been described. The call recommendation model according to the ninth embodiment may be configured to generate a call recommendation model using, for example, the illuminance information, the temperature information, the acceleration information, or the like acquired by the video signal acquired by the other sensor. This is the same as the recommended model generation apparatus 600. Similarly, as the call recommendation model, only one of a model in which calls are frequently generated and a model in which calls are not frequently generated may be generated.

  As described above, the call recommendation model generation apparatus that generates the call recommendation model used for the purpose of knowing whether or not the communication partner in the remote place can communicate is described. Next, using the call recommendation model generation devices 600 to 660, a call suitability notification device that solves the problem of calling a receiver even though a remote communication partner is not in a call-capable state Will be described.

[14th embodiment]
FIG. 24 shows a functional block diagram of a communication system 2000 including the call suitability notification device 700 of the fourteenth embodiment. The functional configuration of the communication system 2000 is the same as that described above. The calling suitability notification device 700 is connected to the communication terminal 2010 on the receiving side, and notifies whether or not the receiving side is currently in a state where the call is possible using the call recommendation model generated by the call recommendation model generating device 600. To do.

  The call suitability notification device 700 includes the acoustic feature quantity calculation device 1, a call recommendation model storage unit 604, and a call recommendation status determination unit 701. The acoustic feature quantity calculation device 1 and the call recommendation model storage unit 604 are the same as the call recommendation model generation device 600.

  The call recommendation status determination unit 701 includes a feature amount extracted from the acoustic signal sequence output by the acoustic feature amount calculation device 1, a feature amount (vector) stored in the call recommendation model storage unit 604, and an occurrence degree T. A call suitability notification information for notifying the sending side of the suitability of the call is output by collating with the corresponding call recommendation model. Appropriateness of the call is determined by comparing the value T of the above-mentioned call occurrence T with a certain threshold value. For example, if the value of T is 10 or more, the call is suitable, and the value of T is -10 or less. If it is not suitable for outgoing calls.

  The call suitability notification information is input to the call terminal 2010 and transmitted to the call terminal 2030 on the transmitting side via the network 2020. The communication terminal 2030 on the transmission side that has received the call suitability notification information displays whether or not the receiver is in a state where a call is possible. The display is performed by calling recommendation information display means (not shown) such as lighting of an LED lamp or the like, or display of a liquid crystal panel.

  The output interval of the call suitability notification information related to the design matter is assumed to be performed at intervals of several seconds to several minutes, for example. The functions of the call suitability notification device 700 and the call recommendation information display unit may be configured integrally with the call terminals (2010, 2030). In addition, the method for identifying the communication terminal on the side of the caller notifying the call suitability notification information is not the main part of the present invention, so the description thereof will be omitted, but the call terminals of a plurality of callers are registered in advance. This can easily be realized.

[Fifteenth embodiment]
FIG. 25 is a functional block diagram of a communication system 2000 including the call suitability notification device 710 according to the fifteenth embodiment. The call adequacy notification device 710 is connected to the communication terminal 2010 on the receiving side, and notifies whether or not the callee is currently in a state where a call is possible using the call recommendation model generated by the call recommendation model generation device 610. To do.

  The call suitability notification device 710 includes an acoustic feature quantity calculation device 1, a specific element sound model storage unit 611, an element sound specification unit 612, a call recommendation model storage unit 613, a call recommendation status determination unit 711, It comprises. The acoustic feature quantity calculation device 1, the specific element sound model storage unit 611, the element sound specification unit 612, and the call recommendation model storage unit 613 are the same as the call recommendation model generation device 610.

  The call recommendation status determination unit 711 generates an element sound output from the element sound identification unit 612 and a call recommendation model in which a voice such as a footstep or a glass breaking sound is associated with an occurrence degree T of a call. The call suitability notification information for notifying the sending side of the suitability of the call by collating is output. The subsequent operation is the same as that of the call suitability notification device 700.

[Sixteenth embodiment]
FIG. 26 shows a functional block diagram of a communication system 2000 including the call suitability notification device 720 of the sixteenth embodiment. The call suitability notification device 720 is connected to the communication terminal 2010 on the receiving side, and notifies whether or not the receiver side is currently in a state where the call is possible using the call recommendation model generated by the call recommendation model generating device 620. To do.

  The call suitability notification device 720 includes an acoustic feature quantity calculation device 1, a specific element sound model database 621, an element sound cluster determination unit 622, a call recommendation model storage unit 623, a call recommendation status determination unit 721, It comprises. The acoustic feature quantity calculation device 1, the specific element sound model database 621, the element sound cluster determination unit 622, and the call recommendation model storage unit 623 are the same as the call recommendation model generation device 620.

  The call recommendation status determination unit 721 checks the suitability of the call by comparing the classification class output from the element sound cluster determination unit 622 with the call recommendation model in which the classification class is associated with the call generation degree T. The call suitability notification information to be notified to the transmitting side is output. The subsequent operation is the same as that of the call suitability notification device 700.

[Seventeenth embodiment]
The call suitability notification device can also be configured using a call recommendation model in which the degree T of occurrence of a call is associated with other information. For example, instead of the above-described feature quantity itself, the call / action information that can be estimated from the feature quantity, the action / behavior classification information that classifies the action / behavior information, and the call occurrence degree T are associated with each other. It may be a recommended model.

  FIG. 27 shows a functional block diagram of the call suitability notification device 730 in that case. The call suitability notification device 730 of the seventeenth embodiment includes the acoustic feature quantity calculation device 1, an association unit 732, a call recommendation model storage unit 633, and a call recommendation status determination unit 731. The acoustic feature quantity calculation device 1 divides a listener's acoustic signal sequence including a plurality of element sounds into short-time frames, and extracts feature quantities for each frame. The associating unit 732 outputs associating information in which the feature amount is associated with the behavior / behavior information on the listener side. The call recommendation model storage unit 633 stores a call recommendation model in which the action / behavior information specified from the feature amount is associated with the degree of ease of occurrence of a call. The call recommendation situation determination unit 731 receives the association information as input, refers to the call recommendation model that matches with the action / behavior information, or is a situation in which a call frequently occurs on the receiver side or a situation in which a call hardly occurs? Is transmitted to the caller side.

  The call suitability notification device 730 notifies whether or not the receiver side is currently in a callable state using the call recommendation model generated by the call recommendation model generation device 630.

  The call suitability notification device 730 includes the acoustic feature amount calculation device 1, an association unit 732, a call recommendation model storage unit 633, and a call recommendation status determination unit 731. The associating unit 732 includes a specific element sound model database 611, an element sound specifying unit 612, an action / behavior specifying model storage unit 631, and an action / behavior specifying unit 632. This configuration is the same as that of the call recommendation model generation device 630.

The call recommendation situation determination unit 731 determines the feature amount output by the associating unit 732 and the operation / value on the receiver side.
Using association information associated with action information as input, refer to the call recommendation model that matches with the action / behavior information to determine whether the call is likely to occur on the receiver side or is not likely to occur. The call success / failure notification information is transmitted to the caller.

  FIG. 28 is a functional block diagram of the call suitability notification device 740. The call suitability notification device 740 uses the call recommendation model generated by the call recommendation model generation device 640 to notify whether or not the receiver side is currently in a callable state.

  The call suitability notification device 740 includes the acoustic feature quantity calculation device 1, an association unit 742, a call recommendation model storage unit 633, and a call recommendation status determination unit 741. The associating unit 742 includes a specific element sound model database 621, an element sound cluster determining unit 622, an action / behavior specifying model storage unit 631, and an action / behavior specifying unit 632. This configuration is the same as that of the call recommendation model generation device 630.

The call recommendation situation determination unit 741 receives associating information that associates the feature amount output by the associating unit 742 with the action / behavior information, and refers to the call recommendation model that matches the action / behavior information. Call adequacy notification information is transmitted to the caller's side, which determines whether the call is frequently occurring on the receiver side or is not likely to occur.

[Eighteenth embodiment]
FIG. 29 shows a functional block diagram of the call suitability notification device 750 of the eighteenth embodiment. The call adequacy notification device 750 notifies whether or not the receiver side is currently in a callable state using the call recommendation model generated by the call recommendation model generation device 650.

  The call suitability notification device 750 includes the acoustic feature quantity calculation device 1, an association unit 752, a call recommendation model storage unit 653, and a call recommendation status determination unit 751. The associating unit 752 includes a specific element sound model database 611, an element sound specifying unit 612, an action / behavior classification model storage unit 651, and an action / behavior cluster specifying unit 652. This configuration is the same as that of the call recommendation model generation device 650.

The call recommendation status determination unit 751 receives the feature amount output from the association unit 752 and the action / behavior classification information as input, and refers to the call recommendation model that matches the action / behavior classification information, so that the call on the receiver side is good. Call adequacy notification information that determines whether the situation occurs or the situation in which a call does not occur so much is transmitted to the caller side.

  FIG. 30 shows a functional block diagram of the call suitability notification device 760. The call suitability notification device 760 uses the call recommendation model generated by the call recommendation model generation device 660 to notify whether or not the receiver side is currently in a callable state.

  The call suitability notification device 760 includes the acoustic feature quantity calculation device 1, an association unit 762, a call recommendation model storage unit 653, and a call recommendation status determination unit 761. The association unit 762 includes a specific element sound model database 621, an element sound cluster determination unit 622, an action / behavior classification model storage unit 651, and an action / behavior cluster determination unit 652. This configuration is the same as that of the call recommendation model generation device 660.

The call recommendation state determination unit 761 refers to the call recommendation model that matches the action / behavior classification information by using the association information that associates the feature amount output by the association unit 762 with the action / behavior classification information as an input. Thus, call adequacy notification information is transmitted to the caller side for determining whether the call is frequently generated on the receiver side or whether the call hardly occurs.

  As described above, according to the calling suitability notification devices 700 to 760 of the present invention, the listener can use the feature amount, element sound, classification class, operation / behavior information, or operation / behavior classification information on the receiver side. It is possible to determine whether or not the telephone is currently in a callable state and notify the determination result (calling suitability notification information) to the transmitter side.

  The processes described in the above apparatuses and methods are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. .

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (12)

A frame dividing unit that divides the input acoustic signal into frames of a predetermined time length;
The acoustic signal of each frame is divided into K sections, p ^- _k is an average value of the index indicating the magnitude of the acoustic signal of the kth section of each frame, and Δp ^- _k is the _kth of each frame. The value defined by the following equation is calculated with the rate of change of p ^- _k in the interval of m, m being a predetermined integer of 2 or more, and if the value is 0 or more, the value is calculated as the rise characteristic of each frame. and gender, and the feature amount extraction unit if the value is less than 0 with the rising characteristic calculation unit to 0 the rising characteristics of each frame,

An acoustic feature quantity calculation device including In the acoustic feature amount calculation apparatus according to claim 1,
The suddenness calculation unit calculates the suddenness indicating the degree of concentration in the time domain of the acoustic signal of each frame.
The time diffusivity calculation unit calculates time diffusivity indicating the degree of diffusion in the time domain of the acoustic signal of each frame.
The narrowband calculation unit calculates the narrowbandness indicating the degree of concentration in the frequency domain of the acoustic signal of each frame,
The band spread calculation unit calculates the band spread indicating the degree of spread in the frequency domain of the acoustic signal of each frame.
The pitch characteristic calculation unit calculates a pitch characteristic indicating the degree of uneven distribution of energy in the frequency domain of the acoustic signal of each frame,
The amplitude unevenness calculation unit calculates the amplitude unevenness indicating the degree of uneven distribution of the amplitude value distribution of the acoustic signal of each frame,
The feature amount extraction unit includes the suddenness calculation unit, the time diffusivity calculation unit, the narrowband calculation unit, the band spreadability calculation unit, the pitch characteristic calculation unit, and the amplitude unevenness. And further comprising at least one of a calculation unit,
Acoustic feature quantity calculation device. The acoustic feature quantity calculation device according to claim 1 or 2,
A specific element sound model database storing specific element sound models of a plurality of specific element sounds;
The feature element calculated by the acoustic feature quantity calculation device and the specific element sound model stored in the specific element sound model database are compared, and the label of the specific element sound model of the most similar model, or the specific element sound An element sound model comparison unit that outputs a labeled acoustic signal sequence in which a label of the model is assigned to the acoustic signal sequence;
The frequency of appearance for each label of the specific element sound model in a histogram frame in which a predetermined number of the frames are collected with the label of the specific element sound model or the labeled acoustic signal sequence output from the element sound model comparison unit as an input. An element sound histogram generator for creating an element sound histogram;
Using the element sound histogram as an input, a specific situation modeling unit that generates a specific situation model corresponding to the specific field using a modeling method for the element sound histogram;
A specific situation model database creation device comprising: The acoustic feature quantity calculation device according to claim 1 or 2,
A specific element sound model database storing specific element sound models of a plurality of specific element sounds;
The feature element calculated by the acoustic feature quantity calculation device and the specific element sound model stored in the specific element sound model database are compared, and the label of the specific element sound model of the most similar model, or the specific element sound An element sound model comparison unit that outputs a labeled acoustic signal sequence in which a label of the model is assigned to the acoustic signal sequence;
The frequency of appearance for each label of the specific element sound model in a histogram frame in which a predetermined number of the frames are collected with the label of the specific element sound model or the labeled acoustic signal sequence output from the element sound model comparison unit as an input. An element sound histogram generator for creating an element sound histogram;
A distribution clustering processing unit for creating an element sound classification obtained by classifying the plurality of element sound histograms according to the shape of the distribution;
With the element sound classification as an input, a situation classification modeling unit that generates a situation classification model using a modeling method for the element sound classification;
A specific situation model database creation device comprising: A specific element sound model database creation device for creating the specific element sound model database of the specific situation model database creation device according to claim 3 or 4,
The acoustic feature quantity calculation device according to claim 1 or 2,
A specific element sound modeling unit that generates a specific element sound model using a modeling method for the feature amount, using the feature amount calculated by the acoustic feature amount calculation device;
A specific element sound model database creation device comprising: A specific element sound model database creation device for creating the specific element sound model database of the specific situation model database creation device according to claim 3 or 4,
The acoustic feature quantity calculation device according to claim 1 or 2,
A feature amount clustering unit that classifies the feature amounts calculated by the acoustic feature amount calculation device and creates a feature amount classification;
An element sound classification modeling unit that generates the element sound classification model using the modeling method for the feature quantity classification, using the feature quantity classification as an input;
A specific element sound model database creation device comprising: The acoustic feature quantity calculation device according to claim 1 or 2,
A specific element sound model database storing the specific element sound model generated by the specific element sound model database creation device according to claim 5;
An element that compares the specific element sound model with the feature amount calculated by the acoustic feature amount calculation device, determines the closest one as the element sound of each short-time acoustic signal, and assigns an element sound label for each frame The sound model comparison section,
Using the labeled acoustic signal sequence as an input, an element sound histogram generating unit that creates an element sound histogram of the label of the specific element sound model and its frequency,
A specific situation model database storing a plurality of specific situation models and situation classification models generated by the specific situation model database creation device according to claim 3;
A situation judgment model comparison that compares the element sound histogram with the specific situation model or the situation classification model, estimates the most similar one as the situation represented by the specific situation model or the situation classification model, and outputs a situation estimation result And
A situation estimation apparatus comprising: The acoustic feature quantity calculation device according to claim 1 or 2,
A call recommendation model storage unit that stores a call recommendation model that correlates the degree of ease of occurrence of a call;
And inputs the feature quantity the acoustic feature quantity calculation device has calculated, or situations where circumstances to or call the feature amount is the call recommending call model with reference to the receiving speaker side may occur a match is not much generated A call adequacy notification device comprising: a call recommendation status determination unit that determines call adequacy notification information and transmits the call adequacy notification information to the caller side. A frame dividing step in which the frame dividing unit divides the input acoustic signal into frames of a predetermined time length;
The feature amount extraction unit divides the acoustic signal of each frame into K sections, and p ^- _k is an average value of an index representing the magnitude of the acoustic signal of the k-th section of each frame, and Δp ^- _k is The value defined by the following equation is calculated with the change rate of p ⁻ _k in the k-th section of each frame, m being a predetermined integer of 2 or more, and if the value is 0 or more, the value is and the rising characteristics of each frame, a feature amount extracting step comprises a rising characteristic calculation unit to 0 the rising characteristics of the respective frame if its value is less than 0,

A method for calculating acoustic features including Program for causing a computer to function as the acoustic feature quantity calculation apparatus according to claim 1 or 2.   The program for functioning a computer as a condition estimation apparatus of Claim 7.   A program for causing a computer to function as the call suitability notification device according to claim 8.

Images