JP5278952B2 - Infant emotion diagnosis apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a diagnostic device of infants' feeling and a method to perform diagnosis of feeling by analyzing interaction between infants' feeling and mother's feeling. <P>SOLUTION: A pitch period of voice data is computed by transmitting the voice data to a sound processing section 12 from an air conduction microphone 1 to which voice of an infant B is input and a bone conduction microphone 2 to which voice of a mother M is input through a voice acquisition section 10. A body section 20 includes: a feeling evaluation section 21 in which fluctuation components of the pitch period are computed and evaluated as feeling evaluation data of the infant B and the mother M; a feeling diagnosis section 22 in which causal analysis and inference is performed about infant B's feeling based on the computed feeling evaluation data of the infant B and the mother M; and a childcare support section 23 in which childcare support for mother based on a diagnostic result of infant B's feeling is performed. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to an infant's emotion that determines an emotion of the infant and the mother based on the voice analysis of the infant and the mother, and analyzes an interaction between the infant's emotion and the mother's emotion to perform emotion diagnosis. The present invention relates to a diagnostic apparatus and method.

  Infants who cannot express their emotions in words like babies generally reveal their emotions by crying, facial expressions and actions. However, there is a problem that a person who is engaged in childcare such as a mother cannot grasp the infant's emotions well and cannot take an appropriate response to the infant.

  In response to these issues, it is possible to analyze infant voices such as crying voices and determine the emotions of the infants by matching the analyzed patterns with standard patterns for emotions such as hunger, boredom, discomfort, sleepiness, and stress. It has been broken.

  For example, in Patent Document 1, a baby's voice is sampled with a microphone, a voice analysis is performed and matching processing with a reference voice pattern is performed to determine a baby's emotion, and the determined emotion corresponds to a preset emotion to be relaxed If you want to do this, you can output a sound with a predetermined sound pattern from the speaker, operate the electric toy with a remote control to drive it with a predetermined operation pattern, or operate the light with a remote control to light it with a predetermined lighting pattern. The point to perform is described. Further, in Patent Document 2, the situation analysis unit estimates the infant's psychological state based on the infant's image, sound, and biological information, and the cause estimation unit based on the estimated psychological state causes past meal and diaper exchange situation data. The point which estimates the cause based on etc. is described.

JP 2005-185630 A JP 2006-127057 A

  In the above-mentioned prior literature, the voice of an infant is analyzed to determine its emotion, but in reality there are various factors involved in the cause of the infant's emotion, and the correlation between the cry and emotion is statistically correlated. Even after analysis, it is difficult to identify the cause of the determined emotion. In Patent Document 2, the cause of the psychological state is estimated by analyzing whether or not it corresponds to a plurality of causes specified in advance (meal, diaper exchange, physical condition, etc.). Multiple causes such as the environment and physical condition of each other overlap, and it is not realistic to narrow down to a specific cause.

  When analyzing the causes of infant emotions, one of the most important factors in the surrounding environment is the presence of the mother. In general, since an infant grows up by raising a mother, the relationship with the mother is considered to be important for the infant, but the prior literature does not consider the relationship with the mother.

  Therefore, the present invention focuses on the relationship with the mother, which is important for infants, and provides an infant emotion diagnosis apparatus and method for analyzing emotions by analyzing the interaction between the infant's emotions and the mother's emotions. It is intended to provide.

An infant emotion diagnosis apparatus according to the present invention includes a voice acquisition unit that acquires voices of infants and mothers, and an emotion evaluation unit that calculates emotion evaluation data based on the acquired voice data of infants and mothers. An emotion diagnosis means for diagnosing an infant's emotion by analyzing the degree of the influence of the interaction between the infant and the mother based on the correlation in the time series data of the emotion evaluation data regarding the infant and the mother calculated It is characterized by having.

The infant emotion diagnosis method according to the present invention acquires the respective voices of the infant and mother, calculates respective emotion evaluation data based on the acquired infant and mother voice data, and relates to the calculated infant and mother. It is characterized in that an infant's emotion diagnosis is performed by analyzing the degree of the influence of the interaction between the infant and the mother based on the correlation in the time series data of the emotion evaluation data .

  In an environment where infants can recognize their mothers, both emotional evaluation data is calculated based on the voices of the infants and mothers, and the emotional evaluation is performed in consideration of the influence of the mother's emotions. Diagnosis can be made, and emotion diagnosis with higher accuracy can be performed.

  In addition, the actual emotional interaction between infants and mothers will be materialized from the results of diagnosis of infants' emotions, and point out how emotional expressions in childcare will affect infants and mothers. Can provide accurate support in line with the actual situation of childcare.

  And the precision of emotion diagnosis can be improved by calculating emotion evaluation data based on the pitch period and / or amplitude of voice data. Here, the pitch period is a period of a waveform that repeatedly appears in the waveform of audio data, and means a certain period corresponding to the vibration period of the vocal cords. Amplitude means sound pressure, and is represented by the magnitude of the vertical axis, which is sound pressure, in the waveform of normal voice data.

It is a schematic block diagram regarding embodiment which concerns on this invention. It is a graph which shows an example of the audio | voice waveform of the acquired infant's audio | voice. It is a graph which shows the pitch period of the audio | voice data shown in FIG. 2 in time series. It is a graph which shows the fluctuation component of the pitch period shown in FIG. 3 in a time series. It is a graph which shows typically time transition of the fluctuation component of an infant and a mother. It is a graph regarding the speech waveform which carried out the speech synthesis | combination by the 2 mass vibration model based on the pitch period shown in FIG. It is a processing flow regarding emotion diagnosis in the main body.

  Hereinafter, embodiments according to the present invention will be described in detail. The embodiments described below are preferable specific examples for carrying out the present invention, and thus various technical limitations are made. However, the present invention is particularly limited in the following description. Unless otherwise specified, the present invention is not limited to these forms.

  FIG. 1 is a schematic configuration diagram relating to an embodiment of the present invention. In an environment where the infant B can recognize the mother M (for example, in the home), the infant B is attached with the air conduction microphone 1 for inputting voice, and the mother is attached with the bone conduction microphone 2 for inputting voice. ing. By attaching a bone-conduction microphone to the mother, noise can be reduced when the mother's voice is input, and if the bone-conduction microphone is attached to the head, it can be an obstacle when the mother performs household chores. It can be in the normal state without becoming.

  The voice of the infant B input from the air conduction microphone 1 is transmitted to the voice acquisition unit 10 and stored in the voice DB 11. Then, the voice of the infant B is transmitted from the voice acquisition unit 10 to the acoustic processing unit 12, and processing for calculating the pitch period from the voice of the infant B is performed. The pitch period samples audio data for each predetermined time interval, and calculates a pitch period for each sampling interval by a known method such as an autocorrelation function. The pitch period calculated in this way is stored in the processing DB 13 in association with the time series.

  FIG. 2 is a graph showing an example of the voice waveform of the acquired baby B voice. The vertical axis represents amplitude and the horizontal axis represents time. FIG. 3 is a graph showing, in time series, results obtained by sampling the voice waveform shown in FIG. 2 every 24 milliseconds and calculating the pitch period. The vertical axis represents the pitch period (indicated by frequency in this graph), and the horizontal axis represents time.

  The sound of the mother M input from the bone-conduction microphone 2 is also transmitted to the sound acquisition unit 10 and stored in the sound DB 11 as in the case of the infant B, and the pitch period is calculated in the sound processing unit 12 and stored in the process DB 13. The pitch periods are stored in time series.

  The main body 20 calculates an evaluation component by calculating a fluctuation component of the pitch cycle as emotion evaluation data of the infant B and the mother M, and the infant B and the mother M based on the calculated emotion evaluation data of the infant B and the mother M. An emotion diagnosis unit 22 that performs cause analysis and inference of emotions, and a childcare support unit 23 that provides childcare support to the mother based on the emotion diagnosis results of the infant B are provided. The main body 20 is connected to a memory 24. The memory 24 stores an evaluation DB 25 for storing emotion evaluation data in time series, causes analysis results and diagnosis results on infant emotions, and various cases. A diagnosis DB 26 for storing diagnosis knowledge about the child and a support DB 27 for storing a childcare support message corresponding to the diagnosis result are stored.

  An image display unit 30 and an audio output unit 31 are attached to the main body unit 20. The image display unit 30 displays audio data, analysis data, and the like, and the audio output unit 31 outputs synthesized audio and the like described later. To do.

  The main body 20 can be configured using a known computer system. The processing performed by the emotion evaluation unit 21, the emotion diagnosis unit 22, and the childcare support unit 23 reads a program stored in the memory 24 and is processed by a processing device such as a CPU. And as the image display part 30 and the audio | voice output part 31, it can comprise with the well-known liquid crystal panel display and speaker connected to a computer system.

The emotion evaluation unit 21 calculates a pitch cycle fluctuation component PPQ1 used as emotion evaluation data by the following equation. This evaluation formula shows how much the pitch period varies with respect to the average pitch period. Here, N is the number of samplings for which the pitch period is calculated, and p _i is the pitch period calculated in each sampling section.

図４は、図３に示すピッチ周期に基づいて算出したゆらぎ成分を時系列で示したグラフである。縦軸にゆらぎ成分をとり、横軸に時間をとっている。このグラフをみると、ゆらぎ成分は、所定の基準値に対して振動するように変動することがわかる。

FIG. 4 is a graph showing fluctuation components calculated based on the pitch period shown in FIG. 3 in time series. The vertical axis represents the fluctuation component, and the horizontal axis represents time. From this graph, it can be seen that the fluctuation component fluctuates so as to vibrate with respect to a predetermined reference value.

  The present inventors have demonstrated that such fluctuations in fluctuation components correlate with emotions such as infant joy and sadness (C. Araki et.al. "An experimental system to analyze or synthesize baby's emotional voice using the varidation of pitch frequencies ", ISUC.2008, 2008.12, IEEE). Therefore, the emotion evaluation unit 21 calculates the fluctuation component of the pitch period of the voices of the infant B and the mother M, and quantifies the change in the emotion of both. The quantified evaluation data is stored in the evaluation DB 25 in association with the time series.

  As a method of quantifying the calculated fluctuation component, for example, the fluctuation range of the fluctuation component of the pitch period is set to 100, and the threshold value is set to 45. The threshold value may be determined by statistically analyzing the collected data. When it is less than 45, it is determined that the emotion is in a “sadness” state, and when it is greater than 45, it is determined that the emotion is in a “joy” state. It is determined that the closer to 100, the stronger the “joy” state, and the closer to 0, the stronger the “sadness” state.

  The emotion diagnosis unit 22 diagnoses the emotion of the infant B from the transition of the emotions of the infant B and the mother M evaluated by the emotion evaluation unit 21 based on the fluctuation component. When performing emotion diagnosis of an infant, the emotion diagnosis is performed by looking at the temporal transition of the emotion evaluation data and paying attention to the correlation with the emotion of the mother M.

  For example, it is determined whether the infant B is in “joy” or “sadness” state, and when it is determined that the determined state is “sadness” state, for example, within a predetermined time before that (for example, 1 to 10 minutes), it is investigated whether or not the mother M is determined to be in the state of “sadness”. When the state of “sadness” of infant B has a strong correlation with the state of “sadness” of mother M before that, the cause of the state of “sadness” of infant B is It is analyzed that it is due to the influence of interaction. Even if it is determined that the infant B is in the “joy” state, the correlation with the “joy” state of the mother M is similarly investigated, and the degree of the influence of the interaction with the mother is analyzed. The

FIG. 5 is a graph schematically showing temporal transitions regarding fluctuation components of the infant B and the mother M. In this example, the threshold is set to 45, and if it is 45 or less, it is determined as a “sadness” state, and if it exceeds 45, it is determined as a “joy” state. The emotion of baby B (displayed with a dotted line) is in a state of “sadness” from time t ₁ to time t ₂ , and thereafter in a state of “joy” from time t ₃ to time t ₄ . On the other hand, the emotion of the mother M (displayed with a solid line) is in the same emotional state immediately before the infant B's emotion is exposed, and it can be seen that there is a strong correlation between the two. Therefore, it can be said that the undulations of the mother's emotions have a strong influence on the infant's emotions, and it is analyzed that the mother's emotions are one of the main causes of the infant's emotions. Then, the cause of the infant's emotion is analyzed by analyzing the correlation between the infant's emotion and the factors such as the magnitude of the emotional undulation of the mother, the time interval of the emotional variation, and the tendency of daily emotional variation.

  In addition, in the state of “sadness” when the infant B is hungry or changing diapers, the correlation between the mother M and the infant B is small even if the mother M is “joyful”. With regard to the relationship between the situation and emotional interaction, a more accurate cause analysis can be performed by recording the situation of both and comparing the recorded situation with the cause of the infant's emotion It can be performed.

  The cause analysis results regarding the infant's emotion as described above are accumulated in the diagnosis DB 26. The diagnosis DB 26 stores cause knowledge about past cases and diagnosis knowledge based on diagnosis results, and inference is performed based on cause analysis of infant emotions using such diagnosis knowledge.

  For example, if a mother “has something sad and kept feeling sad”, an infant “feels that feeling through communication with the mother and feels sad”, conversely, If the mother “feels very happy, smiles and thinks the baby is cute,” the baby will “sad” for the mother ’s “sadness”, such as “the child will be happy”. The hypothesis is that the mother's “joy” should be returned with “joy” as a hypothesis. Based on this hypothesis, the infant's emotions are inferred and diagnosed.

  The diagnosis DB 26 analyzes and patterns how the infant and mother's emotions interact in various cases, and stores emotion patterns related to the various cases. For example, emotional patterns when the mother's mental condition has deteriorated (I want to die anymore, this child is not cute, I don't need a child, etc.), or a good mental state (for this child Quantify various emotional patterns, such as emotional patterns in the case of anything you want to do-etc.) into a database.

  Then, by performing a matching analysis on the similarity relationship between the case to be diagnosed and the emotion pattern, it is possible to perform a more specific and accurate diagnosis.

  The childcare support unit 23 creates childcare support information for the mother based on the diagnosis result regarding the emotion of the infant in the emotion diagnosis unit 22 and displays the information on the image display unit 30. For example, when it is diagnosed that the relationship between the mother and the child is in a good state, the message “Please contact the baby in the current condition” is displayed as a message to the mother. On the other hand, if the infant's emotion is judged to be “sad” and the mother-child relationship is diagnosed, the message to the mother says “Mom ’s emotional state is not so good. Please touch your baby. Please remember and imagine the fun so that it will be brighter. "

  Thus, the message corresponding to the diagnosis result is patterned and stored in the support DB 27 in advance, and the corresponding message is read and displayed according to the diagnosis result.

  The speech synthesizer 14 synthesizes speech based on the pitch period calculated by the acoustic processing unit 12. The synthesized voice is stored in the synthesized voice DB 15.

  Regarding the technology for synthesizing speech from the pitch period, the present inventors have demonstrated that speech synthesis reflecting actual emotions can be performed using a two-mass vibration model of the vocal cords (the above-described present inventors See the paper). A one-mass vibration model may be used as the vocal fold vibration model to be used. In particular, in the case of an infant, synthesized speech that is closer to actual speech can be obtained by using the one-mass vibration model. FIG. 6 is a graph relating to a speech waveform obtained by speech synthesis using a two-mass vibration model based on the pitch period shown in FIG.

  The voice data synthesized in the voice synthesizing unit 14 is transmitted to the main body unit 20 and compared with the actual voice data from the voice acquisition unit 10 to determine how close the synthesized voice is to the actual voice in the emotion evaluation unit 21. Validated. In addition, the childcare support unit 23 outputs a synthesized voice in the “joy” and “sadness” states of the infant from the voice output unit 31 to let the mother hear a voice corresponding to the emotion of the infant, It can also be used for childcare support, such as for the infant to hear a synthesized voice of the mother's “joy” state.

  FIG. 7 is a processing flow related to emotion diagnosis in the main unit 20. First, the fluctuation component of the pitch period is calculated as described above based on the time series data of the pitch period of the infant and mother input from the acoustic processing unit 12 to obtain time series data of the fluctuation component (S100). Based on the time-series data of fluctuation components, the infant and mother are judged to be in “joy” or “sadness” based on a predetermined threshold, and the correlation is based on the temporal transition of the determined emotions. The relationship is analyzed (S101).

  The cause analysis of the infant's emotion is performed based on the correlation between both emotions, and the mother-child relationship is diagnosed by inference (S102). Then, a childcare support message corresponding to the diagnosis result is selected and read (S103), and the read childcare support message is displayed on the display unit (S104).

1 Air conduction microphone 2 Bone conduction microphone
10 Voice acquisition unit
11 Voice DB
12 Sound processor
13 Processing DB
14 Speech synthesis unit
15 Synthetic voice DB
20 Main unit
21 Emotion Evaluation Department
22 Emotion diagnosis department
23 Childcare support department
24 memory
25 Evaluation DB
26 Diagnosis DB
27 Support DB
30 Display section
31 Audio output section

Claims (3)

Voice acquisition means for acquiring voices of infants and mothers, emotion evaluation means for calculating emotion evaluation data based on the acquired voice data of infants and mothers, and emotion evaluation data regarding the calculated infants and mothers Emotions of infants characterized by having emotion diagnosis means for analyzing emotions of infants by analyzing the degree of interaction between infants and mothers based on the correlation in time series Diagnostic device. Acquire voices of infants and mothers, calculate emotional evaluation data based on the acquired voice data of infants and mothers, and calculate correlations in time-series data of emotional evaluation data about infants and mothers. A method for diagnosing an infant's emotion, characterized in that the infant's emotion is diagnosed by analyzing the degree of influence of the interaction between the infant and the mother. A program for causing a computer to function as emotion evaluation means and emotion diagnosis means according to claim 1.

Images