JP5714411B2 - Behavior analysis method and behavior analysis device - Google Patents

Description

  The present invention relates to a behavior analysis method and a behavior analysis apparatus for predicting a behavior by analyzing the emotion and spirit of a subject.

  In recent years, technology development for recognizing a subject's emotion based on the utterance of the subject has been actively performed (for example, Patent Document 1), and such voice emotion recognition technology has been developed together with a robot and a video game, It is applied to lie detectors.

Japanese Patent No. 3676969

  However, in the prior art, since the subject's emotion is analyzed based on an index that is a criterion for emotion created based on the subjectivity of the person, the subject is subject to shifts and fluctuations in subjectivity such as doctors and subjects used for learning. It is very difficult to identify the subject's emotions, psychology, and real intentions with the accuracy required for the medical level.

  It is also difficult to know a person's heart only from physiological indices.

  In view of the above-described problems of the prior art, an object of the present invention is to provide a technique that can analyze a subject's emotional state and mental state and predict a subject's behavior with high accuracy.

An aspect of the behavior analysis method illustrating the present invention includes an input procedure for acquiring a subject's utterance as a speech signal, a calculation procedure for calculating a plurality of speech feature amounts from the speech signal, a plurality of speech feature amounts, and a human physiology. Based on the first table that associates a plurality of physiological information indicating activity with each other, the feature amount of each physiological information in the subject is determined from the calculated plurality of voice feature amounts, and the plurality of voice feature amounts and each physiological information based on the second table associated with a plurality of emotional state of the features and the human, to determine the emotional state in a subject and a feature amount of each physiological information determined with a plurality of speech feature amounts calculated, determined A prediction procedure for predicting the behavior of the subject based on the emotional state .

An aspect of the behavior analysis apparatus illustrating the present invention includes an input unit that acquires a speech of a subject as a voice signal, a calculation unit that calculates a plurality of voice feature amounts from the voice signal, a plurality of voice feature amounts, and human physiology. Based on the first table that associates a plurality of physiological information indicating activity with each other, the feature amount of each physiological information in the subject is determined from the calculated plurality of voice feature amounts, and the plurality of voice feature amounts and each physiological information based on the second table associated with a plurality of emotional state of the features and the human, to determine the emotional state in a subject and a feature amount of each physiological information determined with a plurality of speech feature amounts calculated, determined A predicting unit that predicts the behavior of the subject based on the emotional state .

  According to the present invention, the emotional state and the mental state of the subject can be analyzed, and the behavior of the subject can be predicted with high accuracy.

The block diagram which shows the structure of the action analysis apparatus which concerns on 1st Embodiment. The figure which shows an example of the user interface for performing identity verification and others evaluation The figure which shows an example of the time schedule of voice and physiological information measurement The figure which shows an example of the judgment tree of emotion class determination based on a heart rate and an audio | voice parameter The figure which shows an example of the judgment tree of the emotion class determination based only on a voice parameter The flowchart explaining operation | movement of the behavior analysis apparatus which concerns on 1st Embodiment. The figure which shows an example of the graph of a built-in influence degree and a body reaction influence degree The figure which shows an example of a template (emotional physiological model) The flowchart explaining operation | movement of the behavior analysis apparatus which concerns on 2nd Embodiment. The figure which shows the typical example of the template in each basic mental state A figure showing an example of a display for a healthy person The figure which shows an example in the case of treatment of a person with a personality disorder designated by the court The figure which shows an example before treatment of the court designation personality person with a disability of FIG. The figure which shows an example in the case of indicating a bipolar 2 type | mold obstacle degree numerically The flowchart explaining operation | movement of the action analysis apparatus which concerns on 3rd Embodiment.

<< First Embodiment >>
FIG. 1 is a block diagram showing a configuration of a behavior analysis apparatus 100 according to the first embodiment of the present invention.

  A behavior analysis apparatus 100 illustrated in FIG. 1 includes a computer 1, a microphone 2, a digital camera 3, and a physiological measurement apparatus 4. Thereby, the behavior analysis apparatus 100 of this embodiment performs a mental analysis, behavior prediction, and analysis. In addition, the behavior analysis apparatus 100 analyzes and displays the real sound and the like by an analysis method based on a physiological index (especially, a voice parameter affected by the cranial nervous system).

  The computer 1 includes a CPU 10, a storage unit 11, a memory 12, an input / output interface (input / output I / F) 13, and a bus 14. The CPU 10, the storage unit 11, the memory 12, and the input / output I / F 13 are connected via a bus 14 so that information can be transmitted. Further, the computer 1 is connected to a display device 15 for displaying the progress of audio signal processing and processing results and an input device 16 for receiving input from the user via an input / output I / F 13.

  The CPU 10 is a processor that comprehensively controls the operation of each unit of the computer 1. As will be described later, the CPU 10 executes a behavior analysis program, thereby receiving voice signals of a subject or a doctor received by the microphone 2 via the input / output I / F 13 and a subject measured by the physiological measurement device 4. It operates as a prediction unit that performs analysis of the emotional state of the subject, psychoanalysis, and behavior prediction using the physiological information. Specifically, the CPU 10 uses a plurality of voice parameter values calculated from the voice signal of the subject and the measured physiological information, and a template (or physiological emotion model) that is a model showing a relationship between the physiology and the mind described later. Based on this, the subject's emotional state is analyzed to predict behavior. The CPU 10 displays the results of emotion analysis, mental analysis, and behavior prediction of the subject on the display device 15. A general central processing unit can be used for the CPU 10.

  The storage unit 11 includes the voice data of the speech signal of the subject received by the microphone 2 and the speech signal of the conversation with the doctor, the physiological information data of the subject measured by the physiological measurement device 4, and the CPU 10 based on the data. Emotional state, mental analysis, behavior prediction program for predicting behavior, program for analyzing mental state and true intention, etc. In addition, as will be described later, the storage unit 11 includes a subjective emotion table (speech feature table) necessary for the CPU 10 to perform behavior prediction, and a parameter that lists voice parameters affected by various physiological information for each physiological information. Tables and various decision trees (logic tables) generated based on these tables are stored. Data, programs, tables, decision trees, and the like stored in the storage unit 11 can be appropriately referred from the CPU 10 via the bus 14. A storage device such as a general hard disk device or magneto-optical disk device can be selected and used for the storage unit 11.

  The memory 12 temporarily stores audio data, physiological information, various tables, judgment trees, and other data processed by the CPU 10. As the memory 12, a semiconductor memory such as a volatile memory can be appropriately selected and used.

  The display device 15 displays information on the subject and the like, and also displays analysis results such as the subject's emotional state, behavior prediction, mental state, physiological state, and real intention. A general liquid crystal monitor or the like can be used for the display device 15.

  The input device 16 is a device that receives a request, an inquiry instruction, a diagnosis result, or the like from a doctor or a subject who is a user. As the input device 16, a keyboard, a mouse, or the like can be selected and used as appropriate.

The microphone 2 converts the voice signal which is an electrical signal by receiving sound such as subjects, and outputs to the computer 1. The computer 1 receives the audio signal from the microphone 2 and records it in the storage unit 11. The CPU 10 calculates the values of a plurality of sound parameters (such as F0) (speech feature amount) from the sound data of the subject acquired by the microphone 2 as a calculation unit using a known analysis method such as FFT or wavelet. . The CPU 10 detects vocal cords and voice features (related to voice quality) directly connected to the emotional control part (limbic system, hypothalamus, etc.) of the brain using the voice parameters and various physiological information of the subject to be described later. To do. In addition, the microphone 2 operates as a speaker that outputs a recorded audio signal as a sound wave, for example, when reproducing audio.

  The digital camera 3 captures the face of the subject and transmits the captured image data to the computer 1 via the input / output I / F 13. The computer 1 stores the received image data in the storage unit 11. The CPU 10 displays the image data on the display device 15, causes the subject himself or her doctor to observe changes in the facial expression of the subject, receives the observation result via the input device 16, and records it in the storage unit 11 or the memory 12. Alternatively, the CPU 10 may read the image data and use a known method to recognize the face and obtain the psychological attribute of the subject's facial expression. In the present embodiment, this expression change, for example, complimenting the face, eye movement such as blinking or pupil change, neck tilt, etc. is also used as one of the physiological information. This is because symptoms such as mental disorders appear in the expression.

  The physiological measurement device 4 measures a subject and acquires a plurality of physiological information. The physiological measurement device 4 is an MRI, PET, sphygmomanometer, thermometer, etc., and is appropriately selected and used. The physiological measurement device 4 transmits the acquired physiological information to the computer 1 via the input / output I / F 13. The computer 1 stores the received physiological information in the storage unit 11. The physiological information measured by the physiological measurement apparatus 4 of the present invention is exhaled breath, brain activity, neural activity, receptor, psychotropic drug administration, visceral information, eyeball activity, heartbeat electrocardiogram, built-in, and physical activity.

  Here, the physiological information measured based on exhalation includes an off switch, timing, length, amount, and deep breathing. Since there are research reports that the relationship between breathing and emotion is deeply related to off-switching, it is based on the known off-switch model of breathing (for example, even if breathing is intentionally stopped, it will resume someday) Compare the off switch with the nature of the emotion (for example, you can't be angry forever). Timing is measured because the number of breaths is also influenced by mental factors. In addition, timing is necessary to measure sighs and the like. The length needs to know how much the respiratory state is, and measures the length and depth of the breath. The quantity needs to know the quality of breathing and measures the breathing volume. Deep breathing measures the deep breath and creates an off-model of the emotion due to the deep breath from the human characteristic that when you take a deep breath, you cannot recall negative emotions during that time.

  Physiological information of brain activity refers to brain activity measured by brain waves, MRI, etc. of the emotional control part (limbic system, hypothalamus, mirror reaction, etc.) of the brain during emotional speech. The measured brain activity is compared with the emotional response of the voice by the microphone 2 and other physiological information. Also, it is necessary to know the emotional activity when there is no utterance, and in this embodiment, the electroencephalogram and brain activity in that case are also measured. When measuring with MRI or the like, a microphone equipped with a non-magnetic microphone in a soundproof mask and a soundproof hose that supplies fresh air so that natural breathing can be used to reduce noise. To implement.

  Physiological information of nerve activity refers to the state of sympathetic nerves and parasympathetic nerves and various nerve states measured using a general known method. In the present embodiment, the measured physiological information of the neural activity is compared with the emotional response of the voice by the microphone 2 and other physiological information. It should be noted that physiological information of neural activity can also be measured from other physiological indices such as heartbeat electrocardiogram.

  The physiological information of the receptor indicates the state of the nerve / receptor related to serotonin, emotion, breathing, etc., and is measured by PET or the like. In this embodiment, the physiological information of the receptor is compared with the emotional response of voice and other physiological responses.

  Physiological information on psychotropic drug administration is information on SSRIs, tranquilizers, psychotropic drugs, neuropharmaceuticals, etc., and in this embodiment, the state before and after the administration of the psychotropic drug and the emotional reaction of voice and other physiological reactions And compare.

  Physiological information of internal organs information refers to human built-in activities. For example, serotonin and the like are often present in the intestine, and the influence of a hormone such as serotonin may be fed back to the brain (called biofeedback) via the intestinal activity. Therefore, it is necessary to analyze whether the subject's emotion is an effect of brain activity or an internal activity such as the intestine. In order to screen for other factors, the physiological measurement device 4 measures physiological information related to the visceral activity. . In the present embodiment, for example, an ultrasonic examination device or the like is used as the physiological measurement device 4, and the intestinal activity is measured as a built-in activity.

  Physiological information on eyeball activity includes pupils, blinks, and eye movements. The pupil is opened when interested, and it is the same even if surprised. For example, the number of blinks increases when the user is conscious of shaking or the other party, and decreases when there is attention or interest, so the number of blinks is measured. The measurement of the eyeball activity in the present embodiment is performed by a doctor or the like while viewing the face image of the subject photographed by the digital camera 3, for example.

  In the present embodiment, the physiological information measured by the physiological measurement device 4 is exhaled breath, brain activity, nerve activity, receptor, psychotropic drug, visceral information, and eyeball activity, but is not limited thereto. For example, it is preferable to use electrocardiogram, blood, saliva, defecation / urine, body temperature, hormone, biological substance, transmitter, genetic information, sweating, and skin potential. Thereby, a test subject's emotional state, behavior prediction, etc. are attained more correctly.

  Here, the physiological information of the electrocardiogram is a change in heart rate or a change in heart activity accompanying an emotional change, which is measured by a known general method. The physiological information of blood is information about blood collected before, during, and during an experiment in order to analyze emotional changes caused by depression, stress, and illness. The saliva physiological information is information about saliva measured before, during and after the experiment in order to measure stress and depression. Physiological information on defecation and urine is collected and measured immediately before the experiment in order to measure dietary status and health status. Physiological information of body temperature is information about changes in body temperature that are measured in order to know the influence of emotion due to changes in body temperature, changes in stress and emotions. Physiological information on sweating is measured because reactions such as tension, stress and agitation appear in sweating. Physiological information on skin potential is measured because reactions such as tension and stress also appear on skin potential.

  Physiological information such as biological substances, transmitters, and genetic information is a list of relationships between biological substances such as adrenaline, dopamine, and caffeine, and physical reactions and emotions such as excitement, stress, anxiety, and depression. is there. For example, adrenaline is related to emotions such as excitement, anxiety, struggle, and fear, and body reactions such as heartbeat, pupils, and sympathetic nerves. It is shown that it is related to the emotion of stress.

  Next, the operation of the behavior analysis apparatus 100 according to the present embodiment will be described. However, before describing the operation of the behavior analysis apparatus 100, a subjective emotion table, a parameter table, a decision tree, and generation thereof necessary for the operation of the behavior analysis apparatus 100 will be described.

  First, the subjective emotion table will be described. A subjective emotion table is a method in which a plurality of subjects (for example, 100 or more) listen to utterances uttered by themselves and others, and subjectively evaluate emotions at the time of utterance (identity confirmation or others evaluation) The results are listed in association with each utterance unit.

  Specifically, each of the subjects speaks freely toward the microphone 2. The microphone 2 receives the emitted sound and outputs it to the computer 1 as an audio signal. The computer 1 records the received audio signal as audio data in the storage unit 11 for each subject.

  The CPU 10 receives an instruction for starting an application program for verifying the identity of the subject from the subject via the input device 16. Based on the activation instruction, the CPU 10 activates the application, and displays a user interface for subjectively evaluating emotions by listening to its own utterance as shown in FIG. The subject selects his / her own audio data file and outputs a playback command to the CPU 10 via the input device 16. The CPU 10 reproduces the subject's own voice data in units of utterances. The subject listens to the reproduced voice of his / her utterance unit, and performs subjective pleasure / discomfort evaluation, emotion evaluation, excitement evaluation (emotion level), and the like for each utterance using the input device 16. For example, the subject evaluates a certain utterance as unpleasant, angry 3 or pleasant, joy 2 or the like. The CPU 10 accepts those evaluations by the subject, and records the utterances and the evaluation results corresponding thereto in the storage unit 11 in association with each other.

  Although FIG. 2 shows normality, anger, joy, sadness, anxiety, and pain as emotion evaluation, it is not limited to this, and it is preferable to have emotions other than those. Moreover, although the excitement evaluation which is the degree of the feeling with respect to those feelings was set to 1-3, it is not limited to this, A 10-step evaluation etc. may be sufficient. The utterances to be played may be played back in time series, but are preferably shuffled and played back at random. Furthermore, utterances acquired in the past and utterances of other subjects may be included. Thereby, even if it is evaluation by subjectivity, the more stable subjective emotion table can be acquired. Moreover, you may reproduce | regenerate by continuous audio | voice not only in a speech unit. By being played back with continuous voice, the subject himself / herself can grasp the situation when the voice is uttered, and the emotion at that time can be determined together with the situation.

  Next, a test subject outputs the instruction | command which reproduces another test subject's audio | voice data at random by speech unit, for example, using the said application, and evaluates others to the CPU10 via the input device 16. FIG. CPU10 reproduces | regenerates the audio | voice data of several other test subjects at random for every speech. The subject listens to the reproduced voice of the other person for each utterance, and uses the input device 16 for subjective pleasure / discomfort evaluation, emotion evaluation, and excitement evaluation (emotion level) for each utterance. The CPU 10 accepts those evaluations by the subject, and records the utterances and the evaluation results corresponding thereto in the storage unit 11 in association with each other.

  And in order to acquire the stable evaluation result in the speech and emotion based on subjectivity among the results of the above identity verification and other person evaluation, the CPU 10 utters and evaluates the utterance among the evaluation results by the plurality of subjects. A subjective emotion table is generated by selecting those whose results match 80% or more.

  In the present embodiment, the CPU 10 obtains an action record indicating what action each subject has taken after uttering a sound toward the microphone 2 from a doctor or the like who is a user via the input device 16. The data is recorded in the storage unit 11 in association with the voice data of each subject. This behavior record serves as a sample for comparing with behavior prediction described later and for deriving a behavior prediction pattern. At the same time, the CPU 10 also obtains prior information on the environment / situation that affects emotions and psychology, such as the weather, meal contents, and physical condition, from the doctor, and records them in the storage unit 11 in association with the voice data of each subject. To do. The CPU 10 also obtains and confirms from a doctor or the like whether or not there is a cognitive influence on whether or not an influence (bias) of the surrounding environment has been received when the subject utters a voice.

  Furthermore, in this embodiment, each subject has performed identity verification based on his / her voice data immediately after obtaining the voice data. However, the present invention is not limited to this. The user may be confirmed again after a predetermined time. Similarly, it is preferable to perform another person evaluation again after a predetermined time interval. Thereby, a more stable subjective emotion table can be generated.

  Next, the parameter table will be described. The parameter table causes each of a plurality of subjects to speak, performs the various physiological measurements by the physiological measurement device 4, compares the plurality of voice parameters calculated from the voice signal of the utterance, and various physiological information, and responds to physiological changes. The voice parameters that are affected are listed for each physiological information. However, the difference from the case where the parameter table is the subjective emotion table is that the speech by the subject is performed under a situation where emotions are strongly emphasized in a natural state. In other words, voice and emotion, brain, and nervous system are closely related, and the test is performed in a situation where a specific emotion is clearly recalled to the subject. Here, the situations where emotions are strongly emphasized in a natural state are: “Put the subject in a small room”, “Fix the body so that it does not move”, “Questions that acquaintances do not want to touch the subject so much “Accept” or “Send a video that stimulates the subject's emotions or listen to a novel”.

  In the present embodiment, the situation where the emotion is strongly emphasized in a natural state includes a normal free situation. That is, the previously obtained subjective emotion table is also a parameter table that is subject to the subjective influence among the plurality of parameter tables. This makes it possible to see the difference between the emotional state in the normal case and the emotional state under the stress condition.

  Specifically, for example, the subject is equipped with a blood pressure meter, a thermometer, or the like that constitutes the physiological measurement device 4, and the body is fixed for measurement such as fMRI. In such a state, the digital camera 3 and the physiological measurement device 4 measure various physiological information of the subject based on the time schedule as shown in FIG. 3, and the microphone 2 receives the voice freely emitted by the subject. To do. The microphone 2 outputs the received audio signal to the computer 1. At the same time, the digital camera 3 outputs the captured subject image, and the physiological measurement device 4 outputs the measured physiological information of the subject to the computer 1. The computer 1 records the received audio signal, image, and physiological information in the storage unit 11 for each subject.

  Here, as shown in FIG. 3, in the time schedule of the present embodiment, the digital camera 3 measures the subject's facial expression and the physiological information of the eyeball activity from before the subject starts speaking until after the end. The subject's face is imaged. At the same time, the physiological measurement device 4 measures physical conditions such as electrocardiogram, skin potential, sweating, body temperature, and exhalation. Furthermore, the doctor collects and measures the excrement of the subject immediately before the experiment to check the dietary state and the health state, and the temperature and weather of the surrounding environment are input to each subject via the input device 16. It is recorded in the recording unit 11 in association with the audio data. In addition, before the start of utterance, the doctor confirms and records whether or not a psychotropic drug is administered to the subject.

  In the present embodiment, the physiological measuring device 4 measures the body state as well as the brain state, the nerve state, the neural activity such as the receptor, and the visceral organs such as blood, saliva, visceral state, and hormone after starting the utterance.・ Start measuring the endocrine system until the end of the utterance.

  Note that such measurement is ideally performed for 24 hours, but for example, it may be at least 1 hour. However, when the measurement is performed in a short period such as 1 hour, it is preferable that the measurement is performed at the time of getting up, before and after each meal, and at the time of going to bed. In addition, each measurement may be performed in a normal case without stress and in a stress situation. Alternatively, the measurement at the time of waking up may be performed in a different situation, for example, in a normal case, and the measurement at the time of going to bed may be performed in a different situation, such as in a stress situation. Furthermore, it is preferable that the method of applying stress is changed for each measurement.

  After the measurement, the doctor or the like confirms the behavior of each subject, and records it in the storage unit 11 in association with the speech data of each subject. At the same time, the doctor also checks whether there is any cognitive effect.

  On the other hand, as described above, the CPU 10 analyzes the acquired audio signal by a known method (FFT, wavelet, etc.), and calculates values of a plurality of audio parameters such as F0 and frequency. The CPU 10 uses a general analysis method, and based on temporal changes in the values of these voice parameters and various physiological information, a plurality of voice parameters are set for each physiological information regardless of an environment such as a normal situation or a stress situation. Among them, a speech parameter affected by the physiological information is extracted, and a parameter table composed of the extracted speech parameters is generated. For example, when the body temperature changes greatly above a predetermined threshold value, the CPU 10 determines that the voice parameter (F0 power and frequency displacement Δf) that has been greatly affected by the influence is affected by physiological information such as body temperature. Are extracted and added to the parameter table.

  In the present embodiment, a parameter table for parameters affected by nerve activity, a parameter table for parameters affected by the internal / endocrine system, and the like are generated and recorded in the storage unit 11. Thus, an index of emotional accuracy can be acquired, and abstract emotions can be quantified.

  Note that there is actually a time lag between the change in voice parameters and the change in physiological information. Therefore, it is necessary to capture the change between the voice parameter and the physiological information while considering the time lag.

  In addition, the generation of such a parameter table requires a measurement for at least one month. Preferably, a measurement period of one year is ideal because it is necessary to consider emotional changes accompanying changes such as the four seasons.

  In addition, the subject may perform identity verification and other person evaluation using the voice data acquired in the time schedule shown in FIG. 3 using the user interface shown in FIG. Thereby, a more stable subjective emotion table can be generated.

  Next, a determination tree (logic table) for various emotion class determinations generated based on various parameter tables including a subjective emotion table will be described.

  Specifically, the CPU 10 reads and activates a known decision tree creation program or analysis determination program from the storage unit 11 based on a user instruction from the input device 16. The CPU 10 reads various parameter tables including the subjective emotion table from the storage unit 11 and generates a judgment tree of various physiological information and subjective combinations. That is, the CPU 10 divides the voice parameters into groups that come from physiological basis and groups that come from subjectivity, and makes the logic created from subjective evaluation independent of the voice parameters that are affected by physiological changes, so Judgment trees for various emotion class determinations of combinations of physiological information for displaying the state and subjective parameters including cognitive effects are generated.

  In this embodiment, in generating a decision tree for various emotion class determinations, a parameter table affected by neural activity, a parameter table affected by the internal / endocrine system, a parameter table affected by the body condition, the cranial nervous system Parameter table affected by the internal secretion system, parameter table affected by the internal secretion system and body reaction, nerve, internal secretion system, parameter table affected by the body reaction, parameter table affected by the subjective (subjective emotion Table). However, the present invention is not limited to this. For example, a parameter table affected by nerve activity, a parameter table affected by the internal / endocrine system, a parameter table affected by body condition, a parameter table affected by subjective (subjective emotion table) ) May be used to generate a decision tree.

  4 and 5 show an example of a decision tree for emotion class determination. FIG. 4 shows a decision tree for judging the emotional state of the subject based on the heart rate and the voice parameters. FIG. 5 shows a judgment tree for judging the emotional state of the subject based on the height and power of the voice parameter F0.

  Thus, the subjective emotion table, various parameter tables, and various decision trees are generated. And the behavioral analysis apparatus 100 of this embodiment can diagnose a test subject's emotional state only based on the audio | voice signal input from the microphone 2 by using these.

  Therefore, the operation of the behavior analysis apparatus 100 according to the present embodiment will be described with reference to the flowchart of FIG.

  For example, a user such as a doctor uses the input device 16 to transmit a behavior analysis program start command to the CPU 10. When the CPU 10 receives the activation command, the CPU 10 reads the behavior analysis program from the storage unit 11 and executes it.

  Step S10: The microphone 2 acquires a voice signal of the voice of the subject who is a patient. The microphone 2 outputs an audio signal to the computer 1. The microphone 2 may also acquire the voice of the doctor who is the user together with the subject.

  Step S11: The CPU 10 of the computer 1 analyzes the received audio signal by FFT, wavelet, etc., and calculates a plurality of audio parameter values. The CPU 10 determines the value of physiological information such as neural activity that affects the calculated voice parameter based on the calculated voice parameter and various parameter tables. In this embodiment, in determining the value of physiological information, it is preferable that the CPU 10 can accept manual adjustment by a user such as a doctor and finely adjust the value of physiological information. That is, it is preferable to capture and adjust subtle changes in the audio signal caused by individual differences between subjects that cannot be captured by an application such as a behavior analysis program.

  Step S12: The CPU 10 substitutes the voice parameter and the determined values of various physiological information into a decision tree for determining various emotion classes. The CPU 10 determines the emotional state of the subject based on these various emotion class determination trees, and displays the determination result on the display device 15.

  The CPU 10 according to the present embodiment displays, on the display device 15, at least the built-in influence degree as shown in FIG. 7 a and the body reaction influence degree graph as shown in FIG. The behavior predicted in this way is displayed using a graphical template as shown in FIG.

  Here, FIG. 8 shows an example of a template that shows how emotions are developed by a decision tree for various emotion class determinations, as a concentric diagram. The template shown in FIG. 8 is composed of “pleasant” and “unpleasant” that are primitive emotions toward the center. On the other hand, behavior patterns corresponding to inner emotions are distributed toward the outer periphery. Thereby, the doctor can visualize and confirm a subject's emotional change and mental state. In addition, since the behavior can be predicted toward the outer periphery, the examination progress can be observed by confirming the behavior of the subject and the environmental situation. Furthermore, the doctor can perform psychoanalysis and personality determination from the comparison between the behavior of the subject and the emotional change analysis in the center.

  In addition, the inventor analyzed the emotional state of the singer who sings those music by inputting the data of various music into the behavior analysis apparatus 100 of this embodiment. As a result, for example, in the case of a bright ballad song by a female singer, the behavior analysis apparatus 100 discriminates the emotion of “joy” in the positive lyrics part, and the “angry” emotion in the lyrics part such as a declaration to strengthen. It was determined. On the other hand, in the case of a calm song by a male singer, the behavior analysis apparatus 100 basically discriminates only “normal” emotions, and changes in emotions were not seen much.

  Thus, in this embodiment, a doctor can intuitively grasp a patient's mental state and emotion change in real time, and a true value of a person's emotion is obtained by a physiological index and reliable subjective evaluation means.

  Also, by determining the emotional state of the subject using the various parameter tables including the subjective emotion table and the decision tree for determining the various emotion classes, for example, the subject who says “the stomach hurts” hurts because it really hurts Whether it is due to a physiological (neurological) cause, or a depressive (psychogenic) cause of acting, can be determined from the voice alone.

In addition, according to the present embodiment, based on voice data, it is easy to identify mental states such as emotions and stress, predict behaviors from the change state, psychoanalysis, real intention, personality estimation, and physiological indices Psychiatric analysis and treatment improvement as “X-ray of the heart” at the medical level by cooperation of
<< Second Embodiment >>
The behavior analysis apparatus according to the second embodiment is the same as the behavior analysis apparatus 100 according to the first embodiment. Therefore, the behavior analysis apparatus 100 shown in FIG. 1 is a behavior analysis apparatus according to the present embodiment, and a detailed description of each component is omitted.

  In the second embodiment, a behavioral analysis by the behavioral analysis device 100 is applied to a subject who is a patient by a doctor as a user, and the mental state element of the subject is analyzed from voice parameters, and at the same time from the ICD10 and the DSM4. Physiological index, doctor's diagnosis confirmation. Note that the behavior analysis apparatus 100 according to this embodiment is different from that of the first embodiment in that the determination of the emotional state can be customized for each subject.

  Therefore, the operation of the behavior analysis apparatus 100 including customization of emotion state determination will be described with reference to the flowchart of FIG.

  Note that the storage unit 11 of the behavior analysis apparatus 100 according to the present embodiment has in advance various parameter tables including the subjective emotion table obtained in one embodiment and a determination tree for various emotion class determinations. . However, in the various parameter tables and various emotion class determination decision trees of the present embodiment, for example, the threshold indicating the degree of association between the voice parameters, physiological information, and emotional state as shown in FIGS. Using the behavior analysis device 100, characteristics of subjects who are depressed patients (bias of normality and sadness), characteristics of the patients (bias of anger / joy axis), personality disorder (simultaneous measurement of characteristics of depression and depression) It is assumed that anxiety neurosis (anxiety / uncomfortable bias) and the like are preliminarily analyzed and set based on medical standards such as ICD10 and DSM4. Specifically, doctors decide the analysis items they want from psychoanalysis standards (clinical) based on medical standards such as ICD10 and DSM4, and “depression”, “so”, “anxiety”, “personality disorder” , Etc. is derived. Alternatively, a doctor can ask the subject various questions according to medical standards such as ICD10 and DSM4 to derive thresholds for voice parameters indicating “depression”, “so”, “anxiety”, “personality disorder”, etc. It may be set.

  Step S101: The CPU 10 displays a speech that does not enter the display device 15 such as “name / age”, “Iroha ni hi ni niru ni aka”, “ABCDEFGHIJK”, etc., and makes the subject speak.

  Step S102: The microphone 2 receives an utterance from a subject whose dialogue does not contain an emotion. The microphone 2 transmits the received audio signal to the computer 1. The CPU 10 analyzes the received speech signal that does not contain emotion using FFT, wavelet, etc., and calculates the values of a plurality of speech parameters. The CPU 10 determines the value of physiological information such as neural activity that affects the calculated voice parameter based on the calculated voice parameter value and various parameter tables. The CPU 10 substitutes the voice parameter and the determined physiological information value into a decision tree for various emotion class determinations. The CPU 10 determines the emotional state for psychoanalysis and outputs the determination result.

Step S103: The CPU 10 causes the display device 15 to display a bright expression such as “good weather” that reminds a bright image, and causes the subject to speak. Similarly, the CPU 10 displays dialogues of sad images and anger images on the display device 15 and causes the subject to speak. CPU10 determines the emotional state for divine analysis based on the audio | voice signal of each line by a test subject, and outputs the determination result. CPU10 displays the determination result of step S102 and step S103 on the display device 15, to the physician, indicating the determination result state transition of the emotional state for mental analysis in a subject. Note that it is preferable that the state transition is displayed using a template as shown in FIG.

  Step S104: The CPU 10 accepts the observation result of the state transition of the emotional state of the subject by the doctor based on the template via the input device 16, and “depression”, “so”, “anxiety”, “personality disorder” Measure the degree of mood disorder such as, and determine the subject's basic mental state. For example, if there is more sad reaction than a basic emotional state in a bright image, the degree of “depression” is measured in mood disorders. In addition, the degree of “yes” is measured for a sad reaction and a bright reaction. On the other hand, if there is always a lot of “anger” or a reaction such as “Yes” or “Depression” is high at the same time, a personality disorder or the like can be confirmed. In the case of PTSD and trauma, rapid changes in emotion, depression (bipolar reaction), personality disorder reaction, etc. are detected simultaneously and in real time.

  FIGS. 10A to 10C show examples of template display in each of “depression”, “so”, and “schizophrenia personality disorder”. The width of each emotion on the inner periphery of the template is determined according to the appearance frequency and degree of each emotion in the subject, and the width of each behavior state on the outer periphery is determined based on the emotion of the subject, the predicted behavior, and the actual behavior. It is determined according to the difference. Then, in order to adjust the range of the emotion and the behavioral state for each subject, the doctor determines the subject based on the voice characteristics such as “depression”, “so”, “anxiety”, “personality disorder”, etc. It is preferable to adjust and set the threshold value in the decision tree for various parameter tables and various emotion class determinations.

  In addition, the state transition display of the determination result of the emotional state includes “depression”, “so”, “anxiety”, “personality disorder (personality disorder)” which are basic emotional states / basic mental states, for example, You may display by a bar graph, a degree, or a color as shown in FIGS. Here, in FIGS. 11 to 14, the left channel and the right channel indicate temporal fluctuations of the pleasant / unpleasant value, emotion evaluation value, and excitement value of the subject and the doctor, respectively. FIG. 11 shows a sample for a healthy person. FIG. 12 shows an example of treatment of a person with a personality disorder designated by the court. FIG. 13 shows a state before treatment of the person with a personality disorder designated by the court in FIG. FIG. 14 shows an example when the degree is indicated by a numerical value. In addition, in FIGS. 11 to 14, the display of trauma reaction, the degree of mood disorder, “depressed”, and “real” changes in “so” may be displayed.

  Thus, in this embodiment, a doctor can intuitively grasp a patient's mental state and emotion change in real time, and a true value of a person's emotion is obtained by a physiological index and reliable subjective evaluation means.

  Also, by determining the emotional state of the subject using the various parameter tables including the subjective emotion table and the decision tree for determining the various emotion classes, for example, the subject who says “the stomach hurts” hurts because it really hurts Whether it is due to a physiological (neurological) cause, or a depressive (psychogenic) cause of acting, can be determined from the voice alone.

  Furthermore, since the doctor who is a user can adjust the threshold values of various parameter tables and various emotion class determination trees for each subject, the true value of human emotion can be obtained more accurately.

  In addition, based on voice data, it is easy to identify mental states such as emotions and stresses, predict behavior based on the change state, psychoanalysis, real intention, estimation of personality, and link with physiological indicators at the medical level. Psychoanalysis and treatment improvement can be performed as "mental X-ray".

Also, the mental state element can be detected from the required emotion value regardless of the time change, instead of the conventional time transition method.
<< Third Embodiment >>
The behavior analysis apparatus according to the third embodiment is the same as the behavior analysis apparatus 100 according to the first embodiment. Therefore, the behavior analysis apparatus 100 shown in FIG. 1 is a behavior analysis apparatus according to the present embodiment, and a detailed description of each component is omitted.

  Similar to the behavior analysis device of the second embodiment, the behavior analysis device 100 according to the present embodiment applies the result of emotion analysis to a template that is a model indicating the relationship between the physiology and the mind, and the psychological of the subject. Determine characteristics (real intention, personality, behavior prediction, psychoanalysis, etc.). In other words, the behavior analysis apparatus 100 can be seen with time transitions such as bipolar features (alternate detection by temporal transitions of so-called features and depression features), panic disorder (a situation in which anxiety and pain from normal, and personality disorder features are confused). While detecting the mental state, the threshold value is analyzed and set according to medical standards such as ICD10 and DSM4 to derive the change feature.

  Note that the various parameter tables including the subjective emotion table of this embodiment and the decision tree for various emotion class determinations are generated in advance in the same procedure as in the first embodiment and the second embodiment. The subject of embodiment differs from the case of 1st Embodiment and 2nd Embodiment by the point comprised from the healthy subject and the patient who has a mental illness.

  Further, in generating the various parameter tables of the present embodiment, the point that the doctor dispenses a psychotropic drug to each subject, that is, is affected by the administration, is the first embodiment and the second implementation. Different from the case of form. That is, the CPU 10 gives a psychotropic drug dosing flag, causes the subject to speak, and measures the subject's facial expression observed from the image captured by the digital camera 3 and measured values measured using MRI or the like. Are used to generate various parameter tables and decision trees for various emotion class determinations according to the present embodiment.

  Next, operation | movement of the behavior analysis apparatus 100 of this embodiment is demonstrated, referring the flowchart of FIG. In addition, this embodiment presupposes that the test subject's basic mental state of 2nd Embodiment mentioned above is observed over a long period of time.

  Step S201: The CPU 10 displays the basic mental state observed over a long period of time on the display device 15. The CPU 10 causes the doctor to determine whether or not a disorder such as depression and so on has been confirmed in the subject over a long period of time equal to or longer than a predetermined period in the period in accordance with the doctor's own criteria. The determination result is received via. CPU10 determines that a test subject's basic mental state is normal, when a disorder | damage | failure is not confirmed over the long term beyond a predetermined period (NO side), and complete | finishes the process of action analysis. On the other hand, when the failure is confirmed for a predetermined period or longer, the CPU 10 proceeds to step S202 (YES side). The period according to the doctor's own criteria is a period determined based on the doctor's own experience, such as 1 hour, 1 day, 1 month, 1 year, etc., which is determined according to the personality and situation of the subject. It is preferable. In addition, the term “longer than the predetermined period” in the present embodiment is determined for each content of the disorder and each subject, and is, for example, about 14 days in the case of depression.

  Step S202: The CPU 10 determines the “symptom degree” based on the determination result of the doctor in step S201. The CPU 10 displays the result on the display device 15 as a bar graph, color (attribute), and amount (degree) of the mental state of the subject such as a bipolar state and a panic disorder state as shown in FIGS. To do.

  Step S203: The CPU 10 reflects the change in the value of physiological information affected by the medication on the template based on the physiological comparison matrix that is physiological information of the biological substance, the transmitting substance, and the genetic information, and displays it on the display device 15.

  Step S204: The CPU 10 displays a speech that does not enter the display device 15, such as “name / age”, “Iroha ni hi ni niru ni aka”, “ABCDEFGHIJK”, etc., and causes the subject to speak.

  Step S205: The microphone 2 receives an utterance from a subject whose speech does not enter. The microphone 2 transmits the received audio signal to the computer 1. The CPU 10 analyzes the received speech signal that does not contain emotion using FFT, wavelet, etc., and calculates the values of a plurality of speech parameters. The CPU 10 determines the value of physiological information such as neural activity that affects the calculated voice parameter based on the calculated voice parameter value and various parameter tables. The CPU 10 substitutes the voice parameter and the determined physiological information value into a decision tree for various emotion class determinations. The CPU 10 determines the emotional state for psychoanalysis and outputs the determination result.

  Step S206: The CPU 10 causes the display device 15 to display a bright expression such as “It's a nice weather” to remind a bright image, and to let the subject speak. Similarly, the CPU 10 displays dialogues of sad images and anger images on the display device 15 and causes the subject to speak. CPU10 determines the emotional state for divine analysis based on the audio | voice signal of each line by a test subject, and outputs the determination result. CPU10 displays the determination result of step S205 and step S206 on the display apparatus 15, and shows the state transition of the determination result of the emotional state for divine analysis in a test subject with respect to a doctor. In addition, as a display which shows a state transition, it is preferable to show to a doctor with a template as shown in FIG.

  Step S207: The CPU 10 accepts the observation result of the state transition of the emotional state of the subject by the doctor based on the template via the input device 16, and “depression”, “so”, “anxiety”, “personality disorder” Measure the degree of mood disorder such as, and determine the subject's basic mental state.

  Step S208: The CPU 10 displays various questions according to medical standards such as ICD 10 and DSM 4 on the display device 15 based on the doctor's instructions from the input device 16, and asks the subject questions. The CPU 10 receives an answer from the subject to the question via the input device 16. CPU10 changes a template dynamically according to a test subject's answer, and displays the change on the display apparatus 15. FIG. The CPU 10 causes the doctor to observe a dynamic change of the template according to the question item and the answer to the question item, and to make a personality determination such as trauma or PTSD. In other words, from the relationship with various measured physiological information, emotions (emotional elements that can be confirmed by physiological responses) that come from the brain and body reaction are considered as real sounds, and the temporal appearance characteristics of mental states and comparisons between real sounds and questions, The doctor can estimate the personality of the subject. The CPU 10 receives a diagnosis result of a personality disorder state such as a subject saddist or masochist by a doctor via the input device 16.

  Step S209: The CPU 10 confirms the situation based on the personality determination in step S208 and the state transition of the template, and predicts the behavior of the subject. That is, the CPU 10 predicts the behavior of the subject based on the personality determination by the doctor, the detected real-time emotion and mental state changes, and the state transition of the template corresponding to them. CPU10 displays the comment with respect to the predicted action with a test subject's true intention and personality on the display apparatus 15. FIG. For example, when the doctor determines that the subject is a sadist, the CPU 10 predicts an action that, if the doctor is a sadist, he will feel joy in a certain situation. Then, the CPU 10 accepts the actual behavior and emotional state of the subject confirmed after the personality determination from the doctor as an action confirmation input via the input device 16. Then, when a reaction other than the standard reaction of the template occurs, such as when an emotional state other than joy is detected from the subject, the CPU 10 determines that the subject has personality disorder or alerts the display device 15. It is preferable to display it.

  Step S210: The CPU 10 compares the predicted action with the action confirmation input, calculates the appearance frequency and degree of the emotion that appears in the subject, and sets various parameter tables and various types so that the template is more suitable for each subject. The threshold in the decision tree for emotion class determination is adjusted. That is, for example, when the subject indicates a behavior or emotional state different from the predicted behavior even though the subject has determined that the subject is a sadist, the threshold values in the decision trees for various parameter tables and various emotion class determinations Determines that it is not optimal for the subject. The CPU 10 adjusts the threshold values of the various parameter tables and the determination tree based on the determination result. CPU10 adjusts the width | variety of each emotion and action state in a template according to adjustment of the threshold value. As a result, the accuracy of behavior prediction in the behavior analysis device 100 can be improved.

  Thus, in this embodiment, a doctor can intuitively grasp a patient's mental state and emotion change in real time, and a true value of a person's emotion is obtained by a physiological index and reliable subjective evaluation means.

  Also, by determining the emotional state of the subject using the various parameter tables including the subjective emotion table and the decision tree for determining the various emotion classes, for example, the subject who says “the stomach hurts” hurts because it really hurts Whether it is due to a physiological (neurological) cause, or a depressive (psychogenic) cause of acting, can be determined from the voice alone.

  Furthermore, by adjusting the thresholds of various parameter tables and judgment trees for various emotion class judgments according to the subject and changing the template, the subject's psychoanalysis based on real intention, personality, behavior prediction, and time transition can be more accurately performed. Can be done.

In addition, based on voice data, it is easy to identify mental states such as emotions and stresses, predict behavior based on the change state, psychoanalysis, real intention, estimation of personality, and link with physiological indicators at the medical level. Psychoanalysis and treatment improvement can be performed as "mental X-ray"
<< Modification of Third Embodiment >>
The behavior analysis apparatus according to the modified example of the third embodiment is the same as the behavior analysis apparatus 100 of the first embodiment. Therefore, the behavior analysis apparatus 100 shown in FIG. 1 is a behavior analysis apparatus according to the present embodiment, and a detailed description of each component is omitted.

  The behavior analysis apparatus 100 according to the present embodiment is basically the same as the behavior analysis apparatus according to the third embodiment, and performs a processing operation according to the flowchart shown in FIG. However, the difference between the behavior analysis apparatus 100 of the present embodiment and that of the third embodiment is as follows.

  1) In step S206, the CPU 10 uses a known general method such as a predetermined operator or a decision tree generation from the state transition of the determination result of the emotional state for the god analysis in the subject, and the constancy that the physiological information has. The balance state (for example, trying to keep the blood pressure constant) is analyzed. The predetermined operator is, for example, an operator that linearly expresses a relationship between certain physiological information and a voice parameter that is easily influenced by the physiological information.

  2) In step S210, the CPU 10 causes the doctor to determine the subject's failure based on the comparison between the behavior predicted for the subject and the actual behavior and the analysis result of the balance state described above. The CPU 10 records the combination of the determined failure and the characteristics of the balance change in the failure by reflecting it in the template. Then, the CPU 10 adjusts the threshold values in the various parameter tables and various emotion class determination decision trees based on the recorded templates. This reflects the characteristics of the subject and the diagnostic characteristics of the doctor. Note that the recorded template and threshold information are preferably recorded in the medical chart as information about the subject.

  Further, the CPU 10 applies the template and threshold value of the subject to other subjects, and when consistency is confirmed in the diagnosis of the failure determination by the doctor, the information on the template and the threshold value is used as reference data in the determination of the predetermined failure. As such, a setting for increasing the importance may be made. In addition, for example, when the behavior analysis apparatus 100 is connected to the network, the CPU 10 aggregates the diagnosis results by other doctors and updates the reference data for determining a predetermined failure as needed based on the diagnosis results. It is preferably shared with other doctors.

  Thus, in this embodiment, a doctor can intuitively grasp a patient's mental state and emotion change in real time, and a true value of a person's emotion is obtained by a physiological index and reliable subjective evaluation means.

  Also, by determining the emotional state of the subject using the various parameter tables including the subjective emotion table and the decision tree for determining the various emotion classes, for example, the subject who says “the stomach hurts” hurts because it really hurts Whether it is due to a physiological (neurological) cause, or a depressive (psychogenic) cause of acting, can be determined from the voice alone.

  Furthermore, based on the balance state of homeostasis in physiological information and the failure determination by the doctor, by adjusting the template according to the subject, the thresholds of the decision tree for various parameter tables and various emotion class determinations, The subject's psychoanalysis based on behavior prediction and time transition can be performed more accurately.

In addition, based on voice data, it is easy to identify mental states such as emotions and stresses, predict behavior based on the change state, psychoanalysis, real intention, estimation of personality, and link with physiological indicators at the medical level. Psychoanalysis and treatment improvement can be performed as "mental X-ray".
<< Additional items of embodiment >>
(1) In the above embodiment, the user is a doctor or the like, but the present invention is not limited to this. For example, the subject himself / herself may improve the mental state by repeatedly speaking using the present invention until the normal mental state is reached. In this case, it is preferable that the subject can confirm the determination result such as his / her mental state in real time while speaking, and obtain a sense of security and reliability with respect to the behavior analysis apparatus 100.

  In addition, the behavior analysis apparatus 100 displays announcements and comments that guide the subject's own utterance to change gradually, and changes the voice quality of the subject so that the subject can self-learn mental state changes. It is preferable to make it.

  Moreover, it is preferable that the behavior analysis apparatus 100 induces the subject to make a long-term improvement by reproducing the voice and self-learning the subject or reporting a change every week or every month. Thereby, it is possible to improve the mental state and present the prescription from voice utterance using a system made by a method that has never been made.

  (2) In the above embodiment, the template shown in FIG. 8 is used as the physiological emotion model. However, the present invention is not limited to this, and other known physiological emotion models such as Bion grid may be used.

  Further, the number of layers, emotions, and behavior states in the template shown in FIG. 8 can be changed as appropriate. That is, a template with a more detailed hierarchy, emotion, and action state may be used, or a more simplified template may be used.

  (3) In the above embodiment, the behavior analysis apparatus 100 has the configuration shown in FIG. 1, but the present invention is not limited to this. For example, the microphone 2 is used to acquire a plurality of voice parameters from a voice signal of the subject's utterance, and at the same time, by placing one or more sensors on the neck portion of the subject where the vocal cords are located, You may measure as a new voice parameter. Alternatively, the subject's vocal cords may be imaged through an optical fiber from the nose or the like, for example, with a camera capable of capturing a moving image at 1000 to 2000 fps, and a change in the shape of the vocal cords may be directly measured as a new audio parameter.

  (4) In the above-described embodiment, the case where psychological counseling such as psychoanalysis, behavior prediction, and behavior analysis is applied to interviews and prescriptions in psychiatry and general medicine has been described, but the present invention is not limited thereto. For example, application of behavior analysis devices to robots and automobiles, call centers, entertainment, Internet and mobile phone applications and services, search systems, financial credit management systems and behavior prediction, companies, schools, government agencies, police and military, information Information analysis in collection activities, psychological analysis leading to false discovery, application to organizational group management, application to systems that manage mental health and behavior prediction of organizational members, researchers, employees, managers, etc. Application to systems that control the environment such as homes and offices, airplanes and spacecrafts, means to know the state of mind and behavior of family and friends, application to music and movie distribution, general information retrieval Application to information analysis management and information processing, application to customer sensibility preference market analysis, etc. It is also possible to apply to the application or the like of a system for managing in. For example, when applied to a robot, utterances or actions corresponding to the emotional state and behavior prediction of a person based on the person's utterance may be performed. In addition, when applied to a car, based on the driver's emotional state and behavior prediction based on the driver's utterance, a warning or the like corresponding to them is displayed by voice or a display device, or the operation of the car is restricted. By doing so, an accident or the like may be prevented in advance.

  For example, in the case of a call center, the behavior analysis device 100 analyzes the voice signal of the customer's utterance and displays the customer's current mental state and the result of behavior prediction on the display device 15 to the operator. It is possible to determine whether the product is actually being inquired about, or simply being harassed, and more appropriate customer service is possible.

  (5) In the said 3rd Embodiment, you may carry out continuously with 2nd Embodiment and it is preferable to abbreviate | omit step S204 to step S207 in that case.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 1 Computer, 2 Microphone, 3 Digital camera, 4 Physiological measurement apparatus, 10 CPU, 11 Memory | storage part, 12 Memory, 13 Input / output I / F, 14 Bus, 15 Display apparatus, 16 Input apparatus, 100 Behavior analysis apparatus

Claims (8)

An input procedure for acquiring the speech of the subject as an audio signal;
A calculation procedure for calculating a plurality of audio feature values from the audio signal;
Based on the first table associating the plurality of voice feature quantities with a plurality of physiological information indicating physiological activities of a person, the feature quantities of the physiological information in the subject are calculated from the calculated voice feature quantities. And determining the plurality of sound feature amounts calculated based on the second table in which the plurality of sound feature amounts and the feature amounts of the physiological information are associated with a plurality of emotional states of a person. A prediction procedure for determining an emotional state in the subject from the feature amount of each physiological information, and predicting the behavior of the subject based on the determined emotional state ;
A behavior analysis method comprising: The behavior analysis method according to claim 1 ,
The prediction procedure predicts the behavior of the subject based on the determined transition, appearance frequency, and degree of each emotional state . The behavior analysis method according to claim 1 ,
The prediction procedure, using a model that relates the behavior and emotional state of a person, behavior analysis method characterized by predicting the behavior of the subject from the determined said emotional state. The behavior analysis method according to any one of claims 1 to 3,
The input procedure obtains the behavior result of the subject observed after prediction by the prediction procedure,
The predicting means is based on a comparison between a prediction result and the action result , and a degree of association between the plurality of voice feature amounts in the first table and the plurality of physiological information, and the plurality in the second table . The behavioral analysis method characterized by adjusting the voice feature amount and the degree of association between the feature amount of each physiological information and a plurality of emotional states of the person. The behavior analysis method according to claim 4,
The prediction means adjusts the degree for each subject. The behavior analysis method according to claim 4,
The behavior analysis method, wherein the prediction means adjusts the degree for each emotional state. The behavior analysis method according to any one of claims 1 to 6, further comprising: a display procedure for displaying the subject's physiological activities and emotional states according to the prediction procedure, and a prediction result. . An input unit for acquiring the speech of the subject as an audio signal;
A calculation unit for calculating a plurality of audio feature values from the audio signal;
Based on the first table associating the plurality of voice feature quantities with a plurality of physiological information indicating physiological activities of a person, the feature quantities of the physiological information in the subject are calculated from the calculated voice feature quantities. And determining the plurality of sound feature amounts calculated based on the second table in which the plurality of sound feature amounts and the feature amounts of the physiological information are associated with a plurality of emotional states of a person. A prediction unit that determines an emotional state of the subject from the feature amount of each physiological information, and predicts the behavior of the subject based on the determined emotional state ;
A behavior analysis apparatus comprising:

Images