JP6554422B2 - INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM - Google Patents

Description

  Embodiments described herein relate generally to an information processing apparatus, an information processing method, and a program for estimating a user's emotion.

  2. Description of the Related Art Conventionally, there is a technique for estimating a user's emotion (e.g., laughter, etc.) caused by various actions such as content appreciation or desk work. Such techniques include taking a user's face with a camera and measuring the duration of the user's facial expression (eg, a smile or anxiety face). In addition, in such a technique, there is a technique in which a user's voice is measured with a microphone, a predetermined voice (for example, laughter or cry) is extracted from the user's voice, and a duration of the predetermined voice is measured.

Yoshinori Konishi, 3 others, “Real-time smile estimation”, IPSJ Interaction 2008 Proceedings, 2008. Go Irie, Kota Hidaka, Naoya Miyashita, Takashi Sato, Yukinobu Taniguchi, "Laughter" Scene Detection Method for Quick Viewing of Personal Video Images, Journal of the Institute of Image Information and Television Engineers Vol. 62 (2008) No. 2, pp. 227-233, 2008.

  However, the technique for estimating the emotion by photographing the user's face may not be able to correctly recognize the facial expression when the user moves his head or when the illumination around the user is dark. In addition, the above-mentioned technology may misrecognize the user's "come-laughing" and the like.

  Moreover, there is a possibility that the technology for estimating the emotion by measuring the voice of the user with the microphone cannot correctly recognize the grief that does not make a laughter or the sadness with which the cry was put up.

  In order to solve the above-mentioned subject, an information processor, an information processing method, and a program which can estimate a user's emotion effectively are provided.

According to the embodiment, an information processing apparatus that estimates a user's emotion includes an acquisition unit, a start point detection unit, an end point detection unit, and an emotion estimation unit. The acquisition unit acquires a heart rate and a respiration rate of the user. The start point detection unit detects the start point of the user's emotion based on the heart rate and the respiration rate. The end point detection unit detects the end point of the user's emotion based on the heart rate and the respiration rate. The emotion estimation unit estimates an emotion generated between the start point and the end point. The start point detection unit detects, as the start point, a time point when the heart rate and the change amount of the respiration rate exceed a heart rate threshold and a respiration rate threshold, respectively, and a heart rate before the start point is a reference heart rate And the respiration rate before the start point is output as the reference respiration rate. The end point detection unit detects, as the end point, a time point at which the heart rate returns to the reference heart rate and the respiration rate returns to the reference respiration rate.

  According to the embodiment, the information processing apparatus can estimate the start point and end point of the user's emotion caused by the action based on changes in the heart rate and the respiratory rate. The information processing device estimates emotion based on the heartbeat and respiration between the start point and the end point. Therefore, the information processing apparatus can estimate the user's emotion even when the user's facial expression cannot be recognized or when the user's emotion does not appear in the voice. Therefore, the information processing apparatus can effectively estimate the user's emotions.

FIG. 1 is a block diagram illustrating a configuration example of the information processing apparatus according to the first embodiment. FIG. 2 is a flowchart for explaining an operation example of the change start point detection unit according to the first embodiment. FIG. 3 is a flowchart for explaining an operation example of the change end point detection unit according to the first embodiment. FIG. 4 is a flowchart for explaining an operation example of the emotional change amount calculation unit according to the first embodiment. FIG. 5 is a block diagram showing an exemplary configuration of the information processing apparatus according to the second embodiment. FIG. 6 is a block diagram showing an exemplary configuration of an information processing apparatus according to the third embodiment. FIG. 7 is a block diagram showing an exemplary configuration of an information processing apparatus according to the fourth embodiment.

Hereinafter, embodiments will be described with reference to the drawings.
First Embodiment
The information processing apparatus according to the embodiment estimates the user's emotion based on the user's heart rate and breathing rate. The information processing apparatus continuously measures a user's heart rate and respiration rate for a predetermined period using a sensor or the like attached to a predetermined part of the user. The information processing apparatus estimates the user's emotion based on the plurality of continuously measured heart rates and breathing rates. For example, the information processing apparatus estimates emotion while the user is performing a predetermined action. For example, the predetermined action is watching or desk work of predetermined content. Note that the action of the user whose information processing apparatus estimates the emotion is not limited to a specific action.

  Emotion is also the movement of emotion accompanied by physical expression. For example, emotions include laughter, anger, fear, joy or sadness.

FIG. 1 is a block diagram illustrating a configuration example of the information processing apparatus 100 according to the first embodiment.
As shown in FIG. 1, the information processing apparatus 100 includes a data input unit 101, a processing unit 102, a setting input unit 103, a result output unit 104, and the like.

  The data input unit 101 (acquisition unit) acquires the user's heart rate and respiration rate. For example, the data input unit 101 is connected to a sensor that measures the user's heart rate and respiration rate, and acquires the user's heart rate and respiration rate from the sensor. The data input unit 101 acquires heart rate and respiration rate in time series. For example, the data input unit 101 acquires a heart rate and a respiration rate at predetermined intervals.

  In addition, the data input unit 101 acquires a time stamp indicating a time when the heart rate and the respiration rate are measured. For example, the data input unit 101 acquires a time as a time stamp each time the heart rate and the respiration rate are measured.

  The data input unit 101 includes a plurality of heart rates in a time series (heart rate vector Hr), a plurality of respiration rates in a time series (respiratory rate vector Rs), and a time stamp of each heart rate and respiratory rate (time stamp vector Ts) is output to the processing unit 102.

  The data input unit 101 may be configured to include a sensor that measures the heart rate and breathing rate of the user. Further, the data input unit 101 may acquire a heart rate and a respiration rate from a nonvolatile memory or the like.

  The setting input unit 103 acquires parameters necessary for the processing unit 102 to estimate an emotion. For example, the setting input unit 103 acquires parameters manually input by the operator. For example, the setting input unit 103 may be connected to an operation unit that inputs an operation, and may receive an input of a parameter from the operation unit. The setting input unit 103 may include an operation unit. The setting input unit 103 may be an application of a smartphone. The setting input unit 103 may be a PC or the like.

  The setting input unit 103 supplies parameters to each unit of the processing unit 202, such as the change start point detection unit 111, the change end point detection unit 112, and the emotion change amount calculation unit 113.

  The processing unit 102 estimates the user's emotion based on the heart rate vector Hr, the respiration rate vector Rs and the time stamp vector Ts from the data input unit 101.

  For example, the processing unit 102 includes a CPU, ROM, RAM, NVM, an interface, and the like.

  For example, the functions realized by the processing unit 102 are realized by the CPU executing a program. That is, the program is used for a computer that operates as at least a part of the information processing apparatus 100.

The processing unit 102 includes a change start point detection unit 111, a change end point detection unit 112, an emotion change amount calculation unit 113, and the like.
The change start point detection unit 111 detects a point in time when the emotion starts (change start point Tsc) based on the heart rate vector Hr, the respiration rate vector Rs, and the like.

For example, the change start point detector 111 detects the start point as follows.
First, the change start point detection unit 111 acquires the window width w, the heart rate threshold dHr, and the respiratory rate threshold dRs from the setting input unit 103.

  The change start point detection unit 111 detects a change start point Tsc when the amount of change in the heart rate and the respiration rate exceeds the heart rate threshold dHr and the respiration rate threshold dRs, respectively. That is, the change start point detection unit 111 (difference between the heart rate at the predetermined time (heart rate vector Hr [i]) and the heart rate before the window width w (heart rate vector Hr [i-w]). The amount of change ΔHr) exceeds the heart rate threshold dHr, and the respiration rate (respiratory rate vector Rs [i]) at a predetermined time point and the respiration rate before the window width w (respiratory rate vector Rs [i-w]) When the difference of (respiratory change ΔRs) exceeds the respiratory rate threshold dRs, i is set as the change start point Tsc.

  Further, the change start point detection unit 111 acquires the heart rate and the respiration rate immediately before the change start point as the reference heart rate Hrc and the reference respiration rate Rsc, respectively. That is, the change start point detection unit 111 sets the heart rate vector Hr [Tsc-1] as the reference heart rate Hrc, and sets the respiration rate vector Rs [Tsc-1] as the reference respiration rate Rsc.

  The change start point detector 111 outputs a change start point Tsc, a reference heart rate Hrc, and a reference respiration rate Rsc.

  Note that the change start point detection unit 111 detects the change start point Tsc from the detection start point s. The detection start point s is an argument corresponding to the time to start detection in the time stamp vector Ts. For example, the detection start point s may be a time point immediately after the time point when the emotion estimated immediately before ends. Further, the change start point detection unit 111 may obtain the detection start point s from the setting input unit 103. In addition, the change start point detection unit 111 may set the detection start point s to “1” when performing the first estimation.

  The change end point detection unit 112 detects emotion based on the heart rate vector Hr, the respiration rate vector Rs, the change start point Tsc, the reference heart rate Hrc, the reference respiration rate Rsc, the heart rate threshold coefficient αHr, the respiration rate threshold coefficient αRs, and the like. The end point (change end point Tsg) is detected.

The change end point detection unit 112 detects the time point when the respiration rate returns to the reference respiration rate Rsc and the heart rate returns to the reference heart rate Hrc as the change end point Tsg. For example, the change end point detection unit 112
When the difference between the respiration rate and the reference respiration rate Rsc falls below a predetermined threshold, it is determined that the respiration rate has returned. Similarly, the change end point detection unit 112 determines that the heart rate has returned when the difference between the heart rate and the reference heart rate Hrc falls below a predetermined threshold.

  The change end point detection unit 112 detects, for example, a change end point Tsg as follows. Here, it is assumed that the heart rate rises and then falls and the respiratory rate rises after decreasing after the emotion is generated until the end.

  First, the change end point detection unit 112 acquires the heart rate threshold coefficient αHr and the respiration rate threshold coefficient αRs from the setting input unit 103. The heart rate threshold coefficient αHr and the respiration rate threshold coefficient αRs are coefficients for determining that the heart rate and the respiration rate have returned, respectively.

  For example, when the heart rate threshold coefficient αHr is 1.1 and the reference heart rate Hrc is 60, the change end point detection unit 112 determines that the heart rate is the product of the reference heart rate Hrc and the heart rate threshold coefficient αHr (66 ( If it falls below 60 × 1.1)), it is determined that the heart rate has returned to the reference heart rate Hrc. Similarly, when the respiration rate threshold coefficient αRs is 0.9 and the reference respiration rate Rsc is 20, the change end point detection unit 112 determines that the respiration rate is the product of the reference respiration rate Rsc and the respiration rate threshold coefficient αRs (18 If it exceeds (10 × 0.9)), it is determined that the respiration rate has returned to the reference respiration rate Rsc.

  The change end point detection unit 112 compares the respiration rate (respiratory rate vector Rs [i]) at a predetermined time point with the product of the reference respiration rate Rsc and the respiration rate threshold coefficient αRs (end determination respiration rate Rse). .

  After the time when the respiration rate exceeds the end determination respiration rate Rse, the change end point detection unit 112, the heart rate at a predetermined time (heart rate vector Hr [i]), the reference heart rate Hrc, and the heart rate threshold coefficient αHr. And the product (end determination heart rate Hre) of

  The change end point detection unit 112 detects a point at which the heart rate vector Hr [i] falls below the end determination heart rate Hre as a change end point Tsg. That is, the change end point detection unit 112 sets i as the change end point Tsg when the heart rate vector Hr [i] falls below the end determination heart rate Hre.

  Further, the change end point detection unit 112 extracts an element (heart rate vector Hr ′) from the heart rate vector Hr to the change start point Tsc to the change end point Tsg. That is, the change end point detection unit 112 extracts the heart rate vector Hr [Tsc] from the heart rate vector Hr [Tsc]. Similarly, the change end point detection unit 112 extracts elements (respiration rate vector Rr ′) from the respiration rate vector Rr to the change start point Tsc to the change end point Tsg. That is, the change end point detection unit 112 extracts the respiratory rate vector Rr [Tsc] to the respiratory rate vector Rr [Tsg].

  The change end point detection unit 112 outputs a heart rate vector Hr ′, a respiration rate vector Rs ′, a change start point Tsc, a change end point Tsg, a reference heart rate Hrc, and a next detection start point s (change end point Tsg + 1). .

The emotion change amount calculation unit 113 (emotion estimation unit) estimates the user's emotion (estimated amount difHr) based on the heart rate vector Hr ′, the reference heart rate Hrc, and the like.
For example, the emotion change amount calculation unit 113 estimates the estimated amount difHr as follows.

  First, the emotion change amount calculation unit 113 calculates the difference between the elements of the heart rate vector Hr 'and the reference heart rate Hrc. The emotion change amount calculation unit 113 calculates the sum of the calculated differences, and acquires the estimated amount difHr.

  The emotion change amount calculation unit 113 outputs the change start point Tsc, the change end point Tsg, and the estimated amount difHr to the result output unit 104.

  The emotion change amount calculation unit 113 may also calculate the estimated amount difRs using the respiratory rate vector Rs'. For example, the emotion change amount calculation unit 113 calculates the difference between each element of the respiratory rate vector Rr 'and the reference respiratory rate Rsc. The emotion change amount calculation unit 113 calculates the sum of the calculated differences, and acquires the estimated amount difRs. The emotion change amount calculation unit 113 may output the estimated amount difRs as emotion.

  The result output unit 104 outputs the estimation result of the processing unit 102 (for example, the change start point Tsc, the change end point Tsg, and the estimated amount difHr). For example, the result output unit 104 is connected to a display unit or the like, and displays the estimation result on the display unit. The result output unit 104 is connected to a nonvolatile memory or the like, and stores the estimation result in the nonvolatile memory. The result output unit 104 may include a display unit or a nonvolatile memory.

Next, an operation example of the processing unit 102 will be described.
First, an operation example of the change start point detection unit 111 of the processing unit 102 will be described.

  FIG. 2 is a flowchart for explaining an operation example of the change start point detection unit 111.

First, the change start point detection unit 111 acquires the heart rate vector Hr and the respiration rate vector Rs from the data input unit 101, acquires the window width w, the heart rate threshold dHr, and the respiration rate threshold dRs from the setting input unit 103, The detection start point s is acquired from the setting input unit 103 or the change end point detection unit 112 (S11)
When the heart rate vector Hr, the respiratory rate vector Rs, the window width w, the heart rate threshold value dHr, the respiratory rate threshold value dRs, and the detection start point s are acquired, the change start point detection unit 111 substitutes the detection start point s for the counter i. (S12).

  When the detection start point s is substituted for the counter i, the change start point detection unit 111 determines whether the counter i is equal to or less than the length L (S13). Here, the length L is the number of elements of the heart rate vector Hr or the respiratory rate vector Rs.

  If it is determined that the counter i is equal to or less than the length L (S13, YES), the change start point detection unit 111 calculates a heart rate change amount ΔHr and a respiration change amount ΔRs (S14). When the heart rate change amount ΔHr and the respiratory change amount ΔRs are calculated, the change start point detection unit 111 determines whether the heart rate change amount ΔHr> the heart rate threshold value dHr and the respiratory change amount ΔRs> the respiratory rate threshold value dRs ( S15).

  If it is determined that the heart rate change amount ΔHr> the heart rate threshold value dHr and not the respiratory change amount ΔRs> the respiratory rate threshold value dRs (S15, NO), the change start point detection unit 111 adds 1 to the counter i (S16). ). When 1 is added to the counter i, the change start point detection unit 111 returns to S14.

  If it is determined that heart rate change amount ΔHr> heart rate threshold value dHr and respiration change amount ΔRs> respiration rate threshold value dRs (S15, YES), the change start point detection unit 111 determines the change start point Tsc = i, the reference Heart rate Hrc = heart rate vector Hr [i−1] and reference respiratory rate Rsc = respiration rate vector Rs [i−1] (S17).

If it is determined that the counter i is not less than or equal to the length L (S13, NO), the change start point detection unit 111 sets the change start point Tsc = 0, the reference heart rate Hrc = 0, and the reference respiration rate Rsc = 0 (S18).
After S17 or S18, the change start point detector 111 outputs the change start point Tsc, the reference heart rate Hrc, and the reference respiration rate Rs (S19). When the change start point Tsc, the reference heart rate Hrc, and the reference respiration rate Rs are output, the change start point detection unit 111 ends the operation.

  Next, an operation example of the change end point detection unit 112 of the processing unit 102 will be described.

  FIG. 3 is a flowchart for explaining an operation example of the change end point detection unit 112. Here, the change end point detection unit 112 sets the calculation start point s ′ = 0 as an initial state. The calculation start point s' is a point in time when the determination of whether the heart rate has returned is started.

  First, the change end point detection unit 112 acquires the heart rate vector Hr and the respiration rate vector Rs from the data input unit 101, acquires the heart rate threshold coefficient αHr and the respiration rate threshold coefficient αRs from the setting input unit 103, and starts the change. The change start point Tsc, the reference heart rate Hrc, and the reference respiration rate Rsc are acquired from the point detection unit 111 (S21).

  When the heart rate vector Hr, the respiratory rate vector Rs, the heart rate threshold coefficient αHr, the respiratory rate threshold coefficient αRs, the change start point Tsc, the reference heart rate Hrc, and the reference respiratory rate Rsc are acquired, the change end point detection unit 112 The change start point Tsc + 1 is substituted into the counter i (S22).

  When the change start point Tsc + 1 is substituted into the counter i, the change end point detection unit 112 determines whether the calculation start point s ′ = 0 (S23). If it is determined that the calculation start point s ′ = 0 (S23, YES), the change end point detection unit 112 determines whether respiration rate vector Rs [i]> respiration rate threshold coefficient αRs × reference respiration rate Rsc ( S24).

  If it is determined that respiratory rate vector Rs [i]> respiratory rate threshold coefficient αRs × reference respiratory rate Rsc (S24, YES), the change end point detection unit 112 substitutes the counter i for the calculation start point s ′ (S25) ).

When the counter i is substituted for the calculation start point s ′, the change end point detection unit 112 determines whether the counter i is equal to or less than the length L (S26). If it is determined that the counter i is equal to or less than the length L (S26, YES), the change end point detection unit 112 adds 1 to the counter i (S27). When 1 is added to the counter i, the change start point detection unit 111 returns to S23.
If it is determined that the counter i is not less than or equal to the length L (S26, NO), the change end point detection unit 112 sets the change end point Tsg = 0 and the next detection start point s = L (S28).

  If it is determined that the respiratory rate vector Rs [i]> respiratory rate threshold coefficient αRs × reference respiratory rate Rsc is not satisfied (S24, NO), the change end point detection unit 112 proceeds to S27.

  If it is determined that the calculation start point s ′ is not 0 (S23, NO), the change end point detection unit 112 determines whether the heart rate vector Hr [i]> heart rate threshold coefficient αHr × reference heart rate Hrc (S29) ).

  When it is determined that the heart rate vector Hr [i]> heart rate threshold coefficient αHr × reference heart rate Hrc (S29, YES), the change end point detection unit 112 determines that the change end point Tsg = i and the next detection start point s = I + 1 (S30).

  If it is determined that the heart rate vector Hr [i]> heart rate threshold coefficient αHr × reference heart rate Hrc is not satisfied (S29, NO), the change end point detection unit 112 returns to S27.

  When the change end point Tsg = 0 and the next detection start point s = L (S28) or when the change end point Tsg = i and the next detection start point s = i + 1 (S30), the change end point detection The unit 112 extracts the heart rate vector Hr ′ and the respiratory rate vector Rs ′ (S31).

  After extracting the heart rate vector Hr ′ and the respiration rate vector Rs ′, the change end point detection unit 112 outputs the change end point Tsg, the heart rate vector Hr ′, the respiration rate vector Rs ′ and the next detection start point s ( S32).

  When the change end point Tsg, the heart rate vector Hr ′, the respiration rate vector Rs ′, and the next detection start point s are output, the change end point detection unit 112 ends the operation.

  Next, an operation example of the emotion change amount calculation unit 113 of the processing unit 102 will be described.

  FIG. 4 is a flowchart for explaining an operation example of the emotion change amount calculation unit 113.

  First, the emotion change amount calculation unit 113 acquires a heart rate vector Hr 'and a reference heart rate Hrc from the change end point detection unit 112 (S41). After acquiring the heart rate vector Hr 'and the reference heart rate Hrc, the emotion change amount calculation unit 113 calculates an estimated amount difHr based on the heart rate vector Hr' and the reference heart rate Hrc (S42).

  When the estimated amount difHr is calculated, the emotion change amount calculating unit 113 outputs the estimated amount difHr (S43). After outputting the estimated amount difHr, the emotion change amount calculation unit 113 ends the operation.

  The result output unit 104 acquires the change start point Tsc, the change end point Tsg, and the estimated amount difHr from the processing unit 102. The result output unit 104 displays the change start point Tsc, the change end point Tsg, and the estimated amount difHr on a display unit or the like or stores them in a nonvolatile memory or the like.

  If the detection start point s <L after the end of emotion estimation, the processing unit 102 may perform the above estimation operation again. Further, if the detection start point s = L, the processing unit 102 may end the operation assuming that the estimation of the emotion has been completed up to the last data.

The information processing apparatus configured as described above can specify the change start point and the change end point of emotion based on the heart rate and the respiration rate of the user. Also, the information processing apparatus can estimate the user's emotion from the heart rate or the respiration rate between the change start point and the change end point. Therefore, the information processing apparatus can estimate the user's emotion without using the user's expression or voice.
Second Embodiment
Next, a second embodiment will be described.
The information processing apparatus according to the second embodiment differs from the information processing apparatus according to the first embodiment in that the laughing amount La is estimated as emotion. Therefore, about the other structure, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  The information processing apparatus according to the second embodiment estimates the amount of laugh La of the user. For example, the information processing apparatus acquires a heart rate and a respiration rate of a user who appreciates a comedy show or the like in a theater or the like, and estimates the laughter amount La of the user. The information processing apparatus may assign an ID to each user and estimate the laughter amount La for each user.

FIG. 5 is a block diagram showing a configuration example of the information processing apparatus 200 according to the second embodiment.
As illustrated in FIG. 5, the information processing apparatus 200 includes a data input unit 101, a processing unit 202, a setting input unit 103, an estimation result storage unit 204, an estimation result output unit 205, and the like.

The processing unit 202 includes a change start point detection unit 111, a change end point detection unit 112, and an emotion change amount calculation unit 213. The emotion change amount calculation unit 213 includes a laugh amount calculation unit 214 and the like.
Since the data input unit 101, the setting input unit 103, the change start point detection unit 111, and the change end point detection unit 112 are the same as those in the first embodiment, description thereof is omitted.

  The operator inputs the window width w, heart rate threshold value dHr, and respiration rate threshold value dRs to the setting input unit 103 in consideration of the user's age group and the like. For example, since the younger user has more rapid changes in heart rate and the like, if the user has more in their twenties, the operator inputs the window width w = 1, the heart rate threshold dHr = 1.0, and the respiratory rate threshold dRs = 1.0. Do.

  The laughing amount calculation unit 214 estimates a laughing amount La generated between the change start point Tsc and the change end point Tsg as the user's emotion. For example, the laughing amount calculation unit 214 calculates the estimated amount difHr in the same manner as the emotion change amount calculating unit 113, and sets the calculated estimated amount difHr as the laughing amount La.

  Also, the laughter amount calculation unit 214 may use the estimated amount difRs as the laughter amount La. In addition, the laughing amount calculation unit 214 may estimate the laughing amount La based on the heart rate vector Hr 'and the respiratory rate vector Rs'. The method by which the laughter amount calculation unit 214 estimates the laughter amount La is not limited to a specific method.

  The laughing amount calculation unit 214 outputs the estimated laughing amount La.

The estimation result storage unit 204 obtains the laughing amount La estimated by the laughing amount calculation unit 214. The estimation result storage unit 204 stores the acquired amount of laugh La in a non-volatile memory or the like. Further, the estimation result storage unit 204 may store the ID of each user and the amount of laugh La in association with each other in a non-volatile memory or the like.
The estimation result storage unit 204 may be configured to include a non-volatile memory.

  The estimation result output unit 205 acquires the laughter amount La estimated by the laughter amount calculation unit 214. The estimation result output unit 205 displays the acquired laughing amount La on a display unit or the like.

The information processing apparatus configured as described above can estimate the amount of laughter as the user's emotion from the heart rate or respiratory rate between the change start point and the change end point.
Third Embodiment
Next, a third embodiment will be described.
The information processing apparatus according to the third embodiment differs from the information processing apparatus according to the second embodiment in that the laughing amount La is corrected with the user's expression and voice. Therefore, about the other structure, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

FIG. 6 is a block diagram showing a configuration example of the information processing apparatus 300 according to the third embodiment.
As illustrated in FIG. 6, the information processing apparatus 300 includes a data input unit 101, a processing unit 302, a setting input unit 103, an estimation result storage unit 204, an estimation result output unit 205, a camera 306, a smile level estimation unit 307, and a microphone 308. And a laughing voice estimation unit 309 and the like.

The processing unit 302 includes a change start point detection unit 111, a change end point detection unit 112, an emotion change amount calculation unit 3, and the like. The emotion change amount calculating unit 313 includes a laughing amount calculating unit 214, a laughing amount correcting unit 315, and the like.
The data input unit 101, the setting input unit 103, the change start point detection unit 111, the change end point detection unit 112, the estimation result storage unit 204, the estimation result output unit 205, and the laughing amount calculation unit 214 are the same as those of the second embodiment. The description is omitted because it is similar.

  The camera 306 photographs the user's facial expression. For example, the camera 306 is installed at a position where the user's face can be photographed, such as in front of the user or obliquely above. For example, the camera 306 is a CCD camera or the like. The camera 306 outputs the photographed image to the smile degree estimation unit 307. The camera 306 captures the user's facial expression as needed, and outputs the captured image to the smile degree estimation unit 307.

  The smile level estimation unit 307 estimates the smile level of the user based on the image captured by the camera 306. The smile level estimation unit 307 estimates the smile level laugh of the user from the image using various methods. The method by which the smile level estimation unit 307 estimates the smile level “laugh” is not limited to a specific method.

The smile level estimation unit 307 estimates the smile level laugh with a value of 0 to 100%.
The smile degree estimation unit 307 outputs the estimated smile degree laugh to the laugh amount correction unit 315.

  The microphone 308 measures the user's voice. For example, the microphone 308 is placed at a position where the user's voice can be measured, such as the chest or front of the user. The microphone 308 outputs the measured voice to the laughing voice estimation unit 309. The microphone 308 measures the user's voice at any time, and outputs the measured voice to the laughter estimation unit 309.

The laughter estimation unit 309 estimates the size voice of the user's laughter based on the voice measured by the microphone 308.
For example, the laughter estimation unit 309 extracts the laughter from the voice measured by the microphone 308. The laughter estimation unit 309 estimates the size of the extracted laughter.

  The laughing voice estimation unit 309 outputs the estimated voice size of the laughing voice to the laughing amount correction unit 315.

The laughing amount correction unit 315 corrects the laughing amount La based on the smile level “laugh” estimated by the smile level estimation unit 307 and the laughter level “voice” estimated by the laughter voice estimation unit.
For example, the laughing amount correction unit 315 corrects the laughing amount La as follows.
First, the laughing amount correcting unit 315 extracts a smile level “rough” and a laughing voice level “voice” generated between the change start point Tsc and the change end point Tsg. The laughing amount correcting unit 315 calculates a corrected laughing amount La ′ (corrected laughing amount La ′) from the smile level “laugh” extracted according to the following equation and the loudness level of the laughing voice.

La '= laugh × La + voice
Here, it is assumed that smile degree laugh converts 0 to 100% into a value of 0 to 1.

  Note that the laughing amount correction unit 315 may correct the laughing amount La using the average value of the smile level “rough” and the level of the laughter voice generated between the change start point Tsc and the change end point Tsg. In addition, the laughing amount correction unit 315 may correct the laughing amount La using the smile level “laugh” and the maximum value of the laughing voice “voice” generated between the change start point Tsc and the change end point Tsg.

  The method by which the laughter amount correction unit 315 corrects the laughter amount La is not limited to a specific method.

  The laughing amount correction unit 315 outputs the corrected laughing amount La ′ to the estimation result storage unit 204.

  The information processing apparatus configured as described above can correct the amount of laughter based on the degree of smile and the size of laughter. As a result, the information processing apparatus can realize more accurate estimation of the amount of laugh.

  Further, the information processing apparatus corrects the laughing amount by adding the smile level to the laughing amount. As a result, when the user is not laughing but the laughing amount is estimated to be large, the information processing apparatus can correct the laughing amount to be small and reduce erroneous detection.

Further, the information processing apparatus corrects the amount of laughter by adding the size of the laughter to the amount of laughter. As a result, the information processing apparatus can reflect the laughing voice in the amount of laughing.
Fourth Embodiment
Next, a fourth embodiment will be described.
The information processing apparatus according to the fourth embodiment differs from the information processing apparatus according to the second embodiment in that the stage effect control apparatus is operated according to the amount of laughter La. Therefore, about the other structure, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

FIG. 7 is a block diagram showing an exemplary configuration of an information processing apparatus 400 according to the fourth embodiment.
As FIG. 7 shows, the information processing apparatus 400 includes a data input unit 101, a processing unit 202, a setting input unit 103, a stage effect control device 410, and the like.

The processing unit 202 includes a change start point detection unit 111, a change end point detection unit 112, an emotion change amount calculation unit 213, and the like. The emotion change amount calculation unit 213 includes a laugh amount calculation unit 214 and the like.
Since the data input unit 101, the setting input unit 103, the change start point detection unit 111, the change end point detection unit 112, and the laughter amount calculation unit 214 are the same as those in the second embodiment, description thereof is omitted.

The stage effect control device 410 (stage effect control unit) controls the stage device based on the laughter amount La estimated by the laughter amount calculation unit 214.
Here, the laughing amount calculation unit 214 estimates the laughing amount La of a plurality of users who view the stage.

For example, the stage effect control device 410 controls lighting of the stage.
The stage effect control device 410 controls the illumination of the stage based on the amount of laugh La. For example, the stage effect control device 410 controls the lighting in accordance with the proportion of the user who laughed in the last minute. For example, the stage effect control device 410 may determine that the user who laughs more than the predetermined threshold is a laughing user.

  For example, the stage effect control device 410 turns off the light if the proportion of the laughing user is less than 25%, turns on the light at 200 Lx if it is less than 25 to 50%, and 500 Lx if it is less than 50 to 80%. The illumination is turned on, and if it is 80% or more, the illumination is turned on at 1000 Lx.

  For example, the stage effect control device 410 calculates the percentage of users who laugh every minute and controls the lighting.

  The stage effect control device 410 may control sound, lighting color, seat vibration, and the like. The device controlled by the stage effect control device 410 is not limited to a specific configuration.

  The information processing apparatus configured as described above operates the stage effect control apparatus based on the estimated amount of laugh. As a result, the information processing apparatus can control the stage effect based on the amount of laughter.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components in different embodiments may be combined as appropriate.

  In addition, the method described in each embodiment is, for example, a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, etc.) as a program (software means) that can be executed by a computer (computer). It can be stored in a recording medium such as a DVD, MO, etc., semiconductor memory (ROM, RAM, flash memory, etc.), or transmitted and distributed by a communication medium. The program stored on the medium side includes a setting program that configures software means (including not only the execution program but also a table and data structure) in the computer. A computer that implements this apparatus reads a program recorded on a recording medium, constructs software means by a setting program as the case may be, and executes the above-described processing by controlling the operation by this software means. The recording medium referred to in this specification is not limited to distribution, but includes a storage medium such as a magnetic disk or a semiconductor memory provided in a computer or a device connected via a network.

  100, 200, 300 and 400 ... information processing apparatus, 101 ... data input unit, 102, 202 and 302 ... processing unit, 103 ... setting input unit, 104 ... result output unit, 111 ... change start point detection unit, 112 ... change End point detection unit 113, 213 and 313: emotion change amount calculation unit 204: estimation result storage unit 205: estimation result output unit 214: laughing amount calculation unit 306: camera 307: smile degree estimation unit 308 ... Microphone, 309 ... Laughter estimation unit, 315 ... Laughter amount correction unit.

Claims (7)

An information processing apparatus for estimating a user's emotions, comprising:
An acquisition unit for acquiring the heart rate and the respiration rate of the user;
A start point detection unit that detects a start point of the user's emotion based on the heart rate and the respiration rate;
An end point detection unit that detects an end point of the user's emotion based on the heart rate and the respiration rate;
An emotion estimation unit for estimating an emotion generated between the start point and the end point;
Equipped with a,
The start point detection unit detects, as the start point, a time point when the heart rate and the change amount of the respiration rate exceed a heart rate threshold and a respiration rate threshold, respectively, and a heart rate before the start point is a reference heart rate Output the respiration rate before the start point as a reference respiration rate,
The end point detection unit detects a time point when the heart rate returns to the reference heart rate and the respiration rate returns to the reference respiration rate as the end point.
Information processing device. The emotion estimation unit estimates emotion based on the sum of the difference between the reference heart rate and the heart rate or the sum of the difference between the reference respiration rate and the respiration rate between the start point and the end point.
The information processing apparatus according to claim 1 . The emotion estimation unit estimates the amount of laughter as an emotion.
The information processing apparatus according to claim 1 . A smile level estimation unit that estimates the smile level of the user;
A laughter estimation unit for estimating the size of the user's laughter;
Equipped with
The emotion estimation unit corrects the amount of laugh based on the degree of smile and the size of the laugh.
The information processing apparatus according to claim 3 . A stage effect control unit configured to control a stage effect based on the emotion;
The information processing apparatus according to any one of claims 1 to 4 . An information processing method for estimating a user's emotion, comprising:
Get the user's heart rate and breathing rate,
Detecting a start point of the user's emotion based on the heart rate and the respiration rate;
Detecting an end point of the user's emotion based on the heart rate and the respiration rate ;
Estimating the emotion that has occurred between the start point and the end point,
Detecting the start point is to detect a time point when the amount of change in the heart rate and the respiration rate exceeds a heart rate threshold value and a respiration rate threshold value as the start point, and use the heart rate before the start point as a reference Output as heart rate, and output respiration rate before the start point as reference respiration rate,
Detecting the end point detects a time point when the heart rate returns to the reference heart rate and the respiration rate returns to the reference respiration rate as the end point.
Information processing method. It is a program used for the computer which operate | moves as a part of information processing apparatus in any one of Claims 1 thru | or 5 , Comprising:
The computer,
An acquisition unit, a start point detection unit, an end point detection unit, and an emotion estimation unit;
Program to function as.