WO2023058461A1

WO2023058461A1 - Notification program, notification device, and notification method

Info

Publication number: WO2023058461A1
Application number: PCT/JP2022/035243
Authority: WO
Inventors: 智坂口; 真衣香 ▲高▼橋; 達也宗
Original assignee: ユニ・チャーム株式会社
Priority date: 2021-10-08
Filing date: 2022-09-21
Publication date: 2023-04-13
Also published as: CN118077005A; JP2023056870A

Abstract

A notification program according to the present application causes a computer to execute: an acquisition step for acquiring voice information indicating a voice uttered by a response target to be responded to by a responder; an estimation step for estimating the state of the response target and the probability of being in said state on the basis of the voice information acquired in the acquisition step; and a notification step for notifying of the state of the response target, estimated in the estimation step as a desire or emotion of the response target, together with the accuracy and the recommended action of the responder for the state.

Description

Notification program, notification device and notification method

The present invention relates to a notification program, a notification device, and a notification method.

Conventionally, there is known a technique for estimating the cause of the crying of a corresponding target (infants, etc.) based on the crying of the corresponding target. As an example of such technology, there is known a technology for estimating the cause of an infant's crying based on a signal output from a sensor unit attached to the infant after the infant has started crying. Also, if it is determined that the baby is crying because the diaper is wet, based on the baby's crying, a wet diaper determination screen will be displayed showing that the baby is crying because the diaper is wet. A technique for displaying on a display unit is known.

JP 2016-126367 A JP 2019-095931 A

However, it may not be possible to say that the conventional technology described above enhances the self-affirmation of the respondent regarding the response to the response target.

For example, in the conventional technology described above, it is only assumed that the cause of an infant's crying is identified and the childcare person takes measures according to the cause. For this reason, in the conventional technology described above, for example, the child-rearing person can accumulate a successful experience such as when the child stops crying by deducing the cause of the child's crying, thinking of a coping method, and taking action. Can not. For this reason, it cannot be said that the above-described conventional technology enhances the self-affirmation of the responder regarding the response to the response target.

This application was made in view of the above, and aims to increase the self-affirmation of the respondent regarding the response to the response target.

A notification program according to the present application includes an acquisition procedure for acquiring voice information indicating a voice uttered by a responder to be handled by a responder, a state of the responder based on the voice information acquired by the acquisition procedure, an estimation procedure for estimating the certainty of the state, and notification of the state of the correspondence target estimated by the estimation procedure as the desire or emotion of the correspondence object, together with the accuracy and a recommended action of the respondent for the state. and a notification procedure to be executed by a computer.

According to one aspect of the embodiment, it is possible to increase the self-affirmation of the responder regarding the response to the response target.

FIG. 1 is a diagram (1) illustrating an example of notification processing according to an embodiment. FIG. 2 is a diagram (1) showing an example of a screen of the terminal device according to the embodiment; FIG. 3 is a diagram (2) showing an example of a screen of the terminal device according to the embodiment; FIG. 4 is a diagram (3) showing an example of a screen of the terminal device according to the embodiment; FIG. 5 is a diagram (4) showing an example of a screen of the terminal device according to the embodiment; FIG. 6 is a diagram (2) illustrating an example of notification processing according to the embodiment. FIG. 7 is a diagram illustrating a configuration example of an information processing apparatus according to the embodiment; FIG. 8 is a diagram illustrating an example of a correspondence target information database according to the embodiment; FIG. 9 is a diagram illustrating an example of a responder information database according to the embodiment; FIG. 10 is a diagram illustrating a configuration example of a detection device according to the embodiment; 11 is a flowchart illustrating an example of the flow of notification processing executed by the information processing apparatus according to the embodiment; FIG. FIG. 12 is a diagram illustrating an example of a hardware configuration;

At least the following matters become clear from the description of this specification and the attached drawings.

Acquisition procedure for acquiring voice information indicating a voice uttered by a response target handled by a responder, and obtaining the state of the response target and the probability of being in that state based on the voice information acquired by the acquisition procedure an estimation procedure for estimating; and a notification procedure for notifying the state of the correspondence target estimated by the estimation procedure as the desire or emotion of the correspondence target together with the accuracy and the recommended action of the respondent for the state. A notification program characterized in that it is executed by

According to such a notification program, for example, on the basis of a cry or other sound uttered by a target such as an infant, it is possible to identify the state in which the target is crying and determine the accuracy for each specified state. can be estimated and notified to the responder. As a result, the responder can think about how to deal with each presumed state of the response target and act on it, so that the responder can build up successful experiences such as an infant stopping crying. It is possible to increase the self-affirmation of the respondent regarding the response to In addition, for example, since a notification indicating the degree of certainty of a desire such as eating, excretion, or sleep can be provided to the subject, it is possible to appropriately respond to the subject. In addition, for example, since it is possible to notify the responder of the emotional development of the respondent, it is possible to relieve the responder's anxiety about the delay in the development of the respondent. In addition, for example, since it is possible to present an action to be taken as a countermeasure to the correspondence target to the correspondence person, it is possible to reduce anxiety about the correspondence target crying.

Also, the notification program provides a notification indicating that the corresponding target is not in any of the states when the corresponding target is not estimated to be in any of the states.

According to such a notification program, for example, in a state where the subject is crying without desire or emotion as the main cause, such as occurs in mental leap, the respondent is notified that the crying is caused by mental leap. Therefore, it is possible to reduce the effort of the responder to investigate the cause.

Also, if the accuracy cannot be estimated based on the voice information, the notification program estimates the accuracy based on the history of the state of the correspondence target. Further, when the accuracy cannot be estimated based on the voice information, the accuracy may be estimated based on the history of the state of the response target input by the responder.

According to such a notification program, for example, even if the speech of the correspondence target cannot be read due to background noise, the state of the correspondence target can be estimated. This prevents the person from being unable to obtain the estimation result, and allows the responder to know the behavior that should be addressed, so that the responder can positively respond to the response target.

In addition, the notification program provides the responder with a message according to the responder's response to the state of the response target based on the voice uttered by the response target.

According to such a notification program, for example, when the person to be dealt with stops crying, it is possible to provide a positive message about the behavior of the person responding, thereby increasing the sense of self-affirmation of the person responding.

Also, the notification program provides a message according to the time from when the notification is provided until the voice uttered by the corresponding target satisfies a predetermined condition.

According to such a notification program, for example, when it takes a long time for the respondent to stop crying, it is possible to provide a positive message about the behavior of the respondent, thereby increasing the respondent's sense of self-affirmation. be able to.

In addition, the notification program provides the responder with a message according to the response of the responder to the state of the response target based on the voice uttered by the responder.

According to such a notification program, for example, it is possible to evaluate the response of the respondent to the target of the response and provide a positive message, thereby increasing the self-esteem of the respondent.

In addition, the notification program provides the responder with a message according to the use of the content regarding the status of the response target.

According to such a notification program, for example, when the responder browses the content related to the flow of coping when the respondent cries, the responder can be evaluated and a positive message can be provided. Therefore, the self-affirmation of the respondent can be enhanced.

In addition, the notification program provides the responder with content that poses a question regarding the correspondence relationship between the voice of the response target and the state of the response target.

According to such a notification program, for example, the responder can deepen his/her understanding of the correspondence relationship between the accumulated cry of the respondent and the state of the respondent, thereby relieving the anxiety of the responder when he or she cries. be able to. In addition, when the responder is able to give a correct answer to the accumulated cry of the target to be addressed and the state of the corresponding target to be addressed, the responder can be evaluated and a positive message can be provided. You can increase your self-esteem.

In addition, the notification program provides a user having a predetermined relationship with the responder with a message according to the responder's response to the state of the response target.

According to such a notification program, for example, a message evaluating the respondent who is the mother can be provided to the user who is the father, so that the respondent can obtain social persuasion. In particular, by providing a user who is a father with a message that encourages and praises the respondent who is a mother, the user who is a father can evaluate the respondent who is a mother favorably. Therefore, the respondent can obtain social persuasion.

Also, the notification program provides the responder with content indicating the history of the status of the response target based on the estimation result.

According to such a notification program, for example, the estimated result can be provided to the respondent as a childcare record, so convenience can be improved.

In addition, the notification program provides a predetermined message to the responder if the state of the response target is not cleared for a predetermined time or longer after the notification is provided.

According to such a notification program, for example, if the target does not stop crying for a certain period of time, a message can be provided to the effect that the estimation of the target's condition was incorrect, thereby reducing the respondent's dissatisfaction. can do.

In addition, the notification program provides the responder with a message posted by another responder different from the responder.

According to such a notification program, for example, it is possible to provide the experiences of other coordinators who are raising children, so that the coordinator can obtain a substitute experience.

In addition, the notification program provides the message, which is the amount of information according to the timing of providing to the responder.

According to such a notification program, for example, a long message can be provided when the responder has spare time, so it is possible to provide a message with an appropriate amount of information according to the situation of the responder. .

Also, the notification program provides the message, which is the amount of information according to the state of the response target.

According to such a notification program, for example, when a responder is busy with dealing with a subject to be dealt with, a short message can be provided. message can be provided.

Also, the notification program provides the message posted by the responder to another responder different from the responder.

According to such a notification program, for example, it is possible for the coordinator to provide other coordinators with stories of their own experiences in childcare, so that the experiences on behalf of the coordinator can be inherited.

In addition, the notification program estimates the accuracy using a model that has learned the features of the voice uttered by the corresponding target, and the information regarding the state of the corresponding target after the notification is provided and the voice information indicated The model is trained on speech features.

According to such a notification program, for example, a model can be patterned and personalized by learning the characteristics of the voice uttered by the responder, and the model can be used to estimate the state of the response target. can improve the accuracy of

In addition, after the notification is provided, the notification program receives information about the state of the response target from the responder, and causes the model to learn the received information and the characteristics of the voice indicated by the voice information.

According to such a notification program, for example, model learning can be performed using information about the actual state of the response target, so it is possible to improve the accuracy of estimation.

An example of a form (hereinafter referred to as "embodiment") for implementing the notification program, notification device, and notification method will be described in detail below with reference to the drawings. Note that the notification program, the notification device, and the notification method are not limited by this embodiment. Also, in the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[Embodiment]
[1. About an example of notification processing]
The notification processing realized by the notification program etc. of this embodiment will be described with reference to FIG. FIG. 1 is a diagram (1) illustrating an example of notification processing according to an embodiment. In FIG. 1, it is assumed that the information processing apparatus 10 according to the present embodiment implements the notification processing and the like according to the embodiment.

As shown in FIG. 1, the notification system 1 according to the embodiment includes an information processing device 10, a terminal device 100, and a detection device 200. They are communicably connected to each other by wire or wirelessly via a network N (see, for example, FIG. 6). Network N is, for example, a WAN (Wide Area Network) such as the Internet. Note that the notification system 1 shown in FIG. 1 may include a plurality of information processing devices 10, a plurality of terminal devices 100, and a plurality of detection devices 200. FIG.

The information processing device 10 shown in FIG. 1 is an information processing device that performs notification processing, and is realized by, for example, a server device, a cloud system, or the like. For example, in the example of FIG. 1 , the information processing apparatus 10 provides a user U1 who is raising an infant B1, in other words, when the infant B1 to be dealt with is crying, the information processing apparatus 10 performs various functions such as holding, feeding, and diapering the infant B1. Various types of information are notified to the user U1, who is the person who takes care of the exchange, comfort, and the like.

A terminal device 100 shown in FIG. 1 is an information processing device used by a user who is a responder. The terminal device 100 is implemented by, for example, a smartphone, tablet terminal, notebook PC (Personal Computer), desktop PC, mobile phone, PDA (Personal Digital Assistant), and the like. In addition, the terminal device 100 displays information delivered by the information processing device 10 or the like using a web browser or an application. In the example shown in FIG. 1, the terminal device 100 is a smart phone used by the user U1.

The detection device 200 shown in FIG. 1 is a detection device that detects voice information indicating the state of an infant to be addressed, and various IoT (Internet of Things) devices that collect information from sensing devices including various sensing devices or smart speakers. (Things) equipment. For example, the detection device 200 is implemented by a sound sensor such as a microphone installed near the infant and capable of detecting at least the sound uttered by the infant.

The notification process realized by the information processing apparatus 10 will be described below with reference to FIG. In the following description, the terminal device 100 may be identified with the user U1. That is, the user U1 can be read as the terminal device 100 below.

First, the detection device 200 detects crying of the infant B1 (step S1). For example, the detection device 200 uses a sound sensor to collect sounds emitted by the infant B1, and determines whether or not crying of the infant B1 has been detected.

Next, the detection device 200 transmits audio information indicating the cry of the infant B1 to the terminal device 100 registered as a transmission destination (step S2). For example, the detection device 200 transmits, to the terminal device 100, voice information indicating the voice determined that the infant B1 is crying.

Subsequently, the information processing device 10 acquires voice information transmitted from the terminal device 100 (step S3). Subsequently, information processing device 10 estimates the degree of certainty of each state of infant B1 based on the acquired voice information (step S4). For example, the information processing device 10, based on the voice information, determines the accuracy of each desire of the infant B1 (for example, "hold", "breastfeeding", "change diaper", "sleepy", etc.) and the feeling of the infant B1. The accuracy of each emotion (for example, “lonely”, “anger”, “sad”, “tired”, etc.) is estimated. As a specific example, the information processing device 10 uses model #1 that has been trained so as to output the degree of certainty of a state such as desires and emotions of an infant when voice information is input. The probability of each state of infant B1 is estimated.

Here, any known technique can be applied to model learning, and a learning method appropriately selected according to the information to be acquired may be used. For example, model learning may be performed using various conventional techniques related to machine learning (for example, techniques related to supervised machine learning such as SVM (Support Vector Machine)). In addition, deep learning technology may be used for model learning. For example, various deep learning techniques such as RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network) may be used for model learning.

Subsequently, the information processing device 10 provides a notification indicating the degree of certainty of each state of the infant B1 (step S5). Subsequently, the terminal device 100 displays the notification provided from the information processing device 10 on the screen (step S6). Subsequently, user U1 responds to infant B1 based on the notification displayed on the screen of terminal device 100 (step S7). For example, the user U1 performs actions such as hugging, breastfeeding, and changing diapers. User U1's response includes not only direct contact and communication with infant B1, such as hugging, breastfeeding, and changing diapers, but also direct behavior such as leaving infant B1 alone. It also includes measures that are not taken. In particular, when it is presumed that the target does not require direct action, such as crying in his sleep, the terminal device 100 notifies him of a response that does not take direct action itself, such as leaving him alone.

Subsequently, the information processing device 10 receives information about the condition of the infant B1 from the terminal device 100 (step S8). For example, the information processing device 10 receives information indicating whether or not the infant B1 has stopped crying (in other words, whether or not the estimation result matches the actual state of the infant B1) from the terminal device 100.

Here, an example of the notification displayed on the screen by the terminal device 100 in the above-described embodiment will be described using FIG. FIG. 2 is a diagram (1) showing an example of a screen of the terminal device according to the embodiment; In the example shown in FIG. 2, the information processing device 10 estimates the accuracy of the infant B1 desire "breastfeeding" to be 60% and the accuracy of the infant B1 desire "sleepy" to be 40%, and notifies the user of the estimation results. It is assumed that the terminal device 100 is provided with the data.

First, the terminal device 100 displays a screen C11 for displaying the estimation result of the state of the infant B1 by the information processing device 10. FIG. For example, the terminal device 100 includes an area AR11 for displaying the estimation result, an area AR12 for displaying the specific content of the response, and a button BT11 for notifying the information processing apparatus 10 that the infant B1 has stopped crying. and a button BT12 for notifying the information processing apparatus 10 that the infant B1 will not stop crying. As a specific example, the terminal device 100 associates the text "I'm hungry" that interprets the feeling of the infant B1 having the desire "breastfeeding" with the probability of 60% for the desire "breastfeeding", Display on AR11. Next, the terminal device 100 associates the text "I'm sleepy" that interprets the feeling of the infant B1 having the desire "I'm sleepy" with the desire "I'm sleepy" with a probability of 40%, and displays them in the area AR11. That is, the terminal device 100 displays each state estimated by the information processing device 10 in descending order of accuracy.

In this way, not only is the estimation result of the emotion/state of the target to be notified, but also the infant is metaphorized as in the area AR11 of FIG. By displaying contents that are easy for the responder to take concrete actions, the user U1 who is the responder is supported to estimate the needs of the infant B1 to be addressed, and the user U1 successfully responds to the infant B1. Accuracy can be improved. As a result, the responder can estimate the desire of the target of the response and trace the behavior of taking an appropriate response, so that the responder's sense of self-affirmation can be improved.

Here, in the example of FIG. 2, it is assumed that the button BT11 has been pressed. In this case, the terminal device 100 transmits information indicating that the infant B1 has stopped crying to the information processing device 10, and transitions the screen C11 to a screen C12 displaying a predetermined message. For example, the terminal device 100 displays an area AR13 for displaying a message for changing the user U1's understanding of the crying of the infant B1, and displays content for deepening the user U1's understanding of the crying of the infant B1. A screen C12 including a button BT13 for instructing and a button B14 for closing the notification provided from the information processing apparatus 10 is displayed. To give a specific example, the terminal device 100 displays a message to the effect that the infant B1 is crying not because of an emotion such as "sad" but because of communication with the user U1 in the area AR13. to display.

Note that when the button BT11 is pressed on the screen C11, the terminal device 100 accepts input of information indicating the action taken by the user U1, the condition of the infant B1 estimated by the user U1, and the like, and the information processing device 10 may be sent to Further, when the button BT12 is pressed, the terminal device 100 accepts input of information indicating the condition of the infant B1 estimated by the user U1, and transmits the information to the information processing apparatus 10 together with information indicating that the infant B1 does not stop crying. You may That is, the terminal device 100 processes information indicating whether or not the content of the notification provided by the information processing device 10 matches the actual state of the infant B1, as well as information indicating the actual state of the infant B1. It may be sent to device 10 .

Also, the terminal device 100 may transmit information detected by the detection device 200 to the information processing device 10 . For example, when the detection device 200 detects that the baby B1 has stopped crying, the terminal device 100 transmits information indicating that the baby B1 has stopped crying to the information processing device 10 . Further, when the detection device 200 continues to detect the crying of the infant B1 for a certain period of time (for example, 30 minutes or 1 hour) after the screen C11 is displayed, the terminal device 100 detects that the infant B1 is crying. Information indicating not to stop is transmitted to the information processing device 10 .

In addition, the estimation result of the state of infant B1 by information processing device 10 and the screen displayed by terminal device 100 are not limited to the example in FIG. Here, another example of the screen displayed by the terminal device 100 in the above embodiment will be described with reference to FIG. FIG. 3 is a diagram (2) showing an example of a screen of the terminal device according to the embodiment;

As an example, if infant B1 is not estimated to be in any state, the information processing device 10 determines that infant B1 is in a state of crying without desire or emotion as the main cause, as occurs in mental leap. A notification indicating the possibility is provided to the terminal device 100, and a screen C21 indicating the notification is displayed. For example, the terminal device 100 displays the voice waveform of the infant B1, a message indicating that the target is crying without desire or emotion as the main cause, as occurs in the infant B1 in mental leap. A screen C21 including an area AR21 to be displayed and a button BT21 for instructing display of content including detailed information on the state (for example, information on mental leap) is displayed.

Also, when the button BT12 on the screen C11 shown in FIG. 2 is pressed, or when the button BT11 is not pressed even after a certain period of time (for example, 30 minutes or 1 hour) has elapsed since the screen C11 was displayed (in other words, , the infant B1 does not stop crying after a certain period of time), the terminal device 100 causes the screen C11 shown in FIG. 2 to transition to the screen C22. For example, the terminal device 100 displays a screen C22 including an area AR22 displaying an apology for misestimating the condition of the infant B1 and a message to calm the user U1.

Return to Figure 1 and continue the explanation. The information processing device 10 causes the model #1 to learn the information received from the terminal device 100 and the voice features indicated by the voice information (step S9). For example, information processing apparatus 10 learns model #1 using information indicating whether infant B1 has stopped crying and voice information corresponding to the estimation result as learning data. In addition, the information processing apparatus 10 performs learning on the model #1 using the response taken by the user U1 to stop the infant B1 from crying and the voice information indicating the crying as learning data. As a result, the information processing apparatus 10 patterns and personalizes the model #1 as a model for estimating the state of the infant B1.

Subsequently, the information processing device 10 provides the message and content to the user U1 based on the accumulated voice information of the infant B1 (step S10). For example, the information processing device 10 provides content for the user U1 to deepen his understanding of the infant B1, a message for caring for the user U1, and the like.

Here, an example of content provided by the information processing apparatus 10 in the above-described embodiment will be described using FIG. FIG. 4 is a diagram (3) showing an example of a screen of the terminal device according to the embodiment; In the example of FIG. 4, the information processing device 10 provides content (quiz content) that asks a quiz about the correspondence relationship between the cry of the infant B1 and the state of the infant B1.

First, the terminal device 100 displays a screen C31 showing a quiz about the correspondence relationship between the cry of the infant B1 and the state of the infant B1. For example, the terminal device 100 displays a screen C31 that includes an area AR31 that displays the waveform of infant B1's crying, an area AR32 that displays a quiz, and buttons BT31 to BT33 that respectively correspond to options for answering the quiz. , the crying voice of the infant B1 corresponding to the waveform displayed in the area AR31 is output.

When the user U1 selects any one of the buttons BT31 to BT33 on the screen C31, the terminal device 100 causes the screen C31 to transition to a screen C32 displaying the answer results of the user U1. For example, the terminal device 100 displays a screen C32 including an area AR33 for displaying the result of the answer instead of the area AR32, and a button BT34 for instructing display of the crying waveform of the infant B1 corresponding to each option. do.

When the user U1 selects the button BT34 on the screen C32, the terminal device 100 causes the screen C32 to transition to a screen C33 displaying the waveform of the cry of the infant B1. For example, the terminal device 100 displays an area AR33 that displays the crying waveform of the infant B1 corresponding to the option "I'm hungry", and an area AR34 that displays the waveform of the crying infant B1 corresponding to the option "Cuddle me". A screen C33 containing is displayed.

Note that the quiz content described above can be used not only by user U1, but also by user U2 who has a predetermined relationship with user U1 (for example, if user U1 is the mother of infant B1, the user who is the father of infant B1). It may be provided to the terminal device 101 used by the person U2). In such a case, the terminal device 101 first displays the screen C31 like the terminal device 100 does. Then, when the user U2 selects any one of the buttons BT31 to BT33, the terminal device 101 causes the screen C31 to transition to a screen C34 displaying the result of the user U2's answer. For example, instead of the area AR32, the terminal device 101 includes an area AR34 that displays a comparison between the answer result and the answer result of the user U1, and displays the crying waveform of the infant B1 corresponding to each option. A screen C34 including a button BT35 for instructing is displayed. Here, when the button BT35 is pressed, the terminal device 101 transitions the screen C34 to the screen C33.

Next, an example of a message provided by the information processing device 10 in the above embodiment will be described using FIG. FIG. 5 is a diagram (4) showing an example of a screen of the terminal device according to the embodiment; In the example of FIG. 5, the information processing apparatus 10 provides messages (experience stories, etc.) posted by other users (users raising infants).

In the example shown in FIG. 5, the information processing device 10 provides the terminal device 100 with a message according to the situation of the user U1, and displays it on the screen. For example, when the number of times the information processing apparatus 10 performs the process of estimating the state of the infant B1 (in other words, the number of times the infant B1 cries) is equal to or greater than a predetermined threshold, or when the infant B1 cries after the notification is provided. If it takes more than a predetermined time to stop, or if the message is provided within a predetermined time after the process of estimating the condition of the infant B1, in other words, if it is estimated that the user U1 is tired. , the terminal device 100 displays a screen C41. As a specific example, the terminal device 100 displays a screen C41 including an area AR41 for displaying short messages having a predetermined number of characters or less among messages posted by other users.

Further, when the number of times the information processing device 10 has performed the process of estimating the state of the infant B1 is less than a predetermined threshold value, or when the user U1 is using a predetermined application such as a nursing application, the infant goes to bed. If it is estimated that the user U1 has enough time, the terminal device 100 displays the screen C42. To give a specific example, the terminal device 100 displays an area AR42 for displaying a message having more characters than a predetermined number among messages posted by other users, and an area AR42 for displaying the preamble of the message. button BT41 and a screen C42 is displayed.

Note that the information processing device 10 may provide the terminal device 100 with content for accepting a message posted by the user U1 for other users, and display the content on the screen. For example, the terminal device 100 displays a screen C43 including an area AR43 for inputting a short message with a predetermined number of characters or less and an area AR44 for inputting a message with more characters than the predetermined number. The terminal device 100 then transmits the messages input to the areas AR43 and AR44 to the information processing apparatus 10 .

As described above, the information processing apparatus 10 according to the embodiment estimates the accuracy for each state, such as the desires that the infant desires and the emotions that the infant feels, based on the cries of the infant, and notifies the user who is raising the infant. . As a result, the information processing apparatus 10 according to the embodiment provides a sense of security that the infant can stop crying if the corresponding states are dealt with in descending order of accuracy, and the user himself/herself predicts the baby's desires and emotions. Respondents can accumulate successful experiences, such as taking action on their own, and the infant stopping crying as a result of the action. That is, the information processing apparatus 10 according to the embodiment can enhance the self-affirmation of the responder regarding the response to the response target.

Note that the notification system 1 according to the embodiment is not limited to the connection relationship among the information processing device 10, the terminal device 100, and the detection device 200 described above. For example, the detection device 200 may be connected to the information processing device 10 and the terminal device 100 may not be connected to the information processing device 10 . As a specific example, the detection device 200 transmits voice information detected by itself to the information processing device 10 , receives a notification indicating an estimation result transmitted from the information processing device 10 , and provides the information to the terminal device 100 . do.

Also, the terminal device 100 and the detection device 200 may be individually connected to the information processing device 10 . As a specific example, the detection device 200 transmits audio information detected by itself to the information processing device 10 . Then, the terminal device 100 receives a notification indicating an estimation result transmitted from the information processing device 10 .

Also, the notification system according to the embodiment may be composed of the terminal device 100 and the detection device 200 as shown in FIG. In that case, the detection device 200 executes the processes corresponding to steps S3 to S5 and steps S8 to S10 in FIG. Here, the notification system 2 configured by the terminal device 100 and the detection device 200 will be described using FIG. FIG. 6 is a diagram (2) illustrating an example of notification processing according to the embodiment. In addition, since the terminal device 100 and the detection device 200 shown in FIG. 6 have the same configuration as the notification system 1 shown in FIG. 1, description thereof will be omitted.

As shown in FIG. 6, the detection device 200 detects (acquires) the cry of the infant B1 (step Sa1), and estimates the accuracy of each state of the infant B1 based on the audio information indicating the cry of the infant B1 ( Step Sa2). Subsequently, detection device 200 provides terminal device 100 with a notification indicating the accuracy of each state of infant B1 (step Sa3). Note that the processing of steps Sa1 to Sa3 is the same as steps S1, S4, and S5 in FIG. 1, respectively, so description thereof will be omitted.

Subsequently, the terminal device 100 displays the notification provided by the detection device 200 on the screen (step Sa4). Subsequently, user U1 responds to infant B1 based on the notification displayed on the screen of terminal device 100 (step Sa5).

Subsequently, the detection device 200 receives information about the state of the infant B1 from the terminal device 100 (step Sa6), and uses the received information and the features of the voice indicated by the voice information as a model to be used for estimating each state of the infant. Let it learn (step Sa7). Subsequently, detection device 200 provides message and contents to user U1 based on the accumulated voice information of infant B1 (step Sa8). Note that the processing of steps Sa6 to Sa8 is the same as steps S8 to S10 in FIG. 1, respectively, so description thereof will be omitted.

[2. Configuration of Information Processing Device]
Next, the configuration of the information processing device 10 in the notification system 1 shown in FIG. 1 will be described using FIG. FIG. 7 is a diagram illustrating a configuration example of an information processing apparatus according to the embodiment; As shown in FIG. 7 , the information processing device 10 has a communication section 20 , a storage section 30 and a control section 40 .

(Regarding communication unit 20)
The communication unit 20 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 20 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the terminal device 100, the detection device 200, and the like.

(Regarding storage unit 30)
The storage unit 30 is implemented by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 7 , the storage unit 30 has a correspondence target information database 31 , a correspondence person information database 32 and a model database 33 .

(Regarding the correspondence target information database 31)
The correspondence target information database 31 stores various kinds of information related to a correspondence target (infants) to be dealt with (raised) by a responder (parent or the like). Here, an example of information stored in the correspondence target information database 31 will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of a correspondence target information database according to the embodiment; In the example of FIG. 8, the correspondence target information database 31 includes "correspondence target ID", "corresponding person ID", "detection device ID", "month age", "voice information", "detection date and time", "estimation result", It has items such as "Correct/False Information".

"Corresponding target ID" indicates identification information for identifying the corresponding target. The “responder ID” indicates identification information for identifying the responder who handles the response target. The “detection device ID” indicates identification information for identifying the detection device 200 that detects the audio to be handled. "Moon age" indicates the age in months of the corresponding object. "Voice information" indicates the voice to be handled. The "detection date and time" indicates the date and time when the audio information was detected. "Estimation result" indicates information about the state of the correspondence target estimated based on the voice information. The “correctness information” indicates information transmitted from the responder regarding the estimation result (for example, information indicating whether or not the response target has stopped crying).

That is, in FIG. 8, the response target identified by the response target ID "BID #1" is handled by the responder identified by the responder ID "UID #1", and the voice of the response target is detected by the detection device ID " DID#1”, the age of the corresponding target is “3 months”, and the voice information detected at the detection date and time “detection date and time #1” for the corresponding target is “voice information # 1”, the estimation result based on the speech information is “estimation result #1”, and the correct/incorrect information is “correct/incorrect information #1”.

(Regarding the responder information database 32)
The responder information database 32 stores various types of information about the responders who respond to the response targets. Here, an example of information stored in the responder information database 32 will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of a responder information database according to the embodiment; In the example of FIG. 9, the responder information database 32 has items such as "responder ID", "concerned party information", and "posted message".

"Responder ID" indicates identification information for identifying the responder who will respond to the response target. "Related party information" indicates information related to a user who has a predetermined relationship with the responder. "Posted message" indicates a message posted by the correspondent.

That is, in FIG. 9, the information of the user who has a predetermined relationship with the responder identified by the responder ID "UID #1" is "related person information #1", and the message posted by the responder is " message #1”.

(Regarding model database 33)
The model database 33 stores models that have learned the features of the speech uttered by the corresponding object. For example, the model database 33 stores, for each corresponding target, a model that has been trained to output the degree of certainty of the state of the corresponding target, such as desires and emotions, when voice information is input.

(Regarding the control unit 40)
The control unit 40 is a controller, and various programs stored in a storage device inside the information processing apparatus 10 are stored in the RAM by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). It is realized by executing as Also, the control unit 40 is a controller, and is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 40 according to the embodiment includes an acquisition unit 41, an estimation unit 42, a notification unit 43, a provision unit 44, a reception unit 45, and a learning unit 46, as shown in FIG. implements or performs the information processing functions and operations described in .

(Regarding the acquisition unit 41)
Acquisition unit 41 acquires voice information indicating a voice uttered by a response target handled by a responder. For example, in the example of FIG. 1 , the acquisition unit 41 acquires voice information indicating the cry of the infant B<b>1 and stores it in the correspondence target information database 31 .

(Regarding the estimation unit 42)
Based on the voice information acquired by the acquisition unit 41, the estimation unit 42 estimates the state of the correspondence target and the probability of being in the state. For example, in the example of FIG. 1, the estimation unit 42 estimates the certainty for each state of the infant B1 based on the voice information.

Also, the estimation unit 42 may estimate the certainty of a predetermined desire desired by the corresponding target for each desire. For example, in the example of FIG. 1, the estimation unit 42 estimates the accuracy for each desire of the infant B1 such as "hold", "breastfeed", "change diaper", and "sleepy".

In addition, the estimation unit 42 may estimate the degree of certainty of the emotion felt by the corresponding target for each emotion. For example, in the example of FIG. 1, the estimation unit 42 estimates the accuracy for each emotion of the infant B1, such as "lonely", "anger", "sad", and "tired".

In addition, when the accuracy cannot be estimated based on the voice information, the estimation unit 42 may estimate the accuracy based on the history of the state of the correspondence target. For example, in the example of FIG. 1, when the estimating unit 42 acquires the voice information of the infant B1, the estimation unit 42 refers to the childcare record of the infant B1, and in the childcare record, breastfeeding is performed as a corresponding action within a predetermined time, When it is judged that it is not the timing for the next breastfeeding based on the average time interval of breastfeeding in the childcare record so far, it is judged that infant B1's desire for "feeding" is low, and it cries because it is another desire. It is preferable to estimate the accuracy of the correspondence target by By doing so, it is possible to provide the estimation result even if the estimation cannot be made from the crying alone. It becomes possible for the responder to know the behavior that should be handled, and the responder can positively respond to the response target.

In addition, the estimation unit 42, based on the crying of the infant B1, as a method of estimating that the corresponding target is crying without desire or emotion as the main cause, such as when the infant B1 is in a mental leap, includes, for example, To confirm the degree of similarity with the history of voice information, etc., when infant B1 is not estimated to be in any state, or when the responder does not stop crying even if the responder responds, based on the history of previous responders. Therefore, it is good to estimate whether the target is crying without desire or emotion as the main cause, as occurs in mental leap, and infant B1 is "holding", "feeding", "changing diapers" If it is not presumed to be in a state of having any desire such as "sleepy", it may be presumed that the corresponding object is in a state of crying without desire or emotion as the main cause, as occurs in mental leap.

In addition, when the accuracy cannot be estimated based on the voice information, the estimation unit 42 may estimate the accuracy based on the history of the status of the response target input by the responder. For example, the estimating unit 42 estimates the accuracy based on the history of the status of the response target input by the responder in a predetermined application for recording childcare (childcare diary application, breastfeeding recording application, etc.). As a specific example, the estimating unit 42 determines that the timing at which the corresponding voice information is acquired is the timing for breastfeeding, based on the average of the breastfeeding time intervals in the childcare record input to the childcare diary application. , the accuracy of the desire of the corresponding target "breastfeeding" is estimated higher than other desires.

Note that the estimation unit 42 may estimate the accuracy of each state of the response target based on the history of the voice information and the history of the status of the response target input by the responder in a predetermined application for recording childcare. . For example, when voice information is newly acquired, the estimating unit 42 selects the voice information and the voice information of the response target acquired within a predetermined period from the time when the status of the response target was input to the childcare diary application. Based on the degree of similarity with the history, the probability of the state of the correspondence target is estimated.

Also, the estimation unit 42 may estimate the accuracy using a model that has learned the features of the voice uttered by the corresponding target. For example, in the example of FIG. 1, the estimating unit 42 uses the model #1 that has been trained to output the probability of the infant's desires, emotions, etc. when voice information is input. Estimate the probability of each state of B1.

(Regarding the notification unit 43)
The notification unit 43 notifies the state of the correspondence target estimated by the estimation unit 42 as the desire or emotion of the correspondence target, together with the degree of certainty and the recommended behavior of the respondent for the state. For example, in the example of FIG. 1, the notification unit 43 provides the terminal device 100 with a notification indicating the accuracy of each state of the infant B1 shown in FIG.

In addition, the notification unit 43 may provide the correspondent with a notification indicating the degree of certainty for each desire. For example, in the example of FIG. 1, the notification unit 43 provides a notification indicating an estimation result in which the accuracy of infant B1's desire "breastfeeding" is estimated to be 60% and the accuracy of infant B1's desire "sleepy" is estimated to be 40%. .

In addition, a notification indicating the accuracy of each emotion may be provided to the respondent. For example, the notification unit 43 provides notification indicating the degree of certainty for each emotion to be handled, such as "lonely", "anger", "sad", and "tired".

It should be noted that the notification provided by the notification unit 43 may include the accuracy of the desire desired by the corresponding target and the accuracy of the emotion felt by the corresponding target. For example, the notification unit 43 provides a notification indicating that the accuracy of the desire “hold” of the corresponding object is 60% and the accuracy of the desire “sleepy” is 40%, and that the emotion of the infant is “lonely”. 60%, a notification may be provided indicating that the probability of being "sad" is 40%.

In addition, the notification unit 43 may provide a notification including information (recommended action) for resolving the state of the response target based on the degree of accuracy. For example, in the example of FIG. 1, the notification unit 43 provides a notification including information (for example, information displayed in the area AR12 of FIG. 2) indicating specific contents of the response to the condition of the infant B1.

In addition, when the correspondence target is not estimated to be in any of the states, the notification unit 43 may provide a notification indicating that the correspondence target is in neither state. For example, in the example of FIG. 1, when it is estimated that the infant B1 is in a twilight crying state, the notification unit 43 sends a message indicating that the infant B1 is in a twilight crying state and detailed information about the twilight crying. A notification (for example, screen C21 in FIG. 3) is provided.

(About the providing unit 44)
The providing unit 44 provides the responder with a message according to the responder's response to the state of the response target based on the voice uttered by the response target. For example, the provision unit 44 provides the responder with a message corresponding to the response by the responder based on the voice of the response target detected by the detection device 200 after the notification is provided by the notification unit 43 . As a specific example, the providing unit 44 provides a message that enhances self-affirmation of the respondent.

In addition, the providing unit 44 may provide a message according to the time from when the notification is provided until the voice uttered by the corresponding target satisfies a predetermined condition. For example, the providing unit 44 provides a message that enhances the self-affirmation of the respondent when the time from the notification provided by the notification unit 43 until the respondent stops crying is equal to or less than a predetermined threshold. Further, when the time from when the notification is provided by the notification unit 43 to when the person to be dealt with stops crying is longer than a predetermined threshold, the providing unit 44 Presume that there was, and provide a message of kindness to the responder.

In addition, the providing unit 44 may provide the responder with a message corresponding to the responder's response to the state of the response target, based on the voice uttered by the responder. For example, after the notification is provided by the notification unit 43, when the detection device 200 detects a predetermined voice (that is, a call) of the responder, the provision unit 44 confirms the response of the responder affirmatively. provide a message.

In addition, the providing unit 44 may provide the responder with a message according to the use of the content regarding the status of the response target. For example, in the example of FIG. 1, when the user U1 browses the content for deepening the understanding of the crying of the infant B1 (the content displayed when the button BT13 in FIG. 2 is pressed), or when the user U1 browses the details of the crying at twilight. When content including such information (content displayed when the button BT21 in FIG. 3 is pressed) is viewed, when quiz content is used, the provision unit 44 affirmatively confirms the use of each content by the user U1. provide meaningful messages. Note that the providing unit 44 may provide the responder with a positive message regarding the action when the responder browses the content related to the flow of coping when the subject cries.

In addition, the providing unit 44 may provide the responder with content that poses a question regarding the correspondence relationship between the voice of the response target and the state of the response target. For example, in the example of FIG. 1, the providing unit 44 provides the user U1 with the quiz content shown in FIG.

In addition, the providing unit 44 may provide a user having a predetermined relationship with the responder with a message according to the responder's response to the condition to be addressed. For example, the providing unit 44 provides the user, who is the father, with a message that evaluates the response of the respondent, who is the mother, to the target.

Further, the provision unit 44 may provide the responder with content indicating the history of the state of the response target based on the estimation result by the estimation unit 42 . For example, the provision unit 44 refers to the response target information database 31, and the status of the response target estimated by the estimation unit 42, the estimated date and time, and the response performed by the responder with respect to the status of the response target (for example, Contents (that is, childcare records) indicating information received by the receiving unit 45, which will be described later, are provided to the respondent.

In addition, the provision unit 44 may provide a predetermined message to the responder if the state of the response target has not been resolved for a predetermined period of time after the notification was provided. For example, in the example of FIG. 1, when the button BT11 is not pressed even after a certain period of time has passed since the screen C11 shown in FIG. If it is not detected, the providing unit 44 provides the message shown on the screen C22 of FIG.

In addition, the providing unit 44 may provide the responder with a message posted by another responder different from the responder. For example, in the example of FIG. 1, the providing unit 44 provides the user U1 with messages shown on screens C41 and C42 in FIG.

In addition, the providing unit 44 may provide a message that is an amount of information corresponding to the timing of providing to the responder. For example, in the example of FIG. 1, when the providing unit 44 provides a message within a predetermined period of time after performing the process of estimating the state of infant B1, the number of characters in messages posted by other users is provides a set number of short messages or less. In addition, the provision unit 44 may receive messages posted by other users when the user U1 is using a predetermined application such as a nursing application, or when it is estimated that the infant is asleep. Among them, provide a message having more characters than a predetermined number.

In addition, the providing unit 44 may provide a message that is an amount of information according to the status of the correspondence target. For example, in the example of FIG. 1, when the number of times the process of estimating the state of infant B1 is performed is equal to or greater than a predetermined threshold value, or when infant B1 stops crying after providing the notification, the providing unit 44 If it takes longer than this, a short message with a predetermined number of characters or less among the messages posted by other users is provided. Further, when the number of times the process of estimating the state of infant B1 is performed is less than a predetermined threshold, the providing unit 44 provides a message having more characters than a predetermined number among messages posted by other users. do.

In addition, the providing unit 44 may provide the message posted by the responder to another responder other than the responder. For example, in the example of FIG. 1, the providing unit 44 provides the message input on the screen C43 shown in FIG. 5 to another user other than the user U1.

(Regarding the reception unit 45)
After the notification is provided, the reception unit 45 receives information about the status of the response target from the responder. For example, in the example of FIG. 1, the receiving unit 45 receives information indicating whether or not the infant B1 has stopped crying from the terminal device 100, and stores the information in the corresponding target information database 31. FIG. As a specific example, the reception unit 45 receives information indicating whether the infant B1 has stopped crying, as well as information indicating the response taken by the user U1, the state of the infant B1 estimated by the user U1, and the like. accept.

(Regarding the learning section 46)
The learning unit 46 causes the model to learn information about the state of the response target after the notification is provided and the features of the voice indicated by the voice information. For example, in the example of FIG. 1, the learning unit 46 refers to the correspondence target information database 31, and determines the state of the infant B1 after the notification is provided by the notification unit 43 (for example, whether or not the infant B1 has stopped crying). The model #1 learns the features of the speech indicated by the speech information of .

In addition, the learning unit 46 may cause the model to learn the information received by the receiving unit 45 and the features of the voice indicated by the voice information. For example, in the example of FIG. 1, learning unit 46 learns model #1 using information indicating whether infant B1 has stopped crying and voice information corresponding to the estimation result as learning data. Further, the learning unit 46 learns the model #1 using as learning data the response taken by the user U1 to stop the infant B1 from crying and the voice information indicating the crying.

[3. Configuration of detection device]
Next, the configuration of the detection device 200 in the notification system 2 shown in FIG. 6 will be described using FIG. FIG. 10 is a diagram illustrating a configuration example of a detection device according to the embodiment; As shown in FIG. 10 , the detection device 200 has a communication section 210 , a storage section 220 and a control section 230 .

(Regarding communication unit 210)
The communication unit 210 is implemented by, for example, a NIC. The communication unit 210 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the terminal device 100 and the like.

(Regarding storage unit 220)
The storage unit 220 is realized by, for example, a semiconductor memory device such as a RAM or flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 10 , the storage unit 220 has a correspondence target information database 221 , a correspondence person information database 222 and a model database 223 . The correspondence object information database 221, the correspondence person information database 222, and the model database 223 have the same configurations as the correspondence object information database 31, the correspondence person information database 32, and the model database 33, respectively, and thus the description thereof is omitted.

(Regarding the control unit 230)
The control unit 230 is a controller, and is implemented, for example, by executing various programs stored in a storage device inside the detection device 200 using a RAM as a work area by a CPU, MPU, or the like. Also, the control unit 40 is a controller, and is implemented by an integrated circuit such as an ASIC or FPGA, for example. The control unit 230 according to the embodiment includes an acquisition unit 231, an estimation unit 232, a notification unit 233, a provision unit 234, a reception unit 235, and a learning unit 236, as shown in FIG. 6 realizes or executes information processing functions and actions in the notification system 2 shown in FIG. The acquisition unit 231, the estimation unit 232, the notification unit 233, the provision unit 234, the reception unit 235, and the learning unit 236 Since it has the same configuration as , the description is omitted.

[4. Notification processing procedure]
Next, the procedure of notification processing executed by the information processing apparatus 10 according to the embodiment will be described with reference to FIG. 11 . 11 is a flowchart illustrating an example of the flow of notification processing executed by the information processing apparatus according to the embodiment; FIG.

As shown in FIG. 11, the information processing device 10 determines whether or not the voice information indicating the voice uttered by the corresponding target has been acquired (step S101). If the voice information has not been acquired (step S101; No), the information processing apparatus 10 waits until the voice information is acquired.

On the other hand, if it is determined that the voice information has been acquired (step S101; Yes), the information processing apparatus 10 estimates the probability that the correspondence target is in a predetermined state for each state based on the voice information (step S102). . Subsequently, the information processing device 10 provides the responder with a notification indicating the accuracy (step S103). Subsequently, the information processing device 10 provides a message corresponding to the correspondence of the responder (step S104). Subsequently, the information processing apparatus 10 receives information about the state of the response target from the responder (step S105). Subsequently, the information processing apparatus 10 causes the model to learn the information received from the responder and the characteristics of the voice indicated by the voice information (step S106), and ends the process.

[5. Modification]
The information processing apparatus 10 described above may be embodied in various forms other than the above embodiment.

[5-1. Regarding the degree of development of the corresponding target]
In the above-described embodiment, an example was shown in which the providing unit 44 provides a message according to the response of the responder to the corresponding target, but the function of the providing unit 44 is not limited to such an example. For example, the providing unit 44 may provide a message indicating the degree to which emotion is included in the state of the correspondence target estimated based on the voice information (in other words, the degree of development of the correspondence target). To give a specific example, the providing unit 44, based on the voice information, selects desires such as “hold”, “breastfeeding”, “change diapers”, and “sleepy”, as well as desires such as “sadness” and “anger”. , ``sad'', ``tired'', etc., a message is provided to the responder to the effect that the subject is developing smoothly. As a result, the information processing apparatus 10 can ensure that the subject develops smoothly even when there is no child-rearing-experienced person around the responder and the responder cannot confirm the degree of development of the subject. Since it is possible to convey that there is a presence, it is possible to increase the self-affirmation of the respondent.

[5-2. Regarding the detection device 200]
In the above-described embodiment, an example in which the detection device 200 detects audio information to be handled has been described, but the function of the detection device 200 is not limited to such an example. For example, the detection device 200 may output a sound indicating a notification, message, comment, or the like provided by the information processing device 10 . Further, when the detection device 200 has a display unit (display), the detection device 200 may display notifications, messages, comments, etc. provided by the information processing device 10 on the display unit.

[5-3. About message]
In the above-described embodiment, the providing unit 44 provides a message to the respondent according to the result of estimation by the estimating unit 42 and the use of the provided content. is not limited to For example, when the frequency with which the responder uses the application that implements the notification process according to the above-described embodiment is equal to or less than a predetermined threshold value, in other words, even if the responder does not use the application, the responder cries as the response target. , the providing unit 44 may provide a message praising the corresponding target. Further, when the detecting device 200 detects that the corresponding target has stopped crying before the notification unit 43 provides the notification, the providing unit 44 may provide a message praising the corresponding target.

[5-4. Regarding the reception unit 45]
In the above-described embodiment, an example is shown in which the receiving unit 45 receives information about the status of the response target from the responder on the screen displaying the notification provided by the notification unit 43. However, the function of the receiving unit 45 is such Examples are not limiting. For example, the reception unit 45 determines the timing at which a predetermined time has passed since it was detected that the subject has stopped crying, or the timing at which the user U1 is using a predetermined application such as a nursing application (in other words, the response Information about the status of the response target may be received via the content provided to the response person at the timing when it is estimated that the person is free. In addition, the reception unit 45 receives the notification provided by the notification unit 43 within the period and the actual response through content provided to the responder every predetermined period (for example, content provided every night or every morning). An input of the degree of matching with the target state may be received.

[5-5. others〕
Of the above processes, all or part of the processes described as being performed automatically may be performed manually. Also, all or part of the processes described as being performed manually may be performed automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual and does not necessarily have to be physically configured as illustrated. That is, the specific form of distribution/integration of each device is not limited to the illustrated one. Further, all or part of each component may be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Further, each of the processes described above may be executed in combination as long as there is no contradiction.

[6. Hardware configuration]
Further, the information processing device 10, the terminal device 100, and the detection device 200 in the above-described embodiments are implemented by a computer 1000 configured as shown in FIG. 12, for example. FIG. 12 is a diagram illustrating an example of a hardware configuration; The computer 1000 is connected to an output device 1010 and an input device 1020 , and a bus 1090 connects an arithmetic device 1030 , a cache 1040 , a memory 1050 , an output IF (Interface) 1060 , an input IF 1070 and a network IF 1080 .

The arithmetic device 1030 operates based on programs stored in the cache 1040 and memory 1050, programs read from the input device 1020, and the like, and executes various processes. The cache 1040 is a cache such as a RAM that temporarily stores data used by the arithmetic device 1030 for various arithmetic operations. The memory 1050 is a storage device in which data used for various calculations by the arithmetic unit 1030 and various databases are registered, and is realized by ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, etc. memory.

The output IF 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a monitor and a printer. It may be realized by a standard connector such as HDMI (registered trademark) (High Definition Multimedia Interface). On the other hand, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, scanner, etc., and is realized by, for example, USB.

For example, the input device 1020 includes optical recording media such as CD (Compact Disc), DVD (Digital Versatile Disc), PD (Phase change rewritable disk), magneto-optical recording media such as MO (Magneto-Optical disk), tape media, It may be implemented by a device that reads information from a magnetic recording medium, a semiconductor memory, or the like. Also, the input device 1020 may be realized by an external storage medium such as a USB memory.

The network IF 1080 has a function of receiving data from another device via the network N and sending it to the arithmetic device 1030, and transmitting data generated by the arithmetic device 1030 via the network N to other devices.

Here, the arithmetic device 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, arithmetic device 1030 loads a program from input device 1020 or memory 1050 onto cache 1040 and executes the loaded program. For example, when the computer 1000 functions as the information processing device 10 , the arithmetic device 1030 of the computer 1000 implements the functions of the control unit 230 by executing the program loaded on the cache 1040 .

The embodiments of the present application have been described in detail above based on the drawings. However, these are examples, and the embodiments of the present application can be implemented in other forms with various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the disclosure of the invention. is possible. Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit".

10 information processing device 20 communication unit 30 storage unit 31 correspondence target information database 32 correspondence person information database 33 model database 40 control unit 41 acquisition unit 42 estimation unit 43 notification unit 44 provision unit 45 reception unit 46 learning unit 100 terminal device 200 detection device 210 communication unit 220 storage unit 221 correspondence target information database 222 correspondence person information database 223 model database 230 control unit 231 acquisition unit 232 estimation unit 233 notification unit 234 provision unit 235 reception unit 236 learning unit

Claims

an acquisition procedure for acquiring voice information indicating a voice uttered by a response target to be handled by a responder;
an estimation step for estimating the state of the correspondence target and the probability of being in the state based on the voice information obtained by the obtaining step;
a notification procedure for notifying the state of the correspondence target estimated by the estimation procedure as the desire or emotion of the correspondence target together with the certainty and a recommended action of the respondent for the state;
A notification program characterized by causing a computer to execute
The notification procedure includes:
2. The notification program according to claim 1, further comprising providing a notification indicating that the target is not in any of the states when the target is not presumed to be in any of the states.
The notification program according to claim 1 or 2, wherein the estimation procedure estimates the accuracy based on a history of the state of the correspondence target when the accuracy cannot be estimated based on the voice information.
4. The method according to any one of claims 1 to 3, wherein the estimation procedure estimates the accuracy based on a history of the state of the correspondence target input by the responder when the accuracy cannot be estimated based on the voice information. A notification program according to any one of the preceding.
2. The computer further causes the computer to execute a first provision step of providing the responder with a message corresponding to the response of the responder to the state of the response target based on the voice uttered by the response target. 4. The notification program according to any one of 4.
The first provision procedure includes:
6. The notification program according to claim 5, wherein a message is provided according to the time from when the notification is provided until the voice uttered by the corresponding object satisfies a predetermined condition.
2. The computer further causes the computer to execute a second provision step of providing the responder with a message corresponding to the response of the responder to the state of the response target based on the voice uttered by the responder. 7. The notification program according to any one of 6.
8. The notification according to any one of claims 1 to 7, further causing the computer to execute a third provision step of providing the responder with a message corresponding to the use of the content regarding the status of the response target. program.
9. Among claims 1 to 8, further causing the computer to execute a fourth provision step of providing the responder with content asking a question about the correspondence relationship between the voice of the response target and the state of the response target. A notification program according to any one of the preceding claims.
1. The computer further causes the computer to execute a fifth providing step of providing a message according to the response of the responder to the state of the response target to a user having a predetermined relationship with the responder. 10. The notification program of any one of 9.
11. The computer further causes the computer to execute a sixth provision step of providing the responder with content indicating the history of the state of the response target based on the estimation result of the estimation step. or the notification program according to one.
7. The computer further causes the computer to execute a seventh provision step of providing a predetermined message to the responder if the state of the response target is not resolved for a predetermined time after the notification is provided. 12. The notification program of any one of 1-11.
13. The computer according to any one of claims 1 to 12, further causing the computer to execute an eighth providing procedure of providing the message posted by another responder different from the responder to the responder. notification program.
The eighth providing procedure includes:
14. The notification program according to claim 13, wherein the message is provided in an amount corresponding to the timing of providing the message to the responder.
The eighth providing procedure includes:
15. The notification program according to claim 13 or 14, wherein the message is provided with an amount of information according to the state of the correspondence target.
16. The computer according to any one of claims 1 to 15, further causing the computer to execute a ninth providing step of providing the message posted by the responder to another responder different from the responder. notification program.
The estimation procedure includes:
estimating the accuracy using a model that has learned the features of the voice uttered by the corresponding target;
17. The computer further causes the computer to execute a learning procedure for causing the model to learn information about the state of the target after the notification is provided and features of speech indicated by the speech information. A notification program according to any one of
causing the computer to further execute a reception procedure for receiving information about the status of the response target from the responder after the notification is provided;
The learning procedure includes:
18. The notification program according to claim 17, wherein the model learns the information received by the receiving procedure and the features of the voice indicated by the voice information.
an acquisition unit that acquires voice information indicating a voice uttered by a response target that is handled by a responder;
an estimating unit that estimates the state of the correspondence target and the probability of being in the state based on the audio information acquired by the acquiring unit;
a notification unit that notifies the state of the corresponding target estimated by the estimating unit as the desire or emotion of the corresponding target together with the certainty and the recommended action of the respondent for the state;
A notification device comprising:
A computer implemented notification method comprising:
an acquisition step of acquiring voice information indicating a voice uttered by a target to be handled by a responder;
an estimating step of estimating the state of the correspondence target and the probability of being in the state based on the voice information obtained by the obtaining step;
a notification step of notifying the state of the corresponding target estimated by the estimating step as the desire or emotion of the corresponding target together with the certainty and a recommended action of the respondent for the state;
A notification method comprising: