JP7416245B2 - Learning devices, learning methods and learning programs - Google Patents

Description

The present embodiment relates to a learning device, a learning method, and a learning program for voice selection.

Various methods have been proposed for selecting a voice to be presented to a user from among a plurality of voice candidates. A speech classification model may be used for such selection. Some classification models of this type perform learning by providing information on whether the classified speech is correct or incorrect as training data. Appropriate evaluation of speech is required to generate training data. As a proposal regarding voice evaluation, for example, a method listed in Non-Patent Document 1 is known.

Takuya Nishimoto et al., “Cognitive load measurement method and its evaluation in speech interfaces”, Speech and Linguistic Information Processing 45-5

The embodiments provide a learning device, a learning method, and a learning program that can efficiently collect training data for speech classification.

The learning device according to the embodiment uses training data for a learning model for selecting a voice to be presented to a user from among a plurality of voice candidates based on the user's reaction to a plurality of voices simultaneously presented to the user. Equipped with a learning section that acquires based on. The plurality of sounds are sounds presented from each of a plurality of speakers arranged equidistantly and in different directions from the user and emitting sounds toward the user from different directions .

According to the embodiment, a learning device, a learning method, and a learning program are provided that can efficiently collect teacher data for classifying speech.

FIG. 1 is a diagram illustrating a hardware configuration of an example of a voice generation device according to an embodiment. FIG. 2A is a diagram illustrating an example of speaker arrangement. FIG. 2B is a diagram showing an example of speaker arrangement. FIG. 2C is a diagram showing an example of speaker arrangement. FIG. 2D is a diagram illustrating an example of speaker arrangement. FIG. 3 is a diagram showing an example of the configuration of the familiarity DB. FIG. 4 is a diagram showing the configuration of an example of the user log DB. FIG. 5 is a diagram illustrating an example of the configuration of the appeal text DB. FIG. 6 is a functional block diagram of the audio generation device. FIG. 7A is a flowchart illustrating audio presentation processing by the audio generation device. FIG. 7B is a flowchart showing audio presentation processing by the audio generation device. FIG. 8 is a diagram illustrating an image of a binary classification model using "familiarity", "concentration", and "awareness level change".

Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a diagram illustrating a hardware configuration of an example of a speech generation device including a learning device according to an embodiment. The sound generation device 1 according to the embodiment emits a calling sound that urges the user to wake up when the user is not in an awake state such as a sleepy state.

In the embodiment, it is determined whether the user is in a state of wakefulness based on the "degree of wakefulness." The arousal degree in the embodiment is an index indicating the degree of arousal corresponding to the arousal level. The arousal level corresponds to the cerebral activity level and represents the degree of arousal ranging from sleep to excitement. Arousal level is measured from eye movements, blink activity, electrodermal activity, reaction time to stimulation, etc. The degree of alertness in the embodiment is calculated using any one or a combination of eye movements, eye blink activity, electrodermal activity, reaction time to stimulation, to measure the level of alertness. The degree of arousal is a value that increases from a sleeping state to an excited state, for example. The arousal level may be a continuous numerical value or may be a discrete value such as Level 1, Level 2, . . . . Further, in the case where the degree of alertness is calculated by a combination of the values of eye movement, blink activity, electrodermal activity, and reaction time to stimulation, there are no particular limitations on how to combine them. For example, simply adding up these values, weighted addition, etc. can be used as a combination method.

The audio generation device 1 includes a processor 2, a ROM 3, a RAM 4, a storage 5, a microphone 6, speakers 7a and 7b, a camera 8, an input device 9, a display 10, and a communication module 11. has. The audio generation device 1 is a variety of terminals such as, for example, a personal computer (PC), a smartphone, or a tablet terminal. However, the present invention is not limited thereto, and the voice generation device 1 may be installed in various devices used by users. Note that the audio generation device 1 does not need to have all the configurations shown in FIG. For example, the microphone 6, speakers 7a, 7b, camera 8, and display 10 may be separate devices from the audio generation device 1.

The processor 2 is a control circuit such as a CPU that controls the overall operation of the audio generation device 1. The processor 2 does not need to be a CPU, and may be an ASIC, FPGA, GPU, or the like. The processor 2 does not need to be composed of a single CPU or the like, and may be composed of a plurality of CPUs or the like.

ROM3 is a nonvolatile memory such as a flash memory. The ROM 3 stores, for example, a startup program for the audio generation device 1. The RAM 4 is a volatile memory such as SDRAM. The RAM 4 can be used as a working memory for various processes in the audio generation device 1.

The storage 5 is a storage such as a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The storage 5 stores various programs used by the audio generation device 1. The storage 5 may store a familiarity database (DB), a user log database (DB) 52, a model database 53, a speech synthesis parameter database (DB) 54, and a message database (DB) 55. . These databases will be explained in detail later.

The microphone 6 is a device that converts input audio into an audio signal that is an electrical signal. The audio signal obtained by the microphone 6 may be stored in the RAM 4 or the storage 5, for example. For example, the voice synthesis parameters for synthesizing the calling voice may be obtained from the voice input through the microphone 6.

The speakers 7a and 7b are devices that output audio based on input audio signals. Here, it is desirable that the speaker 7a and the speaker 7b are not close to each other. Further, it is desirable that the speaker 7a and the speaker 7b are arranged in different directions with respect to the user. Furthermore, it is desirable that the distance between the speaker 7a and the user and the distance between the speaker 7b and the user be equal distances.

FIGS. 2A and 2B are diagrams showing examples of arrangement of speakers 7a and 7b. In FIG. 2A, speakers 7a and 7b are arranged in front of user U so as to be equidistant from the user. In FIG. 2B, speakers 7a and 7b are placed in front and behind the user U so as to be equidistant from the user.

Here, the same number of speakers as the number of voices to be presented are placed in the environment where the user is present. In other words, FIG. 1 is an example in which the number of audio presentations is two. On the other hand, the number of audio presentations may be three or more. In this case, three or more speakers will also be arranged. Even when three or more speakers are arranged, it is desirable that the speakers are not close to each other. Further, it is desirable that the respective speakers are arranged in different directions with respect to the user. Furthermore, it is desirable that the distances between each speaker and the user be equal. For example, an arrangement example when the speakers are three speakers 7a, 7b, and 7c is shown in FIGS. 2C and 2D. In FIG. 2C, speakers 7a, 7b, and 7c are placed in front of user U. Moreover, in FIG. 2D, speakers 7a, 7b, and 7c are arranged behind the user U.

The camera 8 captures an image of the user. The user's image obtained by camera 8 may be stored in RAM 4 or storage 5, for example. The user's image is used, for example, to obtain the alertness level or to obtain the user's reaction to the calling voice.

The input device 9 is a mechanical input device such as a button, switch, keyboard, or mouse, or a software input device using a touch sensor. The input device 9 receives various inputs from the user. The input device 9 then outputs a signal according to the user's input to the processor 2.

The display 10 is, for example, a liquid crystal display or an organic EL display. The display 10 displays various images.

The communication module 11 is a device for the voice generation device 1 to communicate. The communication module 11 communicates with, for example, a server provided outside the audio generation device 1. The method of communication by the communication module 11 is not particularly limited. The communication module 11 may perform wireless communication or wired communication.

Next, the familiarity database (DB) 51, user log database (DB) 52, model database (DB) 53, speech synthesis parameter database (DB) 54, and appeal sentence database (DB) 55 will be explained.

FIG. 3 is a diagram showing an example of the configuration of the familiarity DB 51. The familiarity level DB 51 is a database that records the "familiarity level" of users. The familiarity level DB 51 records, in association with, for example, a user ID, a voice label, a familiarity target, a familiarity level, the number of reactions, the number of presentations, and the average value of change in arousal level.

“User ID” is an ID assigned to each user of the voice generation device 1. The user ID may be associated with user attribute information such as a user name.

The "voice label" is a label uniquely attached to each candidate for a calling voice. Any label can be used as the audio label. For example, the name of the familiar target may be used as the audio label.

A "familiar object" is a person with whom the user converses on a daily basis or an object that generates sounds that the user often hears. The familiar object does not necessarily have to be a person.

“Familiarity level” is the user's degree of familiarity with the corresponding familiarity target voice. The degree of familiarity can be calculated from the frequency of communication with the familiar target through SNS etc., the frequency of daily conversations with the familiar target, the frequency of hearing from the familiar target on a daily basis, etc. For example, the value of the degree of familiarity increases as the frequency of communication with the familiar target through SNS etc., the frequency of daily conversations with the familiar target, and the frequency of hearing from the familiar target on a daily basis. Here, the degree of familiarity may be acquired by self-report by the user.

The “number of responses” is the number of times the user responded to the calling voice generated based on the corresponding voice label. The number of presentations is the number of times the calling voice generated based on the corresponding voice label was presented to the user. The response probability can be calculated by dividing the number of responses by the number of presentations. The response probability is the probability that the user will respond to the calling voice generated based on the corresponding voice label.

The "average change in arousal level" is the average value of the amount of change in the user's arousal level with respect to the calling voice generated based on the corresponding voice label. The amount of change in arousal level will be explained later.

FIG. 4 is a diagram showing an example of the configuration of the user log DB 52. As shown in FIG. The user log DB 52 is a database that records logs related to the use of the voice generation device 1 by users. The user log DB 52 stores, for example, log occurrence date and time, user ID, voice label, familiarity target, concentration level, presence or absence of reaction, arousal level, new arousal level, amount of change in arousal level, and correct label. It is recorded in association. The user ID, voice label, and familiarity target are the same as those in the familiarity level DB 51.

The “log occurrence date and time” is the date and time when the voice generation device 1 was used by the user. The log occurrence date and time is recorded, for example, every time a voice calling out to the user is presented.

“Reaction presence/absence” is information on whether or not the user reacts after the calling voice is presented to the user. When there is a reaction from the user, "Yes" is recorded. When there is no response from the user, "None" is recorded.

The "degree of concentration" is the degree of concentration of the user when presenting the calling voice. The degree of concentration can be measured, for example, by estimating the user's posture and behavior during work from an image obtained by the camera 8. The concentration level value is calculated such that it increases each time the user performs a posture or action in which the user is considered to be concentrating, and decreases each time the user performs a posture or action in which the user is considered to be unconcentrated. Further, the degree of dilation of the user's pupils during work can be estimated by estimating the degree of dilation of the user's pupils while working. The value of the degree of concentration is calculated such that it becomes higher when the pupil is more dilated and lower when the pupil is more miosis. The degree of concentration may be a discrete value such as Lv (Level) 1, Lv2, . . . . Note that the concentration degree acquisition method is not limited to a specific method.

The “awakeness level” is the wakefulness level acquired before the voice generation device 1 presents the calling voice.

The "new arousal level" is the newly acquired arousal level after the user's reaction. The new arousal level is not recorded when there is no reaction from the user.

The "amount of change in arousal level" is an amount representing a change in arousal level before and after the user's reaction. For example, the amount of change in arousal level can be obtained from, for example, the difference between the new arousal level and the arousal level. The amount of change in the arousal level may be a ratio between the new arousal level and the arousal level. The amount of change in arousal level is not recorded when there is no reaction from the user.

“Correct label” is a label of correct or incorrect answer for supervised learning. For example, a correct answer is recorded as ○, and an incorrect answer is recorded as ×. The correct answer label will be explained in detail later.

The model DB 53 is a database that records voice label classification models for extracting voice label candidates. In the embodiment, the model is configured to classify correct or incorrect audio labels in a two-dimensional space of familiarity and concentration. The model includes an initial model and a learning model. The initial model is a model that is generated based on the initial values stored in the model DB 53 and is not updated by learning. Here, the initial value is a constant (plane equation coefficient). The classification surface generated using these initial values is the initial model. In the initial model, audio labels with a degree of familiarity greater than the classification surface are classified as correct (〇), and other audio labels are classified as incorrect (×). Further, the learning model is a trained model generated from the initial model. The learning model can be a binary classification model with different classification aspects than the initial model.

The speech synthesis parameter DB 54 is a database that records speech synthesis parameters. The speech synthesis parameters are data used to synthesize the user's familiar speech. For example, the voice synthesis parameter may be feature data extracted from voice data collected through the microphone 6 in advance. Alternatively, speech synthesis parameters obtained or defined by other systems may be recorded in advance. Here, the speech synthesis parameters are associated with speech labels.

FIG. 5 is a diagram showing the configuration of an example of the appeal text DB 55. The appeal text DB 55 is a database that records template data of various appeal texts for encouraging the user's awakening. The calling text is not particularly limited. However, it is preferable that the message includes a message using the user's name. This is to enhance the cocktail party effect, which will be explained later.

Here, the familiarity DB 51, user log DB 52, model DB 53, voice synthesis parameter DB 54, and appeal text DB 55 do not necessarily need to be stored in the storage 5. For example, the familiarity level DB 51, user log DB 52, model DB 53, voice synthesis parameter DB 54, and appeal text DB 55 may be stored in a server separate from the voice generation device 1. In this case, the voice generating device 1 accesses the server using the communication module 11 and acquires necessary information.

FIG. 6 is a functional block diagram of the audio generation device 1. As shown in FIG. 6, the audio generation device 1 includes an acquisition section 21, a determination section 22, a selection section 23, a generation section 24, a presentation section 25, and a learning section 26. The operations of the acquisition unit 21, determination unit 22, selection unit 23, generation unit 24, presentation unit 25, and learning unit 26 are performed by the processor 2 executing a program stored in the storage 5, for example. Realized. The determining unit 22, the selecting unit 23, the generating unit 24, the presenting unit 25, and the learning unit 26 may be realized by hardware separate from the processor 2.

The acquisition unit 21 acquires the user's alertness level. The acquisition unit 21 also acquires the user's reaction to the calling voice. As described above, the alertness level is calculated using any one or a combination of eye movements, eye blink activity, electrodermal activity, and reaction time to stimulation. Here, eye movement, blink activity, and reaction time to stimulation for calculating the alertness level can be measured from an image of the user acquired by the camera 8, for example. Further, the reaction time to the stimulus may be measured from an audio signal acquired by the microphone 6. Additionally, electrodermal activity may be measured, for example, by a sensor worn on the user's arm. In addition, the user's reaction is determined by the presence or absence of a physical reaction of the user, such as the user's head turning toward the speaker 7a or 7b, or the user's line of sight turning toward the speaker 7a or 7b, and the direction of the reaction. can be obtained by measuring from an image obtained by the camera 8, for example. The acquisition unit 21 may be configured to acquire the arousal level calculated outside the voice generation device 1 or the user's reaction through communication.

The determination unit 22 determines whether the user is in an awake state based on the degree of wakefulness acquired by the acquisition unit 21. When determining that the user is awake, the determining unit 22 transmits a voice label selection request to the receiving unit 231 of the selecting unit 23. Here, the determination unit 22 performs the determination by comparing the degree of wakefulness with a predetermined threshold. The threshold value is a threshold value of the degree of wakefulness for determining whether the user is awake, and is stored in the storage 5, for example. Further, the determining unit 22 determines whether there is a reaction from the user based on the information on the user's reaction acquired by the acquiring unit 21 .

When it is determined that the user is not awake, the selection unit 23 selects a voice label of a voice that is a candidate for encouraging the user to wake up. The selection unit 23 includes a reception unit 231 , a model selection unit 232 , a voice label candidate extraction unit 233 , a voice label selection unit 234 , and a transmission unit 235 .

The receiving unit 231 receives a voice label selection request from the determining unit 22 .

The model selection unit 232 selects a model to be used for audio label selection from the model DB 53. The model selection unit 232 selects either the initial model or the learning model based on the degree of applicability. The degree of fit is a value for determining which of the initial model and the learning model has higher accuracy. The degree of applicability will be explained in detail later.

The voice label candidate extracting unit 233 extracts voice labels that are candidates for the inviting voice to be presented to the user from the familiarity level DB 51 based on the model selected by the model selection unit 232 and the user's concentration level.

The voice label selection unit 234 selects a voice label for generating a calling voice to be presented to the user from the voice labels extracted by the voice label candidate extraction unit 233.

The transmitter 235 transmits information on the audio label selected by the audio label selector 234 to the generator 24 .

The generation unit 24 generates a calling voice to encourage the user to wake up based on the voice label received from the transmission unit 235. The generation unit 24 acquires the voice synthesis parameter corresponding to the voice label received from the transmission unit 235 from the voice synthesis parameter DB 54. Then, the generation unit 24 generates a calling voice based on the data of the calling sentence recorded in the calling sentence DB 55 and the voice synthesis parameters.

The presentation unit 25 presents the calling voice generated by the generation unit 24 to the user. For example, the presentation unit 25 uses the speaker 7 to reproduce the calling voice generated by the generation unit 24 .

The learning unit 26 performs learning of the models recorded in the model DB 53. The learning unit 26 performs learning using, for example, binary classification learning using correct labels.

Next, the operation of the audio generation device 1 will be explained. 7A and 7B are flowcharts showing the audio presentation process by the audio generation device 1. The processes in FIGS. 7A and 7B may be performed periodically.

In step S1, the acquisition unit 21 acquires the user's alertness level. The acquisition unit 21 outputs the acquired alertness level to the determination unit 22. Further, the acquisition unit 21 retains the acquired arousal level until the timing of acquiring a response from the user after presentation of the calling voice.

In step S2, the determination unit 22 determines whether the degree of wakefulness acquired by the acquisition unit 21 is less than or equal to a threshold value. In step S2, when it is determined that the arousal level exceeds the threshold value, that is, when the user is in an awake state, the processing in FIGS. 7A and 7B ends. In step S2, when it is determined that the wakefulness level is below the threshold value, that is, when the user is not in a state of wakefulness such as sleepiness, the process moves to step S3.

In step S3, the determination unit 22 transmits a voice label selection request to the selection unit 23. When the voice label selection request is received by the receiving unit 231, the model selecting unit 232 refers to the user log DB 52 and obtains the number of responses. The number of responses is the total number of "response/presence" responses.

In step S4, the model selection unit 232 determines whether the number of reactions is less than a threshold. The threshold value is a threshold value for determining whether or not an available learning model is recorded in the model DB 53. The threshold value is set to 2, for example. In this case, when the number of reactions is 0 or 1, it is determined that the number of reactions is less than the threshold. In step S4, when it is determined that the number of reactions is less than the threshold, the process moves to step S5. If it is determined in step S4 that the number of times there is a reaction is equal to or greater than the threshold, the process moves to step S6.

In step S5, the model selection unit 232 selects an initial value, that is, an initial model from the model DB 53. The model selection unit 232 then outputs the selected initial model to the audio label candidate extraction unit 233. After that, the process moves to step S9.

In step S6, the model selection unit 232 calculates the degree of fit. When calculating the degree of applicability, the model selection unit 232 first obtains all past response and non-reaction logs from the user log DB 52. The model selection unit 232 then calculates the degree of applicability of both the initial model and the learning model. For example, the model selection unit 232 selects a correct answer rate ( Accuracy) can be used as the degree of applicability. The degree of fit is not limited to the correct answer rate, but also includes the precision rate, recall rate, and F value, which are calculated by using the correct or incorrect output results of the model and the presence or absence of a response in the log. -measure) etc. The precision rate is the percentage of data predicted to be correct that actually resulted in a user response of "Yes." Recall rate is the ratio of logs that are predicted to be correct out of logs that actually have user responses. The F value is the harmonic mean of recall and precision. For example, the F value can be calculated from 2Recall·Precision/(Recall+Precision).

In step S7, the model selection unit 232 compares the degree of applicability of the initial model and the learning model, and determines whether the degree of applicability of the learning model is higher. If it is determined in step S7 that the degree of fit of the initial model is higher, the process moves to step S5. In this case, the model selection unit 232 selects the initial value, that is, the initial model. If it is determined in step S7 that the degree of applicability of the learning model is higher, the process moves to step S8.

In step S8, the model selection unit 232 selects a learning model. The model selection unit 232 then outputs the selected learning model to the speech label candidate extraction unit 233. After that, the process moves to step S9.

In step S9, the audio label candidate extraction unit 233 acquires the current user concentration level from the acquisition unit 21.

In step S<b>10 , the voice label candidate extracting unit 233 extracts candidate voice labels to be used for generating the calling voice from the familiarity level DB 51 . The number of extracted candidate audio labels is greater than or equal to a specified number, for example, the number of presentations of calling voices. The voice label candidate extracting unit 233 extracts, for example, from among the voice labels registered in the familiarity level DB 51, all voice labels attached with correct labels for the current concentration level value. A voice label labeled as correct is a voice label that is expected to elicit a reaction from the user upon presentation of the calling voice, and is also expected to increase the user's arousal level.

In step S11, the audio label selection unit 234 selects a specified number of audio labels from among the audio labels extracted by the audio label candidate extraction unit 233, for example, the same number as the number of presentations of calling voices. For example, when selecting a voice label, the voice label selection unit 234 calculates a weighted winning probability based on the number of past presentations. Then, the audio label selection unit 234 selects an audio label by random sampling based on the weighted winning probability. The weighted winning probability can be calculated, for example, according to equation (1). The weighted winning probability may be calculated using a formula different from formula (1).

In step S12, the transmitter 235 transmits information indicating the audio label selected by the audio label selector 234 to the generator 24. The generation unit 24 acquires a speech synthesis parameter corresponding to the received speech label from the speech synthesis parameter DB 54. Then, the generation unit 24 generates a calling voice based on the data of the calling sentence randomly selected from the calling sentence DB 55 and the voice synthesis parameters. Generation of the calling voice may be performed by voice synthesis processing using voice synthesis parameters. After that, the process moves to step S13.

In step S13, the presentation unit 25 simultaneously presents the calling voice generated by the generation unit 24 to the user from the speakers 7a and 7b.

In step S14, the acquisition unit 21 acquires the user's reaction. The acquisition unit 21 then outputs information on the user's reaction to the determination unit 22.

In step S15, the determining unit 22 determines whether there is a reaction from the user. If it is determined in step S15 that there is no reaction from the user, the process moves to step S20. If it is determined in step S15 that there is a reaction from the user, the process moves to step S16.

In step S16, the determination unit 22 requests the acquisition unit 21 to acquire the new arousal level. In response to this, the acquisition unit 21 acquires the new arousal level. Acquisition of the new arousal level may be performed in the same manner as acquisition of the arousal level.

In step S17, the acquisition unit 21 sets a correct label. The acquisition unit 21 sets the correct answer level, for example, as follows.
1) When it is acquired as a reaction that the user turned towards a specific speaker: Audio label corresponding to the audio presented by the relevant speaker: 〇 Other audio labels: ×
2) When it is obtained as a response that the user turned between multiple speakers, etc. Find the angle between the direction the user turned and the direction of each speaker, and calculate the angle of the sound presented on the speaker with the smaller angle. Audio label: 〇 Other audio labels: ×
3) When it is obtained as a reaction that the user turned to one speaker and then turned to another speaker Audio label of the audio presented at the first speaker: 〇 Other audio labels :×
4) If no response was obtained All audio labels: ×

In step S18, the acquisition unit 21 associates the concentration level, response information, arousal level, new arousal level, arousal level change amount, and correct label with the log occurrence date and time, audio label, familiarity target, and familiarity level, and logs the user log. Register in DB52. After that, the process moves to step S19.

In step S19, the learning unit 26 refers to the user log DB 52 and obtains the number of reactions. Then, the learning unit 26 determines whether the number of times there is a reaction is less than a threshold value. The threshold is a threshold for determining whether information necessary for learning has been accumulated. The threshold value is set to 2, for example. In this case, when the number of reactions is 0 or 1, it is determined that the number of reactions is less than the threshold. In step S19, when it is determined that the number of times there is a reaction is less than the threshold value, the processing in FIGS. 7A and 7B ends. If it is determined in step S19 that the number of times there is a reaction is equal to or greater than the threshold value, the process moves to step S20.

In step S20, the learning unit 26 performs binary classification learning. Then, the learning unit 26 records the learning results obtained by performing the binary classification learning in the model DB 53. After that, the processing in FIGS. 7A and 7B ends. In step S20, the learning unit 26 obtains, for example, the correct label recorded in the user log DB 52, the familiarity level associated with the correct label, and the concentration level. Then, the learning unit 26 generates a binary classification model of the audio label in the three-dimensional space of "degree of familiarity", "degree of concentration", and "amount of change in degree of arousal". FIG. 8 is a diagram illustrating an image of a binary classification model using "familiarity", "concentration", and "amount of change in arousal". In the example of FIG. 8, a voice label having a degree of familiarity located in a space above the classification plane P is classified as correct (○). On the other hand, a voice label having a degree of familiarity located in a space below the classification plane P is classified as incorrect (x). Here, various types of binary classification learning using logistic regression, SVN (Support Vector Machine), neural networks, etc. may be used to generate the model.

Here, the reason why the three axes of "familiarity level", "concentration level", and "alertness level change" are adopted in the binary classification model in the embodiment will be explained. People have the characteristic of selectively paying attention to familiar sounds such as conversations of people they are interested in or their own names. This is called the cocktail party effect. In addition, Yumiko Honjo, “Physiological psychological research on attention and arousal,” Kwansei Gakuin University doctoral dissertation, Otsu No. 217, p.187-188, discusses the relationship between attention and arousal, which introduces both selective attention and arousal. A model has been derived. This suggests that there is a relationship between the occurrence of selective attention and the level of arousal. In this way, "familiarity" is considered to influence the likelihood of the cocktail party effect occurring and the change in arousal due to the cocktail party effect, and is therefore adopted as one axis of learning.

Regarding "concentration level", "The brain directs attention and increases concentration through 'efficient selection'", RIKEN News Release, December 8, 2011, [Online] [June 10, 2020] Internet URL: https://www.riken.jp/press/2011/20111208/ reports that in a state of concentration, the information transmitted from the senses to perception is limited. It is presumed that the sound that is perceived when the user's attention is high is the one that is needed by the user or is easier for the user to hear.In this way, "concentration level" increases the likelihood that the user's selective attention will occur. Since it can be thought of as having an effect on which sounds we tend to respond to, it has been adopted as one of the pillars of learning.

The amount of change in arousal degree characterizes the user's reaction in addition to the correct label, that is, whether or not the user responds. Therefore, the "amount of change in arousal level" is adopted as one axis of learning as it is expected to further improve the accuracy of determining correct labels.

As described above, according to the embodiment, when it is determined that the user is not awake, a voice familiar to the user is used to address the user. Therefore, even if the user is drowsy or the like, the cocktail party effect allows the user to hear the calling voice. Therefore, it is expected that the level of alertness will improve in a short period of time. Further, in the embodiment, familiarity and concentration are used to select familiar voices. For this reason, it is possible to make the user hear a calling voice that the user is more likely to respond to.

Further, according to the embodiment, audio labels are classified using a learning model having three axes: familiarity, concentration, and arousal level change. Therefore, as learning progresses, it is expected that voice label candidates that are more suitable for the user will be extracted. Further, according to the embodiment, an audio label for generating audio is selected from the extracted candidates by random sampling based on the number of past presentations. This suppresses the user's habituation and boredom caused by frequent presentation of calling voices with the same voice label. As a result, even if the voice generation device 1 is used for a long period of time, the user's reaction to the calling voice can be easily expected, and as a result, the user's level of alertness is expected to increase.

Furthermore, according to the embodiment, a plurality of speakers placed in the environment present calling voices simultaneously, and a user's reaction to each calling voice is obtained. Then, a correct label is set according to the user's reaction. Thereby, teacher data can be obtained efficiently.

[Modified example]
A modification of the embodiment will be described. In the embodiment, an example is shown in which the selection of a voice label based on the degree of familiarity, the degree of concentration, and the amount of change in arousal level, the generation of a calling voice, and the learning of a learning model are all performed in the voice generation device 1. has been done. However, the selection of the audio label, the generation of the calling voice, and the learning of the learning model may be performed in separate devices.

Further, in the embodiment, three axes of "familiarity", "concentration", and "amount of change in arousal" are employed in the binary classification model. On the other hand, a simpler binary classification model may be used, for example, only "familiarity" or only "familiarity" and "concentration".

Further, in the embodiment, the learning device is used as a learning device for a classification model of a voice label for a calling voice that urges the user to wake up. In contrast, the learning device of the embodiment can be used to learn various models for selecting voices that are easy for the user to recognize.

Each process according to the embodiments described above can also be stored as a program that can be executed by a processor that is a computer. In addition, it can be stored and distributed in a storage medium of an external storage device such as a magnetic disk, an optical disk, or a semiconductor memory. The processor reads the program stored in the storage medium of the external storage device, and its operations are controlled by the read program, thereby being able to execute the above-described processes.

Note that the present invention is not limited to the above-described embodiments, and can be variously modified at the implementation stage without departing from the gist thereof. Moreover, each embodiment may be implemented in combination as appropriate, and in that case, the combined effect can be obtained. Furthermore, the embodiments described above include various inventions, and various inventions can be extracted by combinations selected from the plurality of constituent features disclosed. For example, if a problem can be solved and an effect can be obtained even if some constituent features are deleted from all the constituent features shown in the embodiment, the configuration from which these constituent features are deleted can be extracted as an invention.

1...Sound generation device 2...Processor 3...ROM
4...RAM
5...Storage 6...Microphone (microphone)
7a, 7b... Speaker 8... Camera 9... Input device 10... Display 11... Communication module 21... Acquisition unit 22... Judgment unit 23... Selection unit 24... Generation unit 25... Presentation unit 26... Learning unit 51... Familiarity database (DB )
52...User log database (DB)
53...Model database (DB)
54...Speech synthesis parameter database (DB)
55...Call text database (DB)
231... Receiving unit 232... Model selection unit 233... Voice label candidate extraction unit 234... Voice label selection unit 235... Transmission unit

Claims (4)

a learning unit that acquires training data for a learning model for selecting a voice to be presented to a user from among a plurality of voice candidates based on the user's reactions to a plurality of voices simultaneously presented to the user; Equipped with
In the learning device, the plurality of sounds are sounds presented from each of a plurality of speakers arranged at equal distances and different directions from the user and emitting sounds toward the user from different directions. The learning model includes a degree of familiarity representing the degree to which the user is familiar with each of the plurality of voice candidates, a degree of concentration representing the degree of current concentration of the user, and a degree of arousal from sleep of the user due to the presentation of the voice. In a three-dimensional space consisting of the amount of change in the degree of arousal representing the degree of arousal up to A classification model that classifies into a first voice candidate that is expected and a second voice candidate that is not expected to cause the user's reaction or increase in the user's arousal level due to the presentation of the voice. 1. The learning device according to 1 . The learning unit generates training data for a learning model for selecting a voice to be presented to a user from among a plurality of voice candidates based on the user's reactions to a plurality of voices simultaneously presented to the user. Equipped with obtaining
In the learning method, the plurality of sounds are sounds presented from each of a plurality of speakers arranged at equal distances and different directions from the user and emitting sounds toward the user from different directions. A learning program for causing a processor to function as the learning section of the learning device according to claim 1 or 2 .

Images