JP2023077444A - Voice processing system, voice processing device and voice processing method - Google Patents

Abstract

To reduce stress of listeners.SOLUTION: A voice processing system 1 includes: an acquisition part for acquiring a speech voice signal, the signal of a speech voice of a first user; a voice recognition part for inputting a feature amount to be extracted based on the speech voice signal into a voice recognition model and generating text data including a word string consisting of one or more words; a voice synthesis part for inputting the feature amount extracted based on the text data into a voice synthesis model and generating a synthesized voice signal, the signal of a synthesized voice; and a voice output part for outputting the synthesized voice to a second user.SELECTED DRAWING: Figure 4

Description

The present invention relates to an audio processing system, an audio processing device and an audio processing method.

Conventionally, in order to improve Customer Satisfaction (CS), various types of call centers have been operated in which operators respond to customer complaints by telephone. In this kind of customer service work, "customer harassment," in which customers make coercive behavior or make unreasonable demands toward operators, can lead to mental disorders and high turnover rates among operators. are viewed.

In recent years, a speech conversion system has been considered for companies to protect operators who are employees from such customer harassment. For example, in Patent Document 1, volume and pitch variation are calculated from an input audio signal, and when the volume and pitch variation exceed predetermined values, the volume and pitch are converted so that the volume and pitch variation fall within a predetermined range. It is described that it controls to output as

JP 2004-252085 A

However, for example, simply converting a speaker's uttered voice by the method described in Patent Document 1 does not sufficiently suppress the speaker's (first user)'s emotions, and the listener's (second user)'s stress is not sufficiently suppressed. may not be mitigated. On the other hand, if the speaker's utterance voice output to the listener is converted in order to reduce the listener's stress, the listener may not be able to fully recognize the speaker's emotions, and the listener may not be able to respond appropriately.

Accordingly, the present invention provides a speech processing system, a speech processing device, and a speech processing method that enable the listener's stress to be sufficiently reduced and/or the listener's response to be appropriate.

A speech processing system according to one aspect of the present invention includes an acquisition unit that acquires an uttered voice signal that is a signal of an uttered voice of a first user; , a speech recognition unit that generates text data including a word string consisting of one or more words, and a feature amount extracted based on the text data is input to a speech synthesis model, and a synthesized speech signal A speech synthesis unit for generating a synthesized speech signal, and a speech output unit for outputting the synthesized speech to a second user.

According to this aspect, the text data is generated based on the speech signal of the first user, and the synthesized speech generated based on the text data is output to the second user. Therefore, the second user can hear the synthesized voice that sufficiently suppresses the customer's emotion included in the first user's uttered voice, and the second user's voice resulting from the first user's emotional utterance can be heard. can sufficiently reduce the stress of

In the speech processing system, the emotion recognizing unit includes an utterance voice signal, a feature amount extracted from the utterance voice signal, text data generated from the utterance voice signal, a feature amount extracted from the text data, or at least The speech signal obtained by the acquisition unit and the speech feature quantity extracted from the speech signal are applied to an emotion recognition model machine-learned to output the emotional information of the speaker of the speech signal with a combination of the two as input. , text data generated from the speech signal, a text feature amount corresponding to the text data, or a combination of at least two of these, a first user corresponding to the speech signal acquired by the acquisition unit of emotion information may be generated.

It is a figure showing an example of an outline of voice processing system 1 concerning this embodiment. 2 is a diagram showing an example of a physical configuration of each device that constitutes the audio processing system 1 according to the embodiment; FIG. It is a figure showing an example of functional composition of speech processing unit 10 concerning this embodiment. FIG. 4 is a diagram showing an example of generation of a synthesized speech signal according to the embodiment; It is a figure which shows an example of generation of customer's emotion information which concerns on this embodiment. It is a figure which shows an example of generation of customer's emotion information which concerns on this embodiment. It is a figure showing an example of functional composition of operator terminal 20 concerning this embodiment. It is a figure which shows an example of the screen D1 which concerns on this embodiment. It is a figure which shows an example of the screen D2 which concerns on this embodiment. 6 is a flow chart showing an example of an emotion suppression operation according to the present embodiment; 6 is a flow chart showing an automatic switching operation of the emotion suppression function according to the embodiment; FIG. 11 is a diagram showing an example of generation of a synthesized speech signal according to a modification of the embodiment; It is a figure which shows an example of the screen D3 which concerns on this embodiment.

Embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that, in each figure, the same reference numerals have the same or similar configurations.

The following description assumes that the voice processing system according to the present embodiment is used in a customer service business such as a call center, but the application form of the present invention is not limited to this. This embodiment is applicable to any situation in which a speech signal generated by performing a predetermined process on a signal of a speech voice of a first user (hereinafter referred to as a "speech voice signal") is output to a second user. is also applicable. In the following description, it is assumed that the first user is the customer and the second user is the operator, but this is not the only option.

(Configuration of voice processing system)
<Overall composition>
FIG. 1 is a diagram showing an example of an overview of a speech processing system 1 according to this embodiment. As shown in FIG. 1, the speech processing system 1 includes a speech processing device 10, a terminal (hereinafter referred to as "operator terminal") 20 used by a second user (hereinafter referred to as "operator"), and a first and a terminal (hereinafter referred to as "customer terminal") 30 used by a user (hereinafter referred to as "customer") of.

The voice processing device 10 receives the speech voice signal acquired by the customer terminal 30 via the network 40 . The network 40 may be an external network such as the Internet, or may include an external network and an internal network such as a Local Access Network (LAN). The voice processing device 10 transmits to the operator terminal 20 voice obtained by subjecting the customer's voice signal to predetermined processing. Note that the speech processing device 10 may be configured with one or a plurality of servers.

The operator terminal 20 is, for example, a telephone, smart phone, personal computer, tablet, or the like. The operator terminal 20 outputs voice to the operator based on a voice signal generated by predetermined processing in the voice processing device 10 or an utterance voice signal from the customer terminal 30 .

The customer terminal 30 is, for example, a telephone, smart phone, personal computer, tablet, or the like. The customer terminal 30 picks up the customer's uttered voice with a microphone and transmits the uttered voice signal, which is the signal of the uttered voice, to the voice processing device 10 .

<Physical configuration>
FIG. 2 is a diagram showing an example of the physical configuration of each device that constitutes the speech processing system 1 according to this embodiment. Each device (for example, the voice processing device 10, the operator terminal 20, and the customer terminal 30) includes a processor 10a equivalent to a calculation unit, a RAM (Random Access Memory) 10b equivalent to a storage unit, and a ROM ( 10c, a communication unit 10d, an input unit 10e, a display unit 10f, a camera 10g, an audio input unit 10h, and an audio output unit 10i. These components are connected to each other via a bus so that data can be sent and received. Note that the configuration shown in FIG. 2 is an example, and each device may have a configuration other than these, or may not have some of these configurations.

The processor 10a is, for example, a CPU (Central Processing Unit). The processor 10a is a control unit that controls various processes in each device by executing programs stored in the RAM 10b or ROM 10c. The processor 10a implements the functions of each device and controls the execution of processing by cooperating with other components of each device and programs. The processor 10a receives various data from the input unit 10e and the communication unit 10d, and displays the calculation result of the data on the display unit 10f and stores it in the RAM 10b.

The RAM 10b and ROM 10c are storage units that store data necessary for various processes and data of the process results. Each device may include a large-capacity storage unit such as a hard disk drive in addition to the RAM 10b and ROM 10c. The RAM 10b and ROM 10c may be composed of semiconductor memory elements, for example.

The communication unit 10d is an interface that connects each device to another device. The communication unit 10d communicates with other devices. The input unit 10e is a device for receiving input of data from a user or a device for inputting data from the outside of each device. The input unit 10e may include, for example, a keyboard, mouse, touch panel, and the like. The display unit 10f is a device that displays information under the control of the processor 10a. The display unit 10f may be configured by, for example, an LCD (Liquid Crystal Display).

The camera 10g includes an imaging device that captures a still image or a moving image, and generates a captured image (for example, a still image or a moving image) by capturing a predetermined area. The audio input unit 10h is a device for collecting audio, such as a microphone. The audio output unit 10i is a device that outputs audio, such as a speaker.

A program for executing each device may be stored in a computer-readable storage medium such as the RAM 10b and the ROM 10c and provided, or may be provided via the network 40 connected by the communication unit 10d. . In each device, the processor 10a executes the program, thereby realizing various operations for controlling each device. It should be noted that these physical configurations are examples, and do not necessarily have to be independent configurations. For example, each device may include an LSI (Large-Scale Integration) in which the processor 10a and the RAM 10b and ROM 10c are integrated.

<Functional configuration>
≪Sound processing device≫
FIG. 3 is a diagram showing an example of the functional configuration of the speech processing device 10 according to this embodiment. Speech processing device 10 includes storage unit 101 , transmission/reception unit 102 , speech recognition unit 103 , removal unit 104 , speech synthesis unit 105 , emotion recognition unit 106 , stress recognition unit 107 , control unit 108 , and learning unit 109 .

The storage unit 101 stores various information, programs, algorithms, models, operation logs, and the like. Specifically, the storage unit 101 stores a speech recognition model 101a, a speech synthesis model 101b, an emotion recognition model 101c, a stress recognition model 101d, an emotion suppression switching model 101e, and the like, which will be described later.

The transmission/reception unit 102 transmits and/or receives various information and/or signals to/from the operator terminal 20 and/or the customer terminal 30 . For example, the transmission/reception unit 102 (acquisition unit) acquires an utterance voice signal, which is a signal of the customer's utterance voice collected by the customer terminal 30 . The transmitting/receiving unit 102 transmits a synthesized speech signal and/or an utterance speech signal to the operator terminal 20 . Further, the transmission/reception unit 102 may acquire an operation log by an operator from the operator terminal 20 . The operation log may include information on subjective evaluation of the customer's emotion by the operator (hereinafter referred to as "subjective evaluation information"), "degree of stress" to be described later, and "manual switching history data" to be described later. Further, the transmitting/receiving section 102 may transmit information regarding the customer's emotion (hereinafter referred to as “emotional information”) and the like to the operator terminal 20 .

The speech recognition unit 103 inputs a feature quantity (hereinafter referred to as a “speech feature quantity”) extracted based on the uttered speech signal acquired by the transmission/reception unit 102 to the speech recognition model 101a, and converts the speech recognition model 101a into a feature consisting of one or more words. Generate text data containing word strings. Specifically, the speech recognition unit 103 uses the acoustic model of the speech recognition model 101a to generate a word string from the speech feature amount, and generates the text data according to the analysis result of the word string using the language model. good too. The speech recognition unit 103 may perform preprocessing (for example, digitization of analog signals, noise removal, Fourier transform, etc.) on the uttered speech signal to extract speech features.

The speech recognition model 101a is an algorithm that estimates speech content based on speech signals. The speech recognition model 101a is an acoustic model that models how a certain word is likely to appear as a sound, and/or a model that models how likely a certain word string appears in a specific language. It may also include a language model. For example, a Hidden Markov Model (HMM) and/or a Deep Neural Network (DNN) may be used as the acoustic model. As the language model, for example, a probabilistic language model such as an n-gram language model may be used.

The removal unit 104 detects a specific word string included in the text data generated by the speech recognition unit 103, removes the specific word string, or replaces the specific word string with another word string to obtain text data. generated and output to the speech synthesizing unit 105 . If a specific word string is not detected in the text data generated by the speech recognition section 103 , the removal section 104 may output the text data to the speech synthesis section 105 .

The specific word string may be one or more words that have a negative psychological effect on the listener, such as insulting the listener, denying the listener's personality, making the listener uncomfortable, or the like. Here, each word may include at least one part of speech such as a noun, a verb, an adverb, a particle, an adjective, an auxiliary verb, or a phonetic variation of the part of speech. For example, a specific word string may be a "sentence" such as "You, I'll kill you", or a rough wording such as "Ttsuten" in "I'm in trouble". It may be a "part of a sentence" showing. The removal unit 104 may output to the speech synthesis unit 105 text data in which only a specific word string detected in the text data is replaced with another word string, or may replace a sentence containing the specific word string. Text data in which the entire text data is replaced with another word string may be output to the speech synthesizing unit 105 . The other word string may be blank or the like.

The removing unit 104 may detect a specific word string in the text data and/or replace it with another word string based on the specific word string stored in advance in the storage unit 101 .

Alternatively, the removing unit 104 detects a specific word string in the text data and/or replaces it with another word string with a reduced semantic emotion based on a model learned by machine learning. good too. For example, a specific word string “you” in the text data may be replaced with “you”. Detection and/or replacement of specific word strings in text data with other word strings may be performed based on a model based on machine learning.

Note that, when a specific word string is detected in the text data, the removal unit 104 may generate information (hereinafter referred to as “detection information”) regarding detection of the specific word string. The detection information is, for example, information indicating that the specific word string has been detected (for example, a character string "NG word" or "NG word detection"), information indicating the specific word string, and customer may include at least one of information (hereinafter referred to as "warning information") regarding a warning to The warning information may be, for example, information for notifying the operator that the contents of the customer's speech to the operator may be subject to criminal prosecution such as insult or defamation. The detection information may be transmitted to the operator terminal 20 by the transmitter/receiver 102 . When the detection information is generated, the voice processing device 10 sends warning information to the customer terminal 30 (for example, "There is a risk of insult to the operator of our company. We would appreciate your cooperation as this may be an excessive burden on our operators.") may be output. Such warning information can be used as advance notice against customer harassment.

The speech synthesizing unit 105 inputs a feature amount extracted based on the text data input from the removing unit 104 (hereinafter referred to as "text feature amount") to the speech synthesis model 101b to generate a synthesized speech signal (hereinafter referred to as "Synthetic Speech Signal"). Specifically, the removal unit 104 may predict a speech synthesis parameter based on the text feature quantity, and generate a synthesized speech signal using the predicted speech synthesis parameter. Speech synthesizing section 105 outputs the synthesized speech signal to transmitting/receiving section 102 . The synthesized speech signal can also be said to be a speech signal obtained by reading out the contents of text data.

The speech synthesis model 101b is an algorithm for inputting text data and outputting a synthesized speech signal corresponding to the contents of the text data. For example, the above HMM and/or DNN may be used as the speech synthesis model 101b.

The speech synthesis model 101b may correspond to a plurality of speech types. The speech synthesis unit 105 selects a speech type to be used for a synthesized speech signal from among a plurality of speech types, inputs the selected speech type and text data to the speech synthesis model 101b, and generates a synthesized speech signal of the selected speech type. may be synthesized. The plurality of voice types may be, for example, at least one of low intonation voice, mechanical sound, character voice, entertainer voice, and voice actor voice. The voice synthesizing unit 105 may receive selection of voice type from the operator via the operator terminal 20 .

FIG. 4 is a diagram showing an example of generation of a synthesized speech signal according to this embodiment. In FIG. 4, it is assumed that text data T1 to T3 are generated in speech recognition section 103 based on speech speech signals S1 to S3 acquired by transmission/reception section . For example, in FIG. 4, the removal unit 104 does not detect a specific word string in the text data T1, so the text data T1 is output to the speech synthesis unit 105 as it is. On the other hand, since the removing unit 104 detects a specific word string ("I'll kill you" in T2 and "Ttsuten" in T3) in the text data T2 and T3, it removes or replaces the specific word string. The resulting text data T2′ and T3′ are output to the speech synthesizing unit 105. FIG. For example, in the text data T2', a specific word string in the text data T2 is replaced with blanks (□). Also, in the text data T3', a specific word string "ttsuten" in the text data T3 is replaced with "tou". The speech synthesis unit 105 generates synthesized speech signals S1, S2' and S3' from the text data T1, T2 and T3, respectively.

The emotion recognition unit 106 recognizes the customer's emotion based on at least one of the speech signal acquired by the transmission/reception unit 102, the text data generated by the speech recognition unit 103, and the subjective evaluation information received by the transmission/reception unit 102. to generate emotional information. The emotion recognition unit 106 may generate the customer's emotion information based on voice features (for example, intonation, volume, etc.) extracted based on the speech voice signal. The emotion recognition unit 106 recognizes the customer's emotional information based on the fact that a specific word string is detected in the text data generated based on the utterance voice signal, or that the specific word string has not been detected for a predetermined period of time. can be generated. The emotion recognition unit 106 may generate the customer's emotion information based on the captured image of the customer acquired by the camera 10g. The emotion recognition unit 106 may generate customer's emotion information using the emotion recognition model 101c.

The emotion recognition model 101c receives as input a speech signal, a speech feature quantity extracted from the speech speech signal, text data generated from the speech speech signal, a text feature quantity, or a combination of at least two of these. This is a model that outputs emotion information, which is the emotion of the corresponding customer.

FIG. 5A is an explanatory diagram of the learning process of the emotion recognition model 101c. For example, the learning of the emotion recognition model 101c includes speech features extracted from speech signals, text features extracted from text data, and "subjective evaluation information" by the operator (or extracted from the subjective evaluation information). A plurality of data sets (hereinafter referred to as “data sets”) each including at least one of the feature amounts to be measured may be used. The subjective evaluation information is information obtained by the operator subjectively evaluating the customer's emotion after listening to the customer's speech signal. For example, the operator may rate the customer's anger on multiple levels, such as anger levels 1-10. A data set for learning the emotion recognition model 101c may be generated, for example, as follows. The operator listens to the customer's raw speech signal and annotates the customer's emotion estimated from the speech signal (that is, assigns "subjective evaluation information" to the speech signal). As a result, information is obtained in which the speech signal and the customer's emotion estimated from the speech signal are associated on the time axis. A data set, which is a bundle of such information, is obtained by assigning subjective evaluation information to a plurality of speech signals by a plurality of operators. The emotion recognition model 101c may be supervised machine-learned using such datasets. The data set used for learning the emotion recognition model 101c may include speech signals in addition to or instead of voice features, and may include text data in addition to or instead of text features.

FIG. 5B is an explanatory diagram of estimation processing using the emotion recognition model 101c. For example, as shown in FIG. 5B, the speech feature amount extracted from the speech signal S1 and/or the text feature amount extracted from the text data T1 generated from the speech signal S1 may be input to the emotion recognition model 101c. obtains the output corresponding to the input, that is, the emotional information corresponding to the speech signal. Note that the emotion recognition model 101c may be input with the speech signal S1 in addition to or instead of the voice feature amount, or may be input with the text data T1 in addition to or instead of the text feature amount.

The subjective evaluation information numerically indicates the degree of one or more emotions (for example, at least one of "happiness", "surprise", "fear", "anger", "disgust" and "sadness"). There may be. Alternatively, the affective information may indicate a specific emotion that the customer is likely to be feeling (eg, "anger").

The stress recognition unit 107 generates information about the stress situation of the operator (hereinafter referred to as "stress information"). For example, the stress recognizing unit 107 recognizes the operator's vital data such as heart rate, perspiration, and respiration, or image information such as the operator's line of sight and facial expression collected using a camera, by a conventionally known method. A stress situation may be estimated. For example, the stress recognition unit 107 may estimate the operator's stress situation based on the operator's uttered voice. Specifically, the stress recognizing unit 107 may estimate the operator's stress situation based on changes in the tone or speed of the operator's speech, appearance of words related to apology, speech over the customer's speech, and the like. . For example, the stress recognition unit 107 may estimate the operator's stress situation based on the operation log of the operator terminal 20 . Specifically, the stress recognizing unit 107 may estimate the operator's stress situation according to the movement of the mouse or the like, or the fact that there is no operation input in the scene where the operation should be performed. The stress recognition unit 107 may generate stress information based on the stress recognition model 101d. The stress recognition model 101d receives an utterance voice signal, a voice feature amount extracted from the utterance voice signal, text data generated from the utterance voice signal, a text feature amount, or a combination of at least two of these, and listens to the utterance voice. It is a model that outputs an estimated value of the stress felt by an operator who is For the learning of the stress recognition model 101d, a measured value of stress actually felt by the operator upon listening to the customer's uttered voice may be used. A data set for learning the stress recognition model 101d may be generated as follows, for example. The operator annotates the degree of stress (for example, a level of 1 to 10) felt by listening to the customer's uttered voice (that is, assigns the ``stress level'' felt by the operator to the uttered voice signal). As a result, information is obtained in which the speech signal and the operator's stress when listening to the speech signal are associated on the time axis. A data set, which is a bundle of such information, is obtained by applying stress degrees to a plurality of speech signals by a plurality of operators. The stress perception model 101d may be supervised machine-learned using such datasets.

The control unit 108 performs various controls related to the speech processing device 10 . Specifically, based on the stress information generated by the stress recognizing unit 107, the control unit 108 determines which of the synthetic voice generated by the voice synthesizing unit 105 and the customer's uttered voice to be output in the operator terminal 20. You can switch. The control unit 108 may switch whether to generate a synthesized speech signal based on the speech speech signal based on the stress information. For example, when the stress level indicated by the stress information is greater than or equal to a predetermined threshold value, the control unit 108 may control to output synthesized voice to the operator instead of the customer's uttered voice. On the other hand, if the stress level indicated by the stress information is less than or equal to the predetermined threshold value, the control unit 108 may control to output the uttered voice to the operator. When instruction information for automatic switching of the emotion suppression function is input from the operator, the control unit 108 may perform the switching based on the stress information. The emotion suppression function is a function for outputting synthesized speech to the operator in place of the customer's uttered speech.

The control unit 108 may perform the switching based on emotion information. The control unit 108 may perform the switching based on the output of the emotion suppression switching model 101e. The emotion suppression switching model 101e is a model that receives speech signals, voice features, text data, text features, or a combination of at least two of these as inputs, and outputs timings for switching the emotion suppression function on and off. The emotion suppression switching model 101e may further receive stress information or emotion information. The details of the emotion suppression switching model 101e will be described later.

Further, the control unit 108 may perform the switching based on switching information input by the operator. Here, the switching information is information regarding switching between application (ON) and non-application (OFF) of the customer's emotion suppression function. For example, when the switching information indicates application of the customer's emotion suppression function, the control unit 108 may control to output synthesized speech to the operator. On the other hand, when the switching information indicates that the customer's emotion suppression function is not applied, the control unit 108 may control to output the uttered voice to the operator. When instruction information for manual switching of the emotion suppression function is input from the operator, control unit 108 may perform the switching based on the switching information.

The learning unit 109 may perform learning processing for the emotion recognition model 101c, the stress recognition model 101d, and the emotion suppression switching model 101e.

The speech processing device 10 associates any of the information shown in 1) to 7) below, or a combination of at least two pieces of information on the time axis, and transmits the information to the operator terminal 20 via the transmission/reception unit 102. good. 1) Customer's utterance voice signal, 2) Text data generated from the utterance voice signal, 3) Text data after being processed by the removal unit 104, 4) Detection information, 5) Synthetic voice signal, 6) Customer's utterance customer's emotional information estimated from the voice signal; 7) timing for switching on/off of the emotion suppression function; When the emotion suppression function is on, the voice processing device 10 does not have to send the customer's speech voice signal to the operator terminal 20 . When the emotion suppression function is off, the speech processing device 10 does not have to send the synthesized speech signal to the operator terminal 20 . Regardless of whether the emotion suppression function is on or off, the speech processing device 10 may send both the customer's uttered speech signal and the synthesized speech signal to the operator terminal 20 .

≪Operator terminal≫

FIG. 6 is a diagram showing an example of the functional configuration of an operator terminal according to this embodiment. The operator terminal 20 includes a transmission/reception section 201 , an input reception section 202 and a control section 203 . Note that the functional configuration shown in FIG. 6 is merely an example, and other configurations not shown may be provided.

The transmission/reception unit 201 transmits and/or receives various information and/or signals to/from the voice processing device 10 and/or the customer terminal 30 . For example, the transmitting/receiving unit 201 may receive an utterance voice signal, which is a signal of the customer's utterance voice picked up by the customer terminal 30 . The transceiver 102 may receive the synthesized speech signal from the speech processing device 10 . Further, the transmitting/receiving unit 201 may transmit subjective evaluation information to the speech processing device 10 . Further, the transmitting/receiving unit 201 may receive emotional information of the customer from the speech processing device 10 .

The input reception unit 202 receives input of various information based on the operation of the input unit 10e by the operator. For example, the input reception unit 202 receives subjective evaluation information and stress information from the customer's raw utterance voice signal as part of the work for generating a data set for learning the emotion recognition model 101c and the stress recognition model 101d. You may receive the input of the degree of. Hereinafter, the work of the operator inputting the subjective evaluation information and the degree of stress into the operator terminal 20 will be referred to as "annotation work". Annotation work may be positioned as a business separate from normal call center business. Further, the input reception unit 202 may receive an input of switching information of the customer's emotion suppression function. Further, the input receiving unit 202 may receive an input of instruction information instructing either manual switching or automatic switching of the emotion suppression function.

The control unit 203 performs various controls regarding the operator terminal 20 . For example, the control unit 203 controls display of information and/or images on the display unit 10f. Further, the control unit 203 controls the output of audio in the audio output unit 10i. The control unit 203 may control audio output based on information transmitted from the audio processing device 10 or may control audio output based on information received by the input receiving unit 202 .

The control unit 203 outputs synthesized speech from the speech output unit 10 i based on the synthesized speech signal received from the speech processing device 10 . The control unit 203 may cause the voice output unit 10i to output the voice based on the voice signal from the customer terminal 30. FIG.

Further, based on the emotional information received from the speech processing device 10, the control unit 203 may cause the display unit 10f to display emotional information corresponding to the synthesized speech signal. Further, the control unit 203 may display text data corresponding to the synthesized speech signal received from the speech processing device 10 on the display unit 10f. For example, the control unit 203 may cause the display unit 10f to display a screen D1 including at least one of emotion information, text data, and detection information. Further, the control unit 203 may display the stress information on the display unit 10f. For example, the control unit 203 may display a screen D2 including stress information on the display unit 10f.

FIG. 7 is a diagram showing an example of the screen D1 according to this embodiment. As shown in FIG. 7, on the screen D1, the control unit 203 may cause the display unit 10f to display the emotion information I1 in accordance with the output timing T of the synthetic voice from the voice output unit 10i. By displaying the emotion information I1 at each output timing T of the synthetic voice, the operator can recognize the customer's emotion in real time even when listening to the synthetic voice in which the customer's emotion is suppressed by the emotion suppression function.

Further, on the screen D1, the control unit 203 may cause the display unit 10f to display the content of the text data I2 corresponding to the synthesized speech in accordance with the output timing T of the synthesized speech. By displaying the content of the text data I2, the operator can grasp the content of the customer's utterance visually as well as the synthesized voice.

On screen D1, based on the detection information received from the speech processing device 10, the control unit 203 displays information I3 (for example, "NG word detection”) may be displayed on the display unit 10f. This function is called "NG word non-display function". As a result, it is possible to prevent the operator from directly recognizing the content of the customer's utterance, which causes psychologically adverse effects, so that the operator's stress can be suppressed. In addition, since the operator can be notified of the speech, the operator can appropriately respond to the customer.

Further, on the screen D1, the control unit 203 causes the display unit 10f to display the customer's specific emotional level I4 in chronological order at each output timing T of the synthesized speech based on the emotion information from the speech processing device 10. may For example, in FIG. 7, the customer's "anger" level I4 for each output timing T of the synthesized speech is shown by a line graph. As a result, the operator can easily grasp the transition of the customer's specific emotion (for example, "anger"), so that the operator's satisfaction with the customer can be improved.

In the screen D1, the control unit 203 may display the selection button I5 on the display unit 10f.
The selection button I5 is an interface that allows the operator to select whether the application (ON) or non-application (OFF) of the emotion suppression function is switched automatically or manually. The operator can switch between the "automatic switching mode" and the "manual switching mode" by clicking, tapping, or sliding the selection button I5. In the automatic switching mode, ON/OFF of the emotion suppression function is automatically switched based on, for example, emotion information, stress information, or output from the emotion suppression switching model 101e.

When the "manual switching mode" is selected, the control unit 203 may cause the display unit 10f to display a switching button I6, which is an interface that allows the operator to select application or non-application of the emotion suppression function. The timing at which the operator switches the emotion suppression function on and off is associated with the customer's speech (and/or various feature values extracted based on the speech) on the time axis, and is recorded as "manual switching history data." It is stored in a storage unit (not shown). The "manual switching history data" may further be associated with operator identification information.

The switching button I7 is a button for switching ON/OFF of the "NG word hiding function". When the "NG word hiding function" is off, even if a specific word string is detected in the text data I2, the text data I2 before being processed by the removal unit 104 is displayed on the display unit 10f as it is. Is displayed. When the emotion suppression function is turned on and the NG word hiding function is turned off, the operator does not directly hear the customer's specific word string, so stress is reduced, while the customer's utterance content can be accurately grasped. This makes it possible to grasp the customer's emotions more accurately.

The data set for learning the emotion suppression switching model 101e includes stress information, emotion information, speech voice signal S1, voice feature quantity, text data, text feature quantity, or a combination of at least two of these, and the operator's emotion suppression function. The on/off switching timing may be a bundle of data associated on the time axis. Methods for learning the emotion suppression switching model 101e include, for example, various methods described in 1) to 3) below. 1) The emotion suppression switching model 101e may be learned for each operator. That is, the emotion suppression switching model 101e applied to a certain operator may be learned based only on the "manual switching history data" of the emotion suppression function of that operator. According to this method, the emotion suppression switching model 101e can switch the emotion suppression function at a timing that matches the operator's preference. Alternatively, 2) the emotion suppression switching model 101e applied to a certain operator may be learned based on "manual switching history data" by an unspecified number of operators. According to this method, the amount of data that can be used for learning increases, so the emotion suppression switching model 101e can be learned quickly. Alternatively, 3) the emotion suppression switching model 101e applied to a certain operator may be learned based on "manual switching history data" by operators similar in age, sex, and other characteristics to the operator. According to this method, the amount of data that can be used for learning is increased compared to method 1). It becomes possible to learn the matching switching timing.

FIG. 8 is a diagram showing an example of the screen D2 according to this embodiment. The control unit 203 may display the stress information from the speech processing device 10 on the screen D2. For example, in FIG. 8, as stress information, information indicating an estimated value of stress felt by the operator (eg, "56%") and information indicating a relative evaluation value from the normal state of the operator (eg, "8.1% less than usual") is displayed.

FIG. 12 is a diagram showing an example of the screen D3 according to this embodiment. On the screen D3, the control unit 203 may display an interface I8 for the operator to perform annotation work. For example, while listening to the customer's live voice (sample voice), the operator selects the customer's emotion felt from the sample voice from the interface I8 each time. In FIG. 12, customer sentiment I1 is subjective evaluation information of customer sentiment by the operator. For example, if the operator annotates the sample voice "Please deliver it somehow by this evening" with the emotion "anger", as shown in FIG. Send it to me" and the information "anger" are related on the time axis. Annotation may be performed in units of sentences or may be performed at predetermined time intervals.

(Operation of voice processing system)
FIG. 9 is a flowchart showing an example of an emotion suppression operation according to this embodiment. Note that FIG. 9 is merely an example, and the order of at least some steps (for example, step S106) may be changed, steps not shown may be performed, or some steps may be omitted. may be

The voice processing device 10 acquires an uttered voice signal, which is a signal of the customer's uttered voice picked up by the voice input unit 10h of the customer terminal 30 (S101).

The speech processing device 10 inputs the feature quantity extracted based on the utterance speech signal acquired in S101 to the speech recognition model 101a, and generates text data including a word string consisting of one or more words (S102). .

The speech processing device 10 determines whether or not the text data generated in S102 includes a specific word string (S103). If the text data contains a specific word string, the speech processing device 10 generates text data by removing the specific word string or converting the specific word string into another word string (S104).

The speech processing device 10 inputs the feature amount extracted based on the text data to the speech synthesis model 101b to generate a synthesized speech signal, which is a synthesized speech signal (S105).

The speech processing device 10 extracts a feature amount based on at least one of the speech audio signal acquired in S101, the text data generated in S102, and subjective evaluation information of the customer's emotion input by the operator. is input to the emotion recognition model 101c to generate the customer's emotion information (S106).

The operator terminal 20 outputs synthetic speech from the speech output unit 10i based on the synthesized speech signal generated in S105, and displays emotional information corresponding to the synthetic speech at the output timing T of the synthetic speech on the display unit 10f. (S107, for example, FIG. 7).

The speech processing device 10 determines whether or not to end the process (S108). If the process is not to end (S108: NO), the speech processing device 10 executes the processes S101 to S107 again. On the other hand, when ending the voice conversion process (S108: YES), the voice processing device 10 ends the process.

FIG. 10 is a flowchart showing an automatic switching operation of the emotion suppression function according to this embodiment. Note that FIG. 10 is merely an example, and the order of at least some steps may be changed, steps not shown may be performed, and some steps may be omitted.

The speech processing device 10 generates operator stress information (S201).

The speech processing device 10 determines whether or not the stress information satisfies a predetermined condition (S202). For example, the predetermined condition may be that the stress level indicated by the stress information is greater than or equal to a predetermined threshold.

If the stress information satisfies a predetermined condition (S202: YES), the speech processing device 10 may apply the emotion suppression function (that is, output synthesized speech from the operator terminal 20) (S203). On the other hand, if the stress information does not satisfy the predetermined condition (S202: NO), the speech processing device 10 may not apply the emotion suppression function (that is, output the customer's uttered voice from the operator terminal 20) (S204). ).

The speech processing device 10 determines whether or not to end the process (S205). If the process is not to end (S205: NO), the speech processing device 10 executes the processes S201 to S204 again. On the other hand, when ending the voice conversion process (S205: YES), the voice processing device 10 ends the process. In S201 and S202, the speech processing device 10 may determine whether or not to apply the emotion suppression function based on the emotion information and the output of the emotion suppression switching model 101e.

As described above, according to the speech processing system 1 of the present embodiment, text data is generated based on the customer's uttered voice signal, and synthesized speech generated based on the text data is output to the operator. Therefore, the operator can hear synthesized speech in which the emotions of the customer included in the customer's uttered speech are sufficiently suppressed, and the operator's stress caused by the customer's emotional utterance can be reduced. The inventors of the present invention gave about 50 subjects 1) the customer's uttered voice itself, 2) the customer's uttered voice whose volume was adjusted, 3) the customer's uttered voice whose voice quality was converted, 4) An experiment was conducted in which the customer listened to and compared four types of synthesized speech generated after converting the customer's uttered speech into text, and evaluated the degree of anger felt from the speech on a seven-grade scale. As a result, compared with 2) and 3), 4) showed a remarkable reduction in anger transmitted to the subject.

Further, according to the speech processing system 1 according to the present embodiment, it is possible to notify the operator of the emotional information of the customer in accordance with the timing of outputting the synthesized speech in addition to outputting the synthesized speech. The operator who hears this can recognize the customer's emotions in real time and respond appropriately to the customer.

Further, according to the speech processing system 1 according to the present embodiment, it is possible to determine whether or not to apply the emotion suppression function based on the operator's stress information or the customer's emotional information (that is, whether to apply the synthesized voice or the spoken voice to the operator). output), it is possible to properly balance the operator's stress and the customer's satisfaction.

(Change example)
In the speech processing system 1 described above, the speech recognition unit 103 generates text data including word strings determined as one or more sentences from the speech signal, but the present invention is not limited to this. The speech recognition unit 103 generates text data including a word string consisting of one or more words (parts of speech or morphemes) before the word string recognized from the speech signal is determined as one or more sentences. You may The removal unit 104 may remove a specific word string in text data that has not been determined as the sentence, and the speech synthesis unit 105 may generate a synthesized speech signal from the text data that has not been determined as the sentence.

FIG. 11 is a diagram showing an example of generation of a synthesized speech signal according to a modification of this embodiment. In FIG. 11, it is assumed that text data T41 to T43 are generated in speech recognition section 103 based on speech signal S4 acquired in transmission/reception section . As shown in FIG. 11, the text data T41 to T43 are composed of meaningful morpheme units ("fast", "send", "please") before the sentence "please send it quickly" is fixed. It differs from FIG. 4 in that it is generated. The removal unit 104 determines whether or not a specific word string is included in each of the text data T41 to T43, removes the specific word string, and outputs the result to the speech synthesis unit 105. FIG. The speech synthesizing unit 105 generates synthetic speech signals S41 to S43 from the text data T41 to T43, respectively.

As shown in FIG. 11, by generating text data in units of one or a plurality of morphemes and outputting synthesized speech before finalizing a sentence, the operator's response delay can be reduced by generating text data. A model for determining whether or not a plurality of text data (or synthesized speech) in morpheme units is semantically unnatural may be used.

Also, in order to reduce the response delay, for example, "a~" Filler sounds such as , "well" may be added. As a result, the operator can also prevent the customer's satisfaction from being lowered due to the response delay.

Further, based on the customer's emotion estimated by the emotion recognition unit 106, the speech synthesis unit 105 may select the speech synthesis model 101b that matches the customer's emotion from among the plurality of speech synthesis models 101b. For example, when the customer's emotion estimated by the emotion recognition unit 106 is "excitement", the speech synthesis unit 105 may use the speech synthesis model 101b with a fast pitch and a strong intonation. For example, if the customer's emotion estimated by the emotion recognition unit 106 is "crying", the speech synthesis unit 105 may use the speech synthesis model 101b that outputs a cry-like speech. Alternatively, the speech synthesis unit 105 may change the parameters of the speech synthesis model 101b based on the customer's emotion estimated by the emotion recognition unit 106, and adjust so that the speech matching the customer's emotion is output. An operator who directly hears the live voice of a customer who is furious feels extremely stressed. On the other hand, the operator needs to accurately grasp the customer's emotions in real time in order to perform the customer service work properly. Since the operator does not hear the spoken voice directly, the operator does not feel excessive stress, and by adding the customer's emotion to the synthesized voice, the operator can grasp the customer's emotion in real time through hearing.

(Other embodiments)
In the above embodiment, the customer's uttered voice signal is converted into text and the synthesized voice signal is output to the operator, but the present invention is not limited to this. The speech processing device 10 may input speech features extracted based on the customer's uttered speech signal to the speech conversion model, generate a converted speech signal, and output the converted speech from the operator terminal 20 .

The "speech conversion model" described in the claims includes both a model that converts the speech signal into text and outputs it as synthesized speech, and a model that converts the voice quality and outputs it without converting the speech signal into text. It is a concept that includes By outputting synthesized speech or converted speech to the operator in place of the customer's uttered speech, the stress felt by the operator can be reduced, although the degree of effect varies. On the other hand, it is essential for the operator to grasp the customer's emotions in real time in order to carry out the customer service.

The speech processing device 10 in this modified example inputs the speech feature amount extracted based on the customer's uttered speech signal to the speech conversion model to generate the converted speech signal. The speech processing device 10 generates information in which 1) the converted speech signal and 2) the customer's emotional information estimated from the customer's uttered speech are associated on the time axis, and transmits the information to the operator terminal 20 . The information transmitted by the speech processing device 10 includes a speech audio signal, text data generated from the speech audio signal, text data after being processed by the removing unit 104, detection information, and timing for switching on/off of the emotion suppression function. may be associated.

The operator terminal 20 outputs the signal of the converted speech received from the speech processing device 10 from the speech output unit 10i, and displays information indicating emotional information in accordance with the output timing T of the converted speech from the speech output unit 10i. may be displayed in section 10f. Further, the operator terminal 20 may display the text data on the display section 10f in accordance with the output timing T of the converted voice from the voice output section 10i. The mode of such display may be as illustrated in FIG.

The speech processing device 10 in this modification may generate a signal of converted speech based on the emotion information so that the emotion indicated by the emotion information is reflected in the converted speech. For example, if the emotion indicated by the emotion information is "furious," a speech conversion model with a fast pitch and a strong intonation may be used. For example, if the emotion indicated by the emotion information is "crying", a speech conversion model that outputs a crying voice may be used. The speech processing device 10 may generate a converted speech signal such that the emotion indicated by the emotion information is reflected in the converted speech. The operator does not feel excessive stress by not letting the operator directly hear the uttered voice, but the operator can grasp the customer's emotion in real time through hearing by putting the customer's emotion on the converted voice.

In the speech processing system 1 according to this modification, the annotation work by the operator may be performed on the converted speech during normal call center work by the operator. If the operator annotates the converted voice with "feeling of anger", the speech conversion model may be adjusted in real time to output a softer voice based on the result of the annotation.

In the embodiment described above, a call center where the first user is the customer and the second user is the operator is assumed, but the application scene of the present embodiment is not limited to the call center. For example, the present invention can be applied to any scene such as a web meeting in which the first user's emotion-repressed voice is output to the second user. In other words, the present embodiment can be used not only as a countermeasure against customer harassment, but also as a corporate countermeasure against various types of harassment such as internal power harassment.

In the embodiment described above, the process of "associating emotion information and synthesized speech on the time axis" is performed by matching the output timing of synthetic speech or converted speech, as shown in FIG. As long as it is possible to display the emotional information estimated from the uttered voice, the specific mode is not limited. The process of "associating on the time axis" in the above-described embodiment may be a process of associating based on time information such as hour, minute, and second, or based on information such as how many minutes and seconds have elapsed since the start of the utterance voice information. It may be a process of associating on the basis of a sentence, or a process of associating in units of sentences, in units of words, or in units of morphemes.

In the speech processing system 1 of the embodiment described above, the customer may be prevented from knowing that his/her own speech is being conveyed to the operator with suppressed emotion. That is, the customer may not be able to ascertain whether the emotion suppression function is on or off.

The annotation work may be performed by the operator on the operator terminal 20, or a dedicated application or terminal for the annotation work may be separately prepared.

Moreover, the embodiments described above are for facilitating understanding of the present invention, and are not intended to limit and interpret the present invention. Each element included in the embodiment and its arrangement, materials, conditions, shape, size, etc. are not limited to those illustrated and can be changed as appropriate. Also, it is possible to partially replace or combine the configurations shown in different embodiments. Also, the operator terminal 20 may have the functions described as the functions of the speech processing device 10 . Also, the voice processing device 10 may have the functions described as the functions of the operator terminal 20 .

REFERENCE SIGNS LIST 1 speech processing system 10 speech processing device 20 operator terminal 30 customer terminal 10a processor 10b RAM 10c ROM 10d communication unit 10e input unit 10f display unit 10g Camera 10h Voice input unit 10i Voice output unit 101 Storage unit 102 Transmission/reception unit 103 Voice recognition unit 104 Removal unit 105 Voice synthesis unit 106 Emotion recognition unit 107 STRESS RECOGNIZING UNIT 108 CONTROLLING UNIT 109 LEARNING UNIT 201 TRANSMITTER/RECEIVER UNIT 202 INPUT RECEIVING UNIT 203 CONTROLLER

A speech processing system according to one aspect of the present invention includes an acquisition unit that acquires an uttered voice signal that is a signal of an uttered voice of a first user; , a speech recognition unit that generates text data including a word string consisting of one or more words, and a feature amount extracted based on the text data is input to a speech synthesis model, and a synthesized speech signal A speech synthesizer for generating a synthesized speech signal, a speech output unit for outputting the synthesized speech to a second user, and an emotion recognition unit for generating emotional information of the first user corresponding to the uttered speech signal. and a control unit for switching between outputting either the synthesized voice or the uttered voice from the voice output unit based on the emotional information, wherein the control unit controls whether the emotional information is transmitted to the second user . and outputting the synthesized speech when it is stressful .

A speech processing system according to one aspect of the present invention includes an acquisition unit that acquires an uttered voice signal that is a signal of an uttered voice of a first user; , a speech recognition unit that generates text data including a word string consisting of one or more words, and a feature amount extracted based on the text data is input to a speech synthesis model, and a synthesized speech signal a speech synthesis unit that generates a synthesized speech signal; a speech output unit that outputs the synthesized speech to a second user; a stress recognition unit that generates stress information regarding the stress situation of the second user; a control unit that switches between outputting either the synthesized voice or the uttered voice from the voice output unit based on stress information, wherein the control unit indicates that the stress information indicates a high stress state. output the synthesized speech when the

A speech processing system according to one aspect of the present invention includes an acquisition unit that acquires an uttered voice signal that is a signal of an uttered voice of a first user; , a speech recognition unit that generates text data including a word string consisting of one or more words, and a feature amount extracted based on the text data is input to a speech synthesis model, and a synthesized speech signal a speech synthesis unit for generating a synthesized speech signal; a speech output unit for outputting the synthesized speech to a second user; A control unit for switching between outputting the synthesized voice and the spoken voice, and associating the switching information input by the second user with the spoken voice signal when the switching information is input on the time axis. based on the information, a speech signal, a feature amount extracted from the speech signal, text data generated from the speech signal, a feature amount extracted from the text data, or at least these a learning unit that performs machine learning on an emotional suppression switching model that receives a combination of two as inputs and outputs a timing for switching between the synthesized voice and the uttered voice; inputting the speech signal acquired by the acquiring unit, the feature amount extracted from the speech signal, the text data generated from the speech signal, the feature amount extracted from the text data, or a combination of at least two of these generates a timing for switching between the synthesized speech and the uttered speech .

Claims (12)

an acquisition unit that acquires an utterance audio signal that is a signal of the utterance audio of the first user;
a speech recognition unit that inputs a feature amount extracted based on the speech signal to a speech recognition model to generate text data including a word string consisting of one or more words;
a speech synthesis unit that inputs a feature amount extracted based on the text data to a speech synthesis model to generate a synthesized speech signal that is a synthesized speech signal;
a speech output unit that outputs the synthesized speech to a second user;
An audio processing system comprising: an emotion recognition unit that generates emotion information of a first user corresponding to the speech audio signal;
a display unit that displays the emotional information to the second user;
The display unit displays the emotional information corresponding to the synthesized speech in accordance with the output timing of the synthesized speech by the speech output unit.
2. The audio processing system of claim 1. an emotion recognition unit that generates emotion information of a first user corresponding to the speech audio signal;
a control unit that switches between outputting the synthetic voice or the uttered voice from the voice output unit based on the emotion information;
3. The audio processing system of claim 1 or 2, comprising: The emotion recognition unit receives an utterance voice signal, a feature amount extracted from the utterance voice signal, text data generated from the utterance voice signal, a feature amount extracted from the text data, or a combination of at least two of these. , an emotion recognition model machine-learned to output the emotional information of the speaker of the speech signal, the speech signal acquired by the acquisition unit, the speech feature extracted from the speech signal, and the speech signal By inputting the generated text data, the text feature value corresponding to the text data, or a combination of at least two of these, the first user's emotion information corresponding to the speech signal obtained by the obtaining unit is generated. 4. A speech processing system according to claim 2 or 3. The speech synthesis unit generates the synthetic speech signal based on the emotion information generated by the emotion recognition unit so that the emotion indicated by the emotion information is reflected in the synthetic speech. 5. The audio processing system according to any one of 4. a stress recognition unit that generates stress information about the stress situation of the second user;
a control unit that switches between outputting the synthesized voice or the uttered voice from the voice output unit based on the stress information;
3. The audio processing system of claim 1 or 2, comprising: a control unit that switches between outputting either the synthesized speech or the uttered speech from the speech output unit based on switching information input by the second user;
generating information in which the switching information input by the second user is associated with the speech signal when the switching information is input on a time axis, and generating the speech signal and the speech speech based on the information; A feature quantity extracted from a signal, text data generated from the speech signal, a feature quantity extracted from the text data, or a combination of at least two of these is input, and timing for switching between the synthesized speech and the speech speech is determined. a learning unit that machine-learns the emotion suppression switching model to be output,
The control unit stores, in the emotion suppression switching model, the speech audio signal acquired by the acquisition unit, the feature amount extracted from the speech audio signal, the text data generated from the speech audio signal, and the features extracted from the text data. 7. The speech processing system according to any one of claims 1 to 6, wherein the timing for switching between the synthesized speech and the spoken speech is generated by inputting a quantity, or a combination of at least two of them. an acquisition unit that acquires an utterance audio signal that is a signal of the utterance audio of the first user;
a speech recognition unit that inputs a feature amount extracted based on the speech signal to a speech recognition model to generate text data including a word string consisting of one or more words;
a speech synthesizing unit that inputs a feature amount extracted based on the text data to a speech synthesis model to generate a synthesized speech signal that is a synthesized speech signal that is output to a second user;
A speech processing device comprising: an emotion recognition unit that generates emotion information of a first user corresponding to the speech audio signal;
a transmitting unit that associates the emotional information with a speech voice signal and/or a synthesized voice signal corresponding to the emotional information on a time axis and transmits the emotional information to an external device;
9. The audio processing device according to claim 8. obtaining a speech signal that is a signal of the first user's speech;
A step of inputting the feature amount extracted based on the speech signal into a speech recognition model to generate text data including a word string consisting of one or more words;
a step of inputting a feature quantity extracted based on the text data into a speech synthesis model to generate a synthesized speech signal, which is a synthesized speech signal;
outputting the synthesized speech to a second user;
audio processing methods, including an acquisition unit that acquires an utterance audio signal that is a signal of the utterance audio of the first user;
a speech conversion unit that inputs a feature quantity extracted based on the speech speech signal to a speech conversion model to generate a converted speech signal;
an audio output unit that outputs the converted audio to a second user;
an emotion recognition unit that generates emotion information of a first user corresponding to the speech audio signal;
a display unit for displaying the emotional information corresponding to the converted voice to the second user in accordance with the output timing of the converted voice by the voice output unit;
An audio processing system comprising: an acquisition unit that acquires an utterance audio signal that is a signal of the utterance audio of the first user;
a speech conversion unit that inputs a feature quantity extracted based on the speech speech signal to a speech conversion model to generate a converted speech signal;
an audio output unit that outputs the converted audio to a second user;
an emotion recognition unit that generates a first user's emotion information corresponding to the speech audio signal;
The speech processing system, wherein the speech conversion unit generates a signal of the converted speech based on the emotion information generated by the emotion recognition unit so that the emotion indicated by the emotion information is reflected in the converted speech.

Images