WO2022178933A1 - 基于上下文的语音情感检测方法、装置、设备及存储介质 - Google Patents
基于上下文的语音情感检测方法、装置、设备及存储介质 Download PDFInfo
- Publication number
- WO2022178933A1 WO2022178933A1 PCT/CN2021/082862 CN2021082862W WO2022178933A1 WO 2022178933 A1 WO2022178933 A1 WO 2022178933A1 CN 2021082862 W CN2021082862 W CN 2021082862W WO 2022178933 A1 WO2022178933 A1 WO 2022178933A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- voice
- emotion
- customer service
- segment
- speech
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 109
- 238000004458 analytical method Methods 0.000 claims abstract description 104
- 230000008451 emotion Effects 0.000 claims description 274
- 230000002996 emotional effect Effects 0.000 claims description 42
- 238000000605 extraction Methods 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 20
- 230000009466 transformation Effects 0.000 claims description 19
- 238000000034 method Methods 0.000 claims description 14
- 238000010835 comparative analysis Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 15
- 238000004422 calculation algorithm Methods 0.000 description 13
- 238000007726 management method Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000009977 dual effect Effects 0.000 description 4
- 238000000926 separation method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013481 data capture Methods 0.000 description 2
- 238000013075 data extraction Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
Definitions
- the present application relates to the technical field of semantic analysis, and in particular, to a context-based speech emotion detection method, apparatus, electronic device, and computer-readable storage medium.
- Speech emotion detection has always been a hot field of attention.
- Speech emotion detection is to obtain the speaker's emotion change information from speech files.
- the customer's emotion is detected from the conversation recording between the customer and the customer service, so as to provide the customer with corresponding dialogue sentences according to the customer's emotion.
- the existing speech emotion detection is mostly based on the emotion prediction of the dialogue content, that is, the customer's emotion is judged according to the detailed content of the dialogue between the customer and the customer service, such as the part of speech of the words appearing in the dialogue.
- the inventor realizes that in a real business scenario , we found that customer emotions are also affected by customer service emotions. Therefore, only detecting the customer's emotion based on the dialogue content will cause the problem of low accuracy of emotion detection.
- a context-based speech emotion detection method provided by this application includes:
- the customer service voice segment at the preset first moment is selected as the first voice segment
- the customer voice segment at the preset second moment is selected as the second voice segment, wherein the second moment is at the After the first moment, both the first moment and the second moment are the extracted speech moments;
- the present application also provides a context-based voice emotion detection device, the device comprising:
- a feature extraction module configured to obtain an input voice stream, perform voiceprint feature extraction on the input voice stream, and obtain the voiceprint feature of the input voice stream;
- a voice division module configured to divide the input voice stream into a customer service voice segment and a customer voice segment according to the voiceprint feature
- a time extraction module used for extracting the voice moments of the customer service voice segment and the customer voice segment respectively;
- a voice selection module configured to select the customer service voice segment at the preset first moment as the first voice segment, and select the customer voice segment at the preset second moment as the second voice segment, wherein the The second moment is after the first moment, and both the first moment and the second moment are the extracted speech moments;
- a first detection module configured to perform emotion detection on the first speech segment by using a pre-trained emotion analysis model to obtain customer service emotion
- the second detection module is configured to use the customer service emotion as a parameter of the emotion analysis model, and use the emotion analysis model to perform emotion detection on the second speech segment to obtain customer emotion.
- the present application also provides an electronic device, the electronic device comprising:
- a processor that executes the instructions stored in the memory to achieve the following steps:
- the customer service voice segment at the preset first moment is selected as the first voice segment
- the customer voice segment at the preset second moment is selected as the second voice segment, wherein the second moment is at the After the first moment, both the first moment and the second moment are the extracted speech moments;
- the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the following steps:
- the customer service voice segment at the preset first moment is selected as the first voice segment
- the customer voice segment at the preset second moment is selected as the second voice segment, wherein the second moment is at the After the first moment, both the first moment and the second moment are the extracted speech moments;
- FIG. 1 is a schematic flowchart of a context-based speech emotion detection method provided by an embodiment of the present application
- FIG. 2 is a functional block diagram of a context-based voice emotion detection device provided by an embodiment of the present application
- FIG. 3 is a schematic structural diagram of an electronic device implementing the context-based speech emotion detection method according to an embodiment of the present application.
- the embodiments of the present application provide a context-based speech emotion detection method.
- the execution subject of the context-based speech emotion detection method includes, but is not limited to, at least one of electronic devices that can be configured to execute the method provided by the embodiments of the present application, such as a server and a terminal.
- the context-based voice emotion detection method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform.
- the server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
- the context-based speech emotion detection method includes:
- the input voice stream includes but is not limited to a call recording, for example, when a customer requests customer service after-sales for a product or service, the call recording between the customer and the customer service is recorded.
- the input voice stream can be obtained from a blockchain node that pre-stores the input voice stream by a python statement with a data capture function.
- the efficiency of obtaining the input voice stream can be improved by utilizing the high data throughput of the blockchain.
- the voiceprint feature extraction is performed on the input voice stream to obtain the voiceprint feature of the input voice stream, including:
- the pooled voice stream is fully connected by using the first fully connected layer to obtain a fully connected voice stream;
- the fully-connected voice stream is processed by the second fully-connected layer to obtain the voiceprint feature of the input voice stream.
- the voice stream since the voice stream contains a large amount of voice information, directly analyzing the voice stream will occupy a lot of computing resources, resulting in low analysis efficiency.
- the voice stream is convoluted to extract the voiceprint features of the voice stream. , thereby reducing the amount of data to be analyzed and improving the efficiency of subsequent analysis; however, the voiceprint features of the convolutional voice stream obtained by convolution still have multi-dimensional conditions.
- the dimension of the voiceprint feature in the convolutional voice stream reduces the occupation of computing resources in the subsequent voiceprint feature extraction of the pooled voice stream, and improves the efficiency of voiceprint feature extraction.
- This embodiment of the present application uses a Densenet201 network including double fully connected layers to perform convolution, pooling, and fully connected processing on the input speech stream.
- the Densenet201 network is a densely connected convolutional neural network, including multiple convolutional layers.
- the input of each target convolutional layer in the network is the output of all network layers before the target convolutional layer, so as to reduce the parameters that need to be set, thereby improving the efficiency of the network to process the speech stream.
- This embodiment of the present application utilizes dual fully connected layers to perform dual full connection processing on the pooled voice stream, which can improve the network complexity, thereby improve the accuracy of the obtained voiceprint features of the voice stream, which is beneficial to improve the follow-up based on the voice stream.
- the accuracy of sentiment analysis of the features is beneficial to improve the follow-up based on the voice stream.
- dividing the input voice stream into a customer service voice segment and a customer voice segment according to the voiceprint feature includes:
- the speech segments whose distance value is greater than or equal to the preset distance threshold are collected into customer speech segments.
- the standard customer service voiceprint feature generally refers to a voiceprint feature predetermined according to a customer service voice segment, and the standard customer service voiceprint feature is predetermined by the user.
- the calculating the distance value between the voiceprint feature of the input voice stream and the standard customer service voiceprint feature includes:
- L(X, Y) is the distance value
- X is the standard customer service voiceprint
- Y i is the ith voice segment in the input voice stream.
- the voice segments whose distance values are less than the preset distance threshold in the input voice stream are collected as customer service voice segments, and the voice segments at the moment when the distance value is greater than or equal to the preset distance threshold are collected as customer voice segments.
- customer service voice segments For example, inputting There are voice segment A, voice segment B, voice segment C, and voice segment D in the voice stream.
- the distance between the voiceprint feature of voice segment A and the standard customer service voiceprint feature is 20, and the voiceprint feature of voice segment B is the same as the standard voiceprint feature.
- the distance value of the voiceprint feature of the customer service is 35, the distance between the voiceprint feature of the voice segment C and the standard customer service voiceprint feature is 66, and the distance between the voiceprint feature of the voice segment D and the standard customer service voiceprint feature is 72.
- the distance threshold is set to 50, the voice segment A and the voice segment B are aggregated into the customer service voice segment, and the voice segment C and the voice segment D are aggregated into the customer voice segment.
- the input voice stream is divided into the customer service voice segment and the customer voice segment according to the voiceprint feature, which can realize the separation of the customer service voice segment and the customer voice segment in the input voice stream, which is conducive to the subsequent targeted analysis of different voice segments.
- the speech segment is used for emotion detection, thereby improving the accuracy of emotion detection.
- the voice time refers to the middle time of different voice segments. For example, if the time of the customer service voice segment A is from 9:10 to 9:20, the voice time of the customer service voice segment A is 9 At 15:00, for another example, if the time of the customer voice segment E is from 10:30 to 10:40, the voice time of the customer voice segment E is 10:35.
- the separately extracting the voice moments of the customer service voice segment and the customer voice segment includes:
- b u (t) is the voice moment
- d u is the u-th customer service voice segment or the customer service voice segment in the input voice stream
- i is the customer service voice segment and the customer voice segment in the input voice stream
- t 0 is the start time of the u-th customer service voice segment or the customer service voice segment in the input voice stream
- t 1 is the end time of the u-th customer service voice segment or the customer service voice segment in the input voice stream
- the preset voice segment of the customer service at the first moment is selected as the first voice segment
- the voice segment of the customer at the preset second moment is selected as the second voice segment
- the second moment is at the After the first time, both the first time and the second time are the extracted speech time.
- customer service voice segment 1 For example, there are customer service voice segment 1 with the voice time at 8:10, customer voice segment 2 with the voice time at 8:12, customer service voice segment 3 with the voice time at 8:14, and customer service voice segment 3 with the voice time at 8:16.
- Customer voice segment 4 then customer service voice segment 1 can be selected as the first voice segment, and customer voice segment 2 can be selected as the second voice segment; or, customer service voice segment 3 can be selected as the first voice segment, and customer voice segment 4 can be selected as the second voice segment voice segment.
- the general situation is that the voice of the customer service and the customer appear in turn, for example, the voice of the customer service: what help do you need? Customer: I need to apply for after-sales service for product A. Customer service voice: Do you need to apply for warranty or return service? Customer Voice: I need to apply for warranty service.
- this embodiment of the present application selects The preset customer service voice segment at the first moment is the first voice segment, and the customer voice segment at the second moment after the first moment is selected as the second voice segment, which is conducive to subsequent analysis of the second voice segment based on the first voice segment. to improve the accuracy of emotion detection.
- the pre-trained sentiment analysis model has a convolutional neural network for audio language processing.
- the emotion detection is performed on the first speech segment by using the pre-trained emotion analysis model to obtain the customer service emotion, including:
- the customer service emotion is determined according to the numerical interval in which the customer service emotion value is located.
- the embodiment of the present application utilizes the audio intensity detection tool pre-installed in the emotion analysis model to continuously detect the speech intensity of the first speech segment, and the audio intensity detection tool includes the PocketRTA decibel tester, the SIA SmaartLive decibel test tool. Wait.
- the intonation feature of the first speech segment is extracted by using a pre-trained sentiment analysis model to detect the speech intonation of the first speech segment.
- calculating the speech volume of the first speech segment according to the speech duration and the speech intensity is to calculate the average volume of the first speech segment within the speech duration.
- the following mean value algorithm is used to calculate The average volume:
- L is the average volume
- n is the speech duration
- P t is the speech intensity of the first speech segment at time t.
- ASR Automatic Speech Recognition, automatic speech content recognition
- calculating the speech rate of the first speech segment according to the speech duration and the number of speech words is to calculate the speech rate of the first speech segment within the speech duration of the first speech segment through a rate algorithm.
- the rate algorithm is:
- V is the speech rate
- n is the duration of the speech
- N is the number of words in the speech.
- the calculation of the customer service emotional value according to the voice intonation, the voice volume and the voice speed includes:
- J is the customer service emotional value
- W is the voice intonation
- L is the average volume
- V is the voice speed
- ⁇ is a preset weight coefficient
- the customer service emotional value after calculating and obtaining the customer service emotional value, compare the customer service emotional value with a preset numerical range, and determine the customer service emotion according to the numerical range in which the customer service emotional value is located. For example, when the customer service emotional value is in When the customer service emotion is within the preset numerical interval [a, b), the customer service emotion is determined to be a positive emotion, and when the customer service emotion value is within the preset numerical interval (b, c), the customer service emotion is determined to be a negative emotion.
- the emotion of the customer service is used as a parameter of the emotion analysis model, and emotion detection is performed on the second speech segment by using the emotion analysis model to obtain the emotion of the customer, including:
- the emotion detection is performed on the second speech segment by using the emotion analysis model with parameters to obtain customer emotion.
- performing parameter transformation on the customer service emotion to obtain emotion parameters including:
- Emotion detection is performed on the second speech segment by using the parameter-containing emotion analysis model to obtain customer emotion.
- the embodiment of the present application uses a pre-built word vector transformation model to perform word vector numerical transformation on the customer service emotion to obtain customer service emotion parameters.
- the word vector transformation model includes but is not limited to the word2vec word vector model and the doc2vec word vector model.
- the configuration file can be called from the sentiment analysis model by using a java statement with a file calling function, where the configuration file is a file used to record model data in the sentiment analysis model framework.
- a preset parser is used to parse the configuration file to obtain the configuration item, where the parser includes but is not limited to a CarakanC/C++ parser, a SquirrelFishC++ parser, and a SquirrelFishExtremeC++.
- a python statement with a data extraction function is used to extract the configuration parameters in the configuration item.
- the use of the customer service emotional parameters to assign values to the configuration parameters to obtain assignment parameters including:
- the customer service emotion parameter corresponding to the first identifier uses the customer service emotion parameter corresponding to the first identifier to assign a value to the configuration parameter corresponding to the second identifier to obtain an assignment parameter.
- the first identifier and the second identifier are preset unique identifiers for marking parameter types or names.
- the first identifier of the existing customer service emotional parameter is A; the existing configuration parameter ⁇ , the existing configuration parameter ⁇ and the existing configuration parameter ⁇ are obtained by traversing the three configuration parameters: the second identifier of the configuration parameter ⁇ is C, and the configuration parameter ⁇ The second identifier of ⁇ is A, and the second identifier of the configuration parameter ⁇ is B. Compare and analyze the first identifiers of the three customer service emotional parameters and the second identifiers of the three configuration parameters respectively, and obtain that the first identifier of the customer service emotional parameter is the same as the second identifier of the configuration parameter ⁇ , then use the customer service emotional The parameter assigns a value to the configuration parameter ⁇ .
- step of using the emotion analysis model with parameters to perform emotion detection on the second speech segment to obtain customer emotion is the same as using the emotion analysis model pre-trained in step S5 to perform emotion detection on the first speech segment.
- the steps to perform emotion detection on the segment to obtain the customer service emotion are the same, and will not be repeated here.
- the separation of the customer service voice segment and the customer voice segment in the input voice stream can be realized. It is conducive to the subsequent targeted emotion detection of different speech segments, thereby improving the accuracy of emotion detection; detecting the customer service emotion in the customer service speech segment with the speech time before, and then using the customer service emotion as a parameter
- the customer sentiment in the voice segment is detected, taking into account the influence of the customer service sentiment on the customer sentiment, which is beneficial to improve the accuracy of detecting the customer sentiment in the customer voice segment. Therefore, the context-based speech emotion detection method proposed in this application can solve the problem of low accuracy of emotion detection.
- FIG. 2 it is a functional block diagram of a context-based speech emotion detection apparatus provided by an embodiment of the present application.
- the context-based speech emotion detection apparatus 100 described in this application may be installed in an electronic device. According to the implemented functions, the context-based speech emotion detection apparatus 100 may include a feature extraction module 101 , a speech division module 102 , a time extraction module 103 , a speech selection module 104 , a first detection module 105 and a second detection module 106 .
- the modules described in this application may also be referred to as units, which refer to a series of computer program segments that can be executed by the processor of an electronic device and can perform fixed functions, and are stored in the memory of the electronic device.
- each module/unit is as follows:
- the feature extraction module 101 is configured to acquire an input voice stream, perform voiceprint feature extraction on the input voice stream, and obtain the voiceprint feature of the input voice stream.
- the input voice stream includes but is not limited to a call recording, for example, when a customer requests customer service after-sales for a product or service, the call recording between the customer and the customer service is recorded.
- the input voice stream can be obtained from a blockchain node that pre-stores the input voice stream by a python statement with a data capture function.
- the efficiency of obtaining the input voice stream can be improved by utilizing the high data throughput of the blockchain.
- the feature extraction module 101 is specifically used for:
- the pooled voice stream is fully connected by using the first fully connected layer to obtain a fully connected voice stream;
- the fully-connected voice stream is processed by the second fully-connected layer to obtain the voiceprint feature of the input voice stream.
- the voice stream since the voice stream contains a large amount of voice information, directly analyzing the voice stream will occupy a lot of computing resources, resulting in low analysis efficiency.
- the voice stream is convoluted to extract the voiceprint features of the voice stream. , thereby reducing the amount of data to be analyzed and improving the efficiency of subsequent analysis; however, the voiceprint features of the convolutional voice stream obtained by convolution still have multi-dimensional conditions.
- the dimension of the voiceprint feature in the convolutional voice stream reduces the occupation of computing resources in the subsequent voiceprint feature extraction of the pooled voice stream, and improves the efficiency of voiceprint feature extraction.
- This embodiment of the present application uses a Densenet201 network including double fully connected layers to perform convolution, pooling, and fully connected processing on the input speech stream.
- the Densenet201 network is a densely connected convolutional neural network, including multiple convolutional layers.
- the input of each target convolutional layer in the network is the output of all network layers before the target convolutional layer, so as to reduce the parameters that need to be set, thereby improving the efficiency of the network to process the speech stream.
- This embodiment of the present application utilizes dual fully connected layers to perform dual full connection processing on the pooled voice stream, which can improve the network complexity, thereby improve the accuracy of the obtained voiceprint features of the voice stream, which is beneficial to improve the follow-up based on the voice stream.
- the accuracy of sentiment analysis of the features is beneficial to improve the follow-up based on the voice stream.
- the voice division module 102 is configured to divide the input voice stream into a customer service voice segment and a customer voice segment according to the voiceprint feature.
- the voice division module 102 is specifically configured to:
- the speech segments whose distance value is greater than or equal to the preset distance threshold are collected into customer speech segments.
- the standard customer service voiceprint feature generally refers to a voiceprint feature predetermined according to a customer service voice segment, and the standard customer service voiceprint feature is predetermined by the user.
- the calculating the distance value between the voiceprint feature of the input voice stream and the standard customer service voiceprint feature includes:
- L(X, Y) is the distance value
- X is the standard customer service voiceprint
- Y i is the ith voice segment in the input voice stream.
- the voice segments whose distance values are less than the preset distance threshold in the input voice stream are collected as customer service voice segments, and the voice segments at the moment when the distance value is greater than or equal to the preset distance threshold are collected as customer voice segments.
- customer service voice segments For example, inputting There are voice segment A, voice segment B, voice segment C, and voice segment D in the voice stream.
- the distance between the voiceprint feature of voice segment A and the standard customer service voiceprint feature is 20, and the voiceprint feature of voice segment B is the same as the standard voiceprint feature.
- the distance value of the voiceprint feature of the customer service is 35, the distance between the voiceprint feature of the voice segment C and the standard customer service voiceprint feature is 66, and the distance between the voiceprint feature of the voice segment D and the standard customer service voiceprint feature is 72.
- the distance threshold is set to 50, the voice segment A and the voice segment B are aggregated into the customer service voice segment, and the voice segment C and the voice segment D are aggregated into the customer voice segment.
- the input voice stream is divided into the customer service voice segment and the customer voice segment according to the voiceprint feature, which can realize the separation of the customer service voice segment and the customer voice segment in the input voice stream, which is conducive to the subsequent targeted analysis of different voice segments.
- the speech segment is used for emotion detection, thereby improving the accuracy of emotion detection.
- the time extraction module 103 is configured to extract the voice moments of the customer service voice segment and the customer voice segment respectively.
- the voice time refers to the middle time of different voice segments. For example, if the time of the customer service voice segment A is from 9:10 to 9:20, the voice time of the customer service voice segment A is 9 At 15:00, for another example, if the time of the customer voice segment E is from 10:30 to 10:40, the voice time of the customer voice segment E is 10:35.
- time extraction module 103 is specifically used for:
- b u (t) is the voice moment
- d u is the u-th customer service voice segment or the customer service voice segment in the input voice stream
- i is the customer service voice segment and the customer voice segment in the input voice stream
- t 0 is the start time of the u-th customer service voice segment or the customer service voice segment in the input voice stream
- t 1 is the end time of the u-th customer service voice segment or the customer service voice segment in the input voice stream
- the voice selection module 104 is configured to select the customer service voice segment at the preset first moment as the first voice segment, and select the customer voice segment at the preset second moment as the second voice segment, wherein , the second time is after the first time, and both the first time and the second time are the extracted speech time.
- the preset voice segment of the customer service at the first moment is selected as the first voice segment
- the voice segment of the customer at the preset second moment is selected as the second voice segment
- the second moment is at the After the first time, both the first time and the second time are the extracted speech time.
- customer service voice segment 1 For example, there are customer service voice segment 1 with the voice time at 8:10, customer voice segment 2 with the voice time at 8:12, customer service voice segment 3 with the voice time at 8:14, and customer service voice segment 3 with the voice time at 8:16.
- Customer voice segment 4 then customer service voice segment 1 can be selected as the first voice segment, and customer voice segment 2 can be selected as the second voice segment; or, customer service voice segment 3 can be selected as the first voice segment, and customer voice segment 4 can be selected as the second voice segment voice segment.
- the general situation is that the voice of the customer service and the customer appear in turn, for example, the voice of the customer service: what help do you need? Customer: I need to apply for after-sales service for product A. Customer service voice: Do you need to apply for warranty or return service? Customer Voice: I need to apply for warranty service.
- this embodiment of the present application selects The preset customer service voice segment at the first moment is the first voice segment, and the customer voice segment at the second moment after the first moment is selected as the second voice segment, which is conducive to subsequent analysis of the second voice segment based on the first voice segment. to improve the accuracy of emotion detection.
- the first detection module 105 is configured to perform emotion detection on the first speech segment by using a pre-trained emotion analysis model to obtain customer service emotion.
- the pre-trained sentiment analysis model has a convolutional neural network for audio language processing.
- the first detection module 105 is specifically used for:
- the customer service emotion is determined according to the numerical interval in which the customer service emotion value is located.
- the embodiment of the present application utilizes the audio intensity detection tool pre-installed in the emotion analysis model to continuously detect the speech intensity of the first speech segment, and the audio intensity detection tool includes the PocketRTA decibel tester, the SIA SmaartLive decibel test tool. Wait.
- the intonation feature of the first speech segment is extracted by using a pre-trained sentiment analysis model to detect the speech intonation of the first speech segment.
- calculating the speech volume of the first speech segment according to the speech duration and the speech intensity is to calculate the average volume of the first speech segment within the speech duration.
- the following mean value algorithm is used to calculate The average volume:
- L is the average volume
- n is the speech duration
- P t is the speech intensity of the first speech segment at time t.
- ASR Automatic Speech Recognition, automatic speech content recognition
- calculating the speech rate of the first speech segment according to the speech duration and the number of speech words is to calculate the speech rate of the first speech segment within the speech duration of the first speech segment through a rate algorithm.
- the rate algorithm is:
- V is the speech rate
- n is the duration of the speech
- N is the number of words in the speech.
- the calculation of the customer service emotional value according to the voice intonation, the voice volume and the voice speed includes:
- J is the customer service emotional value
- W is the voice intonation
- L is the average volume
- V is the voice speed
- ⁇ is a preset weight coefficient
- the customer service emotional value after calculating and obtaining the customer service emotional value, compare the customer service emotional value with a preset numerical range, and determine the customer service emotion according to the numerical range in which the customer service emotional value is located. For example, when the customer service emotional value is in When the customer service emotion is within the preset numerical interval [a, b), the customer service emotion is determined to be a positive emotion, and when the customer service emotion value is within the preset numerical interval (b, c), the customer service emotion is determined to be a negative emotion.
- the second detection module 106 is configured to use the customer service emotion as a parameter of the emotion analysis model, and use the emotion analysis model to perform emotion detection on the second speech segment to obtain customer emotion.
- the second detection module 106 is specifically used for:
- the emotion detection is performed on the second speech segment by using the emotion analysis model with parameters to obtain customer emotion.
- performing parameter transformation on the customer service emotion to obtain emotion parameters including:
- Emotion detection is performed on the second speech segment by using the parameter-containing emotion analysis model to obtain customer emotion.
- the embodiment of the present application uses a pre-built word vector transformation model to perform word vector numerical transformation on the customer service emotion to obtain customer service emotion parameters.
- the word vector transformation model includes but is not limited to the word2vec word vector model and the doc2vec word vector model.
- the configuration file can be called from the sentiment analysis model by using a java statement with a file calling function, where the configuration file is a file used to record model data in the sentiment analysis model framework.
- a preset parser is used to parse the configuration file to obtain the configuration item, where the parser includes but is not limited to a CarakanC/C++ parser, a SquirrelFishC++ parser, and a SquirrelFishExtremeC++.
- a python statement with a data extraction function is used to extract the configuration parameters in the configuration item.
- the use of the customer service emotional parameters to assign values to the configuration parameters to obtain assignment parameters including:
- the customer service emotion parameter corresponding to the first identifier uses the customer service emotion parameter corresponding to the first identifier to assign a value to the configuration parameter corresponding to the second identifier to obtain an assignment parameter.
- the first identifier and the second identifier are preset unique identifiers for marking parameter types or names.
- the first identifier of the existing customer service emotional parameter is A; the existing configuration parameter ⁇ , the existing configuration parameter ⁇ and the existing configuration parameter ⁇ are obtained by traversing the three configuration parameters: the second identifier of the configuration parameter ⁇ is C, and the configuration parameter ⁇ The second identifier of ⁇ is A, and the second identifier of the configuration parameter ⁇ is B. Compare and analyze the first identifiers of the three customer service emotional parameters and the second identifiers of the three configuration parameters respectively, and obtain that the first identifier of the customer service emotional parameter is the same as the second identifier of the configuration parameter ⁇ , then use the customer service emotional The parameter assigns a value to the configuration parameter ⁇ .
- step of using the emotion analysis model with parameters to perform emotion detection on the second speech segment to obtain customer emotion is the same as using the emotion analysis model pre-trained in step S5 to perform emotion detection on the first speech segment.
- the steps to perform emotion detection on the segment to obtain the customer service emotion are the same, and will not be repeated here.
- the separation of the customer service voice segment and the customer voice segment in the input voice stream can be realized. It is conducive to the subsequent targeted emotion detection of different speech segments, thereby improving the accuracy of emotion detection; detecting the customer service emotion in the customer service speech segment with the speech time before, and then using the customer service emotion as a parameter
- the customer sentiment in the voice segment is detected, taking into account the influence of the customer service sentiment on the customer sentiment, which is beneficial to improve the accuracy of detecting the customer sentiment in the customer voice segment. Therefore, the context-based speech emotion detection device proposed in this application can solve the problem of low accuracy of emotion detection.
- FIG. 3 it is a schematic structural diagram of an electronic device for implementing a context-based speech emotion detection method provided by an embodiment of the present application.
- the electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a context-based speech emotion detection program 12.
- the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, magnetic disk, CD etc.
- the memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1 .
- the memory 11 may also be an external storage device of the electronic device 1, such as a pluggable mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital) equipped on the electronic device 1. , SD) card, flash memory card (Flash Card), etc.
- the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device.
- the memory 11 can not only be used to store application software installed in the electronic device 1 and various types of data, such as the code of the context-based speech emotion detection program 12, etc., but also can be used to temporarily store data that has been output or will be output.
- the processor 10 may be composed of integrated circuits, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits packaged with the same function or different functions, including one or more integrated circuits.
- Central Processing Unit CPU
- microprocessor digital processing chip
- graphics processor and combination of various control chips, etc.
- the processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and by running or executing the programs or modules stored in the memory 11 (for example, based on Contextual voice emotion detection program, etc.), and call the data stored in the memory 11 to perform various functions of the electronic device 1 and process data.
- the bus may be a peripheral component interconnect (PCI for short) bus or an extended industry standard architecture (Extended industry standard architecture, EISA for short) bus or the like.
- PCI peripheral component interconnect
- EISA Extended industry standard architecture
- the bus can be divided into address bus, data bus, control bus and so on.
- the bus is configured to implement connection communication between the memory 11 and at least one processor 10 and the like.
- FIG. 3 only shows an electronic device with components. Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the electronic device 1, and may include fewer or more components than those shown in the figure. components, or a combination of certain components, or a different arrangement of components.
- the electronic device 1 may also include a power supply (such as a battery) for powering the various components, preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that the power management
- the device implements functions such as charge management, discharge management, and power consumption management.
- the power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components.
- the electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
- the electronic device 1 may also include a network interface, optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.
- a network interface optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.
- the electronic device 1 may further include a user interface, and the user interface may be a display (Display), an input unit (eg, a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface or a wireless interface.
- the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, and the like.
- the display may also be appropriately called a display screen or a display unit, which is used for displaying information processed in the electronic device 1 and for displaying a visualized user interface.
- the context-based speech emotion detection program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, and when running in the processor 10, it can realize:
- the customer service voice segment at the preset first moment is selected as the first voice segment
- the customer voice segment at the preset second moment is selected as the second voice segment, wherein the second moment is at the After the first moment, both the first moment and the second moment are the extracted speech moments;
- the modules/units integrated in the electronic device 1 may be stored in a computer-readable storage medium.
- the computer-readable storage medium may be volatile or non-volatile.
- the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM, Read-Only). Memory).
- the present application also provides a computer-readable storage medium, where the readable storage medium stores a computer program, and when executed by a processor of an electronic device, the computer program can realize:
- the customer service voice segment at the preset first moment is selected as the first voice segment
- the customer voice segment at the preset second moment is selected as the second voice segment, wherein the second moment is at the After the first moment, both the first moment and the second moment are the extracted speech moments;
- modules described as separate components may or may not be physically separated, and components shown as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
- each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.
- the blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm.
- Blockchain essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block.
- the blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- General Health & Medical Sciences (AREA)
- Child & Adolescent Psychology (AREA)
- Signal Processing (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Telephonic Communication Services (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
一种基于上下文的语音情感检测方法，包括：对获取的输入语音流进行声纹特征提取；根据声纹特征将输入语音流划分为客服语音段与客户语音段；提取客服语音段与客户语音段的语音时刻；选取第一时刻下的客服语音段为第一语音段，选取第二时刻下的客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后；利用情感分析模型对第一语音段进行情感检测，得到客服情感；将客服情感作为参数对第二语音段进行情感检测，得到客户情感。此外，还涉及区块链技术，所述输入语音流可存储于区块链的节点，可以解决情感检测的精确度不高的问题。
Description
本申请要求于2021年2月26日提交中国专利局、申请号为CN202110214155.5、名称为“基于上下文的语音情感检测方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本申请涉及语义分析技术领域，尤其涉及一种基于上下文的语音情感检测方法、装置、电子设备及计算机可读存储介质。
语音情感检测一直是人们关注的热门领域，语音情感检测是从语音文件中获得说话人情绪变化信息。例如，从客户与客服的对话录音中检测出客户的情感，以实现根据客户情感对客户提供相应的对话语句。
现有的语音情感检测多为基于对话内容的情感预测，即根据客户与客服之间对话的详细内容，如对话中出现的词语词性等来判断客户情感，发明人意识到在真实的业务场景中，我们发现客户的情感还会受到客服的情感的影响。因此仅基于对话内容对客户的情感进行检测，会造成情感检测的精确度不高的问题。
发明内容
本申请提供的一种基于上下文的语音情感检测方法，包括：
获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；
根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；
分别提取所述客服语音段与所述客户语音段的语音时刻；
选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；
利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；
将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请还提供一种基于上下文的语音情感检测装置，所述装置包括：
特征提取模块，用于获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；
语音划分模块，用于根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；
时刻提取模块，用于分别提取所述客服语音段与所述客户语音段的语音时刻；
语音选取模块，用于选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；
第一检测模块，用于利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；
第二检测模块，用于将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请还提供一种电子设备，所述电子设备包括：
存储器，存储至少一个指令；及
处理器，执行所述存储器中存储的指令以实现如下步骤：
获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；
根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；
分别提取所述客服语音段与所述客户语音段的语音时刻；
选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；
利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；
将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有至少一个指令，所述至少一个指令被电子设备中的处理器执行以实现如下步骤：
获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；
根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；
分别提取所述客服语音段与所述客户语音段的语音时刻；
选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；
利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；
将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
图1为本申请一实施例提供的基于上下文的语音情感检测方法的流程示意图；
图2为本申请一实施例提供的基于上下文的语音情感检测装置的功能模块图；
图3为本申请一实施例提供的实现所述基于上下文的语音情感检测方法的电子设备的结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请实施例提供一种基于上下文的语音情感检测方法。所述基于上下文的语音情感检测方法的执行主体包括但不限于服务端、终端等能够被配置为执行本申请实施例提供的该方法的电子设备中的至少一种。换言之，所述基于上下文的语音情感检测方法可以由安装在终端设备或服务端设备的软件或硬件来执行，所述软件可以是区块链平台。所述服务端包括但不限于：单台服务器、服务器集群、云端服务器或云端服务器集群等。
参照图1所示，为本申请一实施例提供的基于上下文的语音情感检测方法的流程示意图。在本实施例中，所述基于上下文的语音情感检测方法包括：
S1、获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征。
本申请实施例中，所述输入语音流包括但不限于通话录音，例如，客户针对产品或服务提出客服售后时，客户与客服之间的通话录音。
详细地，所述输入语音流可由具有数据抓取功能的python语句从预先存储所述输入语音流的区块链节点中获取。利用区块链对数据的高吞吐性，可提高获取所述输入语音流的 效率。
本申请实施例中，所述对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征，包括：
对不同语音时刻下的输入语音流进行卷积处理，得到卷积语音流；
对所述卷积语音流进行全局最大池化处理，得到池化语音流；
利用第一全连接层对所述池化语音流进行全连接处理，得到全连接语音流；
利用第二全连接层对所述全连接语音流进行全连接处理，得到所述输入语音流的声纹特征。
具体地，由于语音流中包含大量语音信息，直接对语音流进行分析会占用大量计算资源，造成分析效率低下，本申请实施例将语音流进行卷积处理，可提取出语音流的声纹特征，进而减少需要分析的数据量，提高后续分析效率；但卷积得到的卷积语音流的声纹特征依然存在着多维的情况，本申请实施例利用全局最大池化可进一步减少卷积得到的卷积语音流中声纹特征的维度，减少后续对池化语音流进行声纹特征提取时计算资源的占用，提高提取声纹特征的效率。
本申请实施例利用包含双全连接层的Densenet201网络对对所述输入语音流进行卷积、池化及全连接处理，所述Densenet201网络是一种密集连接卷积神经网络，包括多个卷积层，网络中每一目标卷积层的输入均为该目标卷积层之前所有网络层次的输出，以此减少需要设定的参数，从而提高网络对语音流处理的效率。
本申请实施例利用双全连接层级联对池化语音流进行双全连接处理，可提高网络复杂度，进而提高获得的所述语音流的声纹特征的精确度，有利于提高后续基于所述语音流的特征进行情感分析的精确度。
S2、根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段。
本申请实施例中，所述根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段，包括：
计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值；
将所述输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段；
将所述距离值大于或等于所述预设距离阈值的的语音段汇集为客户语音段。
详细地，所述标准客服声纹特征一般是指根据客服语音段预先确定的声纹特征，所述标准客服声纹特征由用户预先给定。
具体地，所述计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值，包括：
利用如下距离算法计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值：
其中，L(X，Y)为所述距离值，X为所述标准客服声纹，Y
i为所述输入语音流中第i个语音段。
本申请实施例将输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段，将距离值大于或等于预设距离阈值的时刻下的语音段汇集为客户语音段，例如，输入语音流中存在语音段A、语音段B、语音段C和语音段D，其中，语音段A的声纹特征与标准客服声纹特征的距离值为20，语音段B的声纹特征与标准客服声纹特征的距离值为35，语音段C的声纹特征与标准客服声纹特征的距离值为66，语音段D的声纹特征与标准客服声纹特征的距离值为72，当预设距离阈值为50时，将语音段A和语音段B汇集为客服语音段，将语音段C和语音段D汇集为客户语音段。
本申请实施例根据所述声纹特征将输入语音流划分为客服语音段与客户语音段，可实 现将输入语音流中客服语音段与客户语音段的分离，有利于后续针对性的对不同的语音段进行情感检测，进而提高情感检测的精确度。
S3、分别提取所述客服语音段与所述客户语音段的语音时刻。
本申请实施例中，所述语音时刻是指不同语音段的中间时刻，例如，客服语音段A的时间为9点10分至9点20分，则所述客服语音段A的语音时刻为9点15分，再例如，客户语音段E的时间为10点30分至10点40分，则所述客户语音段E的语音时刻为10点35分。
详细地，所述分别提取所述客服语音段与所述客户语音段的语音时刻，包括：
利用如下时刻提取算法提取所述客服语音段与客户语音段的语音时刻：
其中，b
u(t)为所述语音时刻，d
u为所述输入语音流中第u个客服语音段或客服语音段，i为所述输入语音流中所述客服语音段与客户语音段的总数量，t
0为所述输入语音流中第u个客服语音段或客服语音段的开始时间，t
1为所述输入语音流中第u个客服语音段或客服语音段的结束时间，
为取中间时刻的运算符号。
S4、选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻。
本申请实施例中，选取预设的第一时下的客服语音段为第一语音段，选取预设的第二时刻下的客户语音段为第二语音段，且所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻。
例如，存在语音时刻为8点10分的客服语音段1，语音时刻为8点12分的客户语音段2，语音时刻为8点14分的客服语音段3，语音时刻为8点16分的客户语音段4，则可选取客服语音段1为第一语音段，选取客户语音段2为第二语音段；或者，选取客服语音段3为第一语音段，选取客户语音段4为第二语音段。
详细地，在客服、客户的对话场景中，一般情况是客服与客户的语音轮流出现，例如，客服语音：您需要什么帮助呢？客户：我需要为产品A申请售后服务。客服语音：您需要申请保修还是退货服务呢？客户语音：我需要申请保修服务。
上述情况下，由于客户的情绪可能会随着客服所说的内容而变化，例如客服的语音内容或语调等让客户产生误解，导致后续客户语音包含的情感为负面情感，因此本申请实施例选取预设的第一时刻下的客服语音段为第一语音段并选取在第一时刻之后的第二时刻的客户语音段为第二语音段，有利于后续基于第一语音段分析第二语音段中客户的情感，以提高情感检测的精确度。
S5、利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感。
本申请实施例中，所述预先训练完成的情感分析模型具有音频语言处理的卷积神经网络。
详细地，所述利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感，包括：
利用所述情感分析模型检测所述第一语音段的语音时长和语音语调；
持续检测所述第一语音段的语音强度，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量；
对所述第一语音段进行语音内容识别，统计语音内容识别结果中用户的语音字数；
根据所述语音时长和所述语音字数计算所述第一语音段的语音语速；
根据所述语音语调、所述语音音量和所述语音语速计算客服情感值；
根据所述客服情感值所在的数值区间确定客服情感。
详细地，本申请实施例利用预先安装于情感分析模型中的音频强度检测工具来持续检测所述第一语音段的语音强度，所述音频强度检测工具包括PocketRTA分贝测试仪、SIA SmaartLive分贝测试工具等。
本申请实施例通过预先训练完成的情感分析模型提取所述第一语音段的语调特征来实现检测所述第一语音段的语音语调。
具体地，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量是计算所述第一语音段在所述语音时长内的平均音量，本申请实施例利用如下均值算法计算所述平均音量：
其中，L为所述平均音量，n为所述语音时长，P
t为所述第一语音段在t时刻的语音强度。
进一步地，本申请实施例中利用ASR(Automatic Speech Recognition，自动语音内容识别)技术对所述第一语音段进行文本转换，得到语音内容识别，并统计语音内容识别的结果中用户的语音字数。
详细地，根据所述语音时长和所述语音字数计算所述第一语音段的语音语速是通过速率算法计算所述第一语音段在所述第一语音段的语音时长内的说话语速，所述速率算法为：
其中，V为所述语音语速，n为所述语音时长，N为所述语音字数。
本申请实施例中，所述根据所述语音语调、所述语音音量和所述语音语速计算客服情感值，包括：
利用如下积极度算法根据所述语音语调、所述语音音量和所述语音语速计算客服情感值：
其中，J为所述客服情感值，W为所述语音语调，L为所述平均音量，V为所述语音语速，α为预设权重系数。
详细地，计算获取所述客服情感值后，将所述客服情感值与预设的数值区间进行比较，根据所述客服情感值所在的数值区间确定客服情感，例如，当所述客服情感值在预设的数值区间[a，b)内时，确定客服情感为积极情感，当所述客服情感值在预设的数值区间(b，c]内时，确定客服情感为消极情感。
S6、将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请实施例中，所述将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感，包括：
对所述客服情感进行参数转化，得到情感参数；
利用所述情感参数对所述情感分析模型进行参数赋值，得到带有参数的情感分析模型；
利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
详细地，所述对所述客服情感进行参数转化，得到情感参数，包括：
对所述客服情感进行词向量数值转化，得到客服情感参数；
获取所述情感分析模型的配置文件；
解析所述配置文件得到配置项，并提取所述配置项中的配置参数；
利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数；
将所述赋值参数输入至所述情感分析模型，得到含参数的情感分析模型；
利用所述含参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请实施例利用预先构建的词向量转化模型对所述客服情感进行词向量数值转化，得到客服情感参数，所述词向量转化模型包括但不限于word2vec词向量模型和doc2vec词向量模型。
具体地，本申请实施例可利用具有文件调用功能的java语句从所述情感分析模型中调用所述配置文件，所述配置文件是所述情感分析模型框架中用于记录模型数据的文件。
本申请实施例利用预设的解析器对所述配置文件进行解析，得到所述配置项，其中，所述解析器包括但不限于CarakanC/C++解析器，SquirrelFishC++解析器和SquirrelFishExtremeC++。
进一步地，本申请实施例利用具有数据提取功能的python语句提取所述配置项中的配置参数。
详细地，所述利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数，包括：
遍历所述客服情感参数并确定所述客服情感参数中的第一标识符；
遍历所述配置参数并确定所述配置参数中的第二标识符；
将所述第一标识符与所述第二标识符进行对比分析；
当所述第一标识符与所述第二标识符不相同时，重新遍历所述配置参数并确定所述配置参数中的第二标识符；
当所述第一标识符与所述第二标识符相同时，利用所述第一标识符对应的客服情感参数对所述第二标识符对应的配置参数进行赋值，得到赋值参数。
本申请实施例中，所述第一标识符和所述第二标识符为预先设定的用于标记参数类型或名称的唯一标识。
例如，存在客服情感参数的第一标识符为A；存在配置参数α、存在配置参数β和存在配置参数γ，遍历三个配置参数得到：配置参数α的第二标识符为C，配置参数β的第二标识符为A，配置参数γ的第二标识符为B。分别将三个客服情感参数的第一标识符与三个配置参数的第二标识符进行对比分析，得到客服情感参数的第一标识符与配置参数β的第二标识符相同，则利用客服情感参数对配置参数β进行赋值。
进一步地，所述利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感的步骤，与步骤S5中利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感的步骤一致，在此不做赘述。
本申请实施例通过提取输入语音流的声纹特征，并根据声纹特征将输入语音流划分为客服语音段与客户语音段，可实现将输入语音流中客服语音段与客户语音段的分离，有利于后续针对性的对不同的语音段进行情感检测，进而提高情感检测的精确度；检测语音时刻在前的客服语音段中的客服情绪，再利用客服情绪作为参数对语言时刻在后的客户语音段中的客户情绪进行检测，考虑到了客服情绪对客户情绪的影响，有利于提高检测客户语音段中客户情绪的精确度。因此本申请提出的基于上下文的语音情感检测方法，可以解决情感检测的精确度不高的问题。
如图2所示，是本申请一实施例提供的基于上下文的语音情感检测装置的功能模块图。
本申请所述基于上下文的语音情感检测装置100可以安装于电子设备中。根据实现的功能，所述基于上下文的语音情感检测装置100可以包括特征提取模块101、语音划分模块102、时刻提取模块103、语音选取模块104、第一检测模块105和第二检测模块106。本申请所述模块也可以称之为单元，是指一种能够被电子设备处理器所执行，并且能够完成固定功能的一系列计算机程序段，其存储在电子设备的存储器中。
在本实施例中，关于各模块/单元的功能如下：
所述特征提取模块101，用于获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征。
本申请实施例中，所述输入语音流包括但不限于通话录音，例如，客户针对产品或服务提出客服售后时，客户与客服之间的通话录音。
详细地，所述输入语音流可由具有数据抓取功能的python语句从预先存储所述输入语音流的区块链节点中获取。利用区块链对数据的高吞吐性，可提高获取所述输入语音流的效率。
本申请实施例中，所述特征提取模块101具体用于：
对不同语音时刻下的输入语音流进行卷积处理，得到卷积语音流；
对所述卷积语音流进行全局最大池化处理，得到池化语音流；
利用第一全连接层对所述池化语音流进行全连接处理，得到全连接语音流；
利用第二全连接层对所述全连接语音流进行全连接处理，得到所述输入语音流的声纹特征。
具体地，由于语音流中包含大量语音信息，直接对语音流进行分析会占用大量计算资源，造成分析效率低下，本申请实施例将语音流进行卷积处理，可提取出语音流的声纹特征，进而减少需要分析的数据量，提高后续分析效率；但卷积得到的卷积语音流的声纹特征依然存在着多维的情况，本申请实施例利用全局最大池化可进一步减少卷积得到的卷积语音流中声纹特征的维度，减少后续对池化语音流进行声纹特征提取时计算资源的占用，提高提取声纹特征的效率。
本申请实施例利用包含双全连接层的Densenet201网络对对所述输入语音流进行卷积、池化及全连接处理，所述Densenet201网络是一种密集连接卷积神经网络，包括多个卷积层，网络中每一目标卷积层的输入均为该目标卷积层之前所有网络层次的输出，以此减少需要设定的参数，从而提高网络对语音流处理的效率。
本申请实施例利用双全连接层级联对池化语音流进行双全连接处理，可提高网络复杂度，进而提高获得的所述语音流的声纹特征的精确度，有利于提高后续基于所述语音流的特征进行情感分析的精确度。
所述语音划分模块102，用于根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段。
本申请实施例中，所述语音划分模块102具体用于：
计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值；
将所述输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段；
将所述距离值大于或等于所述预设距离阈值的的语音段汇集为客户语音段。
详细地，所述标准客服声纹特征一般是指根据客服语音段预先确定的声纹特征，所述标准客服声纹特征由用户预先给定。
具体地，所述计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值，包括：
利用如下距离算法计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值：
其中，L(X，Y)为所述距离值，X为所述标准客服声纹，Y
i为所述输入语音流中第i个语音段。
本申请实施例将输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段，将距离值大于或等于预设距离阈值的时刻下的语音段汇集为客户语音段，例如，输入语音流中存在语音段A、语音段B、语音段C和语音段D，其中，语音段A的声纹特征与标准客服声纹特征的距离值为20，语音段B的声纹特征与标准客服声纹特征的距离值为35，语音段C的声纹特征与标准客服声纹特征的距离值为66，语音段D的声纹特征与标准客 服声纹特征的距离值为72，当预设距离阈值为50时，将语音段A和语音段B汇集为客服语音段，将语音段C和语音段D汇集为客户语音段。
本申请实施例根据所述声纹特征将输入语音流划分为客服语音段与客户语音段，可实现将输入语音流中客服语音段与客户语音段的分离，有利于后续针对性的对不同的语音段进行情感检测，进而提高情感检测的精确度。
所述时刻提取模块103，用于分别提取所述客服语音段与所述客户语音段的语音时刻。
本申请实施例中，所述语音时刻是指不同语音段的中间时刻，例如，客服语音段A的时间为9点10分至9点20分，则所述客服语音段A的语音时刻为9点15分，再例如，客户语音段E的时间为10点30分至10点40分，则所述客户语音段E的语音时刻为10点35分。
详细地，所述时刻提取模块103具体用于：
利用如下时刻提取算法提取所述客服语音段与所述客户语音段的语音时刻：
其中，b
u(t)为所述语音时刻，d
u为所述输入语音流中第u个客服语音段或客服语音段，i为所述输入语音流中所述客服语音段与客户语音段的总数量，t
0为所述输入语音流中第u个客服语音段或客服语音段的开始时间，t
1为所述输入语音流中第u个客服语音段或客服语音段的结束时间，
为取中间时刻的运算符号。
所述语音选取模块104，用于选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻。
本申请实施例中，选取预设的第一时下的客服语音段为第一语音段，选取预设的第二时刻下的客户语音段为第二语音段，且所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻。
例如，存在语音时刻为8点10分的客服语音段1，语音时刻为8点12分的客户语音段2，语音时刻为8点14分的客服语音段3，语音时刻为8点16分的客户语音段4，则可选取客服语音段1为第一语音段，选取客户语音段2为第二语音段；或者，选取客服语音段3为第一语音段，选取客户语音段4为第二语音段。
详细地，在客服、客户的对话场景中，一般情况是客服与客户的语音轮流出现，例如，客服语音：您需要什么帮助呢？客户：我需要为产品A申请售后服务。客服语音：您需要申请保修还是退货服务呢？客户语音：我需要申请保修服务。
上述情况下，由于客户的情绪可能会随着客服所说的内容而变化，例如客服的语音内容或语调等让客户产生误解，导致后续客户语音包含的情感为负面情感，因此本申请实施例选取预设的第一时刻下的客服语音段为第一语音段并选取在第一时刻之后的第二时刻的客户语音段为第二语音段，有利于后续基于第一语音段分析第二语音段中客户的情感，以提高情感检测的精确度。
所述第一检测模块105，用于利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感。
本申请实施例中，所述预先训练完成的情感分析模型具有音频语言处理的卷积神经网络。
详细地，所述第一检测模块105具体用于：
利用所述情感分析模型检测所述第一语音段的语音时长和语音语调；
持续检测所述第一语音段的语音强度，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量；
对所述第一语音段进行语音内容识别，统计语音内容识别结果中用户的语音字数；
根据所述语音时长和所述语音字数计算所述第一语音段的语音语速；
根据所述语音语调、所述语音音量和所述语音语速计算客服情感值；
根据所述客服情感值所在的数值区间确定客服情感。
详细地，本申请实施例利用预先安装于情感分析模型中的音频强度检测工具来持续检测所述第一语音段的语音强度，所述音频强度检测工具包括PocketRTA分贝测试仪、SIA SmaartLive分贝测试工具等。
本申请实施例通过预先训练完成的情感分析模型提取所述第一语音段的语调特征来实现检测所述第一语音段的语音语调。
具体地，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量是计算所述第一语音段在所述语音时长内的平均音量，本申请实施例利用如下均值算法计算所述平均音量：
其中，L为所述平均音量，n为所述语音时长，P
t为所述第一语音段在t时刻的语音强度。
进一步地，本申请实施例中利用ASR(Automatic Speech Recognition，自动语音内容识别)技术对所述第一语音段进行文本转换，得到语音内容识别，并统计语音内容识别的结果中用户的语音字数。
详细地，根据所述语音时长和所述语音字数计算所述第一语音段的语音语速是通过速率算法计算所述第一语音段在所述第一语音段的语音时长内的说话语速，所述速率算法为：
其中，V为所述语音语速，n为所述语音时长，N为所述语音字数。
本申请实施例中，所述根据所述语音语调、所述语音音量和所述语音语速计算客服情感值，包括：
利用如下积极度算法根据所述语音语调、所述语音音量和所述语音语速计算客服情感值：
其中，J为所述客服情感值，W为所述语音语调，L为所述平均音量，V为所述语音语速，α为预设权重系数。
详细地，计算获取所述客服情感值后，将所述客服情感值与预设的数值区间进行比较，根据所述客服情感值所在的数值区间确定客服情感，例如，当所述客服情感值在预设的数值区间[a，b)内时，确定客服情感为积极情感，当所述客服情感值在预设的数值区间(b，c]内时，确定客服情感为消极情感。
所述第二检测模块106，用于将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请实施例中，所述第二检测模块106具体用于：
对所述客服情感进行参数转化，得到情感参数；
利用所述情感参数对所述情感分析模型进行参数赋值，得到带有参数的情感分析模型；
利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
详细地，所述对所述客服情感进行参数转化，得到情感参数，包括：
对所述客服情感进行词向量数值转化，得到客服情感参数；
获取所述情感分析模型的配置文件；
解析所述配置文件得到配置项，并提取所述配置项中的配置参数；
利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数；
将所述赋值参数输入至所述情感分析模型，得到含参数的情感分析模型；
利用所述含参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
本申请实施例利用预先构建的词向量转化模型对所述客服情感进行词向量数值转化，得到客服情感参数，所述词向量转化模型包括但不限于word2vec词向量模型和doc2vec词向量模型。
具体地，本申请实施例可利用具有文件调用功能的java语句从所述情感分析模型中调用所述配置文件，所述配置文件是所述情感分析模型框架中用于记录模型数据的文件。
本申请实施例利用预设的解析器对所述配置文件进行解析，得到所述配置项，其中，所述解析器包括但不限于CarakanC/C++解析器，SquirrelFishC++解析器和SquirrelFishExtremeC++。
进一步地，本申请实施例利用具有数据提取功能的python语句提取所述配置项中的配置参数。
详细地，所述利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数，包括：
遍历所述客服情感参数并确定所述客服情感参数中的第一标识符；
遍历所述配置参数并确定所述配置参数中的第二标识符；
将所述第一标识符与所述第二标识符进行对比分析；
当所述第一标识符与所述第二标识符不相同时，重新遍历所述配置参数并确定所述配置参数中的第二标识符；
当所述第一标识符与所述第二标识符相同时，利用所述第一标识符对应的客服情感参数对所述第二标识符对应的配置参数进行赋值，得到赋值参数。
本申请实施例中，所述第一标识符和所述第二标识符为预先设定的用于标记参数类型或名称的唯一标识。
例如，存在客服情感参数的第一标识符为A；存在配置参数α、存在配置参数β和存在配置参数γ，遍历三个配置参数得到：配置参数α的第二标识符为C，配置参数β的第二标识符为A，配置参数γ的第二标识符为B。分别将三个客服情感参数的第一标识符与三个配置参数的第二标识符进行对比分析，得到客服情感参数的第一标识符与配置参数β的第二标识符相同，则利用客服情感参数对配置参数β进行赋值。
进一步地，所述利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感的步骤，与步骤S5中利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感的步骤一致，在此不做赘述。
本申请实施例通过提取输入语音流的声纹特征，并根据声纹特征将输入语音流划分为客服语音段与客户语音段，可实现将输入语音流中客服语音段与客户语音段的分离，有利于后续针对性的对不同的语音段进行情感检测，进而提高情感检测的精确度；检测语音时刻在前的客服语音段中的客服情绪，再利用客服情绪作为参数对语言时刻在后的客户语音段中的客户情绪进行检测，考虑到了客服情绪对客户情绪的影响，有利于提高检测客户语音段中客户情绪的精确度。因此本申请提出的基于上下文的语音情感检测装置，可以解决情感检测的精确度不高的问题。
如图3所示，是本申请一实施例提供的实现基于上下文的语音情感检测方法的电子设备的结构示意图。
所述电子设备1可以包括处理器10、存储器11和总线，还可以包括存储在所述存储器11中并可在所述处理器10上运行的计算机程序，如基于上下文的语音情感检测程序12。
其中，所述存储器11至少包括一种类型的可读存储介质，所述可读存储介质包括闪存、移动硬盘、多媒体卡、卡型存储器(例如：SD或DX存储器等)、磁性存储器、磁盘、光盘等。所述存储器11在一些实施例中可以是电子设备1的内部存储单元，例如该电子设备1的移动硬盘。所述存储器11在另一些实施例中也可以是电子设备1的外部存储设备，例 如电子设备1上配备的插接式移动硬盘、智能存储卡(Smart Media Card，SMC)、安全数字(Secure Digital，SD)卡、闪存卡(Flash Card)等。进一步地，所述存储器11还可以既包括电子设备1的内部存储单元也包括外部存储设备。所述存储器11不仅可以用于存储安装于电子设备1的应用软件及各类数据，例如基于上下文的语音情感检测程序12的代码等，还可以用于暂时地存储已经输出或者将要输出的数据。
所述处理器10在一些实施例中可以由集成电路组成，例如可以由单个封装的集成电路所组成，也可以是由多个相同功能或不同功能封装的集成电路所组成，包括一个或者多个中央处理器(Central Processing unit，CPU)、微处理器、数字处理芯片、图形处理器及各种控制芯片的组合等。所述处理器10是所述电子设备的控制核心(Control Unit)，利用各种接口和线路连接整个电子设备的各个部件，通过运行或执行存储在所述存储器11内的程序或者模块(例如基于上下文的语音情感检测程序等)，以及调用存储在所述存储器11内的数据，以执行电子设备1的各种功能和处理数据。
所述总线可以是外设部件互连标准(peripheral component interconnect，简称PCI)总线或扩展工业标准结构(extended industry standard architecture，简称EISA)总线等。该总线可以分为地址总线、数据总线、控制总线等。所述总线被设置为实现所述存储器11以及至少一个处理器10等之间的连接通信。
图3仅示出了具有部件的电子设备，本领域技术人员可以理解的是，图3示出的结构并不构成对所述电子设备1的限定，可以包括比图示更少或者更多的部件，或者组合某些部件，或者不同的部件布置。
例如，尽管未示出，所述电子设备1还可以包括给各个部件供电的电源(比如电池)，优选地，电源可以通过电源管理装置与所述至少一个处理器10逻辑相连，从而通过电源管理装置实现充电管理、放电管理、以及功耗管理等功能。电源还可以包括一个或一个以上的直流或交流电源、再充电装置、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。所述电子设备1还可以包括多种传感器、蓝牙模块、Wi-Fi模块等，在此不再赘述。
进一步地，所述电子设备1还可以包括网络接口，可选地，所述网络接口可以包括有线接口和/或无线接口(如WI-FI接口、蓝牙接口等)，通常用于在该电子设备1与其他电子设备之间建立通信连接。
可选地，该电子设备1还可以包括用户接口，用户接口可以是显示器(Display)、输入单元(比如键盘(Keyboard))，可选地，用户接口还可以是标准的有线接口、无线接口。可选地，在一些实施例中，显示器可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED(Organic Light-Emitting Diode，有机发光二极管)触摸器等。其中，显示器也可以适当的称为显示屏或显示单元，用于显示在电子设备1中处理的信息以及用于显示可视化的用户界面。
应该了解，所述实施例仅为说明之用，在专利申请范围上并不受此结构的限制。
所述电子设备1中的所述存储器11存储的基于上下文的语音情感检测程序12是多个指令的组合，在所述处理器10中运行时，可以实现：
获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；
根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；
分别提取所述客服语音段与所述客户语音段的语音时刻；
选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；
利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；
将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
具体地，所述处理器10对上述指令的具体实现方法可参考图1对应实施例中相关步骤的描述，在此不赘述。
进一步地，所述电子设备1集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读存储介质中。所述计算机可读存储介质可以是易失性的，也可以是非易失性的。例如，所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)。
本申请还提供一种计算机可读存储介质，所述可读存储介质存储有计算机程序，所述计算机程序在被电子设备的处理器所执行时，可以实现：
获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；
根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；
分别提取所述客服语音段与所述客户语音段的语音时刻；
选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；
利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；
将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能模块的形式实现。
对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。
因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附关联图标记视为限制所涉及的权利要求。
本申请所指区块链是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链(Blockchain)，本质上是一个去中心化的数据库，是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了一批次网络交易的信息，用于验证其信息的有效性(防伪)和生成下一个区块。区块链可以包括区块链底层平台、平台产品服务层以及应用服务层等。
此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。系统权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第二等词语用来表示名称，而并不表示任何特定的顺序。
最后应说明的是，以上实施例仅用以说明本申请的技术方案而非限制，尽管参照较佳 实施例对本申请进行了详细说明，本领域的普通技术人员应当理解，可以对本申请的技术方案进行修改或等同替换，而不脱离本申请技术方案的精神和范围。
Claims (20)
- 一种基于上下文的语音情感检测方法，其中，所述方法包括：获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；分别提取所述客服语音段与所述客户语音段的语音时刻；选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求1所述的基于上下文的语音情感检测方法，其中，所述对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征，包括：对不同语音时刻下的输入语音流进行卷积处理，得到卷积语音流；对所述卷积语音流进行全局最大池化处理，得到池化语音流；对所述池化语音流进行全连接处理，得到全连接语音流；对所述全连接语音流进行全连接处理，得到所述输入语音流的声纹特征。
- 如权利要求1所述的基于上下文的语音情感检测方法，其中，所述根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段，包括：计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值；将所述输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段；将所述距离值大于或等于所述预设距离阈值的的语音段汇集为客户语音段。
- 如权利要求1所述的基于上下文的语音情感检测方法，其中，所述利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感，包括：检测所述第一语音段的语音时长和语音语调；持续检测所述第一语音段的语音强度，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量；对所述第一语音段进行语音内容识别，统计语音内容识别结果中用户的语音字数；根据所述语音时长和所述语音字数计算所述第一语音段的语音语速；根据所述语音语调、所述语音音量和所述语音语速计算客服情感值；根据所述客服情感值所在的数值区间确定客服情感。
- 如权利要求1至4中任一项所述的基于上下文的语音情感检测方法，其中，所述将所述客服情感作为参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感，包括：对所述客服情感进行参数转化，得到情感参数；利用所述情感参数对所述情感分析模型进行参数赋值，得到带有参数的情感分析模型；利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求5所述的基于上下文的语音情感检测方法，其中，所述对所述客服情感进行参数转化，得到情感参数，包括：对所述客服情感进行词向量数值转化，得到客服情感参数；获取所述情感分析模型的配置文件；解析所述配置文件得到配置项，并提取所述配置项中的配置参数；利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数；将所述赋值参数输入至所述情感分析模型，得到含参数的情感分析模型；利用所述含参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求6所述的基于上下文的语音情感检测方法，其中，所述利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数，包括：遍历所述客服情感参数并确定所述客服情感参数中的第一标识符；遍历所述配置参数并确定所述配置参数中的第二标识符；将所述第一标识符与所述第二标识符进行对比分析，得到对比分析结果；当所述对比分析结果为所述第一标识符与所述第二标识符不相同时，重新遍历所述配置参数并确定所述配置参数中的第二标识符；当所述对比分析结果为所述第一标识符与所述第二标识符相同时，利用所述第一标识符对应的客服情感参数对所述第二标识符对应的配置参数进行赋值，得到赋值参数。
- 一种基于上下文的语音情感检测装置，其中，所述装置包括：特征提取模块，用于获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；语音划分模块，用于根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；时刻提取模块，用于分别提取所述客服语音段与所述客户语音段的语音时刻；语音选取模块，用于选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；第一检测模块，用于利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；第二检测模块，用于将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 一种电子设备，其中，所述电子设备包括：至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如下步骤：获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；分别提取所述客服语音段与所述客户语音段的语音时刻；选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求9所述的电子设备，其中，所述对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征，包括：对不同语音时刻下的输入语音流进行卷积处理，得到卷积语音流；对所述卷积语音流进行全局最大池化处理，得到池化语音流；对所述池化语音流进行全连接处理，得到全连接语音流；对所述全连接语音流进行全连接处理，得到所述输入语音流的声纹特征。
- 如权利要求9所述的电子设备，其中，所述根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段，包括：计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值；将所述输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段；将所述距离值大于或等于所述预设距离阈值的的语音段汇集为客户语音段。
- 如权利要求9所述的电子设备，其中，所述利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感，包括：检测所述第一语音段的语音时长和语音语调；持续检测所述第一语音段的语音强度，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量；对所述第一语音段进行语音内容识别，统计语音内容识别结果中用户的语音字数；根据所述语音时长和所述语音字数计算所述第一语音段的语音语速；根据所述语音语调、所述语音音量和所述语音语速计算客服情感值；根据所述客服情感值所在的数值区间确定客服情感。
- 如权利要求9至12中任一项所述的电子设备，其中，所述将所述客服情感作为参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感，包括：对所述客服情感进行参数转化，得到情感参数；利用所述情感参数对所述情感分析模型进行参数赋值，得到带有参数的情感分析模型；利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求13所述的电子设备，其中，所述对所述客服情感进行参数转化，得到情感参数，包括：对所述客服情感进行词向量数值转化，得到客服情感参数；获取所述情感分析模型的配置文件；解析所述配置文件得到配置项，并提取所述配置项中的配置参数；利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数；将所述赋值参数输入至所述情感分析模型，得到含参数的情感分析模型；利用所述含参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求14所述的电子设备，其中，所述利用所述客服情感参数对所述配置参数进行赋值，得到赋值参数，包括：遍历所述客服情感参数并确定所述客服情感参数中的第一标识符；遍历所述配置参数并确定所述配置参数中的第二标识符；将所述第一标识符与所述第二标识符进行对比分析，得到对比分析结果；当所述对比分析结果为所述第一标识符与所述第二标识符不相同时，重新遍历所述配置参数并确定所述配置参数中的第二标识符；当所述对比分析结果为所述第一标识符与所述第二标识符相同时，利用所述第一标识符对应的客服情感参数对所述第二标识符对应的配置参数进行赋值，得到赋值参数。
- 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现如下步骤：获取输入语音流，对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征；根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段；分别提取所述客服语音段与所述客户语音段的语音时刻；选取预设的第一时刻下的所述客服语音段为第一语音段，选取预设的第二时刻下的所述客户语音段为第二语音段，其中，所述第二时刻在所述第一时刻之后，所述第一时刻和所述第二时刻均为提取到的所述语音时刻；利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感；将所述客服情感作为所述情感分析模型的参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感。
- 如权利要求16所述的计算机可读存储介质，其中，所述对所述输入语音流进行声纹特征提取，得到所述输入语音流的声纹特征，包括：对不同语音时刻下的输入语音流进行卷积处理，得到卷积语音流；对所述卷积语音流进行全局最大池化处理，得到池化语音流；对所述池化语音流进行全连接处理，得到全连接语音流；对所述全连接语音流进行全连接处理，得到所述输入语音流的声纹特征。
- 如权利要求16所述的计算机可读存储介质，其中，所述根据所述声纹特征将所述输入语音流划分为客服语音段与客户语音段，包括：计算所述输入语音流的所述声纹特征与标准客服声纹特征之间的距离值；将所述输入语音流中距离值小于预设距离阈值的语音段汇集为客服语音段；将所述距离值大于或等于所述预设距离阈值的的语音段汇集为客户语音段。
- 如权利要求16所述的计算机可读存储介质，其中，所述利用预先训练完成的情感分析模型对所述第一语音段进行情感检测，得到客服情感，包括：检测所述第一语音段的语音时长和语音语调；持续检测所述第一语音段的语音强度，根据所述语音时长和所述语音强度计算所述第一语音段的语音音量；对所述第一语音段进行语音内容识别，统计语音内容识别结果中用户的语音字数；根据所述语音时长和所述语音字数计算所述第一语音段的语音语速；根据所述语音语调、所述语音音量和所述语音语速计算客服情感值；根据所述客服情感值所在的数值区间确定客服情感。
- 如权利要求16至19中任一项所述的计算机可读存储介质，其中，所述将所述客服情感作为参数，使用所述情感分析模型对所述第二语音段进行情感检测，得到客户情感，包括：对所述客服情感进行参数转化，得到情感参数；利用所述情感参数对所述情感分析模型进行参数赋值，得到带有参数的情感分析模型；利用所述带有参数的情感分析模型对所述第二语音段进行情感检测，得到客户情感。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110214155.5 | 2021-02-26 | ||
| CN202110214155.5A CN112992187B (zh) | 2021-02-26 | 2021-02-26 | 基于上下文的语音情感检测方法、装置、设备及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022178933A1 true WO2022178933A1 (zh) | 2022-09-01 |
Family
ID=76350880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/082862 WO2022178933A1 (zh) | 2021-02-26 | 2021-03-25 | 基于上下文的语音情感检测方法、装置、设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112992187B (zh) |
| WO (1) | WO2022178933A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117711444A (zh) * | 2024-02-05 | 2024-03-15 | 新励成教育科技股份有限公司 | 一种基于口才表达的互动方法、装置、设备及存储介质 |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113903363B (zh) * | 2021-09-29 | 2023-02-28 | 平安银行股份有限公司 | 基于人工智能的违规行为检测方法、装置、设备及介质 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130311184A1 (en) * | 2012-05-18 | 2013-11-21 | Nilay Chokhoba Badavne | Method and system for speech recognition |
| CN109587360A (zh) * | 2018-11-12 | 2019-04-05 | 平安科技（深圳）有限公司 | 电子装置、应对话术推荐方法和计算机可读存储介质 |
| CN111028827A (zh) * | 2019-12-10 | 2020-04-17 | 深圳追一科技有限公司 | 基于情绪识别的交互处理方法、装置、设备和存储介质 |
| CN111341324A (zh) * | 2020-05-18 | 2020-06-26 | 浙江百应科技有限公司 | 一种基于fasttest模型的识别纠错及训练方法 |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103811009A (zh) * | 2014-03-13 | 2014-05-21 | 华东理工大学 | 一种基于语音分析的智能电话客服系统 |
| US10158758B2 (en) * | 2016-11-02 | 2018-12-18 | International Business Machines Corporation | System and method for monitoring and visualizing emotions in call center dialogs at call centers |
| CN107154257B (zh) * | 2017-04-18 | 2021-04-06 | 苏州工业职业技术学院 | 基于客户语音情感的客服服务质量评价方法及系统 |
| CN107452385A (zh) * | 2017-08-16 | 2017-12-08 | 北京世纪好未来教育科技有限公司 | 一种基于语音的数据评价方法及装置 |
| CN111048075A (zh) * | 2018-10-11 | 2020-04-21 | 上海智臻智能网络科技股份有限公司 | 智能客服系统及智能客服机器人 |
| CN109785862A (zh) * | 2019-01-21 | 2019-05-21 | 深圳壹账通智能科技有限公司 | 客服服务质量评价方法、装置、电子设备及存储介质 |
| CN111370030A (zh) * | 2020-04-03 | 2020-07-03 | 龙马智芯（珠海横琴）科技有限公司 | 语音情感检测方法与装置、存储介质、电子设备 |
-
2021
- 2021-02-26 CN CN202110214155.5A patent/CN112992187B/zh active Active
- 2021-03-25 WO PCT/CN2021/082862 patent/WO2022178933A1/zh active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130311184A1 (en) * | 2012-05-18 | 2013-11-21 | Nilay Chokhoba Badavne | Method and system for speech recognition |
| CN109587360A (zh) * | 2018-11-12 | 2019-04-05 | 平安科技（深圳）有限公司 | 电子装置、应对话术推荐方法和计算机可读存储介质 |
| CN111028827A (zh) * | 2019-12-10 | 2020-04-17 | 深圳追一科技有限公司 | 基于情绪识别的交互处理方法、装置、设备和存储介质 |
| CN111341324A (zh) * | 2020-05-18 | 2020-06-26 | 浙江百应科技有限公司 | 一种基于fasttest模型的识别纠错及训练方法 |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117711444A (zh) * | 2024-02-05 | 2024-03-15 | 新励成教育科技股份有限公司 | 一种基于口才表达的互动方法、装置、设备及存储介质 |
| CN117711444B (zh) * | 2024-02-05 | 2024-04-23 | 新励成教育科技股份有限公司 | 一种基于口才表达的互动方法、装置、设备及存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112992187A (zh) | 2021-06-18 |
| CN112992187B (zh) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022116420A1 (zh) | 语音事件检测方法、装置、电子设备及计算机存储介质 | |
| WO2022160449A1 (zh) | 文本分类方法、装置、电子设备及存储介质 | |
| WO2022121171A1 (zh) | 相似文本匹配方法、装置、电子设备及计算机存储介质 | |
| JP2021533397A (ja) | 話者埋め込みと訓練された生成モデルとを使用する話者ダイアライゼーション | |
| CN112560453B (zh) | 语音信息校验方法、装置、电子设备及介质 | |
| CN112527994A (zh) | 情绪分析方法、装置、设备及可读存储介质 | |
| WO2022178933A1 (zh) | 基于上下文的语音情感检测方法、装置、设备及存储介质 | |
| WO2020082734A1 (zh) | 文本情感识别方法、装置、电子设备及计算机非易失性可读存储介质 | |
| JPWO2012147428A1 (ja) | テキストクラスタリング装置、テキストクラスタリング方法、およびプログラム | |
| CN111177350A (zh) | 智能语音机器人的话术形成方法、装置和系统 | |
| CN112667805B (zh) | 一种工单类别确定方法、装置、设备及介质 | |
| CN112579621B (zh) | 数据展示方法、装置、电子设备及计算机存储介质 | |
| WO2022194062A1 (zh) | 疾病标签检测方法、装置、电子设备及存储介质 | |
| JP2022120024A (ja) | オーディオ信号処理方法、モデルトレーニング方法、並びにそれらの装置、電子機器、記憶媒体及びコンピュータプログラム | |
| CN113342955A (zh) | 一种问答语句的处理方法、装置及电子设备 | |
| CN112951233A (zh) | 语音问答方法、装置、电子设备及可读存储介质 | |
| CN113282854A (zh) | 数据请求响应方法、装置、电子设备及存储介质 | |
| WO2021208700A1 (zh) | 语音数据选择方法、装置、电子设备及存储介质 | |
| CN113205814B (zh) | 语音数据标注方法、装置、电子设备及存储介质 | |
| CN112466337A (zh) | 音频数据情绪检测方法、装置、电子设备及存储介质 | |
| CN116450797A (zh) | 基于多模态对话的情感分类方法、装置、设备及介质 | |
| CN116340831A (zh) | 一种信息分类方法、装置、电子设备及存储介质 | |
| WO2022141867A1 (zh) | 语音识别方法、装置、电子设备及可读存储介质 | |
| WO2022141838A1 (zh) | 模型置信度分析方法、装置、电子设备及计算机存储介质 | |
| CN111859985B (zh) | Ai客服模型测试方法、装置、电子设备及存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21927367 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21927367 Country of ref document: EP Kind code of ref document: A1 |