KR20200113432A - Electronic apparatus for multiscale speech emotion recognization and operating method thereof - Google Patents

Abstract

An electronic device and operation method thereof according to various embodiments of the present invention are for multiscale speech emotion recognition. The electronic device may be configured to compress a speech signal into a plurality of scales, assign a weight according to attention to each of the scales, extract speech features of the speech signal based on the weight from the scales, and recognize the emotion of the voice signal based on the speech features.

Description

Electronic device for multi-scale voice emotion recognition and its operation method {ELECTRONIC APPARATUS FOR MULTISCALE SPEECH EMOTION RECOGNIZATION AND OPERATING METHOD THEREOF}

Various embodiments relate to an electronic device for multi-scale voice emotion recognition and a method of operating the same.

In general, the talker expresses the talker's emotions through voice signals. At this time, emotions are not expressed entirely in the voice signal, but are expressed concentrated in a partial area of the voice signal. That is, voice features related to emotions are concentrated in some areas of the voice signal. Accordingly, technology for recognizing emotions from voice signals is being developed.

There is a need for a way to more accurately recognize emotions from voice signals. Various embodiments provide an electronic device for recognizing emotion of a voice signal by converting a voice signal into multiscale, and a method of operating the same.

An operation method of an electronic device according to various embodiments is for multi-scale speech emotion recognition, an operation of compressing a speech signal into a plurality of scales, and a weight according to attention corresponding to each of the scales. An operation of assigning, an operation of extracting a speech characteristic of the speech signal based on the weight from the scales, and an operation of recognizing an emotion of the speech signal based on the speech characteristic.

The electronic device according to various embodiments is for multi-scale voice emotion recognition, and may include an input module for inputting a voice signal, and a processor connected to the input module and configured to recognize the emotion of the voice signal. . According to various embodiments, the processor compresses the speech signal into a plurality of scales, assigns a weight according to attention to each of the scales, and from the scales, based on the weight. It may be configured to extract a voice feature of the voice signal and recognize the emotion based on the voice feature.

According to various embodiments, the electronic device may recognize the emotion of the voice signal by converting the voice signal into multiscale. In this case, as the electronic device compresses the voice signal into a plurality of scales, it may generate compressed signals of various sizes in time. Through this, the electronic device can recognize the emotion of the voice signal by considering a frame change in a relatively long compressed signal and a frame change in a relatively short compressed signal together. In addition, the electronic device may efficiently detect at least one of the frames of the voice signal in which emotion is concentrated. Accordingly, the electronic device can more accurately recognize the emotion from the voice signal.

1 is a diagram for explaining general voice sensibility recognition.
2 is a diagram illustrating an electronic device according to various embodiments.
3 is a diagram illustrating the processor of FIG. 2.
FIG. 4 is a diagram illustrating the multiscale module of FIG. 3.
5 is a diagram illustrating a method of operating an electronic device according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings.

1 is a diagram for explaining general voice sensibility recognition.

Referring to FIG. 1, voice features related to emotion may be extracted from a voice signal. For example, the voice feature may be extracted from the entire section of the voice signal. Here, the voice characteristic may represent a prosodic flow of a voice signal, for example, a change in intonation and stress. As another example, the voice feature may be extracted from unit sections of the voice signal. Here, the unit sections are determined at predetermined time intervals, and may be distinguished from all sections of the voice signal. Here, the voice characteristic may represent at least one of an excitation source or a vocal tract in each of the unit sections.

According to various embodiments, a technique for recognizing emotion of a voice signal by converting a voice signal into multi-scale may be provided. Here, the voice signal may be divided into a plurality of frames in the time domain. In addition, as the audio signal is compressed into a plurality of scales, compressed signals having various sizes in time may be generated. Through this, the emotion of the speech signal can be recognized by considering a frame change in a relatively long compressed signal and a frame change in a relatively short compressed signal together.

2 is a diagram illustrating an electronic device 200 according to various embodiments.

Referring to FIG. 2, an electronic device 200 according to various embodiments may include at least one of an input module 210, an output module 220, a memory 230, or a processor 240.

The input module 210 may receive commands or data to be used for components of the electronic device 200 from outside the electronic device 200. The input module 210 includes at least one of an input device configured to directly input a command or data to the electronic device 200 or a communication device configured to receive commands or data by communicating with an external electronic device wired or wirelessly. It can contain either. For example, the input device may include at least one of a microphone, a mouse, a keyboard, and a camera. For example, the communication device may include at least one of a wired communication device or a wireless communication device, and the wireless communication device may include at least one of a short-range communication device and a long-distance communication device.

The output module 220 may provide information to the outside of the electronic device 200. The output module 220 is at least one of an audio output device configured to audibly output information, a display device configured to visually output information, or a communication device configured to transmit information by wired or wireless communication with an external electronic device. It can contain either. For example, the communication device may include at least one of a wired communication device or a wireless communication device, and the wireless communication device may include at least one of a short-range communication device and a long-distance communication device.

The memory 230 may store data used by components of the electronic device 200. The data may include input data or output data for a program or a command related thereto. For example, the memory 230 may include at least one of a volatile memory or a nonvolatile memory.

The processor 240 may execute a program in the memory 230 to control components of the electronic device 200 and perform data processing or operation. At this time, the processor 240 may convert the voice signal into multi-scale and recognize the emotion of the voice signal. The processor 240 may compress the voice signal into a plurality of scales. The processor 240 may assign a weight according to attention to each of the scales. The processor 240 may extract the speech feature of the speech signal based on the weight from the scales. Through this, the processor 240 may recognize the emotion of the voice signal based on the voice characteristic.

FIG. 3 is a diagram illustrating the processor 240 of FIG. 2. FIG. 4 is a diagram illustrating the multiscale module 310 of FIG. 3.

3, the processor 240 according to various embodiments includes a preprocessing module 300, a multiscale module 310, an attention pooling module 320, and a combination module ( A concatenate module) 330 and an emotion recognition module 360 may be included.

The preprocessing module 300 may perform preprocessing on the voice signal. In this case, the voice signal is based on speech, and the preprocessing module 300 may detect the voice signal input through the input module 210. For example, the preprocessing module 300 may detect a voice signal directly input through an input device. As another example, the preprocessing module 300 may detect a voice signal received from an external electronic device through a communication device. For example, the pre-processing module 300 may perform at least one of denoising, pre-emphasis, and voice activity detection (VAD) on the voice signal. And the preprocessing module 300 converts the speech signal to any one of log scaled mel-spectrogram, mel-spectrogram, pitch, energy, MFCC (mel frequency cepstral coefficient) or LPCC (linear predictive cepstral coefficient), and then zero mean unit variance. Can be normalized with

The multi-scale module 310 may convert a voice signal into multi-scale. The multi-scale module 310 may compress the voice signal into a plurality of scales. Here, the multi-scale module 310 may divide the voice signal into a plurality of frames in a time domain, and compress the frames to reduce the size of the frames over time.

The multi-scale module 310 may include a plurality, that is, N scale modules 311, 313, .... Scales of the scale modules 311, 313,… may be different. Through this, the scale modules 311, 313, ... may generate a plurality of compressed signals by compressing the voice signal at different ratios. For example, the scale modules 311, 313, ... may compress the speech signal by applying a convolutional neural network (CNN) and pooling to the speech signal. Here, the pooling may include at least one of max pooling and average pooling. As an example, the scale modules 311, 313,… may compress a speech signal at different rates according to the number of applications of the convolutional neural network and pooling. The scale modules 311, 313, ... may be connected to the attention pulling module 320, respectively. According to an embodiment, the scale modules 311, 313, ... may be connected in series.

According to an exemplary embodiment, the multi-scale module 310 may include a first scale module 311 and a second scale module 313 connected to the first scale module 313. Similarly, at least one additional scale module (not shown) connected in series from the second scale module 313 may be further included. The first scale module 311 may generate a first compressed signal by compressing an audio signal with a first scale. In addition, the first scale module 311 may transmit the first compressed signal to the attention pulling module 320 and the second scale module 313. The second scale module 313 may generate a second compressed signal by compressing an audio signal with a second scale. In this case, the second scale module 313 may generate a second compressed signal by compressing the first compressed signal transmitted from the first scale module 311. In addition, the second scale module 313 may transmit the second compressed signal to the attention pooling module 320. At this time, if an additional scale module (not shown) is connected to the second schedule module 313, the second scale module 313 may transmit the second compression signal to the additional scale module (not shown).

For example, as shown in FIG. 4, the first scale module 311 includes a first convolutional neural network module 411 and a first pooling module 412, and the second scale module 313, As shown in FIG. 4, a second convolutional neural network module 413 and a second pooling module 414 may be included. Similarly, if at least one additional scale module (not shown) is connected in series from the second scale module 313, the additional scale module also includes the first scale module 311 and the second scale module 313 It can be configured similarly.

The voice signal input from the input module 210 may be converted into a first compressed signal while passing through the first convolutional neural network module 411 and the first pulling module 412 of the first scale module 311. Here, the size of the frames of the voice signal may be reduced to 1/2, for example. In some embodiments, the voice signal returns to the first convolutional neural network module 411 after passing through the first convolutional neural network module 411 and the first pooling module 412, and the first convolutional neural network module 411 ) And the first pulling module 412 may be converted into a first compressed signal. Here, the size of the frames of the voice signal may be reduced to one of 1/4, 1/8, or 1/16, for example. In addition, the first compressed signal may be transmitted from the first pooling module 412 to the attention pooling module 320 and the second convolutional neural network module 413 of the second scale module 313.

The first compressed signal may be converted into a second compressed signal while passing through the second convolutional neural network module 413 and the second pooling module 414 of the second scale module 313. Here, the size of the frames of the first compressed signal may be reduced to 1/2, for example. In other words, the size of the frames of the audio signal can be reduced to, for example, 1/4. In some embodiments, the first compressed signal returns to the second convolutional neural network module 413 after passing through the second convolutional neural network module 413 and the second pooling module 414, and the second convolutional neural network module It may be converted into a second compressed signal while repeatedly passing through 413 and the second pulling module 414. Here, the size of the frames of the first compressed signal may be reduced to one of 1/4, 1/8, and 1/16, for example. In addition, the second compressed signal may be transmitted from the second pooling module 414 to the attention pooling module 320. At this time, if an additional scale module (not shown) is connected to the second schedule module 313, the second compressed signal may be transmitted to an additional scale module (not shown).

The attention pooling module 320 may assign a weight according to attention to each of the scales. At this time, the attention pooling module 320 may calculate the attention of each frame in response to each of the compressed signals. In addition, the attention pulling module 320 may assign a weight according to attention to each frame in response to each of the compressed signals. The attention pooling module 320 includes a plurality, that is, N neural network modules 321, 323, ..., a plurality, that is, N attention modules 331, 333, ..., and a plurality, that is, N weights Modules 341, 343, ... may be included.

The neural network modules 321 and 323 may respectively process compressed signals input from the scale modules 311, 313,… of the multiscale module 310 according to a predetermined neural network structure. For example, the neural network modules 321, 323, ... may process compressed signals according to a long-short term memory (LSTM) cyclic neural network structure. To this end, the neural network modules 321, 323,… may be connected to the scale modules 311, 313, …, respectively.

According to an embodiment, the neural network modules 321, 323, ... are a first neural network module 321 connected to the first scale module 311 and a second neural network module 323 connected to the second scale module 313 ) Can be included. Similarly, when an additional scale module (not shown) is connected to the second scale module 313, the neural network modules 321, 323,… are additional neural network modules (not shown) connected to the additional scale module (not shown). ) May be further included. The first neural network module 321 may process a first compressed signal input from the first scale module 311. The second neural network module 323 may process a second compressed signal input from the second scale module 313.

Attention modules 331, 333,… may calculate attentional concentration in response to each of the compressed signals. At this time, the attention modules 331, 333,… may calculate the attention of each frame. Here, the attention modules 331, 333,… may calculate attention to the voice feature in each of the frames. To this end, the attention modules 331, 333, ... may be connected to the neural network modules 321, 323, ..., respectively.

According to an embodiment, the attention modules 331, 333, ... are a first attention module 331 connected to the first neural network module 321 and a second attention module connected to the second neural network module 323 A module 333 may be included. Similarly, if the neural network modules 321, 323,… further include an additional neural network module (not shown) connected to an additional scale module (not shown), the attention modules 331, 333,… are additional neural network modules. An additional attention module (not shown) connected to (not shown) may be further included. The first attention module 331 may calculate the attention of each frame in response to the first compressed signal. The second attention module 333 may calculate attention of each frame in response to the second compressed signal.

The weighting modules 341, 343, ... may assign weights according to attention, corresponding to each of the compressed signals. In this case, the weight modules 341, 343, ... may assign a weight according to attention to each of the frames. To this end, the weight modules 341, 343,… may be connected to the neural network modules 32, 323,… and the attention modules 331, 333, …, respectively.

According to an embodiment, the weight modules 341, 343, ... are a first weight module 341 and a second neural network module 323 connected to the first neural network module 321 and the first attention module 331 And a second weighting module 343 connected to the second attention module 333. Similarly, if the neural network modules 321, 323,… further include an additional neural network module (not shown), and the attention module 331, 333,… further includes an additional attention module (not shown), The weighting modules 341 and 343 may further include an additional neural network module (not shown) and an additional weighting module (not shown) connected to the additional attention module (not shown). The first weighting module 341 may assign a weight according to attention to each of the frames in response to the first compressed signal. The second weight module 343 may assign a weight according to attention to each of the frames in response to the second compressed signal.

The combining module 350 may extract speech features of the speech signal based on weights from the scales. To this end, the combining module 350 may derive a final result by combining results related to all scales input from the attention pooling module 320. At this time, the combining module 350 may compare frames for all compressed signals. In addition, the combining module 350 may extract a speech feature to which a weight exceeding a predetermined value is assigned from the frames.

The emotion recognition module 360 may recognize the emotion of the voice signal based on the voice characteristic. In this case, the emotion recognition module 360 may select any one of a plurality of emotion labels determined to classify emotions based on voice characteristics. For example, emotion labels may include at least one of anger, disgust, fear, happy, neutral, sad, or surprise.

The electronic device 200 according to various embodiments is for multi-scale voice emotion recognition, is connected to the input module 210 for inputting a voice signal and the input module 210, and is configured to recognize the emotion of the voice signal. The processor 240 may be included.

According to various embodiments, the processor 240 compresses the speech signal into a plurality of scales, assigns a weight according to attention to each of the scales, and based on the weight from the scales. It may be configured to extract a voice feature of a voice signal and recognize an emotion based on the voice feature.

According to various embodiments, the processor 240 may be configured to divide an audio signal into a plurality of frames.

According to various embodiments, the processor 240 includes an attention module 331, 333,… configured to calculate attention of each frame, and a weight according to attention to each of the frames. Weight modules 341, 343, ... may be included.

According to various embodiments, the processor 240 may include a multi-scale module 310 configured to generate a plurality of compressed signals by compressing a voice signal into scales.

According to various embodiments, the multi-scale module 310 compresses the voice signal to generate a first compressed signal of a first scale, and compresses the first compressed signal. , It may include a second scale module 313 configured to generate a second compressed signal of the second scale.

According to various embodiments, the multi-scale module 310 is configured to compress a voice signal by applying a convolutional neural network and pooling, and the pooling may include at least one of max pooling and average pooling.

According to various embodiments, the compressed signals may be classified according to the number of applications of a convolutional neural network and pooling to a speech signal.

The electronic device 200 according to various embodiments is for multi-scale voice emotion recognition, an input module 210 for inputting a voice signal, and a multi-scale module 310 configured to compress the voice signal into a plurality of scales. , In response to each of the scales, the attention pooling module (331, 333, …) configured to assign a weight according to attention, configured to extract the speech feature of the speech signal based on the weight from the scales It may include a combination module 350 that is configured, and an emotion recognition module 360 configured to recognize emotions based on voice characteristics.

According to various embodiments, the multi-scale module 310 compresses the voice signal to generate a first compressed signal of a first scale, and compresses the first compressed signal. , It may include a second scale module 313 configured to generate a second compressed signal of the second scale.

5 is a diagram illustrating a method of operating an electronic device 200 according to various embodiments.

Referring to FIG. 5, the electronic device 200 may detect a voice signal in operation 510. The processor 240 may detect a voice signal input through the input module 210. For example, the processor 240 may detect a voice signal directly input through an input device. As another example, the processor 240 may detect a voice signal received from an external electronic device through a communication device. Here, the preprocessing module 300 may perform preprocessing on the voice signal. For example, the pre-processing module 300 may perform at least one of denoising, pre-emphasis, and voice activity detection (VAD) on the voice signal. And the preprocessing module 300 converts the speech signal to any one of log scaled mel-spectrogram, mel-spectrogram, pitch, energy, MFCC (mel frequency cepstral coefficient) or LPCC (linear predictive cepstral coefficient), and then zero mean unit Can be normalized with variance.

The electronic device 200 may compress the voice signal with a plurality of scales in operation 520. The processor 240 may compress the voice signal in a time domain. In this case, the multi-scale module 310 may divide the frames into a plurality of frames and compress the frames to reduce the size of the frames over time. The multiscale module 310 may generate a plurality of compressed signals by compressing the voice signal at different ratios. For example, the multiscale module 310 may compress the speech signal by applying a convolutional neural network (CNN) and pooling to the speech signal. Here, the pooling may include at least one of max pooling and average pooling. As an example, the multiscale module 310 may compress a speech signal at different rates according to the number of times the convolutional neural network and pooling are applied.

In operation 530, the electronic device 200 may calculate attentional concentration corresponding to each scale. In this case, the processor 240 may process each of the compressed signals according to a predetermined neural network structure. In addition, the processor 240 may calculate the attention of each frame in response to each of the compressed signals. Here, in response to each of the compressed signals, attention to a speech characteristic may be calculated in each of the frames.

In operation 540, the electronic device 200 may assign a weight according to attention, corresponding to each scale. The processor 240 may assign a weight according to attention to each frame in response to each compressed signal.

The electronic device 200 may extract a speech feature based on a weight from all scales in operation 550. The processor 240 may combine results related to all scales to derive a final result. In this case, the processor 240 may compare frames for all compressed signals. Further, the processor 240 may extract a speech feature to which a weight exceeding a predetermined value is assigned from the frames.

The electronic device 200 may recognize the emotion of the voice signal based on the voice characteristic in operation 560. In this case, the processor 240 may select any one of a plurality of emotion labels determined to classify emotions based on voice characteristics. For example, emotion labels may include at least one of anger, disgust, fear, happy, neutral, sad, or surprise. Through this, the electronic device 200 may store the emotion recognition result of the voice signal in the memory 230 or output it through the output module 220.

An operation method of the electronic device 200 according to various embodiments is for multi-scale voice emotion recognition, an operation of compressing a voice signal into a plurality of scales, corresponding to each of the scales, according to attention. An operation of assigning a weight, an operation of extracting a speech characteristic of the speech signal based on the weight from the scales, and an operation of recognizing an emotion of the speech signal based on the speech characteristic.

According to various embodiments, the voice signal may be divided into a plurality of frames.

According to various embodiments, the assigning operation may include an operation of calculating attentional concentration of each of the frames, and an operation of assigning a weight according to the attentional concentration to each of the frames.

According to various embodiments, the compression operation includes an operation of compressing a speech signal to generate a first compressed signal of a first scale, and compressing the first compressed signal to generate a second compressed signal of a second scale. May include actions.

According to various embodiments, the compression operation is configured to compress a speech signal by applying a convolutional neural network and pooling, and the pooling may include at least one of max pooling and average pooling.

According to various embodiments, the compressed signals may be classified according to the number of applications of a convolutional neural network and pooling to a speech signal.

According to various embodiments, the electronic device 200 may recognize the emotion of the voice signal by converting the voice signal into multiscale. In this case, as the electronic device 200 compresses the voice signal into a plurality of scales, the electronic device 200 may temporally generate compressed signals of various sizes. Through this, the electronic device 200 may recognize the emotion of the voice signal by considering a frame change in a relatively long compressed signal and a frame change in a relatively short compressed signal together. In addition, the electronic device 200 may efficiently detect at least one of the frames of the voice signal in which emotion is concentrated. Accordingly, the electronic device 200 may more accurately recognize the emotion from the voice signal.

Various embodiments of the present document and terms used therein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various modifications, equivalents, and/or substitutes of the corresponding embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar elements. Singular expressions may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and/or B", "A, B or C" or "at least one of A, B and/or C" are all of the items listed together. It can include possible combinations. Expressions such as "first", "second", "first" or "second" can modify the corresponding elements regardless of their order or importance, and are only used to distinguish one element from another. The components are not limited. When it is mentioned that a certain (eg, first) component is “(functionally or communicatively) connected” or “connected” to another (eg, second) component, the certain component is It may be directly connected to the component, or may be connected through another component (eg, a third component).

The term "module" used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic blocks, parts, or circuits. A module may be an integrally configured component or a minimum unit or a part of one or more functions. For example, the module may be configured as an application-specific integrated circuit (ASIC).

Various embodiments of this document are implemented as software including one or more instructions stored in a storage medium (eg, memory 230) readable by a machine (eg, electronic device 200). Can be. For example, the processor of the device (for example, the processor 240) may call at least one instruction among one or more instructions stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here,'non-transient' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves). It does not distinguish between temporary storage cases.

According to various embodiments, each component (eg, a module or program) of the described components may include a singular number or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar to that performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repeatedly, or heuristically executed, or one or more of the operations may be executed in a different order, or omitted. , Or one or more other actions may be added.

Claims (15)

In the method of operating an electronic device for multi-scale voice emotion recognition,
Compressing the speech signal into a plurality of scales;
Assigning a weight according to attention to each of the scales;
Extracting a speech feature of the speech signal based on the weight from the scales; And
And recognizing an emotion of the voice signal based on the voice characteristic.
The method of claim 1,
The voice signal is divided into a plurality of frames.
The method of claim 2, wherein the imparting operation,
Calculating the attention of each of the frames; And
And assigning the weight according to the attention to each of the frames.
The method of claim 1, wherein the compression operation,
Compressing the audio signal to generate a first compressed signal of a first scale; And
And compressing the first compressed signal to generate a second compressed signal of a second scale.
The method of claim 1, wherein the compression operation,
Compressing the speech signal by applying a convolutional neural network and pooling,
The pooling includes at least one of max pooling and average pooling.
The method of claim 5,
The compressed signals are classified according to the number of applications of the convolutional neural network and pooling to the speech signal.
In the electronic device for multi-scale voice emotion recognition,
An input module for inputting a voice signal; And
And a processor connected to the input module and configured to recognize emotion of the voice signal,
The processor,
Compressing the speech signal into a plurality of scales,
In response to each of the scales, a weight according to attention is assigned,
From the scales, extracting a speech feature of the speech signal based on the weight,
An electronic device configured to recognize the emotion based on the voice characteristic.
The method of claim 7, wherein the processor,
An electronic device configured to divide the voice signal into a plurality of frames.
The method of claim 8, wherein the processor,
An attention module, configured to calculate attention of each of the frames; And
And a weighting module configured to assign the weight according to the attention to each of the frames.
The method of claim 7, wherein the processor,
And a multiscale module configured to compress the voice signal into the scales to generate a plurality of compressed signals.
The method of claim 10, wherein the multi-scale module,
A first scale module, configured to compress the speech signal to generate a first compressed signal of a first scale; And
And a second scale module configured to compress the first compressed signal to generate a second compressed signal of a second scale.
The method of claim 10, wherein the multi-scale module,
It is configured to compress the speech signal by applying a convolutional neural network and pooling,
The pooling includes at least one of max pooling and average pooling.
The method of claim 12,
The compressed signals are classified according to the number of times the convolutional neural network and pooling are applied to the speech signal.
In the electronic device for multi-scale voice emotion recognition,
An input module for inputting a voice signal;
A multiscale module configured to compress the speech signal into a plurality of scales;
An attention pooling module configured to assign a weight according to attention to each of the scales;
A combining module, configured to extract a speech feature of the speech signal based on the weight from the scales; And
An electronic device comprising an emotion recognition module configured to recognize the emotion based on the voice characteristic.
The method of claim 14, wherein the multi-scale module,
A first scale module, configured to compress the speech signal to generate a first compressed signal of a first scale; And
And a second scale module configured to compress the first compressed signal to generate a second compressed signal of a second scale.

Images