CN111739544B - Voice processing method, device, electronic equipment and storage medium - Google Patents

Abstract

The present disclosure provides a voice processing method, device, electronic equipment and computer-readable storage medium, relating to the technical field of audio processing. The voice processing method includes: receiving a voice signal acquired and sent by an audio collection device; The time-domain signal corresponding to the signal undergoes pitch-shifting processing for adjusting the sampling frequency to obtain a pitch-modified speech signal; the time-domain signal corresponding to the pitch-modified speech signal is maintained for playback time to obtain the target speech signal; wherein, the pitch-modified speech signal The playback time of the subsequent voice signal is the same as the playback time of the voice signal. The present disclosure enables quick and accurate speech intonation modification.

Description

Speech processing method, device, electronic equipment and storage medium

Technical field

The present disclosure relates to the field of audio processing technology, and specifically, to a speech processing method, a speech processing device, an electronic device, and a computer-readable storage medium.

Background technique

In the audio processing process, audio pitch modification processing is a very important function. In related technologies, pitch modification methods mainly include the following: achieving pitch modification of speech audio by changing the sampling rate of playback; using a method combining linear predictive coding technology and differential glottal waves to synthesize pitch-modified speech; or using a method of calculating speech signals. Spectral envelope and pitch-shifting algorithms are used to change the pitch; or delay processing is performed through a delay factor to achieve a pitch-shifting effect.

In the above method, changing the playback sampling rate to achieve pitch change will affect the playback duration of the voice, which may also affect the sound quality of the voice. Moreover, the calculation amount is large and the rapid pitch change of the voice cannot be achieved.

It should be noted that the information disclosed in the above background section is only used to enhance understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to those of ordinary skill in the art.

Contents of the invention

The purpose of the present disclosure is to provide a speech processing method, device, electronic equipment and computer-readable storage medium, thereby overcoming, at least to a certain extent, the problem of being unable to quickly and accurately implement speech pitch changes due to limitations and defects in related technologies.

Additional features and advantages of the disclosure will be apparent from the following detailed description, or, in part, may be learned by practice of the disclosure.

According to one aspect of the present disclosure, a speech processing method is provided, including: receiving a speech signal acquired and sent by an audio collection device; performing pitch modification processing on the time domain signal corresponding to the speech signal for adjusting the sampling frequency to obtain the pitch modification The time domain signal corresponding to the pitch-modified speech signal is maintained for playback time to obtain the target speech signal; wherein the playback time of the pitch-modified speech signal is the same as the playback time of the speech signal.

In an exemplary embodiment of the present disclosure, performing a pitch modification process for adjusting the sampling frequency on the time domain signal corresponding to the speech signal, and obtaining the pitch-modified speech signal includes: modifying the time domain signal corresponding to the speech signal. Carry out frame division; perform windowing processing on the time domain signal corresponding to the framed speech signal to obtain the time domain signal corresponding to the windowed speech signal; perform window processing on the windowed speech signal according to an interpolation algorithm or an extraction algorithm. The corresponding time domain signal is processed to obtain the pitch-modified speech signal.

In an exemplary embodiment of the present disclosure, performing windowing processing on the framed time domain signal includes: using a Hamming window to perform the windowing processing on the time domain signal of the framed speech signal.

In an exemplary embodiment of the present disclosure, processing the time domain signal corresponding to the windowed speech signal according to an interpolation algorithm or an extraction algorithm to obtain the pitch-modified speech signal includes: according to the speech signal The sampling frequency of the signal, the sampling frequency of the pitch-modified speech signal, and the length of each frame of the speech signal determine the pitch-modified speech signal.

In an exemplary embodiment of the present disclosure, the rising pitch of the voice signal corresponds to an increase in the playback time of the pitch-modified voice signal, and the falling pitch of the voice signal corresponds to a decrease in the playback time of the pitch-modified voice signal.

In an exemplary embodiment of the present disclosure, maintaining the playback time of the time domain signal corresponding to the modulated speech signal to obtain the target speech signal includes: determining the time sequence variable and the difference between the two frame speech signals obtained by framing. Comparison results of overlapping lengths; combined with the comparison results, process the length of each frame of the voice signal after the tone change according to the length of each frame of voice signal, and compare the playback time of the voice signal after the tone change with the playback time of the voice signal. The target speech signal is determined at the same time.

In an exemplary embodiment of the present disclosure, combined with the comparison result, the length of each frame of the speech signal after the modulation is processed according to the length of each frame of the speech signal, and the playback time of the modulation of the speech signal is consistent with the length of the speech signal. Determining the target speech signal while the playback time of the speech signals is the same includes: if the timing variable is less than the overlap length, then based on the length of each frame of speech signal, the length of each frame of speech signal after the modulation and The overlap length determines the target speech signal; if the timing variable is greater than or equal to the overlap length, the pitch-modified speech signal is used as the target speech signal.

According to one aspect of the present disclosure, a voice processing device is provided, including: a voice acquisition module for receiving a voice signal acquired and sent by an audio acquisition device; a voice tone modification module for processing a time domain signal corresponding to the voice signal Performing a pitch-shifting process for adjusting the sampling frequency to obtain a pitch-modified speech signal; a time retention module for maintaining the playback time of the time domain signal corresponding to the pitch-modified speech signal to obtain a target speech signal; wherein, the pitch-modified speech signal The playback time of the subsequent voice signal is the same as the playback time of the voice signal.

According to one aspect of the present disclosure, an electronic device is provided, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform via executing the executable instructions. The speech processing method described in any one of the above.

According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above speech processing methods is implemented.

In the speech processing method, device, electronic device and computer-readable storage medium provided by this exemplary embodiment, on the one hand, the time domain signal corresponding to the speech signal sent to the audio processor is used to adjust the sampling frequency. Pitch transposition processing, because it modulates the time domain signal, avoids the problem of introducing harmonics and affecting the voice quality during the processing, and improves the audio quality and accuracy; on the other hand, by converting the time domain of the modulated speech signal The playback time of the signal is maintained to obtain the target speech signal, which avoids the impact on the playback time, so that the speech can be played normally and accurately; on the other hand, because only the time domain signal corresponding to the speech signal is modulated, it avoids The complex calculation process reduces the amount of calculation, improves calculation efficiency, and can quickly achieve voice tone changes.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the present disclosure.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

FIG. 1 schematically shows a schematic diagram of a speech processing method in an exemplary embodiment of the present disclosure.

FIG. 2 schematically illustrates a specific flowchart of pitch transposition processing in an exemplary embodiment of the present disclosure.

FIG. 3 schematically shows a flowchart of playback time maintenance in an exemplary embodiment of the present disclosure.

FIG. 4 schematically shows a block diagram of a speech processing device in an exemplary embodiment of the present disclosure.

FIG. 5 schematically illustrates a block diagram of a speech processing system in an exemplary embodiment of the present disclosure.

FIG. 6 schematically shows a schematic diagram of an electronic device in an exemplary embodiment of the present disclosure.

FIG. 7 schematically illustrates a computer-readable storage medium in an exemplary embodiment of the present disclosure.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concepts of the example embodiments. To those skilled in the art. The described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details described, or other methods, components, devices, steps, etc. may be adopted. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

Pitch-changing methods in related technologies include the following: changing the sampling rate of playback to achieve pitch-changing of voice audio. When the sampling rate is increased for playback, the playback speed of the voice will be accelerated, which will produce a rising pitch effect, but at the same time, the playback time will also be shortened. Yes, when the sampling rate is reduced for playback, the playback speed of the voice will slow down, which will produce the effect of tuning, but the playback time will also become longer. The interpolation method is used to achieve changes in pitch in the frequency domain. For example, if a pitch requires twice the frequency, some frequency components with half the energy of the original frequency point will be interpolated. Using interpolation in the frequency domain to achieve pitch changes will introduce harmonics and affect the sound quality. A method combining linear predictive coding technology and differential glottal waves is used to synthesize pitch-modified speech. The residual signal obtained by passing the speech signal through the inverse filter in the linear predictive coding technology is simulated in more detail using the differential glottal wave model. Obtain high-quality glottal excitation signals to synthesize high-quality pitch-modified speech; or use the cepstral sequence of the speech signal to derive the spectrum envelope, then use the spectrum envelope to separate the excitation component of the speech signal, and pass the excitation component through pitch-modification The algorithm changes its pitch; processes such as calculating the spectrum envelope require Fourier transform and inverse transform of the speech signal, which requires a large amount of calculation and is not suitable for running on a DSP.

In order to solve the above problem, in this exemplary embodiment, a voice processing method is first provided. The voice processing method can be applied to application scenarios of games or other application programs that can use voice interaction. Referring to FIG. 1 , the speech processing method in this exemplary embodiment will be described in detail.

In step S110, the voice signal acquired and sent by the audio collection device is received.

In this exemplary embodiment, the audio collection device may be a microphone on a terminal, and the terminal may be a smartphone, computer, smart watch, smart speaker, or other terminal capable of making calls. Here, a smartphone is used as an example for explanation. In addition, this exemplary embodiment can be applied to games or other applications, in application scenarios that require special processing of the collected voice in order to meet confidentiality or other requirements, that is, voice interaction or voice calls have Pitch-shifting sound effects.

In this exemplary embodiment, voice chat in a game is taken as an example for explanation. On the basis that the voice call has a pitch-changing sound effect, you can first determine whether the pitch-changing sound effect is turned on. This can be determined by judging the state of the control or button used to represent the pitch-changing sound effect. It can also be judged by other methods, which will not be detailed here. describe. If it is detected that the pitch-changing sound effect is turned on, the audio collection device (microphone) can collect the voice signal sent by the user in the mobile game. Further, the microphone can send the collected voice signal to a DSP (Digital Signal Processing, digital signal processor) in the mobile phone, so that the DSP can process the received voice signal.

In step S120, the time domain signal corresponding to the speech signal is subjected to a pitch modulation process for adjusting the sampling frequency to obtain a modulated speech signal.

In this exemplary embodiment, the voice signal may include a time domain signal and a frequency domain signal. Among them, the time domain signal describes the relationship between a mathematical function or a physical signal and time. The time domain waveform of a speech signal can express the change of the speech signal over time. Frequency domain signals refer to converting speech signals into coordinates expressed on the frequency axis. When converting from a time domain signal to a frequency domain signal, it needs to be implemented through Fourier series and Fourier transform.

The main functions of the pitch modification processing may include but are not limited to: performing pitch modification processing on the speech signal in the time domain, that is, processing the time domain signal corresponding to the speech signal to achieve pitch modification. Pitch change refers to raising the pitch of the speech signal (rising pitch) or lowering it (falling pitch). In addition, changes in the pitch of the speech signal can be correlated with the sampling frequency. For example, if the sampling frequency after pitch change increases, the pitch will rise; if the sampling frequency after pitch change decreases, the pitch will fall. Based on this, it can be considered that the pitch shifting process is used to adjust the sampling frequency. The sampling frequency defines the number of samples per second that are extracted from the continuous speech signal and formed into a discrete signal. The specific execution process of step S120 can be shown in Figure 2.

A flowchart of the pitch transposition process is schematically shown in FIG. 2 . Referring to what is shown in Figure 2, it mainly includes steps S210 to S230, where:

In step S210, the time domain signal corresponding to the speech signal is framed.

In this step, in order to maintain the stability of the speech signal and meet the requirements of signal processing, the speech signal can be divided into frames. Framing refers to dividing the speech signal into segments to analyze its characteristic parameters. Each segment is called a frame, and the frame length is generally 20 to 50ms. In this way, for the overall speech signal, what is analyzed is the feature parameter time series composed of the feature parameters of each frame. In this exemplary embodiment, the voice signal collected by the microphone can be represented by x(n). After the speech signal is framed, the length of each frame may be N, and the overlap length (frame shift) between the two frames to prevent discontinuity between the two frames may be W. n in x(n) represents a point in time series, which can be called a time series variable. n is an integer, and n=0,1,1,3,...N-1. The speech signal x(n) is framed, and the obtained framed speech signal can be expressed as x _m (n), where m represents the number of frames, which is the mth frame. The length N of each frame of the speech signal can be 512 , of course, it can also take other values, and there are no special restrictions here. It should be noted that in this step, the time domain signal of the speech signal is framed.

In step S220, windowing processing is performed on the time domain signal corresponding to the framed speech signal to obtain a time domain signal corresponding to the windowed speech signal.

In this step, the time domain signal of the speech signal is still processed. The purpose of windowing is to smooth the less continuous parts of the speech signal (the connection between the last point and the first point) and avoid obvious mutations. That is, windowing is used to smooth the edges of the frame signal. For windowing processing, it is to multiply the original integrand with a specific window function in Fourier integral. This result can achieve the effect of time-frequency localization. Windowing is generally a filter. The system function in the passband is not necessarily a constant value. Windowing is performed in the time domain. The frequency domain shape of the window function is a window. It filters out the components outside the band, which is equivalent to a low-pass filter. ; If it is a rectangular filter, it is equivalent to low-pass filtering, directly filtering out-of-band high-frequency components.

In this exemplary embodiment, when windowing is performed on the time domain signal of the framed speech signal, a Hamming window or a rectangular window may be used for processing. Here, the Hamming window is used as an example for explanation. The shape of the main part of the window function corresponding to the Hamming window is like the shape of sin(x) in the interval from 0 to pi, and the rest is 0. When such a function is multiplied by any other function, only part of it has non-zero values. The Hamming window can make certain corrections to the original speech signal sequence to obtain a better speech signal.

The Hamming window can be expressed specifically by formula (1):

Among them, n is an integer representing a point in the timing (timing parameter), and n=0,1,2,3...N-1.

Using the Hamming window in formula (1) to window the time domain signal of the framed speech signal, the windowed time domain signal as shown in formula (2) can be obtained:

Through steps S210 and S220, preprocessing operations such as framing and windowing are performed on the collected speech signals to eliminate aliasing, high-order harmonic distortion, and distortion caused by the human vocal organs themselves and the equipment for collecting speech signals. High frequency and other factors have an impact on the quality of speech signals; try to ensure that the signals obtained by subsequent speech processing are more uniform and smooth, provide high-quality parameters for signal parameter extraction, and improve the quality of speech processing.

In step S230, the time domain signal corresponding to the windowed speech signal is processed according to an interpolation algorithm or an extraction algorithm to obtain the pitch-modified speech signal.

In this step, the interpolation algorithm and the extraction algorithm are both pitch-shifting algorithms that change the pitch of the speech signal by adjusting the number of sampling points or the sampling frequency of the speech signal, and the pitch-shifting algorithms here are all executed on the time domain signal of the speech signal. . Specifically, the interpolation algorithm refers to inserting zero values (i.e. 0) where interpolation is required to form a new speech signal sequence. The interpolation algorithm may include, for example, but is not limited to linear function interpolation, cubic interpolation, etc., and the interpolation algorithm is used to increase the pitch, that is, to raise the pitch. The specific process may include, for example, zero-filling expansion and interpolation filtering of the speech signal. The extraction algorithm refers to extracting one of every few points in the speech signal to form a sequence of new speech signals in turn, and the purpose of the extraction algorithm is to lower the pitch, that is, lower the tone.

The time domain signal corresponding to the windowed speech signal is processed according to the interpolation algorithm or the extraction algorithm. The specific process of obtaining the pitch-modified speech signal includes: according to the sampling frequency of the speech signal, the pitch-modified speech signal The sampling frequency of the signal and the length of each frame of the speech signal determine the pitch-modified speech signal.

For example, if the sampling frequency of the speech signal before pitch modification is f and the sampling frequency of the speech signal after pitch modification is f ₀ , the speech signal after extraction processing or interpolation processing can be expressed by formula (3):

Among them, n=0,1,2...(N-1)×L+1, [] represents the rounding operation, and mod represents the remainder operation. Where M and L are both positive integers, and/> is the simplest fraction.

Further, after interpolation or extraction, the pitch-modified speech signal shown in formula (4) can be obtained:

y _m (n)＝z _m (Mn) (4)

Among them, n=0,1,2...N×L/M.

It can be seen that when f>f ₀ , M>L, the voice signal after the pitch change will rise; when f<f ₀ , M<L, the voice signal after the pitch change will fall.

In step S130, the playback time of the time domain signal corresponding to the modulated speech signal is maintained to obtain the target speech signal; wherein the playback time of the modulated speech signal is the same as the playback time of the speech signal.

In this exemplary embodiment, if the interpolation algorithm is used to raise the pitch of the speech signal, the playback time of the changed speech signal will increase; if the extraction algorithm is used to lower the pitch of the speech signal, the playback time of the changed speech signal will be reduce. In order to avoid the impact of the pitch change processing on the playback time, playback time maintenance processing can be performed on the voice signal after the pitch change. Playback time preservation refers to processing the time domain signal of the pitch-modified speech signal so that the playback time of the pitch-modified speech signal is the same as the playback time of the pitch-modified speech signal, which avoids pitch changes through sampling rate in related technologies. When the voice playback speed changes, the impact on the voice playback time is caused.

Further, with reference to the schematic flow chart of real-time playback time maintenance in Figure 3, as shown in Figure 3, the playback time maintenance is performed according to the time domain signal corresponding to the modulated speech signal to obtain the target speech signal including steps S310 and steps S320, where:

Step S310: Determine the comparison result between the time series variable and the overlap length between the two framed speech signals obtained by dividing the frame. Specifically, it refers to judging the size relationship between the time series variable n (i.e., a point on the time series) and the overlap length W between the two frames of speech signals obtained by dividing the frame. For example, when n=1,2...W-1, it can be determined that the timing variable n is smaller than the overlap length. When n=W, W+1...N, it can be determined that the timing variable n is greater than or equal to the overlap length.

Step S320, combined with the comparison result, process the length of each frame of the voice signal after the modulation according to the length of each frame of the voice signal, and determine when the playback time of the tone-modified voice signal is the same as the playback time of the voice signal. the target speech signal. That is to say, based on the size relationship between the timing variable n and the overlap length W, the playback time of the speech signal after the pitch change is changed to the playback time of the speech signal before the pitch change. Since there is a corresponding relationship between the playback time and the length of each frame of speech signal, that is, the length of each frame of speech signal is the same, it can be determined that the playback time of the speech signal is the same. Based on this, the pitch-modified speech signals can be spliced together to keep the length of the speech signals consistent. Further, when the length of each frame of the speech signal after the modulation is equal to the length of the original speech signal, that is, when the playback time of the speech signal after the modulation is the same as the playback time of the original speech signal, the speech signal can be determined as the target. voice signal.

Specifically, combined with the comparison results, the length of each frame of the speech signal after the modulation is processed according to the length of each frame of the speech signal, and when the playback time of the speech signal after the modulation is the same as the playback time of the speech signal Determining the target speech signal includes the following two situations: Situation 1. If the timing variable is less than the overlap length, then according to the length of each frame of speech signal, the length of each frame of speech signal after the modulation and the The overlap length determines the target speech signal. For example, assuming that the length of each frame of the speech signal before the pitch change is N, and the length of each frame of the speech signal of the signal y _m (n) after the pitch change becomes N/α, if you want to keep the playback time of the speech signal unchanged , then the length of each frame of the modulated speech signal needs to still be N. If the timing variable is less than the overlap length, it can be based on the overlap length between the two frames, the synthetic displacement (that is, the difference between the length of the speech signal of each frame and the overlap length), and the offset (the starting position of the overlap of the two frames) To determine the target speech signal according to formula (5).

Case 2: If the timing variable is greater than or equal to the overlap length, the pitch-modified speech signal is used as the target speech signal. If the timing variable is greater than or equal to the overlap length and does not exceed the length N of each frame of the speech signal before the change, then after length alignment, the pitch-shifted speech signal can be directly used as the final target speech signal. The target speech signal can be expressed by the formula ( 5) OK.

Among them, W is the overlap length of the two frames, s is the synthetic displacement and s=NW, and _km is the offset. The significance of the offset is: when performing playback time restoration and synthesis of the pitch-modified speech signal, there is overlap between frames, but it cannot be directly superimposed and synthesized, which will cause noise. In order to reduce this phenomenon, the starting position of the overlap of the two frames can be determined, and the starting position can be determined as the offset. Since the offset changes dynamically, when the definition formula (6) is satisfied, the noise can be minimized, and the offset can be as shown in formula (6):

Among them, the offset represents the distance between the optimal matching point and the m-th window.

In this exemplary embodiment, through steps S110 to S130, the time domain signal corresponding to the voice signal is interpolated and extracted while keeping the playback time unchanged, thereby achieving rapid pitch change of the voice signal and avoiding the need for playback. The effect of time. In addition, since the speech signal is interpolated in the time domain, problems affecting the sound quality caused by the introduction of harmonics are avoided, and the quality of the speech signal is improved. Furthermore, since the time domain signal of the speech signal is interpolated and the playback time is restored, there is no need to perform complex operations such as Fourier transformation and inverse transformation on the speech signal, which reduces the amount of calculation and makes the entire pitch modification process straightforward. Running in the DSP without occupying the CPU reduces latency and improves game performance and user experience.

In this exemplary embodiment, a speech processing device is also provided. Referring to Figure 4, the speech processing device 400 mainly includes: a speech acquisition module 401, a speech tone modification module 402 and a time retention module 403, wherein:

The voice acquisition module 401 can be used to receive voice signals acquired and sent by the audio acquisition device;

The speech pitch modification module 402 can be used to perform pitch modification processing on the time domain signal corresponding to the speech signal to adjust the sampling frequency to obtain a pitch-modified speech signal;

The time retention module 403 can be used to maintain the playback time of the time domain signal corresponding to the modulated speech signal to obtain the target speech signal; wherein the playback time of the modulated speech signal is equal to the playback time of the speech signal. same.

In an exemplary embodiment of the present disclosure, the speech tone modification module includes: a framing module, used to frame the time domain signal corresponding to the speech signal; a windowing module, used to frame the framed speech signal The corresponding time domain signal is subjected to windowing processing to obtain a time domain signal corresponding to the windowed speech signal; the pitch change control module is used to process the time domain signal corresponding to the windowed speech signal according to an interpolation algorithm or an extraction algorithm. Perform processing to obtain the pitch-modified speech signal.

In an exemplary embodiment of the present disclosure, the windowing module includes: a windowing control module, configured to use a Hamming window to perform the windowing process on the time domain signal of the framed speech signal.

In an exemplary embodiment of the present disclosure, the pitch change control module includes: a voice determination module, configured to determine the voice signal according to the sampling frequency of the voice signal, the sampling frequency of the voice signal after pitch change, and the length of each frame of voice signal. Pitch-modified speech signal.

In an exemplary embodiment of the present disclosure, the rising pitch of the voice signal corresponds to an increase in the playback time of the pitch-modified voice signal, and the falling pitch of the voice signal corresponds to a decrease in the playback time of the pitch-modified voice signal.

In an exemplary embodiment of the present disclosure, the time keeping module includes: a signal comparison module, used to determine the comparison result of the timing variable and the overlap length between the two frames of speech signals obtained by dividing the frame; a target speech determination module, using In combination with the comparison results, the length of each frame of the speech signal after the modulation is processed according to the length of each frame of the speech signal, and when the playback time of the modulation speech signal is the same as the playback time of the speech signal, the length of the speech signal is determined. target speech signal.

In an exemplary embodiment of the present disclosure, the target speech determination module includes: a first determination module, configured to: if the timing variable is less than the overlap length, according to the length of the speech signal of each frame, the pitch change The length of each frame of speech signal and the overlap length determine the target speech signal; the second determination module is used to use the modified speech signal as the target if the timing variable is greater than or equal to the overlap length. voice signal.

It should be noted that the specific details of each module in the above speech processing device have been described in detail in the corresponding speech processing method, so they will not be described again here.

In addition, a voice processing system is also provided. As shown in Figure 5, the voice processing system 50 mainly includes: a digital signal processor 51 and a central processing unit 52, wherein:

The digital signal processor 51 can be used to modulate the pitch of the speech signal, and maintain the playback time of the modulated speech signal to obtain the target speech signal. Referring to Figure 5, the digital signal processor 51 mainly includes the following modules: a pitch modification module 511, which is used to perform pitch modification processing on the time domain signal corresponding to the speech signal; and a playback time retention module 512, which is used to modify the pitch-modified speech signal. The playback time is maintained so that the playback time of the voice signal after the pitch change is the same as the playback time of the voice signal before the pitch change. Specifically, the pitch modulation module 511 mainly includes a framing module 5111 for framing, a windowing module 5112 for windowing, and a modulation control module 5113 for modulating.

The central processing unit 52 is used for running games or application programs.

In addition, the voice processing system 50 may also include an audio collection device 53 for collecting voice signals and sending the collected voice signals to the digital signal processor 51 .

In this way, the whole process can include: the game runs on the mobile phone's CPU. When the user turns on the pitch-changing sound effect of the voice call, the microphone first collects the voice signal and sends the collected voice signal to the DSP; then, the pitch-changing module responds to the voice signal The time domain signal is processed with rising or falling pitch; again, the playback time of the voice signal processed by the pitch change module will become longer or shorter, so the voice signal is transmitted to the playback time retention module to maintain the playback time before and after the pitch change. remains unchanged; further, the voice of the module that has passed the playback time is sent by the DSP to the game process running on the CPU. In this way, during game chat, the pitch of the voice signal changes, but the playback time of the voice signal does not change, enabling the voice pitch change effect to be quickly and accurately achieved. Since the algorithm for changing pitch and maintaining playback time can run on the DSP, it does not occupy the CPU, does not affect game performance and user experience, and can improve processing efficiency.

It should be noted that although several modules or units of equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into being embodied by multiple modules or units.

Furthermore, although various steps of the methods of the present disclosure are depicted in the drawings in a specific order, this does not require or imply that the steps must be performed in that specific order, or that all of the illustrated steps must be performed to achieve the desired results. result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, etc.

In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

Those skilled in the art will understand that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention can be implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "Circuit", "Module" or "System".

An electronic device 600 according to this embodiment of the invention is described below with reference to FIG. 6 . The electronic device 600 shown in FIG. 6 is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

As shown in Figure 6, electronic device 600 is embodied in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: the above-mentioned at least one processing unit 610, the above-mentioned at least one storage unit 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).

Wherein, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 performs various exemplary methods according to the present invention described in the above-mentioned "Example Method" section of this specification. Implementation steps. For example, the processing unit 610 may perform steps as shown in FIG. 1 .

The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and/or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

Storage unit 620 may also include a program/utility 6204 having a set of (at least one) program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples, or some combination, may include the implementation of a network environment.

Bus 630 may be a local area representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or using any of a variety of bus structures. bus.

The display unit 640 may be a display with a display function to display the processing results obtained by the processing unit 610 executing the method in this exemplary embodiment through the display. Displays include but are not limited to liquid crystal displays or other displays.

Electronic device 600 may also communicate with one or more external devices 800 (e.g., keyboard, pointing device, Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 600, and/or with Any device (eg, router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. This communication may occur through input/output (I/O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or can be implemented by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to cause a computing device (which may be a personal computer, a server, a terminal device, a network device, etc.) to execute a method according to an embodiment of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the method described above in this specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the The terminal device performs the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section above in this specification.

Referring to Figure 7, a program product 700 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be used on a terminal device, For example, run on a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus or device.

The program product may take the form of any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium that can send, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural Programming language—such as "C" or a similar programming language. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In situations involving remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as provided by an Internet service. (business comes via Internet connection).

Furthermore, the above-mentioned drawings are only schematic illustrations of processes included in methods according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal sequence of these processes. In addition, it is also easy to understand that these processes may be executed synchronously or asynchronously in multiple modules, for example.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (7)

1. A speech processing method, characterized by comprising: After turning on the pitch-changing sound effect, receive the voice signal acquired and sent by the audio collection device; The time domain signal corresponding to the speech signal is divided into frames, the time domain signal corresponding to the framed speech signal is windowed, and the time domain signal corresponding to the windowed speech signal is added according to an interpolation algorithm or an extraction algorithm. Perform pitch-shifting processing to adjust the sampling frequency to obtain the pitch-shifted speech signal; According to the corresponding relationship between the playback time and the length of each frame of speech signal, the pitch-modified speech signal is spliced so that the length of the speech signal remains consistent, and the time domain signal corresponding to the pitch-modified speech signal is maintained in playback time. The target speech signal is obtained when the playback time of the modulated speech signal is the same as the playback time of the speech signal; Wherein, maintaining the playback time of the time domain signal corresponding to the pitch-modified speech signal, and obtaining the target speech signal when the playback time of the pitch-modified speech signal is the same as the playback time of the speech signal includes: Determine the comparison result between the time series variable and the overlap length between the two frames of speech signals obtained by dividing the time domain signal corresponding to the speech signal into frames; If the timing variable is less than the overlap length, when the offset is such that the noise synthesized by the playback time reduction of the modulated speech signal is minimized, according to the offset, the length of each frame of the speech signal and the overlap length, The difference between and the overlap length between the two frames of speech signals after the pitch change determines the target speech signal; the offset represents the distance between the optimal matching point and the m-th window; If the timing variable is greater than or equal to the overlap length and does not exceed the length of each frame of the speech signal before the pitch change, then after length alignment, the speech signal after the pitch change is used as the target speech signal. 2. The speech processing method according to claim 1, characterized in that windowing the framed time domain signal includes: A Hamming window is used to perform the windowing process on the time domain signal of the framed speech signal. 3. The speech processing method according to claim 1, characterized in that the time domain signal corresponding to the windowed speech signal is processed according to an interpolation algorithm or an extraction algorithm, and the pitch-modified speech signal obtained includes: : The pitch-modified speech signal is determined according to the sampling frequency of the speech signal, the sampling frequency of the pitch-modified speech signal, and the length of each frame of the speech signal. 4. The speech processing method according to claim 1, characterized in that the rising tone of the voice signal corresponds to the increase in the playback time of the voice signal after the tone change, and the falling tone of the voice signal corresponds to the playback of the voice signal after the tone change. Time decreases. 5. A voice processing device, characterized by comprising: The voice acquisition module is used to receive the voice signal acquired and sent by the audio acquisition device after turning on the pitch-changing sound effect; A speech tone modification module, used to frame the time domain signal corresponding to the speech signal, add windows to the time domain signal corresponding to the framed speech signal, and perform windowing on the windowed speech according to an interpolation algorithm or an extraction algorithm. The time domain signal corresponding to the signal undergoes pitch modification processing to adjust the sampling frequency to obtain the pitch-modified speech signal; The time maintenance module is used to splice the pitch-modified speech signals according to the corresponding relationship between the playback time and the length of each frame of speech signal so that the length of the speech signal remains consistent, and to perform the time domain signal corresponding to the pitch-modified speech signal. The playback time is maintained, and the target voice signal is obtained when the playback time of the pitch-modified voice signal is the same as the playback time of the voice signal; wherein the playback time of the pitch-modified voice signal is equal to the playback time of the voice signal. same; Wherein, maintaining the playback time of the time domain signal corresponding to the pitch-modified speech signal, and obtaining the target speech signal when the playback time of the pitch-modified speech signal is the same as the playback time of the speech signal includes: Determine the comparison result between the time series variable and the overlap length between the two frames of speech signals obtained by dividing the time domain signal corresponding to the speech signal into frames; If the timing variable is less than the overlap length, when the offset is such that the noise synthesized by the playback time reduction of the modulated speech signal is minimized, according to the offset, the length of each frame of the speech signal and the overlap length, The difference between and the overlap length between the two frames of speech signals after the pitch change determines the target speech signal; the offset represents the distance between the optimal matching point and the m-th window; If the timing variable is greater than or equal to the overlap length and does not exceed the length of each frame of the speech signal before the pitch change, then after length alignment, the speech signal after the pitch change is used as the target speech signal. 6. An electronic device, characterized in that it includes: processor; and memory for storing executable instructions for the processor; Wherein, the processor is configured to execute the speech processing method according to any one of claims 1-4 by executing the executable instructions. 7. A computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the speech processing method according to any one of claims 1-4 is implemented.