CN113066503B - Audio frame adjusting method, device, equipment and readable storage medium - Google Patents

Audio frame adjusting method, device, equipment and readable storage medium Download PDF

Info

Publication number
CN113066503B
CN113066503B CN202110276427.4A CN202110276427A CN113066503B CN 113066503 B CN113066503 B CN 113066503B CN 202110276427 A CN202110276427 A CN 202110276427A CN 113066503 B CN113066503 B CN 113066503B
Authority
CN
China
Prior art keywords
harmonic
audio frame
amplitude
harmonics
modification coefficient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110276427.4A
Other languages
Chinese (zh)
Other versions
CN113066503A (en
Inventor
关迪聆
劳振锋
陈传艺
孙洪文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Kugou Computer Technology Co Ltd
Original Assignee
Guangzhou Kugou Computer Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Kugou Computer Technology Co Ltd filed Critical Guangzhou Kugou Computer Technology Co Ltd
Priority to CN202110276427.4A priority Critical patent/CN113066503B/en
Publication of CN113066503A publication Critical patent/CN113066503A/en
Application granted granted Critical
Publication of CN113066503B publication Critical patent/CN113066503B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/003—Changing voice quality, e.g. pitch or formants
    • G10L21/007—Changing voice quality, e.g. pitch or formants characterised by the process used
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/003—Changing voice quality, e.g. pitch or formants
    • G10L21/007—Changing voice quality, e.g. pitch or formants characterised by the process used
    • G10L21/013—Adapting to target pitch

Abstract

The application discloses a method, a device and equipment for adjusting an audio frame and a readable storage medium, and relates to the field of audio processing. The method comprises the following steps: acquiring an audio frame to be adjusted; extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame; determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch characteristics; and adjusting the amplitude of the harmonic wave in the audio frame according to the harmonic wave modification coefficient to obtain an adjusted audio frame. Therefore, the periodic components in the audio frame are modified, and the non-periodic components corresponding to the sound identification are not modified, so that the effect of modifying the sound tension and keeping the original identification is achieved, and the accuracy of adjusting the sound tension is improved.

Description

Audio frame adjusting method, device, equipment and readable storage medium
Technical Field
The embodiment of the application relates to the field of audio processing, in particular to an audio frame adjusting method, an audio frame adjusting device, audio frame adjusting equipment and a readable storage medium.
Background
In the singing voice synthesis technology, the equalizer can be used for modifying the characteristics of different frequency bands, so that the singing voice with rich content and strong expressive force can be generated. The tension belongs to one of expressive force, and is specifically used for indicating the fullness and definition of sound in a specified frequency band range, wherein the higher the tension is, the higher the fullness of sound is, and the higher the definition of sound is; the weaker the tension, the lower the fullness of the sound and the lower the clarity of the sound. For example, when the frequency of a certain frequency band of the audio content is 30-60Hz, the sound corresponding to the frequency is more clumsy and has no larger loudness; by adjusting the frequency of the frequency band to 100-200Hz, the sound can be adjusted to a state with plump hearing.
In the related art, when modifying the sound characteristics, an equalizer is generally used to filter different sound bands. Then, in each frequency band obtained by filtering, the sound amplitude of the designated frequency band is adjusted, so that the effect of modifying the sound effect is achieved.
The inventor finds that in the process of realizing the application, the design of the equalizer is generally fixed, and the designated frequency band of singing voice is filtered according to the center frequency, the bandwidth and the quality factor, so that the periodic component and the non-periodic component of the designated frequency band can be acted simultaneously, the non-periodic component representing the voice recognition degree of the user is synchronously modified, the original tone of the singing voice is influenced, and the accuracy of modifying the voice effect of the singing voice is reduced.
Disclosure of Invention
The embodiment of the application provides a method, a device, equipment and a readable storage medium for adjusting an audio frame, which are used for modifying periodic components related to voice tension irrespective of voice recognition degree of a user in the audio frame by adjusting harmonic amplitude in the audio frame, so that non-periodic components related to voice recognition degree of the user are prevented from being modified, and accuracy of modifying voice tension is improved. The technical scheme is as follows:
In one aspect, there is provided a method for adjusting an audio frame, the method comprising:
acquiring an audio frame to be adjusted;
extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame;
determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch feature; the harmonic waves of the audio frame comprise fundamental waves and higher harmonics, and the fundamental waves are first harmonics;
and adjusting the amplitude of the harmonic wave in the audio frame according to the harmonic wave modification coefficient to obtain an adjusted audio frame.
In an alternative embodiment, when the tension of the audio frame is to be reduced, the harmonic modification coefficient corresponding to the fundamental wave is used for increasing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for reducing the amplitude of the higher harmonic;
when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing or maintaining the amplitude of the higher harmonic.
In an alternative embodiment, the harmonic modification coefficients corresponding to the higher harmonics are used to enhance or maintain the amplitude of the higher harmonics, including:
The harmonic modification coefficients corresponding to the second harmonic of the higher harmonics are used for maintaining the amplitude of the second harmonic, and the harmonic modification coefficients corresponding to other higher harmonics of the higher harmonics are used for enhancing the amplitude of the corresponding higher harmonic.
In an alternative embodiment, the determining, based on the pitch characteristic, a harmonic modification factor that amplitude adjusts a harmonic of the audio frame includes:
determining a number of harmonics included in the audio frame based on the pitch characteristics;
a harmonic modification coefficient that adjusts harmonics in the target audio frame is determined based on the waveform frequency of the harmonics and the number of harmonics.
In an alternative embodiment, the determining the harmonic modification factor for adjusting the harmonic in the target audio frame based on the waveform frequency of the harmonic and the number of harmonics includes:
determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in a first adjustment direction, and determining a harmonic modification coefficient for adjusting the amplitude of the higher harmonic in a second adjustment direction, wherein the harmonic modification coefficient of the higher harmonic is determined based on the waveform frequency and the harmonic quantity;
The first adjustment direction and the second adjustment direction are different adjustment directions, the adjustment directions including at least one of an increasing adjustment direction, a decreasing adjustment direction, and a maintaining adjustment direction of the amplitude.
In an alternative embodiment, the adjustment of the audio frame is aimed at reducing the audio frame tension;
the determining, in a first adjustment direction, a harmonic modification coefficient for adjusting the amplitude of the fundamental wave, and determining, in a second adjustment direction, a harmonic modification coefficient for adjusting the amplitude of the higher harmonic, includes:
determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in the enhancement adjustment direction;
and determining a harmonic modification coefficient for adjusting the amplitude of the higher harmonic in the reduction adjustment direction based on the waveform frequency of the higher harmonic and the harmonic quantity.
In an alternative embodiment, the determining the harmonic modification factor for adjusting the amplitude of the fundamental wave in the enhancement adjustment direction includes:
determining a first preset value as a first harmonic modification coefficient of the fundamental wave, wherein the first harmonic modification coefficient of the fundamental wave is larger than 1, and the first harmonic modification coefficient of the fundamental wave is used for adjusting the amplitude of the fundamental wave;
The determining, based on the waveform frequency of the higher harmonic and the harmonic quantity, a harmonic modification coefficient for adjusting the amplitude of the higher harmonic in the decreasing adjustment direction includes:
and determining a first harmonic modification coefficient of the higher harmonic based on the waveform frequency of the higher harmonic and the harmonic quantity, wherein the first harmonic modification coefficient of the higher harmonic is used for adjusting the amplitude of the higher harmonic.
In an alternative embodiment, the adjustment of the audio frame is aimed at increasing the audio frame tension;
the determining, in a first adjustment direction, a harmonic modification coefficient for adjusting the amplitude of the fundamental wave, and determining, in a second adjustment direction, a harmonic modification coefficient for adjusting the amplitude of the higher harmonic, includes:
determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in the reduction adjustment direction;
determining a harmonic modification coefficient for adjusting the amplitude of the second harmonic in the maintenance adjustment direction;
and determining harmonic modification coefficients for adjusting the amplitudes of other higher harmonics according to the enhancement adjustment direction based on the waveform frequencies of the other higher harmonics and the harmonic quantity, wherein the other higher harmonics are higher harmonics except the second harmonic.
In an alternative embodiment, the determining the harmonic modification factor for adjusting the amplitude of the fundamental wave in the decreasing adjustment direction includes:
determining a second preset value as a second harmonic modification coefficient of the fundamental wave, wherein the second harmonic modification coefficient of the fundamental wave is smaller than 1, and the second harmonic modification coefficient of the fundamental wave is used for adjusting the amplitude of the fundamental wave;
the determining the harmonic modification coefficient for adjusting the amplitude of the second harmonic in the maintenance adjustment direction includes:
determining a third preset value 1 as a second harmonic modification coefficient of the second harmonic;
the method for determining the harmonic modification coefficient for adjusting the amplitude of other higher harmonics according to the waveform frequency of other higher harmonics and the harmonic quantity and the enhancement adjustment direction comprises the following steps:
and determining second harmonic modification coefficients of other higher harmonics based on the waveform frequencies of the other higher harmonics and the harmonic quantity, wherein the second harmonic modification coefficients of the other higher harmonics are used for adjusting the amplitudes of the other higher harmonics.
In another aspect, there is provided an apparatus for adjusting an audio frame, the apparatus comprising:
the acquisition module is used for acquiring the audio frame to be adjusted;
The extraction module is used for extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame;
a determining module for determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch characteristic; the harmonic waves of the audio frame comprise fundamental waves and higher harmonics, and the fundamental waves are first harmonics;
and the adjusting module is used for adjusting the amplitude of the harmonic wave in the audio frame according to the harmonic wave modification coefficient to obtain an adjusted audio frame.
In another aspect, a computer device is provided, where the computer device includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or an instruction set, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a method for adjusting an audio frame according to any one of the embodiments of the present application.
In another aspect, a computer readable storage medium is provided, where at least one instruction, at least one program, a code set, or an instruction set is stored, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a method for adjusting an audio frame according to any one of the embodiments of the present application.
In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method for adjusting an audio frame according to any one of the above embodiments.
The technical scheme provided by the embodiment of the application has the beneficial effects that at least:
and determining a harmonic modification coefficient of the audio frame according to the pitch characteristic of the audio frame, so as to adjust the amplitude of the harmonic based on the harmonic modification coefficient, and obtain the adjusted audio frame. The harmonic wave is a component obtained by decomposing the periodic non-sinusoidal alternating current, namely the periodic component in the sound, and the periodic component is related to the sound tension, so that the modification of the periodic component in the audio frame is realized, and the non-periodic component corresponding to the sound identification is not modified. Therefore, the effect of modifying the sound tension and keeping the original identification is achieved, and the accuracy of adjusting the sound tension is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic illustration of an implementation environment provided by an exemplary embodiment of the present application;
fig. 2 is a flowchart of a method for adjusting an audio frame according to an exemplary embodiment of the present application;
fig. 3 is a flowchart of a method for adjusting an audio frame according to another exemplary embodiment of the present application;
fig. 4 is a flowchart of a method for adjusting an audio frame according to another exemplary embodiment of the present application;
FIG. 5 is a frequency domain schematic diagram of a second audio content that is adjusted based on the first audio content of FIG. 4;
fig. 6 is a block diagram illustrating an audio frame adjusting apparatus according to an exemplary embodiment of the present application;
fig. 7 is a schematic diagram of a server according to an exemplary embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.
The method for adjusting the audio frame provided by the embodiment of the application can be applied to a terminal side, a server side and a terminal and a server together. The method for adjusting the audio frame is schematically described by taking a method for realizing data interaction between the terminal and the server through a communication network.
Fig. 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application, where the implementation environment includes a terminal 110 and a server 120, and the terminal 110 and the server 120 are connected through a communication network 130 as shown in fig. 1;
the terminal 110 is installed with an application program having an audio adjustment function, such as: song recording programs, video recording programs, multimedia playing programs, instant messaging applications, social programs, etc., as embodiments of the present application are not limited in this regard. After the user selects the first audio content in the application program, the first audio content is selected as the content which needs to be subjected to sound tension adjustment. In some embodiments, the first audio content is uploaded to the server 120 by the terminal 110 through the communication network 130, or the first audio content may be an online file selected by the user in the application program, and the server 120 directly obtains the first audio content from the server where the online file is located. Optionally, the terminal 110 receives a selection operation instruction of the user for the first audio content, and transmits the selection operation instruction to the server 120 through the communication network 130.
After acquiring the first audio content, the server 120 decodes the first audio content to obtain an audio frame, and performs sound processing on the audio frame. Illustratively, taking an audio frame as an example, the server 120 extracts a pitch feature of the audio frame and determines a harmonic modification amplitude of the audio frame based on the pitch feature; the server 120 amplitude adjusts harmonics in the audio frame with the harmonic modification amplitude. When the server 120 adjusts each frame of audio frame or each frame of key frame, the second audio content adjusted based on the first audio content can be obtained. The server 120 feeds the second audio content back to the terminal 110 through the communication network 130.
It should be noted that, the above-mentioned terminal 110 may be implemented as a mobile terminal such as a mobile phone, a tablet computer, a wearable device, a portable laptop computer, or a desktop computer, which is not limited in this embodiment of the present application.
The server 120 may be implemented as one server or may be implemented as a server cluster formed by a plurality of servers, and the server 120 may be implemented as a physical server or may be implemented as a cloud server.
In connection with the above-mentioned implementation environment, the method for adjusting an audio frame according to the embodiment of the present application is described, and fig. 2 is a flowchart of a method for adjusting an audio frame according to an exemplary embodiment of the present application, where the method is applied to a server for description, and as shown in fig. 2, the method includes:
in step 201, an audio frame to be adjusted is acquired.
In some embodiments, the audio frame to be adjusted is an audio frame in the first audio content.
Optionally, the method for obtaining the first audio content includes at least one of the following methods:
first, the server receives the first audio content uploaded by the terminal.
In some embodiments, when the terminal uploads the first audio content to the server, the terminal may further indicate to the server an adjustment requirement for adjusting the first audio content, and illustratively, the terminal selects an adjustment parameter in the application program, so that the server adjusts the first audio content based on the adjustment parameter. In the embodiment of the application, when the user selects the adjustment requirement, the user selects the pitch adjustment requirement to customize the tension modification effect in the pitch adjustment process.
Second, the server receives the audio content link sent by the terminal, and acquires the first audio content from the other servers based on the audio content link.
In some embodiments, when receiving the audio content link sent by the terminal, the server obtains the first audio content from the server corresponding to the link website based on the audio content link. Optionally, the terminal may further indicate to the server an adjustment requirement for adjusting the first audio content when sending the audio content link to the server.
Thirdly, when the execution subject is implemented as a terminal, the terminal may acquire the first audio content from the local multimedia library; alternatively, the terminal downloads the first audio content from the server.
In the embodiment of the application, the first audio content is uploaded to the server by the terminal, and the first audio content is locally stored by the terminal; or the first audio content is the audio content recorded by the terminal in real time. Such as: the method comprises the steps that a song recording application program is installed in a terminal, the terminal records sound signals through the song recording application program to obtain first audio content (song recording content), after encoding, an encoded file is uploaded to a server, after the encoded file is received by the server, the encoded file is decoded to obtain each audio frame in the first audio content, and therefore the first audio content is adjusted based on the audio frames. Step 202, extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame.
The pitch feature is used to indicate sounds of various pitch levels within the audio content. In some embodiments, pitch may also affect the tension of sound, which is one of the expressive forces of sound. Tension belongs to one of the sound expressive force, and is specifically used for indicating the fullness and definition of sound in a specified frequency band range, wherein the higher the tension is, the higher the fullness of sound is, and the higher the definition of sound is; the weaker the tension, the lower the fullness of the sound and the lower the clarity of the sound.
After the server acquires the first audio content, frequency domain analysis is carried out on the first audio content, and a frequency domain diagram corresponding to the first audio content is obtained. Subsequent modifications to the audio frames of the first audio content are all adjusted on the basis of the frequency domain map. Such as: and analyzing and obtaining corresponding pitch characteristics from a frequency domain diagram aiming at the audio frame, wherein the frequency domain diagram is used for representing the relation between the frequency of the harmonic wave in the audio frame and the amplitude of the harmonic wave, and the amplitude of the harmonic wave is used for determining the sound tension corresponding to the audio frame.
Illustratively, for an audio frame in the first audio content, a probability-based YIN (Probabilistic YIN, pyin) algorithm is utilized to extract a pitch feature f in the audio frame 0 . The Pyin algorithm is an algorithm for detecting a pitch, and generally the Pyin algorithm detects a pitch based on probability, thereby extracting a pitch within a given frequency domain.
In some embodiments, the server sequentially extracts the pitch of the audio frames in the first audio content and sequentially adjusts the audio frames based on the extracted pitch characteristics. Illustratively, the server first extracts a pitch feature corresponding to the first frame of audio frames and adjusts the first frame of audio frames based on the pitch feature; and after the adjustment is finished, extracting the pitch characteristic of the second frame of audio frame to adjust the second frame of audio frame, and the like until the adjustment of the audio frame in the first audio content is finished.
Step 203, determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch characteristics.
In some embodiments, harmonics within an audio frame include fundamental and higher harmonics, where fundamental refers to the first harmonic within the audio frame. Optionally, harmonics other than the fundamental wave in the audio frame are higher harmonics, such as: second harmonic, third harmonic, … ….
In some embodiments, higher harmonics are used to indicate harmonics of the waveform frequency that are integer multiples of the fundamental frequency, where harmonics equal to or greater than twice the fundamental frequency are referred to as higher harmonics. Illustratively, the second harmonic corresponds to twice the fundamental frequency and the third harmonic corresponds to three times the fundamental frequency. For example, when the frequency of the fundamental wave is 300Hz, that is, the frequency of the first harmonic (which may be also referred to as the 0 th harmonic) is 300Hz, the frequency corresponding to the second harmonic (which may be also referred to as the 1 st harmonic) is 600Hz, and the frequency corresponding to the third harmonic (which may be also referred to as the 2 nd harmonic) is 900Hz; optionally, the location information of harmonics of the higher harmonics in the audio frame is determined at integer multiples of the fundamental frequency, in some embodiments, the higher harmonics are determined at integer multiples of the fundamental frequency, in particular several harmonics in the audio frame, and the order of the higher harmonics is determined. For example, when the frequency of the fundamental wave is 300Hz, if the frequency of a certain higher harmonic is 1200Hz, the frequency is four times the frequency corresponding to the fundamental wave, that is, the fourth harmonic (also referred to as the 3 rd harmonic).
In some embodiments, the determination of the harmonic modification coefficients described above includes at least one of:
first, when the tension of the audio frame is to be reduced, the harmonic modification coefficient corresponding to the fundamental wave is used for increasing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for reducing the amplitude of the higher harmonic. Schematically, the harmonic modification coefficient corresponding to the fundamental wave is greater than 1, and the harmonic modification coefficient corresponding to the higher harmonic is less than 1.
Second, when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing or maintaining the amplitude of the higher harmonic.
Wherein the enhancement or maintenance of the amplitude of the higher harmonics is indicative of the enhancement or maintenance of the amplitude of the harmonics for a single higher harmonic. Illustratively, the harmonic modification coefficient corresponding to the fundamental wave is smaller than 1, and the harmonic modification coefficient corresponding to the higher harmonic is larger than or equal to 1.
In some embodiments, harmonic modification coefficients corresponding to second ones of the higher harmonics are used to maintain magnitudes of the second harmonics, and harmonic modification coefficients corresponding to other ones of the higher harmonics are used to enhance magnitudes of the respective harmonics, wherein the other higher harmonics are used to indicate higher harmonics other than the second harmonic.
In some embodiments, in determining the above harmonic modification coefficients, it is first necessary to determine the waveform frequency and the number of harmonics in the audio frame, thereby determining the harmonic modification coefficients based on the number of harmonics.
In some embodiments, the pitch feature f of the audio frame is first based on 0 The number of harmonics M contained in the audio frame is determined, so that a harmonic modification coefficient that adjusts harmonics in the target audio frame is determined based on the number of harmonics M.
When the harmonic quantity is determined, the specific process is as follows:
acquiring preset sampling rate data f s The sampling rate data f s For representing the sampling rate at which pitch samples are taken in an audio frame. Pitch feature f corresponding to audio frame based on extraction 0 And sample rate data f s The number of harmonics M contained in the audio frame is determined, and a determination formula of the number of harmonics M can be referred to as the following formula 1.
Equation 1:
in the formula 1, M is the number of harmonics corresponding to the audio frame; f (f) s Sampling rate data corresponding to the audio frame; f (f) 0 Is the pitch characteristic corresponding to the audio frame. Alternatively, the above formula 1 is preset.
In some embodiments, harmonic modification coefficients that adjust harmonics in the target audio frame are determined based on the waveform frequency and the number of harmonics of the harmonics.
Alternatively, the first harmonic in an audio frame is called a fundamental wave, the remaining harmonics are called overtones or higher harmonics, the fundamental wave mainly affecting the pitch of the sound, and the higher harmonics mainly affecting the expressivity, i.e. the tension, of the sound. The high-tension sound penetration force is strong, the energy of the overtone part is sufficient in the frequency domain, the low-tension sound penetration force is weak, and the energy of the overtone part is weak in the frequency domain. In the audio adjustment process, the amplitude corresponding to the harmonic wave is modified, so that the effect of modifying fundamental wave energy and higher harmonic energy can be achieved, and audio content with richer content and better expressive force can be generated.
Determining a harmonic modification coefficient corresponding to the audio frame based on the waveform frequency of the harmonic wave and the harmonic quantity M, and adjusting the harmonic amplitude according to the harmonic modification coefficient to obtain a harmonic modification amplitude O p In some embodiments, the harmonic amplitude corresponds to two modifications: a first modification and a second modification. Correspondingly, the value of p is 1 or 2. Under the first modification mode, the value of p is 1, and the corresponding harmonic modification amplitude is O 1 The method comprises the steps of carrying out a first treatment on the surface of the In the second modification mode, the value of p is 2, and the corresponding harmonic modification amplitude is O 2 . The first modification and the second modification are described in detail later.
Illustratively, for the kth harmonic in the audio frame, a harmonic modification coefficient a for the kth harmonic is determined based on the number of harmonics M p Harmonic modification coefficient a based on kth harmonic p Determining a harmonic modification amplitude O of a kth harmonic p K is more than or equal to 0 and less than M, and k is an integer. Aiming at the situation that fundamental waves and higher harmonics exist in an audio frame, the fundamental waves are first harmonics in the audio frame, and the higher harmonics are second harmonics, third harmonics, fourth harmonics, … and M-1 harmonics in the audio frame. In the embodiment of the present application, since k is an integer from 0 to M-1, the kth harmonic is actually used to indicate the k+1 th harmonic, for example, when k=1, it is actually used to represent the second harmonic in the audio frame, and when k=2, it is actually used to represent the third harmonic in the audio frameSubharmonics, and so on.
When the harmonic modification coefficient is a 1 When the coefficient in the first modification mode is correspondingly adopted to take the value; when the harmonic modification coefficient is a 2 And when the coefficient is correspondingly valued in the second modification mode. The first modification and the second modification will be described in detail later.
And 204, adjusting the amplitude of the harmonic wave in the audio frame according to the harmonic wave modification coefficient to obtain an adjusted audio frame.
Optionally, the server obtains harmonic modification coefficients a corresponding to M harmonics in the audio frame p After that, the harmonic modification amplitude value O corresponding to M harmonics is determined p The harmonic modification amplitude of the audio frame is applied to the audio frame to adjust the amplitude of the harmonic.
In some embodiments, when all of the N audio frames to be adjusted in the first audio content are adjusted, the adjusted second audio content is obtained.
In some embodiments, the server stores the second audio content and the corresponding relation with the first audio content in the database, and the server feeds back the second audio content to the terminal, so that the terminal plays the second audio content.
Or after obtaining the second audio content, the server sends a prompt message to the terminal, where the prompt message is used to instruct the terminal to select a storage condition of the first audio content, that is, the terminal may select to store the first audio content, may also select to use the second audio content to replace the first audio content, and may also select to store the first audio content and the second audio content simultaneously. In some embodiments, the server first sends the second audio content to the terminal for playing, and then sends the prompt message to the terminal.
In summary, according to the method for adjusting an audio frame provided by the embodiment, the harmonic modification coefficient of the audio frame is determined according to the pitch characteristic of the audio frame, so that the amplitude of the harmonic is adjusted based on the harmonic modification coefficient, and the adjusted audio frame is obtained. The harmonic wave is a component obtained by decomposing the periodic non-sinusoidal alternating current, namely the periodic component in the sound, and the periodic component is related to the sound tension, so that the modification of the periodic component in the audio frame is realized, and the non-periodic component corresponding to the sound identification is not modified. Therefore, the effect of modifying the sound tension and keeping the original identification is achieved, and the accuracy of adjusting the sound tension is improved.
In an alternative embodiment, the modification to the harmonic wave includes a first modification manner and a second modification manner, and fig. 3 is a flowchart of a method for adjusting an audio frame according to another exemplary embodiment of the present application, and the method is illustrated by using the method in a server as an example, and as shown in fig. 3, the method includes:
step 301, an audio frame to be adjusted is acquired.
In some embodiments, the audio frame to be adjusted is an audio frame in the first audio content.
The manner of acquiring the audio frame to be adjusted in step 201 is described in detail above, and will not be described here again.
Step 302, extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame.
The pitch feature is used to indicate sounds of various pitch levels within the audio content. In some embodiments, pitch may also affect the tension of sound, which is one of the expressive forces of sound. Tension is set in the pitch range, and the fullness, definition, certainty and elasticity of sound make it have abundant internal holding power in audio frequency content broadcast process.
The process of extracting the pitch feature is described in detail in the above step 202, and will not be described here again.
Step 303, applying a first modification mode to the harmonics in the audio frame to obtain a first harmonic modification amplitude.
In some embodiments, the adjustments are made in different adjustment directions as the amplitude adjustments are made to the fundamental and higher harmonics in the audio frame. Illustratively, the harmonic modification coefficients for adjusting the amplitude of the fundamental wave are determined in a first adjustment direction, and the harmonic modification coefficients for adjusting the amplitude of the higher harmonic are determined in a second adjustment manner, wherein the harmonic modification coefficients of the higher harmonic are determined based on the waveform frequency and harmonic number of the higher harmonic. The first adjustment direction and the second adjustment direction are different adjustment directions, wherein the adjustment directions include at least one of an increasing adjustment direction, a decreasing adjustment direction, and a maintaining adjustment direction of the amplitude.
In some embodiments, the modification to the pitch includes a first modification and a second modification. Optionally, in the first modification, the adjusting target of the audio frame is to reduce the tension of the audio frame; in a second modification, the adjustment of the audio frame is aimed at enhancing the audio frame tension. The first modification mode is a modification mode for enhancing the fundamental frequency component and reducing other higher harmonic components; the second modification means a modification in which the fundamental frequency component is reduced, the second harmonic (i.e., the 1 st harmonic) is unchanged, and the other higher harmonic components are enhanced.
Determining a first harmonic modification coefficient a corresponding to a kth harmonic based on the waveform frequency of the harmonic and the number of harmonics M 1 。
In a first modification mode (namely, the adjustment target of the audio frame is to reduce the tension of the audio frame), determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in the enhancement adjustment direction; and determining a harmonic modification coefficient for adjusting the amplitude of the higher harmonic based on the waveform frequency and the harmonic quantity of the higher harmonic so as to reduce the adjustment direction.
Illustratively, for a fundamental wave in an audio frame, a first preset value b is determined as a first harmonic modification coefficient a of the fundamental wave 1 Wherein the fundamental wave is used to indicate the first harmonic in the audio frame, i.e. the corresponding harmonic when k=0; it is noted that the first preset value b is a value greater than 1, and the first harmonic modification coefficient of the fundamental wave is used for adjusting the amplitude of the fundamental wave.
For higher harmonics in an audio frame, determining a first harmonic modification coefficient a of the higher harmonic based on the waveform frequency of the higher harmonic and the number of harmonics M 1 The higher harmonics refer to harmonics other than the fundamental wave, that is, harmonics corresponding to integers having k greater than 0 and less than M. First harmonic modification system of higher harmonicsThe number is used to adjust the amplitude of the higher harmonics. Optionally, the corresponding k value is determined based on the waveform frequency of the higher harmonic, where k is used to represent what harmonic the higher harmonic is in the audio frame, where what harmonic the higher harmonic is in the audio frame is determined by a multiple relationship between the waveform frequency of the higher harmonic and the waveform frequency of the fundamental wave. For example, the frequency corresponding to the fundamental wave (corresponding to the 0 th harmonic in the audio frame) is 300Hz, and the frequency corresponding to the third harmonic is 900Hz, so the third harmonic is the 2 nd harmonic in the audio frame, that is, the value of k is 2, and so on.
Determining a first harmonic modification coefficient a for the above expression 1 See equation 2.
Equation 2:
in equation 2, k is used to represent the kth harmonic; m is used to represent the number of harmonics in the audio frame; b is a first preset value, and b is more than 1; h is a fourth preset value, which refers to a preset degree coefficient, and the value range of h is more than 0 and less than 1; a, a 1 The method is used for representing a first harmonic modification coefficient, namely the harmonic modification coefficient obtained in the first modification mode; the above formula 2 may be stored in the server in advance. It should be noted that the first preset value b and the fourth preset value h are values preset by the user.
The server determines the first harmonic modification coefficient a corresponding to each harmonic in the N frames of audio frames according to the formula 2 1 。
And step 304, adjusting the amplitude of the harmonic wave in the audio frame by using the first harmonic wave modification amplitude value to obtain the adjusted second audio content.
Optionally, after obtaining the number M of harmonics of the audio frame, the server extracts the harmonic amplitude o of the kth harmonic in the audio frame by using an engineering interface of the open source libllsm2 k Extracting the harmonic amplitude o of the kth harmonic k Other ways may also be utilized, and the application is not limited in this regard.
Determining a first harmonic modification coefficient a of a kth harmonic 1 Harmonic amplitude o with the kth harmonic k Is determined as the first harmonic modification amplitude O of the kth harmonic 1 The method comprises the steps of carrying out a first treatment on the surface of the Please refer to formula 3.
Equation 3: o (O) 1 ＝a 1 ×o k ，k＝0，...，M-1
In formula 3, a 1 A first harmonic modification coefficient for representing a k-th harmonic correspondence; o (o) k The value range of k is 0-k < M, k is an integer, and M is used for representing the number of the harmonic waves corresponding to the audio frame; o (O) 1 For representing the first harmonic modification amplitude corresponding to the kth harmonic. The above formula 3 may be stored in the server in advance.
Modifying the first harmonic amplitude O using the engineering interface of the open source libllsm2 1 And restoring to the first audio content to obtain the second audio content. It should be noted that the degree of tension reduction is determined by the first preset value b and the fourth preset value h in the formula 2.
Based on the above process, the server obtains first harmonic modification coefficients a corresponding to M harmonics in the audio frame 1 Thereby obtaining the first harmonic modification amplitude value O corresponding to the M harmonics 1 And applying the harmonic modification amplitude value of the corresponding frame to the current harmonic amplitude value in the audio frame to carry out amplitude adjustment until the audio frame to be adjusted is completely adjusted, so as to obtain the adjusted second audio content.
In step 305, a second modification is applied to the harmonics in the audio frame to obtain a second harmonic modification amplitude.
In a second modification mode (namely, the adjustment target of the audio frame is to increase the tension of the audio frame), determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in a decreasing adjustment direction; determining a harmonic modification coefficient for adjusting the amplitude of the second harmonic in a maintenance adjustment direction; and determining harmonic modification coefficients for adjusting the amplitudes of other higher harmonics, which are higher harmonics except the second harmonic, according to the waveform frequency and the harmonic quantity of the other higher harmonics in the enhancement adjustment direction.
Determination based on harmonic quantity MSecond harmonic modification coefficient a corresponding to kth harmonic 2 。
For fundamental waves in the audio frame, determining a second preset value d as a second harmonic modification coefficient a of the fundamental waves 2 Wherein the fundamental wave is used to indicate the corresponding harmonic in the audio frame when k=0; it is noted that the range of the second preset value d is 0 < d < 1, and the second harmonic modification coefficient of the fundamental wave is used for adjusting the amplitude of the fundamental wave.
For the second harmonic (also referred to as the 1 st harmonic) in the audio frame, a third preset value e is determined as the second harmonic modification coefficient a of the second harmonic 2 Optionally, the third preset value e is 1, which is used to indicate that the harmonic component of the second harmonic is unchanged, and the original amplitude is maintained.
For other higher harmonics in the audio frame, determining a second harmonic modification coefficient a of the other higher harmonics based on the waveform frequency of the harmonic and the number of harmonics M 2 The other harmonics are harmonics other than the second harmonic, that is, harmonics corresponding to integers having k greater than 1 and less than M. The second harmonic modification coefficients of the other higher harmonics are used to adjust the amplitude of the other higher harmonics.
Determining a second modification coefficient a for the above expression 2 See equation 4.
Equation 4:
in equation 4, k is used to represent the kth harmonic; m is used to represent the number of harmonics in the audio frame; d is a second preset value, and d is more than 0 and less than 1; e is a third preset value, and e=1; j is a fifth preset value, and j is more than 0; a, a 2 For representing a second harmonic modification factor; the above formula 4 may be stored in the server in advance. It should be noted that the second preset value d, the third preset value e and the fifth preset value j are values preset by the user.
The server determines the second harmonic modification coefficient a corresponding to each harmonic in the N frames of audio frames according to the formula 4 2 。
And 306, adjusting the amplitude of the harmonic wave in the audio frame by using the second harmonic wave modification amplitude value to obtain an adjusted second audio content.
Optionally, after obtaining the number M of harmonics of the audio frame, the server extracts the harmonic amplitude o of the kth harmonic in the audio frame by using an engineering interface of the open source libllsm2 k Extracting the harmonic amplitude o of the kth harmonic k Other ways may also be utilized, and the application is not limited in this regard.
Determining a second harmonic modification coefficient a of a kth harmonic 2 Harmonic amplitude o with the kth harmonic k And determining the product as a second harmonic modification amplitude O of the kth harmonic 2 The method comprises the steps of carrying out a first treatment on the surface of the See equation 5 for details.
Equation 5: o (O) 2 ＝a 2 *o k ，k＝0，...，M-1
In formula 5, a 2 A second harmonic modification coefficient for representing a k-th harmonic correspondence; o (o) k The value range of k is 0-k < M, k is an integer, and M is used for representing the number of the harmonic waves corresponding to the audio frame; o (O) 2 And the second harmonic modification amplitude corresponding to the kth harmonic is represented. The above formula 5 may be stored in the server in advance.
Modifying the second harmonic amplitude O using the engineering interface of the open source libllsm2 2 And restoring to the first audio content to obtain the second audio content. It should be noted that the degree of increasing the tension is determined by the second preset value d and the fifth preset value j in the formula 4.
Based on the above process, the server obtains second harmonic modification coefficients a corresponding to M harmonics in the audio frame 2 Thereby obtaining the second harmonic modification amplitude value O corresponding to the M harmonics 2 And applying the harmonic modification amplitude value of the corresponding frame to the current harmonic amplitude value of the audio frame to carry out amplitude adjustment until all N frames of audio frames are adjusted, and obtaining adjusted second audio content.
In summary, according to the method for adjusting an audio frame provided by the embodiment, the harmonic modification coefficient of the audio frame is determined according to the pitch characteristic of the audio frame, so that the amplitude of the harmonic is adjusted based on the harmonic modification coefficient, and the adjusted audio frame is obtained. The harmonic wave is a component obtained by decomposing the periodic non-sinusoidal alternating current, namely the periodic component in the sound, and the periodic component is related to the sound tension, so that the modification of the periodic component in the audio frame is realized, and the non-periodic component corresponding to the sound identification is not modified. Therefore, the effect of modifying the sound tension and keeping the original identification is achieved, and the accuracy of adjusting the sound tension is improved.
In an alternative embodiment, fig. 4 is a flowchart of a method for adjusting an audio frame according to another exemplary embodiment of the present application, and the method is applied to a server, and as shown in fig. 4, the method includes:
in step 401, first audio content is acquired.
The first audio content includes audio frames to be adjusted.
The manner of obtaining the first audio content in step 201 is described in detail above, and will not be described here again.
Step 402, extracting the pitch of the audio frame according to the audio frame in the first audio content to obtain the pitch characteristic of the audio frame.
After the server acquires the first audio content, frequency domain analysis is carried out on the first audio content, and a frequency domain diagram corresponding to the first audio content is obtained. Subsequent modifications to the audio parameters of the first audio content are all adjusted on the basis of the frequency domain map.
The process of extracting the pitch feature is described in detail in the above step 202, and will not be described here again.
Step 403, determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch characteristics.
After the server determines the harmonic quantity of the audio frame, extracting harmonic amplitude corresponding to each harmonic; for the kth harmonic component: determining a first harmonic modification coefficient a that reduces tension in a first audio content using equation 2 1 In this process, the first preset value b is set to 2, andthe fourth preset value h is set to 0.5;
alternatively, the second harmonic modification coefficient a that increases the tension of the first audio content is determined using equation 4 2 In this process, the second preset value d is set to 0.5, and the fifth preset value j is set to 1.5.
And step 404, adjusting the amplitude of the harmonic in the audio frame according to the harmonic modification coefficient to obtain an adjusted audio frame and obtain an adjusted second audio content.
The first harmonic modification amplitude and the second harmonic modification amplitude are respectively applied to the first audio content by using an engineering interface of the open source libllsm2, which is schematically shown in fig. 5, and fig. 5 shows a frequency domain schematic diagram of the second audio content obtained by adjusting the first audio content. The ordinate of the spectrum diagram represents the amplitude, i.e. pitch, corresponding to each time point of the audio content in dB; the abscissa represents the frequency corresponding to the audio content, which is inversely related to time, i.e. a line segment within the spectral diagram may be represented as a change in the magnitude of the sound amplitude over time. In this embodiment, the first peak is taken as the fundamental tone, that is, the harmonic corresponding to k=0, the other peaks are called overtones, the above-mentioned method for adjusting the audio frame is applied to mainly adjust the tension of the overtones of the first audio content, and the fundamental tone is reserved; the solid line 501 is a line segment of the server that performs frequency domain analysis on the first audio content after receiving the first audio content, the dotted line 502 is a line segment that reduces the harmonic amplitude in the first audio content, and the dotted line 503 is a line segment that increases the harmonic amplitude in the first audio content.
And the server adjusts the first audio content according to the corresponding harmonic modification coefficient and the harmonic modification amplitude value to obtain the second audio content. And feeding the second audio content back to the terminal, and correspondingly processing the second audio content in the terminal by the user.
In summary, according to the method for adjusting an audio frame provided by the embodiment, the harmonic modification coefficient of the audio frame is determined according to the pitch characteristic of the audio frame, so that the amplitude of the harmonic is adjusted based on the harmonic modification coefficient, and an adjusted audio frame is obtained, wherein the harmonic is a component obtained by decomposing the periodic non-sinusoidal alternating current, namely a periodic component in sound, and the periodic component is related to the sound tension, so that the periodic component in the audio frame is modified, and the non-periodic component corresponding to the sound identification is not modified. Therefore, the effect of modifying the sound tension and keeping the original identification is achieved, and the accuracy of adjusting the sound tension is improved.
Fig. 6 is a block diagram of an audio frame adjusting apparatus according to an exemplary embodiment of the present application, as shown in fig. 6, including: an acquisition module 610, an extraction module 620, a determination module 630, and an adjustment module 640;
An obtaining module 610, configured to obtain an audio frame to be adjusted;
an extracting module 620, configured to extract a pitch of the audio frame, so as to obtain a pitch feature of the audio frame;
a determining module 630, configured to determine a harmonic modification coefficient for adjusting a magnitude of a harmonic of the audio frame based on the pitch characteristic; the harmonic waves of the audio frame comprise fundamental waves and higher harmonics, and the fundamental waves are first harmonics;
and the adjusting module 640 is configured to adjust the amplitude of the harmonic in the audio frame according to the harmonic modification coefficient, so as to obtain an adjusted audio frame.
In an alternative embodiment, the adjusting module 640 is further configured to, when the tension of the audio frame is to be reduced, increase the amplitude of the fundamental wave by using a harmonic modification coefficient corresponding to the fundamental wave, and reduce the amplitude of the higher harmonic by using a harmonic modification coefficient corresponding to the higher harmonic;
when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing or maintaining the amplitude of the higher harmonic.
In an alternative embodiment, the adjustment module 640 is further configured to maintain the amplitude of the second harmonic with respect to the harmonic modification coefficients corresponding to the second harmonic of the higher harmonics, and to enhance the amplitude of the corresponding higher harmonic with respect to the harmonic modification coefficients corresponding to the other higher harmonics of the higher harmonics.
In an alternative embodiment, the determining module 630 is further configured to determine a number of harmonics included in the audio frame based on the pitch characteristic;
a harmonic modification coefficient that adjusts harmonics in the target audio frame is determined based on the waveform frequency of the harmonics and the number of harmonics.
In an alternative embodiment, the audio frame includes a fundamental wave and a higher harmonic, the fundamental wave is a first harmonic, the determining module 630 is further configured to determine a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in a first adjustment direction, and determine a harmonic modification coefficient for adjusting the amplitude of the higher harmonic in a second adjustment direction, where the harmonic modification coefficient of the higher harmonic is determined based on the waveform frequency and the harmonic quantity; the first adjustment direction and the second adjustment direction are different adjustment directions, the adjustment directions including at least one of an increasing adjustment direction, a decreasing adjustment direction, and a maintaining adjustment direction of the amplitude.
In an alternative embodiment, the audio frame is adjusted to reduce the audio frame tension, and the determining module 630 is further configured to determine a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in the enhancement adjustment direction; and determining a harmonic modification coefficient for adjusting the amplitude of the higher harmonic in the reduction adjustment direction based on the waveform frequency of the higher harmonic and the harmonic quantity.
In an alternative embodiment, the method further comprises:
the determining module 630 is configured to determine a first preset value as a first harmonic modification coefficient of the fundamental wave, where the first harmonic modification coefficient of the fundamental wave is greater than 1, and the first harmonic modification coefficient of the fundamental wave is used to adjust an amplitude of the fundamental wave;
the determining module 630 is further configured to determine a first harmonic modification coefficient of the higher harmonic based on the waveform frequency of the higher harmonic and the harmonic number, where the first harmonic modification coefficient of the higher harmonic is used to adjust the amplitude of the higher harmonic.
In an alternative embodiment, the audio frame is adjusted to increase the audio frame tension, and the determining module 630 is further configured to determine a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in the decreasing adjustment direction; determining a harmonic modification coefficient for adjusting the amplitude of the second harmonic in the maintenance adjustment direction; and determining harmonic modification coefficients for adjusting the amplitudes of other higher harmonics according to the enhancement adjustment direction based on the waveform frequencies of the other higher harmonics and the harmonic quantity, wherein the other higher harmonics are higher harmonics except the second harmonic.
In an optional embodiment, the determining module 630 is further configured to determine a second preset value as a second harmonic modification coefficient of the fundamental wave, where the second harmonic modification coefficient of the fundamental wave is smaller than 1, and the second harmonic modification coefficient of the fundamental wave is used to adjust the amplitude of the fundamental wave;
the determining module 630 is further configured to determine a third preset value 1 as a second harmonic modification coefficient of the second harmonic;
the determining module 630 is further configured to determine a second harmonic modification coefficient of the other higher harmonic based on the waveform frequency of the other higher harmonic and the harmonic number, where the second harmonic modification coefficient of the other higher harmonic is used to adjust the amplitude of the other higher harmonic.
In summary, according to the adjusting device for an audio frame provided in the embodiment, the harmonic modification coefficient of the audio frame is determined according to the pitch characteristic of the audio frame, so that the amplitude of the harmonic is adjusted based on the harmonic modification coefficient, and the adjusted audio frame is obtained. The harmonic wave is a component obtained by decomposing the periodic non-sinusoidal alternating current, namely the periodic component in the sound, and the periodic component is related to the sound tension, so that the modification of the periodic component in the audio frame is realized, and the non-periodic component corresponding to the sound identification is not modified. Therefore, the effect of modifying the sound tension and keeping the original identification is achieved, and the accuracy of adjusting the sound tension is improved.
It should be noted that: the adjusting device for audio frames provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the device for adjusting an audio frame provided in the above embodiment and the method embodiment for adjusting an audio frame belong to the same concept, and detailed implementation processes of the device and the method embodiment are detailed and will not be described herein.
Fig. 7 is a schematic diagram illustrating a structure of a server according to an exemplary embodiment of the present application. The server may be the server shown in fig. 1. Specifically, the present application relates to a method for manufacturing a semiconductor device.
The server 120 includes a central processing unit (CPU, central Processing Unit) 701, a system Memory 704 including a random access Memory (RAM, random Access Memory) 702 and a Read Only Memory (ROM) 703, and a system bus 705 connecting the system Memory 704 and the central processing unit 801. The server 120 also includes a basic input/output system (I/O system, input Output System) 706, which facilitates the transfer of information between various devices within the computer, and a mass storage device 707 for storing an operating system 713, application programs 714, and other program modules 715.
The basic input/output system 706 includes a display 708 for displaying information and an input device 709, such as a mouse, keyboard, or the like, for a user to input information. Wherein both the display 708 and the input device 709 are coupled to the central processing unit 701 through an input output controller 710 coupled to the system bus 705. The basic input/output system 706 may also include an input/output controller 710 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input output controller 710 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer readable media provide non-volatile storage for the server 120. That is, the mass storage device 707 may include a computer readable medium (not shown) such as a hard disk or compact disc read only memory (CD-ROM, compact Disc Read Only Memory) drive.
Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, erasable programmable read-only memory (EPROM, erasable Programmable Read Only Memory), electrically erasable programmable read-only memory (EEPROM, electrically Erasable Programmable Read Only Memory), flash memory or other solid state memory devices, CD-ROM, digital versatile disks (DVD, digital Versatile Disc) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The system memory 704 and mass storage device 707 described above may be collectively referred to as memory.
According to various embodiments of the application, server 120 may also operate by being connected to remote computers on a network, such as the Internet. I.e., server 120 may be connected to the network 712 through a network interface unit 711 coupled to the system bus 705, or other types of networks or remote computer systems (not shown) may be coupled using the network interface unit 711.
The memory also includes one or more programs, one or more programs stored in the memory and configured to be executed by the CPU.
The embodiment of the application also provides a computer device, which comprises a processor and a memory, wherein at least one instruction, at least one section of program, code set or instruction set is stored in the memory, and the at least one instruction, the at least one section of program, the code set or the instruction set is loaded and executed by the processor to realize the audio frame adjusting method provided by the method embodiments.
Embodiments of the present application also provide a computer readable storage medium having at least one instruction, at least one program, a code set, or an instruction set stored thereon, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method for adjusting an audio frame provided by the above method embodiments.
Alternatively, the computer-readable storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), solid state disk (SSD, solid State Drives), or optical disk, etc. The random access memory may include resistive random access memory (ReRAM, resistance Random Access Memory) and dynamic random access memory (DRAM, dynamic Random Access Memory), among others. The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.

Claims (10)

1. A method of adjusting an audio frame, the method comprising:
Acquiring an audio frame to be adjusted;
extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame;
determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch feature; the harmonic waves of the audio frame comprise fundamental waves and higher harmonics, and the fundamental waves are first harmonics; when the tension of the audio frame is to be reduced, the harmonic modification coefficient corresponding to the fundamental wave is used for increasing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for reducing the amplitude of the higher harmonic;
when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing the amplitude of the higher harmonic; or when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing the amplitude of part of higher harmonic and maintaining the amplitude of the other part of higher harmonic;
and adjusting the amplitude of the harmonic wave in the audio frame according to the harmonic wave modification coefficient to obtain an adjusted audio frame.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, the harmonic modification coefficient corresponding to the second harmonic in the higher harmonics is used for maintaining the amplitude of the second harmonic, and the harmonic modification coefficients corresponding to other higher harmonics in the higher harmonics are used for enhancing the amplitude of the corresponding harmonic.
3. The method of claim 1, wherein the determining harmonic modification coefficients for amplitude adjustment of harmonics of the audio frame based on the pitch characteristics comprises:
determining a number of harmonics included in the audio frame based on the pitch characteristics;
a harmonic modification coefficient that adjusts harmonics in the audio frame is determined based on the waveform frequency of the harmonics and the number of harmonics.
4. A method according to claim 3, wherein the adjustment of the audio frame is aimed at reducing audio frame tension;
the determining a harmonic modification coefficient for adjusting harmonics in the audio frame based on the waveform frequency of the harmonics and the number of harmonics, includes:
Determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in an enhanced adjustment direction;
and determining a harmonic modification coefficient for adjusting the amplitude of the higher harmonic with a reduced adjustment direction based on the waveform frequency of the higher harmonic and the number of harmonics.
5. The method of claim 4, wherein the determining the harmonic modification factor that adjusts the amplitude of the fundamental wave in the enhancement adjustment direction comprises:
determining a first preset value as a first harmonic modification coefficient of the fundamental wave, wherein the first harmonic modification coefficient of the fundamental wave is larger than 1, and the first harmonic modification coefficient of the fundamental wave is used for adjusting the amplitude of the fundamental wave;
the harmonic modification coefficient for adjusting the amplitude of the higher harmonic is determined based on the waveform frequency of the higher harmonic and the harmonic quantity in order to reduce the adjustment direction, and the method comprises the following steps:
and determining a first harmonic modification coefficient of the higher harmonic based on the waveform frequency of the higher harmonic and the harmonic quantity, wherein the first harmonic modification coefficient of the higher harmonic is used for adjusting the amplitude of the higher harmonic.
6. A method according to claim 3, wherein the adjustment of the audio frame is aimed at increasing the audio frame tension;
The determining a harmonic modification coefficient for adjusting harmonics in the audio frame based on the waveform frequency of the harmonics and the number of harmonics, includes:
determining a harmonic modification coefficient for adjusting the amplitude of the fundamental wave in a reduced adjustment direction;
determining a harmonic modification coefficient for amplitude adjustment of the second harmonic in a maintenance adjustment direction;
and determining harmonic modification coefficients for adjusting the amplitudes of other higher harmonics, which are higher harmonics except the second harmonic, according to the waveform frequencies of the other higher harmonics and the harmonic quantity in an enhancement adjustment direction.
7. The method of claim 6, wherein the determining the harmonic modification factor that adjusts the amplitude of the fundamental wave in the decreasing adjustment direction comprises:
determining a second preset value as a second harmonic modification coefficient of the fundamental wave, wherein the second harmonic modification coefficient of the fundamental wave is smaller than 1, and the second harmonic modification coefficient of the fundamental wave is used for adjusting the amplitude of the fundamental wave;
the harmonic modification coefficient for adjusting the amplitude of the second harmonic to maintain the adjustment direction includes:
Determining a third preset value 1 as a second harmonic modification coefficient of the second harmonic;
the method for determining the harmonic modification coefficient for adjusting the amplitude of other higher harmonics according to the waveform frequency of the other higher harmonics and the harmonic quantity and the enhancement adjustment direction comprises the following steps:
and determining second harmonic modification coefficients of other higher harmonics based on the waveform frequencies of the other higher harmonics and the harmonic quantity, wherein the second harmonic modification coefficients of the other higher harmonics are used for adjusting the amplitudes of the other higher harmonics.
8. An apparatus for adjusting an audio frame, the apparatus comprising:
the acquisition module is used for acquiring the audio frame to be adjusted;
the extraction module is used for extracting the pitch of the audio frame to obtain the pitch characteristic of the audio frame;
a determining module for determining a harmonic modification coefficient for amplitude adjustment of a harmonic of the audio frame based on the pitch characteristic; the harmonic waves of the audio frame comprise fundamental waves and higher harmonics, and the fundamental waves are first harmonics; when the tension of the audio frame is to be reduced, the harmonic modification coefficient corresponding to the fundamental wave is used for increasing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for reducing the amplitude of the higher harmonic;
When the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing the amplitude of the higher harmonic; or when the tension of the audio frame is to be enhanced, the harmonic modification coefficient corresponding to the fundamental wave is used for reducing the amplitude of the fundamental wave, and the harmonic modification coefficient corresponding to the higher harmonic is used for enhancing the amplitude of part of higher harmonic and maintaining the amplitude of the other part of higher harmonic;
and the adjusting module is used for adjusting the amplitude of the harmonic wave in the audio frame according to the harmonic wave modification coefficient to obtain an adjusted audio frame.
9. A computer device comprising a processor and a memory, wherein the memory has stored therein at least one program that is loaded and executed by the processor to implement the method of adjusting an audio frame according to any of claims 1 to 7.
10. A computer-readable storage medium, wherein at least one program is stored in the storage medium, the at least one program being loaded and executed by a processor to implement the method of adjusting an audio frame according to any one of claims 1 to 7.
CN202110276427.4A 2021-03-15 2021-03-15 Audio frame adjusting method, device, equipment and readable storage medium Active CN113066503B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110276427.4A CN113066503B (en) 2021-03-15 2021-03-15 Audio frame adjusting method, device, equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110276427.4A CN113066503B (en) 2021-03-15 2021-03-15 Audio frame adjusting method, device, equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN113066503A CN113066503A (en) 2021-07-02
CN113066503B true CN113066503B (en) 2023-12-08

Family

ID=76561156

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110276427.4A Active CN113066503B (en) 2021-03-15 2021-03-15 Audio frame adjusting method, device, equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN113066503B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6993480B1 (en) * 1998-11-03 2006-01-31 Srs Labs, Inc. Voice intelligibility enhancement system
CN102881283A (en) * 2011-07-13 2013-01-16 三星电子（中国）研发中心 Method and system for processing voice
CN103258539A (en) * 2012-02-15 2013-08-21 展讯通信（上海）有限公司 Method and device for transforming voice signal characteristics

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6704711B2 (en) * 2000-01-28 2004-03-09 Telefonaktiebolaget Lm Ericsson (Publ) System and method for modifying speech signals

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6993480B1 (en) * 1998-11-03 2006-01-31 Srs Labs, Inc. Voice intelligibility enhancement system
CN102881283A (en) * 2011-07-13 2013-01-16 三星电子（中国）研发中心 Method and system for processing voice
CN103258539A (en) * 2012-02-15 2013-08-21 展讯通信（上海）有限公司 Method and device for transforming voice signal characteristics

Also Published As

Publication number Publication date
CN113066503A (en) 2021-07-02

Similar Documents

Publication Publication Date Title
US10037313B2 (en) Automatic smoothed captioning of non-speech sounds from audio
CN109922268B (en) Video shooting method, device, equipment and storage medium
CN109147805B (en) Audio tone enhancement based on deep learning
US10141010B1 (en) Automatic censoring of objectionable song lyrics in audio
WO2015092492A1 (en) Audio information processing
US20130266147A1 (en) System and method for identification of highly-variable vocalizations
CN110928518B (en) Audio data processing method and device, electronic equipment and storage medium
CN112420015A (en) Audio synthesis method, device, equipment and computer readable storage medium
CN112423019B (en) Method and device for adjusting audio playing speed, electronic equipment and storage medium
US20190172477A1 (en) Systems and methods for removing reverberation from audio signals
JP2005522744A (en) How to identify audio content
CN113066503B (en) Audio frame adjusting method, device, equipment and readable storage medium
WO2023226572A1 (en) Feature representation extraction method and apparatus, device, medium and program product
US20230050519A1 (en) Speech enhancement method and apparatus, device, and storage medium
CN110853606A (en) Sound effect configuration method and device and computer readable storage medium
US20130322645A1 (en) Data recognition and separation engine
CN111028860B (en) Audio data processing method and device, computer equipment and storage medium
CN113766307A (en) Techniques for audio track analysis to support audio personalization
US10971161B1 (en) Techniques for loss mitigation of audio streams
CN113593519A (en) Text speech synthesis method, system, device, equipment and storage medium
US20140214412A1 (en) Apparatus and method for processing voice signal
KR20210086086A (en) Equalizer for equalization of music signals and methods for the same
CN113436644B (en) Sound quality evaluation method, device, electronic equipment and storage medium
US11516545B1 (en) Optimization tools and techniques for audio and audiovisual content
CN111128228B (en) Frequency spectrum extraction method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant