WO2019233359A1 - 对音乐进行通透处理的方法及设备 - Google Patents

对音乐进行通透处理的方法及设备 Download PDF

Info

Publication number
WO2019233359A1
WO2019233359A1 PCT/CN2019/089756 CN2019089756W WO2019233359A1 WO 2019233359 A1 WO2019233359 A1 WO 2019233359A1 CN 2019089756 W CN2019089756 W CN 2019089756W WO 2019233359 A1 WO2019233359 A1 WO 2019233359A1
Authority
WO
WIPO (PCT)
Prior art keywords
music
probability
training data
permeability
transparent
Prior art date
Application number
PCT/CN2019/089756
Other languages
English (en)
French (fr)
Inventor
姚青山
秦宇
喻浩文
卢峰
Original Assignee
安克创新科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 安克创新科技股份有限公司 filed Critical 安克创新科技股份有限公司
Priority to US17/059,158 priority Critical patent/US11887615B2/en
Publication of WO2019233359A1 publication Critical patent/WO2019233359A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/003—Changing voice quality, e.g. pitch or formants
    • G10L21/007—Changing voice quality, e.g. pitch or formants characterised by the process used
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0091—Means for obtaining special acoustic effects
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/091—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/155—Musical effects
    • G10H2210/265—Acoustic effect simulation, i.e. volume, spatial, resonance or reverberation effects added to a musical sound, usually by appropriate filtering or delays
    • G10H2210/281—Reverberation or echo
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/311—Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation

Definitions

  • Embodiments of the present invention relate to the field of sound, and more specifically, to a method and device for transparently processing music.
  • Sound quality is a subjective evaluation of audio quality. Generally, the sound quality is divided into dozens of indicators. The transparency of the music in these indicators is an important indicator of the sound quality of the music. It refers to the effects of reverberation and echo in music. There is an appropriate echo Give the music a sense of space, forming the effect of reverberation around the beam. For certain types of music, such as symphonies and music with natural style, enhanced transparency will produce better sound quality, but not all types of music are suitable for enhanced transparency, so determine which music is suitable for transparency Enhancement, how to set the enhancement parameters has become the main problem of permeability adjustment.
  • the current sound quality adjustment methods are mainly adjusted by the user. For example, the user manually selects whether to perform reverberation on the music and selects a predetermined set of parameters to generate a reverberation effect in a specific environment, such as Reverberation effects such as small rooms, bathrooms, etc. are brought to the user's operational complexity and affect the user's experience.
  • Embodiments of the present invention provide a method and a device for automatically adjusting the permeability of music, which can adjust the permeability of music based on deep learning, eliminating user operations, thereby improving the user experience.
  • a method for transparently processing music including:
  • the transparency enhancement parameter being used to perform transparency processing on the music to be played.
  • the method before the inputting the feature to the transparent probability neural network, the method further includes:
  • the transparent probability neural network is obtained through training.
  • each of the training data in the training data set is music data, and each of the training data has a feature and a transparent probability.
  • the characteristics of the training data are obtained in the following manner:
  • Feature extraction is performed on each frame after the divided frames to obtain the features of the training data.
  • the permeable probability of the training data is obtained in the following manner:
  • the permeable probability of the training data is obtained according to the scores of all the evaluators.
  • the obtaining the transparent probability of the training data according to the scores of all evaluators includes:
  • An average value of the scores of all the evaluators is determined as the transparent probability of the training data.
  • the determining a permeability enhancement parameter corresponding to the permeability probability includes:
  • the permeability enhancement parameter corresponding to the permeability probability is determined.
  • mapping relationship is preset as:
  • the permeability enhancement parameter is p0.
  • mapping relationship is determined in the following manner:
  • mapping relationship is determined according to the magnitude relationship of t (i).
  • the determining the mapping relationship according to a magnitude relationship of t (i) includes:
  • the permeability enhancement parameter corresponding to the probability s is p + ⁇ p * n.
  • the method further includes:
  • a method for transparently processing music including:
  • the feature is input to a permeation enhancement neural network to obtain permeation enhancement parameters, and the permeation enhancement parameters are used to permeate the music to be played.
  • the method before the inputting the feature to the permeability enhanced neural network, the method further includes:
  • the penetration enhanced neural network is obtained through training, wherein each training data in the training data set is music data, and each training data has features and recommended penetration enhancement parameters.
  • a device for transparently processing music is provided.
  • the device is configured to implement the steps of the method in the foregoing first aspect or any implementation manner, and the device includes:
  • An acquisition module for acquiring characteristics of music to be played
  • a transparent probability determining module configured to input the feature to a transparent probability neural network to obtain a transparent probability of the music to be played;
  • a transparency enhancement parameter determination module is configured to determine a transparency enhancement parameter corresponding to the permeable probability, and the transparency enhancement parameter is used to transparently process the music to be played.
  • a device for transparently processing music is provided.
  • the device is configured to implement the steps of the method according to the foregoing second aspect or any implementation manner, and the device includes:
  • An acquisition module for acquiring characteristics of music to be played
  • a determining module is configured to input the feature to a permeability enhancement neural network to obtain a transparency enhancement parameter, and the transparency enhancement parameter is used to transparently process the music to be played.
  • a device for transparently processing music which includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the computer program.
  • a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect or any implementation manner are implemented.
  • a permeability enhanced neural network can be constructed, and specifically, a penetration probability neural network is constructed in advance based on deep learning and a mapping relationship between the permeability probability and the transparency enhanced parameter is constructed, so that the playback Music is automatically transparent. This process greatly simplifies the operation of the user while ensuring the sound quality of the music, thereby improving the user experience.
  • FIG. 1 is a schematic flowchart of obtaining a permeable probability of training data according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of calculating a transparent probability based on an evaluator's score according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of determining a mapping relationship in an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a method for transparently processing music in an embodiment of the present invention.
  • FIG. 5 is another schematic flowchart of a method for transparently processing music in an embodiment of the present invention.
  • FIG. 6 is a schematic block diagram of a device for transparently processing music in an embodiment of the present invention.
  • FIG. 7 is another schematic block diagram of a device for transparently processing music in an embodiment of the present invention.
  • FIG. 8 is another schematic block diagram of a device for transparently processing music in an embodiment of the present invention.
  • Deep learning is a machine learning method that uses deep neural networks to learn features of data with complex models, and intelligently organizes low-level features of data to form more advanced abstract forms. Because deep learning has strong feature extraction and modeling capabilities for complex data that is difficult to abstract and model manually, deep learning is an effective implementation method for tasks such as adaptive adjustment of sound quality that are difficult to model manually.
  • a transparent probability neural network is constructed based on deep learning.
  • the transparent probabilistic neural network is trained based on the training data set.
  • the training data set includes a large amount of training data, and a single training data is described in detail below.
  • the training data is music data, including the characteristics of the training data, which can be used as the input of the neural network; and the permeability probability of the training data, which can be used as the output of the neural network.
  • the original music waveform is a time-domain waveform
  • the time-domain waveform may be framed, followed by feature extraction for each frame after the framed frame to obtain the characteristics of the training data.
  • feature extraction may be performed through Short-Time Fourier Transform (STFT), and the extracted feature may be Mel Frequency Frequency Cepstrum Coefficient (MFCC).
  • STFT Short-Time Fourier Transform
  • MFCC Mel Frequency Frequency Cepstrum Coefficient
  • the manner of feature extraction in this article is only schematic, and other features, such as amplitude spectrum, log spectrum, energy spectrum, etc. can also be obtained, which are not listed here one by one.
  • the extracted features may be expressed in the form of a feature tensor, such as an N-dimensional feature vector; or the extracted features may also be expressed in other forms, which is not limited herein.
  • the permeable probability of the training data can be obtained by referring to the method shown in FIG. 1, and the process includes:
  • the original music waveform is a time-domain waveform
  • the time-domain waveform can be framed and feature extracted for each frame to obtain frequency-domain features. Enhance processing is performed on some of the frequency points, and attenuation processing is performed on some of the frequency points to complete the transparent processing. It can then be restored to the time domain to obtain processed training data.
  • the lifting multiple at a certain frequency point f can be expressed as p (f).
  • the set of parameters for performing the transparency processing can be expressed as p, including the multiples of improvement at each frequency point, and p can also be referred to as the permeability parameter or the transparency enhancement parameter.
  • the reviewer will transparently process the music (that is, the processed training data obtained in S101). Compared with the music that has not undergone the pass through processing (ie, training data), it is determined whether the sound quality of the music after the pass through processing becomes better. In other words, the score indicates whether the sound quality of the processed training data is better than the sound quality of the training data in terms of subjectivity of the evaluator.
  • the evaluator can listen to the transparent music (that is, the processed training data obtained in S101) and the untransparent music (that is, the training data), and evaluate according to whether the sound quality of the transparent music is better or worse. Scoring. For example, if the reviewer thinks the sound quality of the transparent music is better, it is scored as 1, otherwise it is scored as 0. In this way, you can get the score of all the reviewers in a group of reviewers.
  • the scores of the 7 reviewers 1 to 7 are 1, 0, 1, 1, 0, 1, 1 in order.
  • the scores of all people on this music are averaged to form an evaluation value.
  • This evaluation value is hereinafter referred to as the "permeability probability". The larger the value, the more suitable the music is to be transparent.
  • the average value of the scores of all the evaluators obtained in S102 may be determined as the transparent probability, that is, the proportion of “1” among all the scores may be defined as the transparent probability. It can be understood that the range of the transparent probability is from 0 to 1.
  • the average of the scores of multiple evaluators can be used as the evaluation value (permeability probability). It can be understood that the larger the value, the more suitable the permeation processing is.
  • the permeability probability is 71.4%.
  • features can be obtained through feature extraction, and the transparent probability can be obtained with reference to a similar process as shown in FIGS. 1-2.
  • the permeable neural network is trained until convergence, and then the trained PN neural network can be obtained.
  • the embodiment of the present invention also constructs a mapping relationship between the permeable probability and the permeable enhancement parameter.
  • the mapping relationship may be preset.
  • the transparency enhancement parameter is represented as P and the permeability probability is represented as s.
  • the mapping relationship can be set in advance as:
  • mapping relationship can be determined through a subjective experiment of Just Noticeable Difference (JND).
  • the permissible probability can be obtained by referring to the foregoing process of FIG. 1 to FIG. 2, which is expressed as s.
  • This process can be implemented with reference to FIG. 3, for a piece of non-permeable music, it is subjected to multiple permeation treatments, the permeation parameters are p, p + ⁇ p, p + ⁇ p * 2, ..., p + ⁇ p * n, p + ⁇ p * (n + 1). Subsequently, the corresponding subjective feeling can be obtained according to the comparison of the sound quality of the two transparently processed music.
  • t (0) is obtained by comparing the sound quality of the music processed according to the permeability parameter p with the sound quality of the non-permeated music, and comparing the sound quality of the music processed according to the permeability parameter p + ⁇ p * i with The sound quality of the music processed by the permeability parameter p + ⁇ p * (i-1) is t (i).
  • the music processed according to the permeability parameter p + ⁇ p * i is represented as YY (i). Specifically, a plurality of evaluators listen to untransparent music and YY (0) and score them, and calculate t (0) based on the average of the scores.
  • YY (i) and YY (i-1) listen to YY (i) and YY (i-1) and score them, and calculate t (i) based on the average of the scores. Among them, if the assessor thinks that the sound quality of YY (i) is better than the sound quality of YY (i-1), the score is 1; otherwise, the score is 0.
  • the corresponding relationship is obtained according to the process shown in FIG. 3, so that the mapping relationship between the transmissive probability and the transmittance enhancement parameter can be established.
  • the obtained different permeable enhancement parameters may be averaged.
  • the permeable probability of music 1 and music 2 are both s1.
  • the penetration enhancement parameter P p + ⁇ p * n2 corresponding to s1 is obtained.
  • the mapping relationship it can be determined that the transmissibility probability s1 in the mapping relationship corresponds to p + ⁇ p * (n1 + n2) / 2.
  • the “average” used herein is a value obtained by averaging a plurality of terms (or values).
  • the average calculated in the above embodiment may be an arithmetic average.
  • the "average” can also obtain the result value through other calculation methods, such as a weighted average, in which the weights of different items can be equal or different, and the embodiment of the present invention does not limit the average method.
  • the embodiment of the present invention constructs a permeability probability neural network and a mapping relationship between the permeability probability and the permeability enhancement parameter.
  • an embodiment of the present invention may also provide a penetration enhanced neural network.
  • the input of the penetration enhanced neural network is a feature of music data
  • the output is a penetration enhanced parameter.
  • the penetration enhanced neural network is recommended.
  • Permeability enhancement parameters that permeate music data.
  • the permeation enhanced neural network may be obtained through training based on a training data set.
  • Each training data in the training data set is music data, and each training data has features and recommended penetration enhancement parameters.
  • For each training data its features can be obtained by feature extraction.
  • the trained penetration enhanced neural network can be obtained through training until convergence.
  • the permeability-enhanced neural network has an intermediate parameter: the probability of permeability. That is to say, the permeable neural network can obtain the permeable probability based on the characteristics of the input music data, and then obtain the permeable enhanced parameter according to the permeable probability as the output of the permeable neural network.
  • this process may refer to the aforementioned transparency probability neural network and the mapping relationship between the permeability probability and the permeability enhancement parameter, which will not be repeated here.
  • An embodiment of the present invention provides a method for transparently processing music. As shown in FIG. 4, a flowchart of the method includes:
  • the feature is input to a permeation enhancement neural network to obtain permeation enhancement parameters, where the permeation enhancement parameters are used to permeate the music to be played.
  • a permeability-enhanced neural network may have an intermediate variable, which is the probability of permeability.
  • the permeable probability can be obtained based on the aforementioned permeable probability neural network, and the permeable enhancement parameter can be obtained according to the permeable probability.
  • the method may further include: obtaining the penetration enhanced neural network through training based on a training data set, where each training data in the training data set is music data, and each training data is Features and recommended penetration enhancement parameters.
  • the characteristics of the training data can be obtained in the following ways: obtaining the time-domain waveform of the training data; framing the time-domain waveform; performing feature extraction on each frame after the framing to obtain the training The characteristics of the data.
  • the transparency enhancement parameters of the training data can be obtained in the following ways: performing a transparent processing on the training data to obtain processed training data; obtaining a score of each of a group of reviewers, and the rating Indicates whether the sound quality of the processed training data is subjectively better than the sound quality of the training data; obtain the permeability probability of the training data according to the scores of all the reviewers; The mapping relationship between the permeability probability and the permeability enhancement parameter determines the permeability enhancement parameter corresponding to the permeability probability.
  • the mapping relationship may be set in advance: if the permeability probability is greater than a threshold value, the permeability enhancement parameter is p0.
  • the permeability enhancement neural network may include a permeability probability neural network and a mapping relationship between the permeability probability and the permeability enhancement parameter.
  • S220 may include: inputting the feature to the permeability probability.
  • the neural network obtains the permeable probability of the music to be played, and obtains the permeable enhancement parameter corresponding to the permeable probability based on the mapping relationship between the permeable probability and the permeable enhancement parameter.
  • FIG. 5 A flowchart of another method for transparently processing music provided by an embodiment of the present invention may be shown in FIG. 5, which includes:
  • the transparent probability neural network in S2201 may be the aforementioned trained transparent probability neural network. It is understood that the foregoing training process is generally performed on the server side (that is, the cloud).
  • S210 may include obtaining features to be played through feature extraction. Alternatively, S210 may include a feature of receiving music to be played from the opposite end.
  • the peer is the client; if the process in FIG. 4 or FIG. 5 is executed by the client, the peer is the server.
  • FIG. 4 or FIG. 5 can be executed on the server side (that is, the cloud) or on the client side (such as a client application). These two cases will be described below in conjunction with FIG. 5 .
  • the music to be played is the client's local music.
  • S210 may include: receiving the music to be played from the client, obtaining a time-domain waveform of the music to be played, framing the time-domain waveform, and performing feature extraction on each frame to obtain its characteristics.
  • S210 may include: receiving music information of the music to be played from the client, where the music information may include at least one of a song title, an artist, an album, and the like.
  • the music to be played is obtained from the music database on the server side, and its characteristics are obtained by framing the time domain waveform of the music to be played and extracting features for each frame.
  • S210 may include a feature of receiving music to be played from a client.
  • the client may frame the time-domain waveform of the music to be played and extract features from each frame to obtain its features, and then the client sends the obtained features to the server.
  • the features in S210 are obtained through feature extraction, and the process of feature extraction can be performed on the server or the client.
  • a permeability enhancement parameter corresponding to the permeability probability of S2201 may be obtained.
  • the server can send the transparency enhancement parameter to the client, so that the client can transparently process the local music to be played according to the transparency enhancement parameter. In this way, the transparently processed music can be played locally on the client.
  • the user plays the music to be played online, that is, the music to be played is stored on the server side, for example, it may be stored in a music database on the server side.
  • S210 may include: receiving music information of the music to be played from the client, where the music information may include at least one of a song title, an artist, an album, and the like.
  • the music to be played is obtained from the music database on the server side, and its characteristics are obtained by framing the time domain waveform of the music to be played and extracting features for each frame.
  • S2202 may obtain a permeability enhancement parameter corresponding to the permeability probability of S2201 based on the foregoing mapping relationship.
  • the server can perform transparent processing on the music to be played according to the transparent enhancement parameter. In this way, the transparently processed music can be played online.
  • the client executes:
  • the client may be a mobile terminal such as a smart phone, a tablet computer, or a wearable device.
  • S210 may include: if the music to be played is local music, the client may frame the time domain waveform of the music to be played and extract features from each frame to obtain its features. If the music to be played is the music stored on the server, the client can send the music information of the music to be played to the server.
  • the music information here can include at least one of the song title, artist, album, etc., and then from the server After receiving the music to be played, the client can then frame the time-domain waveform of the music to be played and extract features from each frame to obtain its features.
  • the client may send the music information of the music to be played to the server, and then receive the characteristics of the music to be played from the server.
  • the server can obtain the music to be played from the music database according to the music information, frame the time-domain waveform of the music to be played, and extract the features of each frame to obtain its features. Then, the server can obtain the features Send to client. It can be seen that the features in S210 are obtained through feature extraction, and the process of feature extraction can be performed on the server or the client.
  • the music information described in the embodiment of the present invention is merely exemplary, and it may include other information, such as duration, format, etc., which are not listed here one by one.
  • the client can obtain the trained transparent probability neural network from the server, so in S2201, the client can use its locally trained transparent probability neural network to obtain the The probability of being transparent when playing music.
  • the foregoing mapping relationship may be determined on the server side.
  • the client may obtain the mapping relationship from the server side.
  • the foregoing mapping relationship may be directly stored in the client in advance, as in the foregoing implementation manner of the foregoing predetermined mapping relationship.
  • the client can obtain a penetration enhancement parameter corresponding to the penetration probability of S2201 based on the mapping relationship.
  • the client can transparently process the local music to be played according to the transparency enhancement parameter. In this way, the transparently processed music can be played locally on the client.
  • a transparent probability neural network can be constructed in advance based on deep learning, so that the transparent processing can be automatically performed on the music to be played. This process greatly simplifies the operation of the user while ensuring the sound quality of the music, thereby improving the user experience.
  • FIG. 6 is a schematic block diagram of a device for transparently processing music according to an embodiment of the present invention.
  • the device 30 shown in FIG. 6 includes an obtaining module 310 and a determining module 320.
  • the obtaining module 310 is configured to obtain characteristics of music to be played.
  • the determining module 320 is configured to input the features to a permeation enhancement neural network to obtain permeation enhancement parameters, and the permeation enhancement parameters are used to permeate the music to be played.
  • the device 30 shown in FIG. 6 may be a server side (that is, the cloud).
  • the device 30 may further include a training module for obtaining the permeation enhanced neural network through training based on a training data set, where each training data in the training data set is music data, and each The training data has features and recommended penetration enhancement parameters.
  • the permeation-enhanced neural network may have an intermediate variable as a permeability probability.
  • FIG. 7 is another schematic block diagram of a device for transparently processing music according to an embodiment of the present invention.
  • the device 30 shown in FIG. 7 includes an obtaining module 310, a permeable probability determining module 3201, and a permeable enhanced parameter determining module 3202.
  • the obtaining module 310 is configured to obtain characteristics of music to be played.
  • the transparent probability determining module 3201 is configured to input the feature to a transparent probability neural network to obtain the transparent probability of the music to be played.
  • the transparency enhancement parameter determining module 3202 is configured to determine a transparency enhancement parameter corresponding to the permeable probability, and the transparency enhancement parameter is used to transparently process the music to be played.
  • the device 30 shown in FIG. 7 may be a server (ie, the cloud).
  • the device 30 may further include a training module, configured to obtain the transparent probability neural network through training based on the training data set.
  • each training data in the training data set is music data, and each training data has a feature and a transparent probability.
  • the characteristics of the training data can be obtained in the following ways: obtaining the time-domain waveform of the training data; framing the time-domain waveform; performing feature extraction on each frame after the framing to obtain the training The characteristics of the data.
  • the transparent probability of the training data can be obtained in the following ways: performing a transparent processing on the training data to obtain processed training data; obtaining a score of each of a group of reviewers, and the rating It indicates whether the sound quality of the processed training data is better than the sound quality of the training data subjectively; the transparent probability of the training data is obtained according to the scores of all the reviewers. For example, the average value of the scores of all the evaluators may be determined as the transparent probability of the training data.
  • the permeability enhancement parameter determining module 3202 may be specifically configured to determine the communication corresponding to the permeability probability according to a mapping relationship between a pre-built permeability probability and a permeability enhancement parameter. Penetration enhancement parameters.
  • the mapping relationship may be set in advance: if the permeability probability is greater than a threshold, the permeability enhancement parameter is p0.
  • the penetration enhancement parameter corresponding to the penetration probability s is p + ⁇ p * n.
  • the device 30 shown in FIG. 6 or FIG. 7 may be a server side (that is, the cloud).
  • the device 30 may further include a sending module for sending the transparency enhancement parameter to the client.
  • the client can perform transparent processing on the music to be played based on the transparent enhancement parameter; and play the transparent processed music.
  • the device 30 shown in FIG. 6 or FIG. 7 may be a client.
  • the device 30 may further include a transparent processing module and a playback module.
  • the transparent processing module is configured to perform transparent processing on the music to be played based on the transparent enhanced parameter
  • the playback module is configured to play the transparent processed music.
  • the device 30 shown in FIG. 6 or FIG. 7 can be used to implement the foregoing method for transparently processing music shown in FIG. 4 or FIG. 5. To avoid repetition, details are not described herein again.
  • an embodiment of the present invention further provides another device for transparently processing music, including a memory, a processor, and a computer program stored on the memory and running on the processor.
  • a processor executes the program, the steps of the method shown in FIG. 4 or FIG. 5 are implemented.
  • the processor may obtain the characteristics of the music to be played; input the characteristics to the permeation enhancement neural network to obtain permeation enhancement parameters, and the permeation enhancement parameters are used to perforate the music to be played.
  • the processor may obtain the characteristics of the music to be played; input the characteristics to a permeability probability neural network to obtain the permeable probability of the music to be played; and determine the corresponding to the permeable probability Permeability enhancement parameter, which is used to permeate the music to be played.
  • the device for transparently processing music in the embodiment of the present invention may include: one or more processors, one or more memories, input devices, and output devices, and these components are implemented through a bus system and / or other forms Connection mechanism interconnected. It should be noted that the device may also have other components and structures as required.
  • the processor may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the device to perform desired functions.
  • CPU central processing unit
  • the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory.
  • the volatile memory may include, for example, a random access memory (RAM) and / or a cache memory.
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may run the program instructions to implement a client function (implemented by the processor) in the embodiments of the present invention described below, and / Or other desired function.
  • client function implemented by the processor
  • Various application programs and various data can also be stored in the computer-readable storage medium.
  • the input device may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
  • the output device may output various information (for example, images or sounds) to the outside (for example, a user), and may include one or more of a display, a speaker, and the like.
  • an embodiment of the present invention also provides a computer storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the steps of the method shown in FIG. 4 or FIG. 5 may be implemented.
  • the computer storage medium is a computer-readable storage medium.
  • a permeability enhanced neural network can be constructed, and specifically, a penetration probability neural network is constructed in advance based on deep learning and a mapping relationship between the permeability probability and the transparency enhanced parameter is constructed, so that the playback Music is automatically transparent. This process greatly simplifies the operation of the user while ensuring the sound quality of the music, thereby improving the user experience.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present invention is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in various embodiments of the present invention.
  • the foregoing storage media include: U disks, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks or compact discs and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Auxiliary Devices For Music (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种对音乐进行通透处理的方法及设备。方法包括：获取待播放音乐的特征(S210)；将特征输入至通透增强神经网络，得到通透增强参数，通透增强参数用于对待播放音乐进行通透处理(S220)。通过构建通透增强神经网络，具体地基于深度学习预先构建通透概率神经网络并且构建可通透概率与通透增强参数之间的映射关系，从而可以对待播放音乐自动地进行通透处理。该过程极大地简化用户的操作的同时，保证音乐的音质，从而提升了用户体验。

Description

对音乐进行通透处理的方法及设备
本申请要求于2018年6月5日提交的、申请号为201810583109.0、发明名称为“对音乐进行通透处理的方法及设备”的中国发明专利申请的优先权。
技术领域
本发明实施例涉及声音领域，并且更具体地，涉及一种对音乐进行通透处理的方法及设备。
背景技术
音质是人对音频质量的主观评价。一般地音质被划分成几十个指标，这些指标中音乐的通透性(transparency)是音乐音质的一项重要指标，它指的是音乐中类似混响和回音的效果，有适当的回音会使音乐具有空间感，形成余音绕梁的效果。某些类型的音乐，如交响乐、有大自然风格的音乐，通透度被增强会产生更好的音质效果，但并不是所有类型音乐都适合通透度增强，因此判断哪些音乐适合通透度增强，增强参数如何设置，就成了通透度调整的主要问题。
目前的音质调节(如通透性调节)方法，主要是由用户自己调节，如用户手动选择是否对音乐进行混响效果处理，选择事先给定的一组参数产生特定环境的混响效果，如产生小房间，浴室之类的混响效果，这样给用户带来了操作复杂度，影响了用户的体验。
发明内容
本发明实施例提供了一种对音乐的通透性进行自动调节的方法及设备，可以基于深度学习实现对音乐的通透性进行调节，免去用户操作，从而提升了用户的体验。
第一方面，提供了一种对音乐进行通透处理的方法，包括：
获取待播放音乐的特征；
将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概 率；
确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
在本发明的一种实现方式中，在所述将所述特征输入至通透概率神经网络之前，还包括：
基于训练数据集，通过训练得到所述通透概率神经网络。
在本发明的一种实现方式中，所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及可通透概率。
在本发明的一种实现方式中，通过以下方式得到所述训练数据的特征：
获取所述训练数据的时域波形；
对所述时域波形进行分帧；
对所述分帧后的每帧进行特征提取得到所述训练数据的所述特征。
在本发明的一种实现方式中，通过以下方式得到所述训练数据的可通透概率：
对所述训练数据进行通透处理，得到处理后的训练数据；
获取一组评测者中每个评测者的打分，所述打分表示所述处理后的训练数据的音质在所述评测者的主观上是否优于所述训练数据的音质；
根据所有评测者的打分得到所述训练数据的所述可通透概率。
在本发明的一种实现方式中，所述根据所有评测者的打分得到所述训练数据的所述可通透概率，包括：
将所述所有评测者的打分的均值确定为所述训练数据的所述可通透概率。
在本发明的一种实现方式中，所述确定与所述可通透概率对应的通透增强参数，包括：
根据预先构建的可通透概率与通透增强参数之间的映射关系，确定与所述可通透概率对应的所述通透增强参数。
在本发明的一种实现方式中，所述映射关系被预先设定为：
若所述可通透概率大于阈值，则所述通透增强参数为p0。
在本发明的一种实现方式中，通过以下方式确定所述映射关系：
对可通透概率为s的未通透音乐进行多个通透处理，通透增强参数依次为：p+Δp*i，i＝0,1,2…；
获取所述多个通透处理对应的多个主观感受t(i)，其中t(i)是基于多个评测者对按照通透参数p+Δp*i处理后的音乐的音质与按照通透参数p+Δp*(i-1)处理后的音乐的音质进行比较打分得到的；
根据t(i)的大小关系确定所述映射关系。
在本发明的一种实现方式中，所述根据t(i)的大小关系确定所述映射关系，包括：
若满足t(n+1)<t(n)，且t(j+1)>t(j)，j＝0,1,…,n-1，则确定所述映射关系中与可通透概率s所对应的通透增强参数为p+Δp*n。
在本发明的一种实现方式中，还包括：
基于所述通透增强参数对所述待播放音乐进行通透处理；
播放所述通透处理后的音乐。
第二方面，提供了一种对音乐进行通透处理的方法，包括：
获取待播放音乐的特征；
将所述特征输入至通透增强神经网络，得到通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
在本发明的一种实现方式中，在将所述特征输入至通透增强神经网络之前，还包括：
基于训练数据集，通过训练得到所述通透增强神经网络，其中所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及推荐的通透增强参数。
第三方面，提供了一种对音乐进行通透处理的设备，所述设备用于实现前述第一方面或任一实现方式所述方法的步骤，所述设备包括：
获取模块，用于获取待播放音乐的特征；
可通透概率确定模块，用于将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概率；
通透增强参数确定模块，用于确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
第四方面，提供了一种对音乐进行通透处理的设备，所述设备用于实现前述第二方面或任一实现方式所述方法的步骤，所述设备包括：
获取模块，用于获取待播放音乐的特征；
确定模块，用于将所述特征输入至通透增强神经网络，得到通透增强参 数，所述通透增强参数用于对所述待播放音乐进行通透处理。
第五方面，提供了一种对音乐进行通透处理的设备，包括存储器、处理器及存储在所述存储器上且在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现前述第一方面或第二方面或任一实现方式所述方法的步骤。
第六方面，提供了一种计算机存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现前述第一方面或第二方面或任一实现方式所述方法的步骤。
由此可见，本发明实施例中可以构建通透增强神经网络，具体地基于深度学习预先构建通透概率神经网络并且构建可通透概率与通透增强参数之间的映射关系，从而可以对待播放音乐自动地进行通透处理。该过程极大地简化用户的操作的同时，保证音乐的音质，从而提升了用户体验。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例的得到训练数据的可通透概率的示意性流程图；
图2是本发明实施例中基于评测者打分计算可通透概率的示意图；
图3是本发明实施例中确定映射关系的示意图；
图4是本发明实施例中对音乐进行通透处理的方法的示意性流程图；
图5是本发明实施例中对音乐进行通透处理的方法的另一示意性流程图；
图6是本发明实施例中对音乐进行通透处理的设备的示意性框图；
图7是本发明实施例中对音乐进行通透处理的设备的另一示意性框图；
图8是本发明实施例中对音乐进行通透处理的设备的再一示意性框图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是 全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。
深度学习(Deep Learning)是一种机器学习方法，其应用深层神经网络对具有复杂模型的数据进行特征学习，并将数据低层次特征进行智能组织，形成更高级抽象形式。由于深度学习对人工难以抽象并建模的复杂数据具有较强的特征提取和建模能力，对音质自适应调整这类较难进行人工建模的任务，深度学习是一种有效的实现方法。
本发明实施例中基于深度学习构建了一种通透概率神经网络。该通透概率神经网络是根据训练数据集进行训练得到的。其中，训练数据集中包括大量的训练数据，下面对单个训练数据进行详细阐述。
训练数据是音乐数据，包括该训练数据的特征，其可以作为神经网络的输入；还包括该训练数据的可通透概率，其可以作为神经网络的输出。
示例性地，对于训练数据，其原始音乐波形为时域波形，可以对该时域波形进行分帧，随后对分帧后的每帧进行特征提取从而得到该训练数据的特征。可选地，作为一例，可以通过短时傅里叶变换(Short-Time Fourier Transform，STFT)进行特征提取，所提取的特征可以为梅尔频率倒谱系数(Mel Frequency Cepstrum Coefficient，MFCC)。应理解，本文对特征提取的方式仅是示意性的，并且也可以得到其他的特征，如幅度谱、对数谱、能量谱等，这里不再一一罗列。可选地，本发明实施例中，所提取的特征可以表示为特征张量的形式，例如表示为N维特征向量；或者，所提取的特征也可以表示为其他的形式，此处不作限定。
示例性地，可以参照如图1所示的方法得到训练数据的可通透概率，该过程包括：
S101，对训练数据进行通透处理，得到处理后的训练数据。
对于训练数据，其原始音乐波形为时域波形，可以对时域波形进行分帧并对每帧进行特征提取后得到频域特征。对其中的某些频点进行增强处理，对某些频点进行衰减处理，从而完成通透处理。随后可以将其还原至时域从而得到处理后的训练数据。
其中，某个频点f处的提升倍数可以表示为p(f)。可理解，进行通透处理的参数集合可以表示为p，包括各个频点处的提升倍数，p也可以被称为通透参数或通透增强参数等。
S102，获取一组评测者中每个评测者的打分。
由于并非所有的音乐都适合进行通透处理，并且通透效果取决于用户的主观感受，因此这里进行一个主观实验，评测者将通透处理后的音乐(即S101得到的处理后的训练数据)与未进行通透处理的音乐(即训练数据)进行比较，判断通透处理后的音乐的音质是否变得更好。也就是说，打分表示处理后的训练数据的音质在评测者的主观上是否优于训练数据的音质。
具体地，评测者可以听通透后的音乐(即S101得到的处理后的训练数据)和未通透的音乐(即训练数据)，根据通透后的音乐的音质变好还是变差进行评价打分。例如，如果评测者认为通透后的音乐的音质变好，则打分为1，否则打分为0。如此便可以得到一组评测者中所有评测者的打分。
如图2所示，测评者1至测评者7这7个测评者的打分依次为1、0、1、1、0、1、1。
将所有人对该音乐的打分进行平均形成评价值，这个评价值后称为“可通透概率”，这个值越大，说明该音乐越适合做通透处理。
S103，根据所有评测者的打分得到该训练数据的可通透概率。
示例性地，可以将S102得到的所有评测者的打分的均值确定为该可通透概率，也就是说，可以将所有打分中，“1”所占的比例定义为可通透概率。可理解，该可通透概率的取值范围为0至1。本发明实施例中，可以将多个评测者的打分的平均作为评价值(可通透概率)，可理解，该值越大，说明越适合进行通透处理。
如图2所示，可以通过计算平均5/7，得到可通透概率为71.4％。
如此，针对每一个训练数据，均可以通过特征提取得到特征，并参照图1-图2类似的过程得到可通透概率。将所提取的特征作为输入，并将可通透概率作为输出，对通透概率神经网络进行训练直到收敛，便可以得到训练好的通透概率神经网络。
本发明实施例还构建了可通透概率与通透增强参数之间的映射关系。
作为一种实现方式，该映射关系可以是预先设定好的。例如，将通透增强参数表示为P，将可通透概率表示为s，可以预先设定该映射关系为：
Figure PCTCN2019089756-appb-000001
其中，s0可以称为可通透概率阈值，其为0至1之间的某值，例如，s0＝0.5或0.6等，s0也可以为其他值，本发明对此不限定。可见，若可通透概率大于阈值，则对应的通透增强参数P＝p0，其中p0是一组已知的固定的参数集，其表示在至少一个频点处的提升倍数，不同频点处的提升倍数可以相等或不等，本发明对此不限定。若可通透概率小于或等于阈值，则对应的通透增强参数P＝0，即表示不进行通透处理。
作为另一种实现方式，可以通过最小可觉差(Just Noticeable Difference，JND)主观实验来确定该映射关系。
针对某未通透音乐，可以参照前述图1至图2的过程得到其可通透概率，表示为s。确定映射关系的过程可以包括：对可通透概率为s的未通透音乐进行多个通透处理，通透参数依次为：p+Δp*i(i＝0,1,2…)；获取多个通透处理对应的多个主观感受t(i)，其中t(i)是基于多个评测者对按照通透参数p+Δp*i处理后的音乐的音质与按照通透参数p+Δp*(i-1)处理后的音乐的音质进行比较打分得到的；根据t(i)的大小关系确定该映射关系。
该过程可以参照图3来实现，针对某未通透音乐，对其进行多个通透处理，通透参数分别为p、p+Δp、p+Δp*2、…、p+Δp*n、p+Δp*(n+1)。随后，可以按照相邻两个通透处理后的音乐的音质的比较从而得到对应的主观感受。
如图3中，通过比较按照通透参数p处理后的音乐的音质与未通透音乐的音质得到t(0)，通过比较按照通透参数p+Δp*i处理后的音乐的音质与按照通透参数p+Δp*(i-1)处理后的音乐的音质得到t(i)。以下为了描述的方便，将按照通透参数p+Δp*i处理后的音乐表示为YY(i)。具体地，由多个测评者听未通透音乐以及YY(0)并进行打分，根据打分的平均计算得到t(0)。由多个测评者听YY(i)以及YY(i-1)并进行打分，根据打分的平均计算得到t(i)。其中，若测评者认为YY(i)的音质优于YY(i-1)的音质，则打分为1，否则打分为0。
进一步地，可以根据t(i)的大小关系确定该映射关系。具体地，若满足t(n+1)<t(n)，且t(j+1)>t(j)，j＝0,1,…,n-1。则可以确定该映射关系中与可通透概率s所对应的通透增强参数P＝p+Δp*n。
针对大量的未通透音乐，均按照如图3所示的过程得到对应关系，这样便可以建立可通透概率与通透增强参数之间的映射关系。
其中，若不同的未通透音乐的可通透概率相等，则它们可能会得到不同的对应关系，此时可将得到的多个不同的通透增强参数作平均。举例来说，音乐1和音乐2的可通透概率均为s1。通过图3所示的过程，针对音乐1得到s1对应的通透增强参数P＝p+Δp*n1。通过图3所示的过程，针对音乐2得到s1对应的通透增强参数P＝p+Δp*n2。则在建立映射关系时，可以确定该映射关系中可通透概率s1对应于p+Δp*(n1+n2)/2。
比较上述两种不同的实现方式，可理解，通过JND主观实验确定映射关系需要耗费大量的人力，消耗更长的时间，然而该实现方式充分考虑了人的主观因素，从而得到的映射关系更能够反应真实的人的听觉感受。在实际应用中，可以结合多种因素考虑使用上述何种实现方式，如精度、人力成本等。
应注意，本文中所使用的“平均”是将多个项(或值)进行均值计算得到结果值。例如，上述实施例中计算的平均可以为算术平均。然而，可理解，“平均”也可以通过其他计算方式得到结果值，如加权平均，其中不同项的权重可以相等或不等，本发明实施例对平均的方式不作限定。
基于以上的描述，本发明实施例构建了通透概率神经网络以及可通透概率与通透增强参数之间的映射关系。可替代地，本发明实施例也可以提供一种通透增强神经网络，该通透增强神经网络的输入为音乐数据的特征，输出为通透增强参数，具体地为该通透增强神经网络推荐对音乐数据进行通透处理的通透增强参数。示例性地，通透增强神经网络可以基于训练数据集经过训练得到。训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及推荐的通透增强参数。针对每个训练数据，可以通过特征提取得到其特征。针对每个训练数据，可以参照前述图1至图3的相关描述，得到通透增强参数。从而可以将训练数据的特征作为输入，将训练数据的通透增强参数作为输出，通过训练直到收敛得到训练好的通透增强神经网络。
作为另一种理解，可以认为该通透增强神经网络具有中间参数：可通透概率。也就是说，该通透增强神经网络可以基于输入的音乐数据的特征得到可通透概率，再根据该可通透概率得到通透增强参数作为该通透增强神经网络的输出。具体地，该过程可以参见前述的通透概率神经网络以及可通透概率与通透增强参数之间的映射关系，这里不再赘述。
本发明实施例提供了一种对音乐进行通透处理的方法，如图4所示为该 方法的流程图，包括：
S210，获取待播放音乐的特征；
S220，将所述特征输入至通透增强神经网络，得到通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
示例性地，通透增强神经网络可以具有中间变量，该中间变量为可通透概率。例如，可以基于前述的通透概率神经网络得到可通透概率，并根据可通透概率得到通透增强参数。
示例性地，在S220之前，还可以包括：基于训练数据集，通过训练得到所述通透增强神经网络，其中所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及推荐的通透增强参数。
其中，可以通过以下方式得到所述训练数据的特征：获取所述训练数据的时域波形；对所述时域波形进行分帧；对所述分帧后的每帧进行特征提取得到所述训练数据的所述特征。
其中，可以通过以下方式得到所述训练数据的通透增强参数：对所述训练数据进行通透处理，得到处理后的训练数据；获取一组评测者中每个评测者的打分，所述打分表示所述处理后的训练数据的音质在所述评测者的主观上是否优于所述训练数据的音质；根据所有评测者的打分得到所述训练数据的可通透概率；根据预先构建的可通透概率与通透增强参数之间的映射关系，确定与可通透概率对应的通透增强参数。
可选地，映射关系可以被预先设定为：若所述可通透概率大于阈值，则所述通透增强参数为p0。
可选地，可以通过以下方式确定所述映射关系：对可通透概率为s的未通透音乐进行多个通透处理，通透参数依次为：p+Δp*i，i＝0,1,2…；获取所述多个通透处理对应的多个主观感受t(i)，其中t(i)是基于多个评测者对按照通透参数p+Δp*i处理后的音乐的音质与按照通透参数p+Δp*(i-1)处理后的音乐的音质进行比较打分得到的；若满足t(n+1)<t(n)，且t(j+1)>t(j)，j＝0,1,…,n-1，则确定所述映射关系中与可通透概率s所对应的通透增强参数为p+Δp*n。
作为一种实现方式，通透增强神经网络可以包括通透概率神经网络以及可通透概率与通透增强参数之间的映射关系，相应地，S220可以包括：将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概率，并 基于可通透概率与通透增强参数之间的映射关系，得到与可通透概率所对应的通透增强参数。
本发明实施例所提供的另一种对音乐进行通透处理的方法的流程图可以如图5所示，其包括：
S210，获取待播放音乐的特征；
S2201，将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概率；
S2202，确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
S2201中的通透概率神经网络可以是前述的训练好的通透概率神经网络，可理解，前述的训练过程一般在服务器端(即云端)执行。
S210可以包括通过特征提取得到待播放音乐的特征。或者，S210可以包括从对端接收待播放音乐的特征。其中，若图4或图5的过程由服务器端执行，则对端为客户端；若图4或图5的过程由客户端执行，则对端为服务器端。
也就是说，图4或图5所示的流程可以在服务器端(即云端)执行，也可以在客户端(如客户端应用程序)执行，下面将结合图5对这两种情形分别进行描述。
服务器端执行：
作为一个示例，待播放音乐是用户的客户端本地音乐。
S210可以包括：从客户端接收该待播放音乐，获取该待播放音乐的时域波形，对时域波形进行分帧并对每帧进行特征提取得到其特征。
或者，S210可以包括：从客户端接收待播放音乐的音乐信息，这里的音乐信息可以包括歌名、歌手、专辑等中的至少一项。根据该音乐信息从服务器端的音乐数据库中获取该待播放音乐，通过对该待播放音乐的时域波形进行分帧并对每帧进行特征提取得到其特征。
或者，S210可以包括：从客户端接收待播放音乐的特征。例如，客户端可以对该待播放音乐的时域波形进行分帧并对每帧进行特征提取得到其特征，随后客户端将所得到的特征发送至服务器端。
可见，S210中的特征是通过特征提取得到的，其中特征提取的过程可以在服务器端或客户端执行。
示例性地，S2202中，可以基于前述的映射关系，得到与S2201的可通透概率所对应的通透增强参数。
进一步地，可理解，在S2202之后，服务器端可以将该通透增强参数发送至客户端，以便客户端根据该通透增强参数对其本地的待播放音乐进行通透处理。这样可以在客户端对通透处理后的音乐进行本地播放。
作为另一个示例，用户在线播放待播放音乐，即该待播放音乐存储在服务器端，例如可以存储在服务器端的音乐数据库中。
S210可以包括：从客户端接收待播放音乐的音乐信息，这里的音乐信息可以包括歌名、歌手、专辑等中的至少一项。根据该音乐信息从服务器端的音乐数据库中获取该待播放音乐，通过对该待播放音乐的时域波形进行分帧并对每帧进行特征提取得到其特征。
示例性地，S2202，可以基于前述的映射关系，得到与S2201的可通透概率所对应的通透增强参数。
进一步地，可理解，在S2202之后，服务器端可以根据该通透增强参数对该待播放音乐进行通透处理。这样便可以对通透处理后的音乐进行在线播放。
客户端执行：
可选地，客户端可以为智能手机、平板电脑、可穿戴设备等移动终端。
S210可以包括：若待播放音乐为本地音乐，则客户端可以对该待播放音乐的时域波形进行分帧并对每帧进行特征提取得到其特征。若待播放音乐为存储在服务器端的音乐，则客户端可以向服务器端发送待播放音乐的音乐信息，这里的音乐信息可以包括歌名、歌手、专辑等中的至少一项，并随后从服务器端接收该待播放音乐，之后客户端可以对该待播放音乐的时域波形进行分帧并对每帧进行特征提取得到其特征。或者，若待播放音乐为存储在服务器端的音乐，则客户端可以向服务器端发送待播放音乐的音乐信息，并随后从服务器端接收待播放音乐的特征。其中，服务器端可以根据音乐信息，从音乐数据库中获取该待播放音乐，对该待播放音乐的时域波形进行分帧并对每帧进行特征提取得到其特征，随后服务器端将所得到的特征发送至客户端。可见，S210中的特征是通过特征提取得到的，其中特征提取的过程可以在服务器端或客户端执行。
可理解，本发明实施例中所述的音乐信息仅仅是示例性的，其可以包括 其他信息，诸如时长、格式等，这里不再一一罗列。
在图5所示的过程之前，客户端可以从服务器端获取训练好的通透概率神经网络，从而在S2201中，客户端可以使用存储在其本地的训练好的通透概率神经网络，得到待播放音乐的可通透概率。
类似地，作为一例，前述的映射关系可以是在服务器端进行确定的，在图5所示的过程之前，客户端可以从服务器端获取映射关系。作为另一例，前述的映射关系可以是直接预先存储在客户端中的，如前述的预先设定的映射关系的实现方式。进而在S2202中，客户端可以基于该映射关系，得到与S2201的可通透概率所对应的通透增强参数。
可理解，进一步地，在S2202之后，客户端可以根据该通透增强参数对其本地的待播放音乐进行通透处理。这样可以在客户端对通透处理后的音乐进行本地播放。
由此可见，本发明实施例中可以基于深度学习预先构建通透概率神经网络，从而可以对待播放音乐自动地进行通透处理。该过程极大地简化用户的操作的同时，保证音乐的音质，从而提升了用户体验。
图6是本发明实施例的对音乐进行通透处理的设备的一个示意性框图。图6所示的设备30包括获取模块310和确定模块320。
获取模块310用于获取待播放音乐的特征。
确定模块320用于将所述特征输入至通透增强神经网络，得到通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
作为一种实现方式，图6所示的设备30可以为服务器端(即云端)。可选地，该设备30还可以包括训练模块，用于基于训练数据集，通过训练得到所述通透增强神经网络，其中所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及推荐的通透增强参数。
示例性地，该通透增强神经网络可以具有中间变量为可通透概率。
图7是本发明实施例的对音乐进行通透处理的设备的另一个示意性框图。图7所示的设备30包括获取模块310、可通透概率确定模块3201和通透增强参数确定模块3202。
获取模块310用于获取待播放音乐的特征。
可通透概率确定模块3201用于将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概率。
通透增强参数确定模块3202用于确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
图7所示的设备30可以为服务器端(即云端)。可选地，该设备30还可以包括训练模块，用于基于训练数据集，通过训练得到所述通透概率神经网络。
示例性地，所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及可通透概率。
其中，可以通过以下方式得到所述训练数据的特征：获取所述训练数据的时域波形；对所述时域波形进行分帧；对所述分帧后的每帧进行特征提取得到所述训练数据的所述特征。
其中，可以通过以下方式得到所述训练数据的可通透概率：对所述训练数据进行通透处理，得到处理后的训练数据；获取一组评测者中每个评测者的打分，所述打分表示所述处理后的训练数据的音质在所述评测者的主观上是否优于所述训练数据的音质；根据所有评测者的打分得到所述训练数据的所述可通透概率。例如，可以将所述所有评测者的打分的均值确定为所述训练数据的所述可通透概率。
关于训练模块训练得到通透概率神经网络可以参见前述结合图1和图2部分的实施例的相关描述，为避免重复，这里不再赘述。
作为一种实现方式，通透增强参数确定模块3202可以具体用于：根据预先构建的可通透概率与通透增强参数之间的映射关系，确定与所述可通透概率对应的所述通透增强参数。
作为一例，映射关系可以被预先设定为：若所述可通透概率大于阈值，则所述通透增强参数为p0。
作为另一例，可以通过以下方式确定所述映射关系：对可通透概率为s的未通透音乐进行多个通透处理，通透参数依次为：p+Δp*i，i＝0,1,2…；获取所述多个通透处理对应的多个主观感受t(i)，其中t(i)是基于多个评测者对按照通透参数p+Δp*i处理后的音乐的音质与按照通透参数p+Δp*(i-1)处理后的音乐的音质进行比较打分得到的；根据t(i)的大小关系确定所述映射关系。例如，若满足t(n+1)<t(n)，且t(j+1)>t(j)，j＝0,1,…,n-1，则确定所述映射关系中与可通透概率s所对应的通透增强参数为p+Δp*n。该过程可以参见前述结合图3部分的实施例的相关描述，为避免重复，这里不再赘述。
作为一种实现方式，图6或图7所示的设备30可以为服务器端(即云端)。该设备30还可以包括发送模块，用于将通透增强参数发送至客户端。进而客户端可以基于该通透增强参数对待播放音乐进行通透处理；播放通透处理后的音乐。
作为一种实现方式，图6或图7所示的设备30可以为客户端。该设备30还可以包括通透处理模块和播放模块。该通透处理模块用于基于该通透增强参数对待播放音乐进行通透处理，该播放模块用于播放通透处理后的音乐。
图6或图7所示的设备30能够用于实现前述图4或图5所示的对音乐进行通透处理的方法，为避免重复，这里不再赘述。
如图8所示，本发明实施例还提供了另一种对音乐进行通透处理的设备，包括存储器、处理器及存储在所述存储器上且在所述处理器上运行的计算机程序，处理器执行所述程序时实现前述图4或图5所示的方法的步骤。
具体地，处理器可以获取待播放音乐的特征；将所述特征输入至通透增强神经网络，得到通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。作为一种实现方式，处理器可以获取待播放音乐的特征；将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概率；确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
示例性地，本发明实施例中的对音乐进行通透处理的设备可以包括：一个或多个处理器、一个或多个存储器、输入装置以及输出装置，这些组件通过总线系统和/或其它形式的连接机构互连。应当注意，该设备根据需要也可以具有其他组件和结构。
所述处理器可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元，并且可以控制所述设备中的其它组件以执行期望的功能。
所述存储器可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多 个计算机程序指令，处理器可以运行所述程序指令，以实现下文所述的本发明实施例中(由处理器实现)的客户端功能以及/或者其它期望的功能。在所述计算机可读存储介质中还可以存储各种应用程序和各种数据，例如所述应用程序使用和/或产生的各种数据等。
所述输入装置可以是用户用来输入指令的装置，并且可以包括键盘、鼠标、麦克风和触摸屏等中的一个或多个。
所述输出装置可以向外部(例如用户)输出各种信息(例如图像或声音)，并且可以包括显示器、扬声器等中的一个或多个。
另外，本发明实施例还提供了一种计算机存储介质，其上存储有计算机程序。当所述计算机程序由处理器执行时，可以实现前述图4或图5所示的方法的步骤。例如，该计算机存储介质为计算机可读存储介质。
由此可见，本发明实施例中可以构建通透增强神经网络，具体地基于深度学习预先构建通透概率神经网络并且构建可通透概率与通透增强参数之间的映射关系，从而可以对待播放音乐自动地进行通透处理。该过程极大地简化用户的操作的同时，保证音乐的音质，从而提升了用户体验。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作 为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (17)

  1. 一种对音乐进行通透处理的方法，其特征在于，包括：
    获取待播放音乐的特征；
    将所述特征输入至通透概率神经网络，得到所述待播放音乐的可通透概率；
    确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
  2. 根据权利要求1所述的方法，其特征在于，在所述将所述特征输入至通透概率神经网络之前，还包括：
    基于训练数据集，通过训练得到所述通透概率神经网络。
  3. 根据权利要求2所述的方法，其特征在于，所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及可通透概率。
  4. 根据权利要求3所述的方法，其特征在于，通过以下方式得到所述训练数据的特征：
    获取所述训练数据的时域波形；
    对所述时域波形进行分帧；
    对所述分帧后的每帧进行特征提取得到所述训练数据的所述特征。
  5. 根据权利要求3所述的方法，其特征在于，通过以下方式得到所述训练数据的可通透概率：
    对所述训练数据进行通透处理，得到处理后的训练数据；
    获取一组评测者中每个评测者的打分，所述打分表示所述处理后的训练数据的音质在所述评测者的主观上是否优于所述训练数据的音质；
    根据所有评测者的打分得到所述训练数据的所述可通透概率。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所有评测者的打分得到所述训练数据的所述可通透概率，包括：
    将所述所有评测者的打分的均值确定为所述训练数据的所述可通透概率。
  7. 根据权利要求1所述的方法，其特征在于，所述确定与所述可通透概率对应的通透增强参数，包括：
    根据预先构建的可通透概率与通透增强参数之间的映射关系，确定与所述可通透概率对应的所述通透增强参数。
  8. 根据权利要求7所述的方法，其特征在于，所述映射关系被预先设定为：
    若所述可通透概率大于阈值，则所述通透增强参数为p0。
  9. 根据权利要求7所述的方法，其特征在于，通过以下方式确定所述映射关系：
    对可通透概率为s的未通透音乐进行多个通透处理，通透参数依次为：p+Δp*i，i＝0,1,2…；
    获取所述多个通透处理对应的多个主观感受t(i)，其中t(i)是基于多个评测者对按照通透参数p+Δp*i处理后的音乐的音质与按照通透参数p+Δp*(i-1)处理后的音乐的音质进行比较打分得到的；
    根据t(i)的大小关系确定所述映射关系。
  10. 根据权利要求9所述的方法，其特征在于，所述根据t(i)的大小关系确定所述映射关系，包括：
    若满足t(n+1)<t(n)，且t(j+1)>t(j)，j＝0,1,…,n-1，则确定所述映射关系中与可通透概率s所对应的通透增强参数为p+Δp*n。
  11. 根据权利要求1至10中任一项所述的方法，其特征在于，还包括：
    基于所述通透增强参数对所述待播放音乐进行通透处理；
    播放所述通透处理后的音乐。
  12. 一种对音乐进行通透处理的方法，其特征在于，包括：
    获取待播放音乐的特征；
    将所述特征输入至通透增强神经网络，得到通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
  13. 根据权利要求12所述的方法，其特征在于，在将所述特征输入至通透增强神经网络之前，还包括：
    基于训练数据集，通过训练得到所述通透增强神经网络，其中所述训练数据集中的每个训练数据均为音乐数据，且每个训练数据均具有特征以及推荐的通透增强参数。
  14. 一种对音乐进行通透处理的设备，其特征在于，所述设备用于实现前述权利要求1至11中任一项所述的方法，所述设备包括：
    获取模块，用于获取待播放音乐的特征；
    可通透概率确定模块，用于将所述特征输入至通透概率神经网络，得到 所述待播放音乐的可通透概率；
    通透增强参数确定模块，用于确定与所述可通透概率对应的通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
  15. 一种对音乐进行通透处理的设备，其特征在于，所述设备用于实现前述权利要求12或13所述的方法，所述设备包括：
    获取模块，用于获取待播放音乐的特征；
    确定模块，用于将所述特征输入至通透增强神经网络，得到通透增强参数，所述通透增强参数用于对所述待播放音乐进行通透处理。
  16. 一种对音乐进行通透处理的设备，包括存储器、处理器及存储在所述存储器上且在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至13中任一项所述方法的步骤。
  17. 一种计算机存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至13中任一项所述方法的步骤。
PCT/CN2019/089756 2018-06-05 2019-06-03 对音乐进行通透处理的方法及设备 WO2019233359A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/059,158 US11887615B2 (en) 2018-06-05 2019-06-03 Method and device for transparent processing of music

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810583109.0A CN109119089B (zh) 2018-06-05 2018-06-05 对音乐进行通透处理的方法及设备
CN201810583109.0 2018-06-05

Publications (1)

Publication Number Publication Date
WO2019233359A1 true WO2019233359A1 (zh) 2019-12-12

Family

ID=64821872

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/089756 WO2019233359A1 (zh) 2018-06-05 2019-06-03 对音乐进行通透处理的方法及设备

Country Status (3)

Country Link
US (1) US11887615B2 (zh)
CN (2) CN109119089B (zh)
WO (1) WO2019233359A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109119089B (zh) * 2018-06-05 2021-07-27 安克创新科技股份有限公司 对音乐进行通透处理的方法及设备
US12001950B2 (en) 2019-03-12 2024-06-04 International Business Machines Corporation Generative adversarial network based audio restoration

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130297539A1 (en) * 2012-05-07 2013-11-07 Filip Piekniewski Spiking neural network object recognition apparatus and methods
CN107329996A (zh) * 2017-06-08 2017-11-07 三峡大学 一种基于模糊神经网络的聊天机器人系统与聊天方法
CN107888843A (zh) * 2017-10-13 2018-04-06 深圳市迅雷网络技术有限公司 用户原创内容的混音方法、装置、存储介质及终端设备
CN108022591A (zh) * 2017-12-30 2018-05-11 北京百度网讯科技有限公司 车内环境中语音识别的处理方法、装置和电子设备
CN109119089A (zh) * 2018-06-05 2019-01-01 安克创新科技股份有限公司 对音乐进行通透处理的方法及设备

Family Cites Families (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000019027A (ja) * 1998-07-01 2000-01-21 Kazuhiro Muroi 風呂用温度状況報知器
CN2523161Y (zh) * 2001-11-27 2002-11-27 黄瑞书 改进的耳机
CN1264381C (zh) * 2001-11-27 2006-07-12 黄瑞书 改进的耳机
FR2841355B1 (fr) * 2002-06-24 2008-12-19 Airbus France Procede et dispositif pour elaborer une forme abregee d'un terme quelconque qui est utilise dans un message d'alarme destine a etre affiche sur un ecran du poste de pilotage d'un aeronef
US7555715B2 (en) * 2005-10-25 2009-06-30 Sonic Solutions Methods and systems for use in maintaining media data quality upon conversion to a different data format
JP2007266800A (ja) * 2006-03-28 2007-10-11 Hitachi Ltd 情報再生装置
US7307207B1 (en) * 2006-10-10 2007-12-11 Davis Gregg R Music page turning apparatus
JP2009055541A (ja) * 2007-08-29 2009-03-12 Canon Inc 動画再生装置
US8467893B2 (en) * 2008-01-14 2013-06-18 Telefonaktiebolaget Lm Ericsson (Publ) Objective measurement of audio quality
US20090238371A1 (en) * 2008-03-20 2009-09-24 Francis Rumsey System, devices and methods for predicting the perceived spatial quality of sound processing and reproducing equipment
US20140081682A1 (en) * 2009-09-09 2014-03-20 Dopa Music Ltd. Method for providing background music
CN102034472A (zh) * 2009-09-28 2011-04-27 戴红霞 一种基于嵌入时延神经网络的高斯混合模型的说话人识别方法
WO2012036912A1 (en) * 2010-09-03 2012-03-22 Trustees Of Princeton University Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers
US8406449B2 (en) * 2010-09-28 2013-03-26 Trash Amps LLC Portable audio amplifier with interchangeable housing and storage compartment
CN102610236A (zh) * 2012-02-29 2012-07-25 山东大学 一种改善喉振话筒语音音质的方法
CN104078050A (zh) * 2013-03-26 2014-10-01 杜比实验室特许公司 用于音频分类和音频处理的设备和方法
CN103489033A (zh) * 2013-09-27 2014-01-01 南京理工大学 融合自组织映射与概率神经网络的增量式学习方法
CN104751842B (zh) * 2013-12-31 2019-11-15 科大讯飞股份有限公司 深度神经网络的优化方法及系统
US10055411B2 (en) * 2015-10-30 2018-08-21 International Business Machines Corporation Music recommendation engine
US10068556B2 (en) * 2015-11-18 2018-09-04 Pandora Media, Inc. Procedurally generating background music for sponsored audio
CN105931658A (zh) * 2016-04-22 2016-09-07 成都涂鸦科技有限公司 一种自适应场景的音乐播放方法
CN105869611B (zh) * 2016-06-03 2022-11-15 陈世江 弦乐器音质训化装置
CN205666052U (zh) * 2016-06-03 2016-10-26 陈世江 弦乐器音质训化装置
US9584946B1 (en) * 2016-06-10 2017-02-28 Philip Scott Lyren Audio diarization system that segments audio input
CN106782603B (zh) * 2016-12-22 2020-08-11 云知声（上海）智能科技有限公司 智能语音评测方法及系统
CN107126615A (zh) * 2017-04-20 2017-09-05 重庆邮电大学 基于脑电信号的音乐诱导睡眠方法及系统
CN107886967B (zh) * 2017-11-18 2018-11-13 中国人民解放军陆军工程大学 一种深度双向门递归神经网络的骨导语音增强方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130297539A1 (en) * 2012-05-07 2013-11-07 Filip Piekniewski Spiking neural network object recognition apparatus and methods
CN107329996A (zh) * 2017-06-08 2017-11-07 三峡大学 一种基于模糊神经网络的聊天机器人系统与聊天方法
CN107888843A (zh) * 2017-10-13 2018-04-06 深圳市迅雷网络技术有限公司 用户原创内容的混音方法、装置、存储介质及终端设备
CN108022591A (zh) * 2017-12-30 2018-05-11 北京百度网讯科技有限公司 车内环境中语音识别的处理方法、装置和电子设备
CN109119089A (zh) * 2018-06-05 2019-01-01 安克创新科技股份有限公司 对音乐进行通透处理的方法及设备

Also Published As

Publication number Publication date
US20210217429A1 (en) 2021-07-15
CN113450811A (zh) 2021-09-28
CN113450811B (zh) 2024-02-06
US11887615B2 (en) 2024-01-30
CN109119089A (zh) 2019-01-01
CN109119089B (zh) 2021-07-27

Similar Documents

Publication Publication Date Title
US11875807B2 (en) Deep learning-based audio equalization
US11456005B2 (en) Audio-visual speech separation
Zhao et al. A deep learning based segregation algorithm to increase speech intelligibility for hearing-impaired listeners in reverberant-noisy conditions
CN104768049B (zh) 一种用于同步音频数据和视频数据的方法、系统及计算机可读存储介质
WO2019233361A1 (zh) 对音乐进行音量调节的方法及设备
CN113921022B (zh) 音频信号分离方法、装置、存储介质和电子设备
CN112967732B (zh) 调整均衡器的方法、装置、设备和计算机可读存储介质
JP2016061968A (ja) 音声処理装置、音声処理方法およびプログラム
WO2019233359A1 (zh) 对音乐进行通透处理的方法及设备
CN113178204A (zh) 一种单通道降噪的低功耗方法、装置及存储介质
US20160196828A1 (en) Acoustic Matching and Splicing of Sound Tracks
Kon et al. Estimation of late reverberation characteristics from a single two-dimensional environmental image using convolutional neural networks
CN111402922B (zh) 基于小样本的音频信号分类方法、装置、设备及存储介质
CN107545898B (zh) 一种区分说话人语音的处理方法及装置
WO2017177629A1 (zh) 远讲语音识别方法及装置
CN113327594A (zh) 语音识别模型训练方法、装置、设备及存储介质
JP6681264B2 (ja) 音声加工装置、及びプログラム
CN113395577A (zh) 变声播放方法和装置、存储介质及电子设备
US20230343312A1 (en) Music Enhancement Systems
US20240353993A1 (en) Emotion estimation method, content determination method, recording medium, emotion estimation system, and content determination system
Araújo et al. Genetic algorithm to estimate the input parameters of Klatt and HLSyn formant-based speech synthesizers
CN119181372A (zh) 语音增强方法、装置及电子设备
WO2024010501A1 (en) Adjusting an audio experience for a user
CN116758897A (zh) 方言语音指令的识别方法、装置及电子设备
CN118571268A (zh) 基于网络模型的自动调音方法、装置、设备和计算机介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19814589

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19814589

Country of ref document: EP

Kind code of ref document: A1