CN111724812A - Audio processing method, storage medium and music practice terminal - Google Patents
Audio processing method, storage medium and music practice terminal Download PDFInfo
- Publication number
- CN111724812A CN111724812A CN201910222788.3A CN201910222788A CN111724812A CN 111724812 A CN111724812 A CN 111724812A CN 201910222788 A CN201910222788 A CN 201910222788A CN 111724812 A CN111724812 A CN 111724812A
- Authority
- CN
- China
- Prior art keywords
- user
- preset
- music
- information
- pitch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/90—Pitch determination of speech signals
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/076—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of timing, tempo; Beat detection
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/081—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for automatic key or tonality recognition, e.g. using musical rules or a knowledge base
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/091—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/90—Pitch determination of speech signals
- G10L2025/906—Pitch tracking
Abstract
The invention provides an audio processing method, a storage medium and a music practice terminal, wherein the audio processing method comprises the following steps: acquiring user audio information acquired by a microphone; extracting music features from the user audio information; comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information; and feeding back deviation information to a user. The invention can enable the user to obtain the deviation information between the user audio information of the performance and the preset audio information, so as to carry out targeted exercise according to the deviation information, thereby being beneficial to rapidly improving the exercise effect of the user.
Description
Technical Field
The invention relates to mechanical technology, in particular to an audio processing method, a storage medium and a music practice terminal.
Background
When a music learner exercises songs in the past, karaoke and other karaoke devices are mostly used for selecting vocal accompaniment patterns of the exercise songs for exercise. The general song accompaniment mode only plays the musical performance part of the song and is matched with the lyric caption displayed on the screen for the learner to sing. The song accompaniment mode needs to be searched by a learner, is difficult to correctly master the tone or rhythm of the song, is easy to cause the conditions of inaccurate pitch, robbery shooting or dragging shooting, causes the trouble of the learner and is not suitable for the music learner to practice singing. If the professional lecturer teaches the singing skill in the professional singing training course, an expensive learning fee is required, and the threshold of music learning is improved.
Disclosure of Invention
The present invention is directed to solve at least one of the above technical drawbacks, and in particular, to provide an audio processing method, a storage medium and a music practice terminal that can improve the efficiency of music practice.
The audio processing method is applied to the music practice terminal and comprises the following steps:
acquiring user audio information acquired by a microphone;
extracting music features from the user audio information;
comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information;
and feeding back deviation information to a user.
Preferably, the musical features include pitch and tempo; the music features are extracted from the user audio information; and comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information, wherein the deviation information comprises the following steps:
extracting pitches from the user audio information according to a preset pitch acquisition period; comparing the extracted pitch with a preset pitch of a song corresponding to the user audio information to obtain deviation information of the pitch;
extracting beats from the user audio information according to a preset beat acquisition cycle; and comparing the extracted beat with a preset beat of a music corresponding to the user audio information to obtain beat deviation information.
Preferably, before acquiring the user audio information collected by the microphone, the method further includes:
and acquiring a prompt instruction for prompting the preset music characteristic, and sending the preset music characteristic to a music characteristic prompt module.
Preferably, after comparing the extracted pitch with a preset pitch of a song corresponding to the user audio information to obtain deviation information of the pitch, the method further includes:
and if the pitch deviation information in the preset pitch acquisition periods is larger than a preset pitch deviation threshold value, controlling the music characteristic prompting module to increase the volume of playing the preset original sound.
Preferably, after comparing the extracted beat with a preset beat of a song corresponding to the user audio information to obtain deviation information of the beat, the method further includes:
and if the deviation information of the beats in a plurality of preset beat acquisition periods is greater than a preset beat deviation threshold value, controlling the music characteristic prompting module to increase the volume of the metronome.
Preferably, the feeding back the deviation information to the user includes:
recording the times that the pitch deviation information in the preset pitch acquisition period is greater than a preset pitch deviation threshold value, and if the times exceed a first preset time, prompting a user to repeatedly exercise or recommending a similar song of a song corresponding to the user audio information to the user; or the like, or, alternatively,
recording the times that the deviation information of the beats in a plurality of preset beat acquisition periods is greater than a preset beat deviation threshold value, and prompting the user to repeatedly exercise or recommend the same type of tracks of the tracks corresponding to the audio information of the user to the user if the times exceed a second preset time.
Preferably, the feeding back the deviation information to the user includes:
obtaining an exercise segment with the maximum pitch deviation according to the pitch deviation information, and obtaining an exercise segment with the maximum beat deviation according to the beat deviation information;
determining key exercise segments according to the exercise segments with the maximum pitch deviation and the exercise segments with the maximum beat deviation;
intercepting the key exercise segment and feeding back the key exercise segment to a user.
Preferably, the musical features include a tone and a range; the extracting music features from the user audio information comprises: extracting musical features of tone and gamut from the user audio information;
after comparing the extracted music features with preset music features of the music corresponding to the user audio information and obtaining deviation information, the method further comprises the following steps:
obtaining the likelihood between the user audio information and the corresponding song according to the deviation information of the pitch and the deviation information of the beat;
and recommending the matched tracks to the user according to the likelihood, the tone and the tone domain.
The invention also proposes a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the audio processing method of any of the preceding claims.
The present invention also provides a music practice terminal, including:
a microphone, one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the audio processing method of any of the preceding claims.
The invention has the following beneficial effects:
1. the invention can enable the user to obtain the deviation information between the user audio information of the performance and the preset audio information, so as to carry out targeted exercise according to the deviation information, thereby being beneficial to rapidly improving the exercise effect of the user.
2. The preset music characteristics can be sent to the music characteristic prompt module before the user sings or plays so that the user can be familiar with the music characteristics such as pitch, beat and the like of the music to be singed or played in advance, or the reference of the music characteristics such as pitch, beat and the like is provided for the user when the user sings or plays, so that the user can improve the exercise accuracy according to the prompt and further improve the exercise effect.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic perspective view of an embodiment of the music practice apparatus of the present invention;
FIG. 2 is a schematic perspective view of another embodiment of the music practice apparatus of the present invention;
FIG. 3 is a schematic view of a portion of a control panel of an embodiment of the music exercise device of the present invention;
FIG. 4 is a flowchart illustrating an audio processing method according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating an audio processing method according to another embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative only and should not be construed as limiting the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, elements, and/or components, but do not preclude the presence or addition of one or more other features, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
The present invention provides an audio processing method, which can be applied to the music practice terminal of the embodiment shown in fig. 1 or fig. 2. When the user uses the music practice terminal, the user can sing through the microphone 2 on the headset 1 shown in fig. 1, or the user can sing through the microphone 2 shown in fig. 2 or play through a musical instrument, so that the music practice terminal can obtain the user audio information of the user singing or playing; the music practice terminal compares the acquired user audio information with the standard song information prestored in the music practice terminal or on the cloud server, so that the deviation between the user audio information and the standard song information can be acquired, the user can know the feedback information of singing or playing, and the targeted practice can be performed repeatedly by the user. Wherein the display means 3 is operable to feed back said deviation information to the user, and to display information relating to the music exercise; when the display device 3 is a touch display screen, the user can also input a touch instruction through the display device 3. The camera device 4 can be used for recording singing or playing process of the user and acquiring facial information of the user to realize functions of face identification, head height identification and the like.
In an embodiment of the audio processing method of the present invention, as shown in fig. 4, the method includes the following steps:
step S10: acquiring user audio information acquired by a microphone;
step S20: extracting music features from the user audio information;
step S30: comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information;
step S40: and feeding back deviation information to a user.
Wherein each step is as follows:
step S10: and acquiring user audio information collected by a microphone.
The audio information of the user can be audio information of singing, lines and the like of the user, and can also be audio information of a musical instrument played by the user and the like. In some circumstances, the user audio information collected by the microphone may include environmental noise, background music, and sounds other than the user's sounds, so the step of filtering the original audio collected by the microphone may be further included in this step to obtain more accurate user audio information. When the music practice terminal of the present invention plays the background audio information or prompts the audio information through the audio playing device, the audio information played outside may interfere the microphone to collect the user audio information, and therefore, in some embodiments of the present invention, the acquiring the user audio information collected by the microphone may include:
the method comprises the steps of acquiring original audio information collected by a microphone, and identifying user audio information from the original audio information.
When the user audio information is identified from the original audio information, the original sound of the reference track can be obtained from the track selected by the user for performance, and then the original sound is filtered from the original audio information, so that the interference of the original sound on the user audio information is reduced. In some music practice devices, it is preferable to be equipped with earphones, so that the interference caused by the audio information played outside can be reduced, and the user can obtain clearer background sound and prompt sound through the earphones, thereby enhancing the immersion of the user and being beneficial to improving the concentration degree of the user in practice.
The time period for acquiring the audio information of the user acquired by the microphone can be the time for the user to perform a complete song, and the whole song can also be divided into a plurality of time periods, so that when the user performs the performance of the current time period, the feedback result of the previous time period can be acquired in time, and the performance deviation can be corrected quickly by the user.
Step S20: music features are extracted from the user audio information.
The musical features may include pitch, intensity, duration, timbre, tempo, melody, register of the user, and so forth. The music characteristics such as pitch, tone intensity, duration, timbre can be extracted from the physical characteristics of user audio information, also can be with user audio information converts time domain signal or frequency domain signal, follow again time domain signal or frequency domain signal extract melody and extract. Since the processing of the time domain signal or the frequency domain signal of the audio has certain requirements on the performance of the device or the connection speed with the server, in some embodiments of the present invention, only the music features of pitch and beat can be extracted to speed up the extraction of the music features from the user audio information.
In some embodiments of the present invention, different music features may be extracted from the user audio information at different time points. For example, the whole song is divided into a plurality of time periods, so that when the next time period is entered, the music characteristics of the pitch of the user are extracted from the acquired user audio information of the previous time period, and the music characteristics are fed back to the user quickly; and when the whole song sings, extracting the music characteristics of the beat and the range of the user from the user audio information of the whole song sung by the user so as to provide a more accurate comparison analysis result for the user. Therefore, the time point for feeding back the deviation information to the user can be in the process of singing the song or after the whole song is singed.
Step S30: and comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information.
The song corresponding to the user audio information may be a song or a musical sound played by the user's choosing, or a song or a musical sound determined by the system. Some of the predetermined musical features in the original sound of the song may be known musical features, such as tempo, determined at the completion of composition; part of the preset music characteristics can be analyzed and extracted in advance and stored in association with the corresponding song so as to be called when comparison is carried out, for example, the tone and the range and the like.
In some embodiments, the musical characteristic comprises pitch; the music features are extracted from the user audio information; and comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information, wherein the deviation information comprises the following steps:
extracting pitches from the user audio information according to a preset pitch acquisition period; and comparing the extracted pitch with a preset pitch of the music corresponding to the user audio information to obtain deviation information of the pitch.
This embodiment only follows extract the pitch in the user's audio information to obtain the deviation information of pitch, can obtain the deviation information of pitch fast in singing or performance process, can reduce the feedback speed of deviation information greatly, be favorable to the user to revise the pitch deviation fast in the exercise process, pointed improvement exercise efficiency.
In other embodiments of the present invention, the musical characteristic may include a tempo; the music features are extracted from the user audio information; and comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information, wherein the deviation information comprises the following steps:
extracting beats from the user audio information according to a preset beat acquisition cycle; and comparing the extracted beat with a preset beat of a music corresponding to the user audio information to obtain beat deviation information.
Similarly, the beat is only extracted from the user audio information in the embodiment to obtain the deviation information of the beat, the deviation information of the beat can be quickly obtained in the singing or playing process, the feedback speed of the deviation information is greatly reduced, the beat deviation can be quickly corrected in the practice process by the user, and the practice efficiency is pertinently improved.
In some embodiments of the present invention, the musical features may also include pitch and beat; the music features are extracted from the user audio information; and comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information, wherein the deviation information comprises the following steps:
extracting pitches from the user audio information according to a preset pitch acquisition period; comparing the extracted pitch with a preset pitch of a song corresponding to the user audio information to obtain deviation information of the pitch;
extracting beats from the user audio information according to a preset beat acquisition cycle; and comparing the extracted beat with a preset beat of a music corresponding to the user audio information to obtain beat deviation information.
The embodiment can simultaneously feed back the pitch deviation information and the beat deviation information to the user, and can also feed back only one of the pitch deviation information or the beat deviation information to the user according to user selection or system preset; or firstly feeding back one kind of deviation information to the user, and then feeding back another kind of deviation information or simultaneously feeding back two kinds of deviation information to the user when the preset condition of the system is met or the control instruction of the user is met. This embodiment can increase the deviation dimension to user's feedback when taking into account feedback efficiency to in the user reaches the exercise effect faster, improves exercise efficiency.
As shown in connection with the results of the embodiment shown in fig. 1-3, a control panel may be provided on the music practice device, and a standard tone alert key 5 may be provided on the control panel for enabling the user to input an instruction to trigger an alert tone height and an instruction to end the alert tone height; the control panel can also be provided with a metronome prompt key 6 for a user to input an instruction for triggering prompt tempo and an instruction for finishing prompt tempo. When the music features include pitch and beat, the embodiment can respectively prompt the beat and the pitch according to the user selection, so that the user can contact according to the personalized requirements. In the illustrated embodiment, a volume adjusting device may be further provided, so that the user can adjust the volume of the background played by the music practice device or adjust the volume of the background played by the earphone according to the requirement.
The preset beat collecting period and the preset pitch collecting period in the invention can be the time length of the whole music piece, and the music piece can also be divided into a plurality of time segments, and each time segment is used as the preset beat collecting period and the preset pitch collecting period so as to analyze beat deviation or pitch height in each time segment. The preset beat collecting period and the preset pitch collecting period may be the same period, that is: the user audio information collected by the microphone in a time period can be used for analyzing and extracting pitches and also can be used for analyzing and extracting beats so as to compare the extracted pitches and beats with preset pitches and beats of the music corresponding to the user audio information respectively. The preset beat collecting period and the preset pitch collecting period may also be different periods, for example, the preset beat collecting period may be 10 seconds, that is: collecting user audio information once every 10 seconds when music begins to play, and extracting beats from the user audio information to obtain deviation information of the beats in each 10 seconds; the preset pitch capture period may be 8 seconds, i.e.: user audio information is collected once every 8 seconds when music begins to play, and pitch is extracted from the user audio information to obtain pitch deviation information in every 8 seconds.
Step S40: and feeding back deviation information to a user.
The deviation information fed back to the user may be displayed on the display device 3, or may be fed back by means of an indicator lamp, a buzzer, or the like. For example, the pitch and the beat in the preset music feature are displayed on the time axis in one preset color, and the pitch and the beat in the user audio information are displayed on the time axis in another preset color; when the deviation between the pitch and the beat in the preset music characteristics and the pitch and the beat in the user audio information exceeds the specified deviation, marking the deviation on the time axis, and flashing an indicator light to prompt the attention of the user; or simultaneously feedback by increasing the volume of the metronome, etc.
As described above, the time point for feeding back the deviation information to the user may be after the whole music performance is completed, so as to obtain a relatively comprehensive analysis result, and save the performance consumption of the device processor during the performance of the user, and avoid the abnormal situations such as the stutter of the music practice device. The invention can also divide the whole song into a plurality of time periods, and after entering the next time period, the music characteristics of the pitch of the user are extracted from the audio information of the user in the previous time period so as to obtain the deviation information of the previous time period in the next time period, thereby being beneficial to the user to quickly adjust the deviation. Of course, in some embodiments, the time for feeding back the deviation to the user may also be determined according to the user instruction; for example, when the user inputs a prompt instruction through the standard sound prompt key 5 or the metronome prompt key 6, the user audio information is extracted and the deviation analysis is performed; and when the deviation information is obtained, immediately feeding back the deviation information to a user.
The invention can enable the user to obtain the deviation information between the user audio information of the performance and the preset audio information, so as to carry out targeted exercise according to the deviation information, thereby being beneficial to rapidly improving the exercise effect of the user.
In some embodiments of the present invention, as shown in fig. 5, before acquiring the user audio information collected by the microphone, the method further includes:
step S01: and acquiring a prompt instruction for prompting the preset music characteristic, and sending the preset music characteristic to a music characteristic prompt module.
Predetermine music characteristic and can include multiple music characteristics such as pitch, beat, duration, correspondingly, music characteristic prompt module does also include the sub-prompt module that is used for indicateing music characteristics such as pitch, beat, duration respectively to be used for respectively indicateing corresponding music characteristic to the user. The prompting instruction can be triggered when the user selects the preset music for playing or practicing, and can also be triggered through the prompting switch in the process of playing or practicing the preset music, so that the corresponding sub-prompting module prompts the music characteristics of the current music to the user. In other embodiments, the prompt instruction may be triggered automatically when the deviation of the user's pitch or tempo exceeds a predetermined limit. The embodiment can send the preset music characteristics to the music characteristic prompt module before the user sings or plays so that the user is familiar with the music characteristics such as pitch, beat and the like of the music to be singed or played in advance, or the user is provided with references to the music characteristics such as pitch, beat and the like when singing or playing, so that the user can improve the exercise accuracy and the exercise effect according to the prompt.
For example, after selecting a song, before singing a song or playing a song, the user can input prompt instructions for prompting the tempo and the pitch through the standard tone prompt key 5 and the metronome prompt key 6 so as to be familiar with the tempo and the pitch of the selected song in advance, so that the exercise of the user is more targeted, and the exercise effect is improved. When the user starts singing or playing, the prompting instruction can be sent again through the instruction input devices such as the standard sound prompting key 5, the metronome prompting key 6 or the touch display screen, so that the prompting information and the playing of the song are kept synchronous. Therefore, the "before" in the preceding period before the user audio information collected by the microphone is acquired refers to the time sequence relationship between the preset music feature in the period of time pre-prompted in the step and the period of audio that the user needs to practice. For example, when the song is played to the 10 th second, before the audio information of the user in the 11 th to 20 th seconds is acquired, the preset music characteristics in the 11 th to 20 th seconds are sent to the music characteristic prompt module for prompt; therefore, the "before" may be before the performance is started or before a certain time period is performed during the performance.
For an ordinary user, the pitch deviation is often large in the first sentence for starting singing, and the pitch deviation can be displayed firstly, so that the user can adjust the pitch deviation firstly, and then the beat deviation is displayed when the pitch deviation is small. Therefore, in an embodiment of the present invention, the feeding back the deviation information to the user includes:
extracting a pitch from the user audio information according to a first preset pitch acquisition period; comparing the extracted pitch with a preset pitch of a song corresponding to the user audio information to obtain deviation information of a first preset pitch;
if the deviation information of the first preset pitch is smaller than a pitch deviation threshold value, extracting beats from the user audio information; and comparing the extracted beat with a preset beat of a music corresponding to the user audio information to obtain beat deviation information, and feeding back the beat deviation information to the user.
In another embodiment of the present invention, after comparing the extracted pitch with a preset pitch of a song corresponding to the user audio information to obtain pitch deviation information, the method further includes:
and if the pitch deviation information in the preset pitch acquisition periods is larger than a preset pitch deviation threshold value, controlling the music characteristic prompting module to increase the volume of playing the preset original sound.
The preset pitch collecting periods can comprise a plurality of preset pitch collecting periods of continuous preset number; for example, if the pitch deviation information of the user in five consecutive preset pitch acquisition periods is greater than a preset pitch deviation threshold, the music feature prompt module is controlled to increase the volume of playing the preset original sound. The preset pitch collection periods can also comprise a plurality of discontinuous preset pitch collection periods within a preset period of time; for example, if the preset pitch collecting period is 10 seconds, and the deviation information of the pitch within four preset pitch collecting periods is greater than the preset pitch deviation threshold value within 1 minute, the music feature prompting module is controlled to increase the volume of playing the preset original sound.
When the user is at a plurality of when the deviation information of the pitch in the preset pitch collection period is greater than the preset pitch deviation threshold value, it indicates that the pitch error rate that the user sings or plays in the preset pitch collection period is more, so this embodiment improves the volume of the preset acoustic sound played by the music feature prompt module, so that the user can hear the pitch in the preset acoustic sound more clearly, and the user can adjust the pitch according to the preset acoustic sound. The volume of playing the preset original sound can be increased from playing the preset original sound to playing the preset original sound, or the volume of the played preset original sound is gradually increased from small to large, so that the reference to the user is enhanced.
Of course, the invention may also provide the following embodiments: comparing the extracted pitch with a preset pitch of a corresponding song of the user audio information to obtain deviation information of the pitch, and further comprising:
and if the pitch deviation information in the preset pitch acquisition periods is less than a preset pitch deviation threshold value, controlling the music characteristic prompting module to reduce the volume of playing the preset original sound.
If the deviation information of the pitches in the preset pitch acquisition periods is smaller than the preset pitch deviation threshold, it indicates that the error rate of the pitches singing or playing by the user in the preset pitch acquisition periods is very low, and the pitches are relatively accurate, so that the volume of playing the preset original sound can be gradually reduced, the dependence of the user on the preset original sound is reduced, and the exercise effect is improved.
Further, after comparing the extracted beat with a preset beat of a song corresponding to the user audio information to obtain deviation information of the beat, the method may further include:
and if the deviation information of the beats in a plurality of preset beat acquisition periods is greater than a preset beat deviation threshold value, controlling the music characteristic prompting module to increase the volume of the metronome.
Similarly, the preset pitch collecting periods may include a plurality of preset pitch collecting periods of a continuous preset number, or a plurality of discontinuous preset pitch collecting periods within a preset period of time. When the deviation information of the beats of the user in the preset beat acquisition periods is larger than the preset beat deviation threshold value, it indicates that the beat error rate of singing or playing of the user in the preset pitch acquisition periods is more, so that the volume of the metronome is increased by the embodiment, the user can hear the sound of the metronome more clearly, and the metronome can keep pace with the user. The volume of the metronome can be increased from the time of not using the metronome to the time of using the metronome, or the volume of the metronome is gradually increased from small to large so as to enhance the reference of users.
In another embodiment of the present invention, the feeding back the deviation information to the user may include:
recording the times that the pitch deviation information in the preset pitch acquisition period is greater than a preset pitch deviation threshold value, and if the times exceed a first preset time, prompting a user to repeatedly exercise or recommending a similar song of a song corresponding to the user audio information to the user; or the like, or, alternatively,
recording the times that the deviation information of the beats in a plurality of preset beat acquisition periods is greater than a preset beat deviation threshold value, and prompting the user to repeatedly exercise or recommend the same type of tracks of the tracks corresponding to the audio information of the user to the user if the times exceed a second preset time.
When a user in a plurality of acquisition periods of a song (namely the preset pitch acquisition period or the preset beat acquisition period), the occurring performance deviation value exceeds the set deviation value (namely the preset pitch deviation threshold value or the preset beat deviation threshold value), and the exceeding times are more, the user is not familiar with the performance of the type of song.
In another embodiment of the present invention, the feeding back the deviation information to the user may include:
recording a first number of times that deviation information of pitches in a plurality of preset pitch acquisition periods is larger than a preset pitch deviation threshold value in a preset time period; and if the first time exceeds a first preset time, recording a second time that deviation information of the beats in the preset beat acquisition period is greater than a preset beat deviation threshold, and if the second time exceeds the second preset time, prompting the user to repeatedly practice or recommending the same type of tracks of the music corresponding to the audio information of the user to the user.
Whether the first number exceeds the first preset number is judged firstly, if yes, the second number continues to be judged to exceed the second preset number, if yes, the pitch and the beat of the user performance are not accurate, and the song possibly has great difficulty for the user, so that the user can be prompted to repeatedly exercise the song, or exercise the music score in the preset time period in the song, or recommend the same song of the song corresponding to the user audio information to the user, and the efficient exercise effect is achieved.
In another embodiment of the present invention, the feeding back the deviation information to the user may further include:
obtaining an exercise segment with the maximum pitch deviation according to the pitch deviation information, and obtaining an exercise segment with the maximum beat deviation according to the beat deviation information;
determining key exercise segments according to the exercise segments with the maximum pitch deviation and the exercise segments with the maximum beat deviation;
intercepting the key exercise segment and feeding back the key exercise segment to a user.
The embodiment can count the exercise segment with the maximum sound height deviation and the exercise segment with the maximum beat deviation in the exercise process of the user, and feed back the obtained determined focus exercise segment to the user, so that the user can repeatedly exercise the segment with the maximum deviation, and the exercise effect can be favorably improved.
When the invention is applied to singing practice, the invention also provides the following embodiments: the musical features include timbre and range; the extracting music features from the user audio information comprises: extracting musical features of tone and gamut from the user audio information;
after comparing the extracted music features with preset music features of the music corresponding to the user audio information and obtaining deviation information, the method further comprises the following steps:
obtaining the likelihood between the user audio information and the corresponding song according to the deviation information of the pitch and the deviation information of the beat;
and recommending the matched tracks to the user according to the likelihood, the tone and the tone domain.
According to the embodiment, the likelihood between the user audio information and the corresponding tracks can be obtained according to the deviation information of the pitch and the deviation information of the beat performed by the user; when the likelihood is higher, the user is indicated to sing the song more accurately, and a song similar to the song can be recommended to the user. Further, the present embodiment may also combine the timbre and the range of the user, so that the similar tracks recommended to the user more match the timbre and the range of the user. And recommending the matched tracks to the user according to the likelihood, the tone and the sound range, analyzing a plurality of tracks sung or played by the user history to obtain the likelihood of each song, and selecting the tracks matched with the user tone and the sound range from the songs of which the likelihoods exceed the preset likelihood. And recommending the matched tracks to the user according to the likelihood, the tone colors and the tone domains, sorting the likelihood of a plurality of tracks sung or played by the user history, and selecting the tracks matched with the tone colors and the tone domains of the user from the plurality of tracks with the top ranking.
According to the likelihood, the tone and the audio field, the embodiment can obtain the track which is sung most accurately by the user, and is beneficial to recommending the track which is most adept by the user to the user, so that the interest of achievement and exercise of the user is improved.
Based on the above audio processing method, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the audio processing method of any one of the preceding claims.
The present invention also provides a music practice terminal, including:
a microphone, one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the audio processing method of any of the preceding claims.
As in the embodiment of the music practicing device shown in fig. 1 and 2, the music practicing device may further include:
and the radio frequency identification module 7 is used for collecting certificate information through radio frequency and identifying the identity of a user according to the certificate information. For example, when the user brings the identification card close to the rfid module 7, the music practicing device of the present invention can recognize the user identity to read the user's historical performance information or match the user with a corresponding performance track.
The music practice device of the present invention may further comprise a fingerprint recognition module 8 for collecting a fingerprint image of the user and recognizing the user's identity based on the fingerprint image. The music practice device of the invention can also comprise a height adjusting module which can be combined with the head height of the user identified by the camera device 4, and can realize the purpose of controlling the height of the display device 3 according to the head height of the user, so that the display device 3 can adapt to users with different heights, and the user experience is improved.
The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. An audio processing method, which is applied to a music practice terminal, comprises the following steps:
acquiring user audio information acquired by a microphone;
extracting music features from the user audio information;
comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information;
and feeding back deviation information to a user.
2. The audio processing method of claim 1, wherein the musical features include pitch and tempo; the music features are extracted from the user audio information; and comparing the extracted music characteristics with preset music characteristics of the music corresponding to the user audio information to obtain deviation information, wherein the deviation information comprises the following steps:
extracting pitches from the user audio information according to a preset pitch acquisition period; comparing the extracted pitch with a preset pitch of a song corresponding to the user audio information to obtain deviation information of the pitch;
extracting beats from the user audio information according to a preset beat acquisition cycle; and comparing the extracted beat with a preset beat of a music corresponding to the user audio information to obtain beat deviation information.
3. The audio processing method according to claim 2, wherein before the obtaining the user audio information collected by the microphone, further comprising:
and acquiring a prompt instruction for prompting the preset music characteristic, and sending the preset music characteristic to a music characteristic prompt module.
4. The audio processing method according to claim 3, wherein after comparing the extracted pitch with a preset pitch of a music corresponding to the user audio information to obtain pitch deviation information, the method further comprises:
and if the pitch deviation information in the preset pitch acquisition periods is larger than a preset pitch deviation threshold value, controlling the music characteristic prompting module to increase the volume of playing the preset original sound.
5. The audio processing method according to claim 3, wherein after comparing the extracted beat with a preset beat of a music corresponding to the user audio information to obtain beat deviation information, the method further comprises:
and if the deviation information of the beats in a plurality of preset beat acquisition periods is greater than a preset beat deviation threshold value, controlling the music characteristic prompting module to increase the volume of the metronome.
6. The audio processing method of claim 5, wherein the feeding back the deviation information to the user comprises:
recording the times that the pitch deviation information in the preset pitch acquisition period is greater than a preset pitch deviation threshold value, and if the times exceed a first preset time, prompting a user to repeatedly exercise or recommending a similar song of a song corresponding to the user audio information to the user; or the like, or, alternatively,
recording the times that the deviation information of the beats in a plurality of preset beat acquisition periods is greater than a preset beat deviation threshold value, and prompting the user to repeatedly exercise or recommend the same type of tracks of the tracks corresponding to the audio information of the user to the user if the times exceed a second preset time.
7. The audio processing method of claim 2, wherein the feeding back the deviation information to the user comprises:
obtaining an exercise segment with the maximum pitch deviation according to the pitch deviation information, and obtaining an exercise segment with the maximum beat deviation according to the beat deviation information;
determining key exercise segments according to the exercise segments with the maximum pitch deviation and the exercise segments with the maximum beat deviation;
intercepting the key exercise segment and feeding back the key exercise segment to a user.
8. The audio processing method according to claim 2, wherein the musical features include timbre and range; the extracting music features from the user audio information comprises: extracting musical features of tone and gamut from the user audio information;
after comparing the extracted music features with preset music features of the music corresponding to the user audio information and obtaining deviation information, the method further comprises the following steps:
obtaining the likelihood between the user audio information and the corresponding song according to the deviation information of the pitch and the deviation information of the beat;
and recommending the matched tracks to the user according to the likelihood, the tone and the tone domain.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out an audio processing method according to any one of claims 1 to 8.
10. A music practice terminal, comprising:
a microphone, one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the audio processing method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910222788.3A CN111724812A (en) | 2019-03-22 | 2019-03-22 | Audio processing method, storage medium and music practice terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910222788.3A CN111724812A (en) | 2019-03-22 | 2019-03-22 | Audio processing method, storage medium and music practice terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111724812A true CN111724812A (en) | 2020-09-29 |
Family
ID=72563498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910222788.3A Pending CN111724812A (en) | 2019-03-22 | 2019-03-22 | Audio processing method, storage medium and music practice terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111724812A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096689A (en) * | 2021-04-02 | 2021-07-09 | 腾讯音乐娱乐科技（深圳）有限公司 | Song singing evaluation method, equipment and medium |
| CN113096486A (en) * | 2021-04-13 | 2021-07-09 | 陕西理工大学 | Piano teaching device convenient to remote learning |
| CN114333497A (en) * | 2022-01-11 | 2022-04-12 | 平安科技（深圳）有限公司 | Music partner training method, device, equipment and medium |
| WO2023132653A1 (en) * | 2022-01-05 | 2023-07-13 | Samsung Electronics Co., Ltd. | Method and device for managing audio based on spectrogram |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090165633A1 (en) * | 2007-12-28 | 2009-07-02 | Nintendo Co., Ltd., | Music displaying apparatus and computer-readable storage medium storing music displaying program |
| CN102110435A (en) * | 2009-12-23 | 2011-06-29 | 康佳集团股份有限公司 | Method and system for karaoke scoring |
| CN104657438A (en) * | 2015-02-02 | 2015-05-27 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105810211A (en) * | 2015-07-13 | 2016-07-27 | 维沃移动通信有限公司 | Audio frequency data processing method and terminal |
| CN105824861A (en) * | 2015-09-18 | 2016-08-03 | 维沃移动通信有限公司 | Audio recommending method and mobile terminal |
| CN105825844A (en) * | 2015-07-30 | 2016-08-03 | 维沃移动通信有限公司 | Sound repairing method and device |
-
2019
- 2019-03-22 CN CN201910222788.3A patent/CN111724812A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090165633A1 (en) * | 2007-12-28 | 2009-07-02 | Nintendo Co., Ltd., | Music displaying apparatus and computer-readable storage medium storing music displaying program |
| CN102110435A (en) * | 2009-12-23 | 2011-06-29 | 康佳集团股份有限公司 | Method and system for karaoke scoring |
| CN104657438A (en) * | 2015-02-02 | 2015-05-27 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105810211A (en) * | 2015-07-13 | 2016-07-27 | 维沃移动通信有限公司 | Audio frequency data processing method and terminal |
| CN105825844A (en) * | 2015-07-30 | 2016-08-03 | 维沃移动通信有限公司 | Sound repairing method and device |
| CN105824861A (en) * | 2015-09-18 | 2016-08-03 | 维沃移动通信有限公司 | Audio recommending method and mobile terminal |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096689A (en) * | 2021-04-02 | 2021-07-09 | 腾讯音乐娱乐科技（深圳）有限公司 | Song singing evaluation method, equipment and medium |
| CN113096486A (en) * | 2021-04-13 | 2021-07-09 | 陕西理工大学 | Piano teaching device convenient to remote learning |
| WO2023132653A1 (en) * | 2022-01-05 | 2023-07-13 | Samsung Electronics Co., Ltd. | Method and device for managing audio based on spectrogram |
| CN114333497A (en) * | 2022-01-11 | 2022-04-12 | 平安科技（深圳）有限公司 | Music partner training method, device, equipment and medium |
| CN114333497B (en) * | 2022-01-11 | 2023-08-25 | 平安科技（深圳）有限公司 | Music partner training method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111724812A (en) | Audio processing method, storage medium and music practice terminal | |
| CN102664016B (en) | Singing evaluation method and system | |
| CN1463419A (en) | Synchronizing text/visual information with audio playback | |
| CN104992712B (en) | It can identify music automatically at the method for spectrum | |
| US10235898B1 (en) | Computer implemented method for providing feedback of harmonic content relating to music track | |
| CN111081272A (en) | Song climax fragment identification method and device | |
| JP2002116754A (en) | Tempo extraction device, tempo extraction method, tempo extraction program and recording medium | |
| CN109658909A (en) | A kind of intelligence small drum system and its implementation | |
| CN105895079B (en) | Voice data processing method and device | |
| CN106327949A (en) | Method and device for training music rhythm | |
| Rao | Audio signal processing | |
| JP3588596B2 (en) | Karaoke device with singing special training function | |
| Narang et al. | Acoustic Features for Determining Goodness of Tabla Strokes. | |
| CN110299049B (en) | Intelligent display method of electronic music score | |
| JP2010085656A (en) | Register specifying system and program | |
| EP0367191B1 (en) | Automatic music transcription method and system | |
| CN210142417U (en) | Music interaction equipment | |
| TWM575598U (en) | Electronic score | |
| JP2004144867A (en) | Singing practice assisting system of karaoke device | |
| CN112489607A (en) | Method and device for recording songs, electronic equipment and readable storage medium | |
| JP2016071187A (en) | Voice synthesis device and voice synthesis system | |
| JP4612329B2 (en) | Information processing apparatus and program | |
| WO2023032319A1 (en) | Information processing device, information processing method, and information processing system | |
| JP6011506B2 (en) | Information processing apparatus, data generation method, and program | |
| KR102077269B1 (en) | Method for analyzing song and apparatus using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |