CN105513583B - song rhythm display method and system - Google Patents
song rhythm display method and system Download PDFInfo
- Publication number
- CN105513583B CN105513583B CN201510827808.1A CN201510827808A CN105513583B CN 105513583 B CN105513583 B CN 105513583B CN 201510827808 A CN201510827808 A CN 201510827808A CN 105513583 B CN105513583 B CN 105513583B
- Authority
- CN
- China
- Prior art keywords
- beat
- audio data
- value
- acquiring
- time interval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000033764 rhythmic process Effects 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000005236 sound signal Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 6
- 238000013139 quantization Methods 0.000 claims description 6
- 230000000295 complement effect Effects 0.000 abstract description 3
- 230000000007 visual effect Effects 0.000 abstract description 3
- 238000001228 spectrum Methods 0.000 description 21
- 238000005070 sampling Methods 0.000 description 15
- 238000004364 calculation method Methods 0.000 description 7
- 230000033001 locomotion Effects 0.000 description 7
- 238000000354 decomposition reaction Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000000630 rising effect Effects 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 238000010606 normalization Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/36—Accompaniment arrangements
- G10H1/361—Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/36—Accompaniment arrangements
- G10H1/40—Rhythm
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/071—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for rhythm pattern analysis or rhythm style recognition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/076—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of timing, tempo; Beat detection
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/021—Indicator, i.e. non-screen output user interfacing, e.g. visual or tactile instrument status or guidance information using lights, LEDs or seven segments displays
- G10H2220/081—Beat indicator, e.g. marks or flashing LEDs to indicate tempo or beat positions
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/021—Indicator, i.e. non-screen output user interfacing, e.g. visual or tactile instrument status or guidance information using lights, LEDs or seven segments displays
- G10H2220/086—Beats per minute [BPM] indicator, i.e. displaying a tempo value, e.g. in words or as numerical value in beats per minute
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Auxiliary Devices For Music (AREA)
Abstract
the invention provides a method and a system for displaying song rhythm, wherein the method comprises the following steps: acquiring audio data of the song according to a preset acquisition period; acquiring the frequency amplitude of the audio data; acquiring a beat value of audio data; and displaying the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value. According to the method, the beat value of the audio data is obtained, the audio data is displayed together with the frequency amplitude, and the display speed of the audio amplitude is determined according to the beat value; the rhythm, the beat and the frequency of the music can be vividly and visually displayed, and the rhythm of the music can be perfectly reflected by the three components in a complementary manner; meanwhile, better visual enjoyment can be brought to the viewers, and the music atmosphere is activated.
Description
Technical Field
the invention relates to the field of audio signal processing, in particular to a method and a system for displaying song rhythm.
background
The music player in the prior art not only supports the playing of music, but also displays the height of the music in the form of dynamic music frequency spectrum during the playing process of the music. In the music frequency spectrum in the prior art, after audio data (PCM data) of music is acquired in real time and the audio data is subjected to fourier transform to transform an audio signal from a time domain to a frequency domain, each discrete frequency corresponds to an amplitude. The music frequency spectrum determines the height of the frequency spectrum animation display by using the amplitude; however, such music spectrum only shows the height and intensity of music (the higher the treble, the right peak is and the higher the bass, the left peak is), and the characteristics of music are many, such as rhythm, beat and timbre of music, which cannot be shown in the general music spectrum. Especially, the rhythm and the beat of the music supplement each other to form a framework of the music together to support the rhythm of the music, so that the rhythm and the beat of the music are very important for the music; however, the conventional music frequency spectrum cannot show the defects of the conventional music frequency spectrum and the conventional music frequency spectrum, so that a dynamic frequency spectrum which more accords with the music characteristic cannot be presented to a user, and the requirement of the user for grasping the music rhythm according to the frequency spectrum in the music appreciation process cannot be met.
The application numbers are: 200910162906.2 discloses a singing and playing computer real-time analysis system, which is used to draw a time-frequency-intensity real-time spectrogram according to short-time fast-reading Fourier transform during the processing of music signals, wherein the spectrogram can reflect the stability of sound, the jitter range, the resonance peak position, the intensity, the adhesion condition of the slip tone in the middle of the fundamental tone, the strong and weak contrast between the fundamental tone and each overtone and the time-dependent progress of frequency, so as to represent the rhythm change of music.
the above patent does not express the change of music rhythm according to the rhythm and beat characteristics of music, or can not express the music rhythm sense in the most intuitive and accurate way; therefore, it is necessary to provide a method and system for acquiring music spectrum based on the rhythm and beat of music.
disclosure of Invention
The technical problem to be solved by the invention is as follows: a method and a system for displaying song rhythm are provided, and audio data can be displayed according to a beat value and a frequency amplitude.
In order to solve the technical problems, the invention adopts the technical scheme that:
a method of displaying song tempos, comprising:
Acquiring audio data of a song according to a preset acquisition period;
Acquiring a beat value of the audio data;
acquiring a beat value of the audio data according to the acquisition period and the time interval;
and displaying the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value.
The invention provides another technical scheme as follows:
a song tempo display system comprising:
the preset module is used for acquiring audio data of the song according to a preset acquisition cycle;
the first acquisition module is used for acquiring the frequency amplitude of the audio data;
The second acquisition module is used for acquiring the beat value of the audio data;
and the display module is used for displaying the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value.
the invention has the beneficial effects that: the invention provides a display method and a system of song rhythm, which are characterized in that the beat value of audio data is obtained, the audio data is displayed by combining with frequency amplitude, and the display speed of the audio amplitude is determined by the beat value; the rhythm, the beat and the frequency of the music can be vividly and visually displayed, and the rhythm of the music can be perfectly reflected by the three components in a complementary manner; meanwhile, better visual enjoyment can be brought to the viewers, and the music atmosphere is activated.
Drawings
FIG. 1 is a flow chart of a song rhythm display method according to the present invention;
fig. 2 is a flowchart illustrating a method for displaying song tempos according to an embodiment of the present invention;
FIG. 3 is a block diagram of the structural components of a song tempo display system of the present invention;
FIG. 4 is a music-frequency interface diagram of an electrocardiogram in accordance with an embodiment of the present invention;
FIG. 5 is a diagram of a phonogram music frequency spectrum interface moving left and right according to an embodiment of the present invention;
FIG. 6 is a music spectrum interface diagram of a 3D animated fret diagram according to an embodiment of the present invention;
FIG. 7 is a spectrogram according to an embodiment of the present invention.
description of reference numerals:
1. presetting a module; 2. a first acquisition module; 3. a second acquisition module; 4. and a display module.
Detailed Description
in order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
The most key concept of the invention is as follows: the method comprises the steps of obtaining a beat value of audio, displaying audio data according to the beat value and a frequency amplitude, and determining the display speed of the audio amplitude according to the beat value in the displaying process.
the technical terms related to the invention are explained as follows:
referring to fig. 1 and 2, the present invention provides a method for displaying song tempos, including:
acquiring audio data of a song according to a preset acquisition period;
acquiring the frequency amplitude of the audio data;
acquiring more than two beat points in the audio data;
acquiring the time interval between every two adjacent beat points;
acquiring a beat value of the audio data according to the acquisition period and the time interval;
and displaying the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value.
further, the beat value is linearly proportional to the display speed of the audio amplitude.
further, the linear proportional relationship is a direct proportional relationship.
From the above description, if the tempo is faster, the change speed of the amplitude is faster, and vice versa, the display speed of the beat value and the audio amplitude is in a linear and proportional relationship, which conforms to the viewing habit of people and can more intuitively reflect the fluctuation of the tempo of music.
Further, the audio data includes a music score corresponding to the audio, and a tempo value of the audio data is obtained according to the music score.
as can be seen from the above description, each beat point is marked on the music score, and the beat value of the audio data can be obtained directly according to the time interval between adjacent beat points. The music score shown in fig. 7 has four beats, 4 beats per cycle, one beat of quarter note, and the playing time of each bar is t. For example, in the first section, there are four notes, each note is a beat, and the time interval (beat value) between adjacent notes is t/4; and 11 notes exist in the second bar, the first three notes form a beat, the time interval (the beat value) between the first note and the second note of the second bar is t/8, and the rest is done, so that the time intervals (the beat values) of other notes are calculated.
referring to fig. 4, fig. 5 and fig. 6, further, the "displaying the audio data according to the beat value and the frequency amplitude" specifically includes:
displaying the audio data by using an electrocardiogram, wherein the potential change of the electrocardiogram represents the change of the frequency amplitude value, and the speed of the potential change represents the beat value;
or displaying the audio data by using a left-right moving sound column graph, wherein the length of the sound column represents the frequency amplitude, and the moving speed of the sound column represents the beat value;
or displaying the audio data by using a sound column graph with elevation change, wherein the elevation height of the sound column represents the audio amplitude, and the elevation speed of the sound column represents the beat value;
Or displaying the audio data by using the flicker change of the light, wherein the brightness of the light represents the audio amplitude, and the flicker speed of the light represents the beat value.
the above description shows that the invention can show the rhythm change of the audio data in different expression forms, and the invention has various forms, rich and colorful contents and vivid images, and brings a brand new audio-visual experience to people through the combination of the appreciation of frequency spectrum and the auditory enjoyment of audio.
further, the step of obtaining more than two beat points in the audio data; acquiring the time interval between every two adjacent beat points; the step value of the audio data obtained according to the acquisition cycle and the time interval "is specifically:
performing wavelet transform processing on the audio data to obtain a coefficient value of the audio data;
carrying out peak value detection on the coefficient value of the audio data to obtain more than two beat points;
acquiring the time interval of two adjacent beat points in the original audio signal;
Acquiring a time interval with the maximum occurrence probability;
and acquiring a beat value according to the acquisition period and the time interval with the maximum occurrence probability.
As can be seen from the above description, based on the fact that the sampling rate is fixed, for example, 48K audio data is collected per second, the time interval for each piece of audio data to be transmitted to the application layer for processing is almost the same, and therefore, the beat value of each piece of audio data can be approximately recognized as the same due to the short time interval, but even within the same time interval, the number of notes appearing is different, which is the manifestation of the rhythm speed. By performing processing such as wavelet transform and peak detection processing on audio data, the tempo value of the audio data, that is, the rhythm of music can be extracted.
The sampling rate of the original signal is reduced by one time after each wavelet transform is performed on the signal in the audio data, and the interval between the beat values in the invention refers to the beat interval in the original audio signal, so that the transformed beat interval needs to be transformed to the beat interval value in the original signal, and a beat interval histogram is constructed according to the beat interval value, so as to ensure the accuracy of the obtained beat interval.
further, before the step of performing wavelet transform processing on the audio data to obtain detail coefficient values of the audio data, the method further includes:
and normalizing the quantization bit number of the audio data.
According to the description, after the audio data are obtained, normalization preprocessing is firstly carried out, the data length of each sampling point represented by quantization digits can be converted into standard quantization to be represented, so that the influence of certain audio data on the data calculation result is prevented from being too large, the calculation of the audio data can be simplified, and the calculation efficiency is improved.
Further, after the "obtaining the beat value", the method further includes:
constructing a beat interval histogram according to the time interval between every two adjacent beat values;
smoothing the beat interval histogram;
And acquiring the time interval with the highest occurrence probability in the smoothed beat interval histogram to obtain a beat value mean value.
As can be seen from the above, the time interval with the highest frequency of occurrence is extracted from the smoothed beat interval histogram, and the accuracy of the extracted beat value can be improved.
further, the step value average value obtaining by obtaining the time interval with the highest occurrence probability in the smoothed beat interval histogram specifically includes:
Acquiring a time interval with the highest occurrence probability in the beat interval histogram after smoothing processing;
According to the formula: and obtaining a beat value mean value, wherein the beat value is the preset time interval with the highest acquisition cycle/probability.
Further, after the "obtaining the mean value of the beat values", the method further includes:
Acquiring an initial beat point of the audio data;
and acquiring all the beat points of the audio data according to the initial beat point and the beat value mean value.
As can be seen from the above, since the positions of the beat points are determined by the initial beat points and the beat values, the initial beat points are the average of the first possible beat point positions obtained after wavelet transformation, that is, the initial beat value average, we can obtain the positions of all the beat points according to the initial beat value average plus several times of beat values, and assuming that the current beat is Bv, the initial beat value average is P1, the position of the nth beat point is: pn ═ P1+ (n-1) Bv.
referring to fig. 3, another technical solution provided by the present invention is:
A song tempo display system comprising:
The system comprises a preset module 1, a processing module and a control module, wherein the preset module is used for acquiring audio data of a song according to a preset acquisition cycle;
the first obtaining module 2 is configured to obtain a frequency amplitude of the audio data;
the second obtaining module 3 is configured to obtain a beat value of the audio data;
And the display module 4 is configured to display the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value.
from the above description, the beneficial effects of the present invention are: the invention provides a display system of song rhythm, obtain the beat value of the audio data through the second acquisition module 3, and then combine the frequency amplitude that the first acquisition module 2 obtains together, demonstrate the audio data through the display module 4, confirm the display speed of the said audio amplitude from the said beat value; the rhythm, the beat and the frequency of the music can be vividly and visually displayed, and the rhythm of the music can be perfectly reflected by the three components in a complementary manner; meanwhile, better visual enjoyment can be brought to the viewers, and the music atmosphere is activated.
further, the second obtaining module 3 includes:
the first acquisition unit is used for acquiring more than two beat points in the audio data;
a second acquisition unit configured to acquire a time interval between every two adjacent beat points;
and the third acquisition unit is used for acquiring the beat value of the audio data according to the acquisition cycle and the time interval.
further, the first obtaining unit is specifically configured to perform wavelet transform processing on the audio data to obtain a coefficient value of the audio data; performing peak value detection on the coefficient value of the audio data to acquire more than two beat points;
the second obtaining unit is specifically configured to obtain a time interval between two adjacent beat points in the original audio signal; acquiring a time interval with the maximum occurrence probability;
and the third acquisition unit is specifically used for acquiring the beat value according to the acquisition cycle and the time interval with the maximum occurrence probability.
according to the description, the beat value is obtained according to the time interval between the adjacent beat points and the acquisition cycle of the audio data, and a basis is provided for showing the rhythm and the music height of the audio data by combining the beat value and the frequency amplitude.
Referring to fig. 1, fig. 2, fig. 4, fig. 5 and fig. 6, a first embodiment of the present invention is:
there is provided an audio frequency spectrum capable of exhibiting a music tempo, comprising:
S1: acquiring audio data of a song according to a preset acquisition period;
In the playing process of music, recording the music, setting the acquisition period (sampling rate) to be 48K, acquiring PCM audio data, and sending the PCM audio data to an application layer for processing. Because the acquisition period is fixed, the time from acquisition to processing is also fixed, that is, the number of beats acquired is fixed, in the present invention, we extract the number of notes (the number of beats) included in the fixed number of beats, that is, the beat value, which represents the rhythm of music.
S2: acquiring the frequency amplitude of the audio data;
Specifically, it is preferable that the first main board is responsible for performing Fourier transform (Fourier transform) on the audio data to obtain the frequency amplitude of the audio data. The fourier transform is capable of transforming the audio data from the time domain to the frequency domain (the so-called frequency domain, which may be represented by the amplitude corresponding to the frequency on the X-axis and the frequency on the Y-axis); then, segmenting the frequency of the X axis, and dividing the width of each frequency band in a nonlinear way; according to each discrete frequency, a frequency amplitude is corresponding, and in the music frequency spectrum adopting the sound column to display the frequency amplitude, the height of the animation sound column (the height of the sound column rising) is determined by the frequency amplitude, and the sound column falls back to the initial point position after finishing displaying the height. Therefore, the maximum frequency amplitude is obtained in each divided frequency band, and all the frequency amplitudes of the audio data are sent to the second main board; and the height fluctuation of the animation sound column is determined by the second main board according to the frequency amplitude.
S3: extracting music rhythm;
s31: carrying out normalization pretreatment on the acquired audio data;
because PCM audio data is subjected to Fourier transform, when analog audio data is converted into digital audio data, audio signals are discretized in amplitude, the data length of each sampling point is represented by quantization bit numbers (8 bits and 16 bits are commonly used), the quantization bit numbers are different, the data ranges which can be represented by the sampling points are different, for example, the data range which can be represented by the 8-bit sampling point is 0-255, and the data range which can be represented by the 16-bit sampling point is-32768-32767; therefore, for unified processing, after the audio data is acquired, normalization processing needs to be performed on the audio data, and data of sampling points are normalized to [ -1,1 ];
Assuming that the sample point sequence is S [ ], and the normalized sample point sequence is Ns [ ], the formula can be used:
Ns[]＝(S[]-S min)/(S max-S min)*(D max-D min)+D min；
wherein Smax is the maximum value in the sequence of sample points; smin is the minimum value in the sampling point sequence, Dmax is the maximum value expected in the sequence after the sampling points are normalized, and Dmin is the minimum value expected in the sequence after the sampling points are normalized; and the data of the sampling points are normalized to prevent certain audio data from influencing the data calculation result too much, simplify the calculation of the audio data and improve the calculation efficiency.
S32: performing wavelet transformation on the preprocessed audio data;
performing discrete wavelet transform on the preprocessed audio data signals, and generating approximate coefficients and detail coefficients after the audio data signals are subjected to wavelet transform for one time; the approximation coefficients preserve the low frequency component of the signal, while the detail coefficients preserve the high frequency component of the signal; and (3) continuously carrying out wavelet decomposition on the approximation coefficient of the low-frequency component to obtain a new approximation coefficient and a detail coefficient, and continuously repeating the decomposition process for n times of wavelet decomposition to obtain n groups of coefficient values. Here, preferably n is 4, resulting in 4 sets of coefficient values.
after the audio data signals are subjected to wavelet transformation, time domain information cannot be lost, and the tone quality effect of the audio data signals can be ensured; meanwhile, the sampling rate of the audio data signal is reduced by one time after each wavelet decomposition, so that the data volume of all processing can be reduced, and the calculation efficiency of the algorithm is improved.
S33: carrying out peak value detection on the audio data after wavelet transformation to obtain a beat point;
since the tempo of music is likely to occur at the peak point of the wavelet decomposition coefficients, it is necessary to apply a peak detection algorithm to the coefficient values in the n sets of coefficient values obtained in S32, respectively, to obtain tempo points;
the method specifically comprises the following steps:
(1) full-wave correction: absolute value conversion is carried out on all data in the audio data signal;
(2) windowing and extreme value solving: for short-time analysis, a window must be added; a window with smooth transition characteristics at both ends should be used in theory, with the purpose of reducing discontinuity of the audio signal at the beginning and end of the audio data frame;
And windowing all corrected audio data, sliding a window while solving a local extremum in the window, continuously solving the extremum, determining a final peak value point by considering that the local extremum is in a window range and the local extremum has the same times, if the communication times are more than 90% of the window width, considering the extremum as a peak value, and simultaneously recording the position of the peak value so as to obtain all peak values of the audio data, wherein the peak value position is assumed to be a beat point position.
S34: acquiring the time interval of the beat points, and constructing a histogram;
after the peak values of n groups of wavelet coefficients of the audio data are detected, the time interval between every two adjacent peak value positions needs to be calculated, namely the time interval IOI between every two adjacent beat points is calculated;
Specifically, after each wavelet transform is performed on the audio data signal, the sampling rate of the original audio data signal is reduced by one time, and the required beat point time interval IOI refers to an interval value in the original audio data signal. Therefore, it is necessary to first transform the transformed time interval values into the time interval values in the original audio data signal.
Supposing that the current interval value is a value obtained by performing nth wavelet transform on the original audio data signal and is recorded as IOI 2; when the time interval value of the original audio data signal is recorded as IOI1, IOI1 is IOI2 × 2 n;
then, according to the probability of the interval occurrence of the beat points, a beat interval histogram is constructed;
S35: smoothing the beat interval histogram, and calculating to obtain a beat value;
the accuracy of the beat value directly obtained according to the constructed beat interval histogram is not high, and in order to improve the accuracy of the beat value, the beat interval histogram needs to be smoothed;
specifically, the weight of the time interval of a beat point is introduced first, and the weight of the time interval of the beat point is estimated by judging the similarity degree of other beat point intervals adjacent to a certain beat point interval. Here, two parameters need to be determined: firstly, comparing with adjacent time intervals, and preferably selecting four beat time intervals around a certain time interval according to historical experience; second, determining whether a time interval is close can define a deviation value, and if the difference between two time intervals is smaller than the deviation value, they can be regarded as the same time interval.
at this time, the following situations may also occur in the histogram: first, the histogram may have more than one extreme (time interval with the same probability of occurrence), how to choose; second, the time interval with the highest probability of occurrence occurs in the low density region; we can simply select the maximum value of this extremum to calculate the beat value, but this maximum value does not represent the main distribution of beat interval values. Therefore, if we do not consider the distribution of the histogram and simply select the maximum value, the calculated beat value will be inaccurate.
to solve the above problem, we can convolve the histogram with a gaussian function to smooth the histogram; the gaussian function may highlight areas of high density; after the smoothing processing, the time interval of the beat point can be judged by obtaining the maximum amplitude;
the time interval of the beat points is obtained as follows:
Obtaining the time interval with the maximum occurrence probability in the smooth histogram, and according to a formula:
The beat value Bv is the time interval with the maximum acquisition period/occurrence probability; and obtaining the mean value of the beat values.
s36: acquiring all beat points in audio data;
the position of the beat point is determined by the initial beat point and the beat value in the audio data, and the initial beat point is the average value of the position of the first possible beat point obtained after wavelet transformation; after the beat value mean value is calculated each time, all beat point positions can be obtained according to the initial beat point and a plurality of times of beat value mean values. The method specifically comprises the following steps:
if the current beat is Bv and the initial beat point is P1, the position of the nth beat point is:
Pn＝P1+(n-1)Bv。
s4: displaying the audio data according to the beat value and the frequency amplitude;
And determining the display speed of the audio amplitude according to the beat value, wherein the optimal beat value is in linear and direct proportional relation with the display speed of the audio amplitude.
the audio data may be presented using a wide variety of spectrograms,
as shown in fig. 4, if an electrocardiograph is used to display the audio data, the change of the electrical potential of the electrocardiogram represents the change of the amplitude of the frequency, and the speed of the change of the electrical potential represents the beat value;
as shown in fig. 5, if the audio data is displayed by using a left-right moving fret spectrogram, the length of the fret represents the frequency amplitude, and the moving speed of the fret represents the beat value;
As shown in fig. 6, if the audio data is displayed by using a spectrogram of a sound column with a variable elevation, the elevation height of the sound column represents the audio amplitude, and the elevation speed of the sound column represents the beat value;
The audio data may also be presented using a flash change in light, wherein the intensity of the light represents the audio amplitude and the speed at which the light flashes represents the beat value.
Specifically, we explain in detail a 3D fret spectrogram with elevation change:
In the spectrogram, the rising height of a sound column is determined according to the amplitude of audio, and the frequency spectrum is changed from small to large from left to right; the stronger the music intensity is, the higher the sound column rises, otherwise, the lower the sound column rises; the stronger the music high pitch, the more the right side columns are, and the stronger the music low pitch, the more the left side columns are;
The rising and falling speeds of the sound column are determined according to the beat value, and the higher the rhythm is, the higher the rising and falling speeds are; the slower the tempo, the slower the speed of rise and fall.
in the process of frequency spectrum display, according to the frequency amplitude of the obtained audio data, the sound column with the height of 0 rises to the height corresponding to the frequency amplitude through the upward throwing motion, and then returns to the height of 0 through the downward throwing motion; calculating to obtain a value of V0 according to a motion formula of S ═ V0 × (t + (1/2) × a × (t) × (S is a rising path determined by the maximum frequency amplitude, V0 is an initial velocity, a is an acceleration preset according to an empirical value, t is time, and time t of motion is determined according to a beat value of the obtained audio data), wherein the initial velocity V0 determines the velocity of "upward throwing" and "downward throwing" of the sound column, and specifically, the faster the rhythm is, the shorter the time of "upward throwing motion" and "downward throwing motion" is; conversely, the longer the movement time.
various 3D animations which are edited by self-definition can be preferably added into the music spectrum, and the music atmosphere can be played by the dancing accompanying of characters while the pitch of the sound columns changes along with the music rhythm.
referring to fig. 3, the second embodiment of the present invention is:
a digital audio-visual system comprises a set-top box and a display screen, wherein the set-top box is connected with the display screen; the set top box comprises a first main board and a second main board; the first mainboard comprises a preset module 1 and a first acquisition module 2;
the second motherboard includes a second obtaining module 3, and the second obtaining module 3 may specifically include a first obtaining unit, a second obtaining unit, and a third obtaining unit;
the preset module 1 is used for acquiring audio data of a song according to a preset acquisition cycle and a preset frequency spectrum display mode;
the first obtaining module 2 is configured to obtain a frequency amplitude of the audio data, and send the frequency amplitude to a second motherboard;
The second obtaining module 3 is configured to obtain a beat value of the audio data, and specifically, perform wavelet transform processing on the audio data through the first obtaining unit to obtain a coefficient value of the audio data; carrying out peak value detection on the detail coefficient value of the audio data to obtain more than two beat points;
acquiring the time interval of two adjacent beat points in the original audio signal through the second acquisition unit; acquiring a time interval with the maximum occurrence probability;
and acquiring a beat value according to the acquisition cycle and the time interval with the maximum occurrence probability through a third acquisition unit.
The display screen is used for displaying the audio data by using a spectrogram of a 3D animation according to the beat value and the frequency amplitude in a preset display mode, wherein the display speed of the audio amplitude is determined according to the beat value.
in conclusion, the song rhythm display and the system thereof provided by the invention can extract the rhythm of the audio data, display the audio data by combining the frequency amplitude, realize vividly and visually displaying the rhythm, the rhythm and the frequency of the music and perfectly embody the rhythm of the music; moreover, the rhythm and the frequency show a linear direct proportional relation, so that the fluctuation of the rhythm of the music is more intuitively reflected; furthermore, various presentation forms for showing the rhythm change of the audio data are provided, the presentation forms are vivid, more choices are provided for people, and brand new audio-visual experience is brought.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.
Claims (12)
1. a method for displaying song tempos, comprising:
acquiring audio data of a song according to a preset acquisition period;
acquiring the frequency amplitude of the audio data;
acquiring a beat value of the audio data;
Displaying the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value;
The step of acquiring the beat value of the audio data is specifically as follows:
Acquiring more than two beat points in the audio data;
acquiring the time interval between every two adjacent beat points;
and acquiring the beat value of the audio data according to the acquisition period and the time interval.
2. the method as claimed in claim 1, wherein the audio data includes audio music score, and the tempo value of the audio data is obtained according to the music score.
3. the method of claim 1, wherein the tempo value is linearly proportional to the speed of displaying the audio amplitude.
4. A method of displaying the tempo of a song according to claim 3, wherein said linear scaling relationship is a direct scaling relationship.
5. The method for displaying song tempo according to claim 1, wherein said "displaying said audio data according to said tempo value and frequency amplitude" specifically comprises:
displaying the audio data by using an electrocardiogram, wherein the potential change of the electrocardiogram represents the change of the frequency amplitude value, and the speed of the potential change represents the beat value;
Or using a left-right moving sound column graph to display the audio data, wherein the length of the sound column represents the frequency amplitude, and the moving speed of the sound column represents the beat value;
or displaying the audio data by using a sound column graph with elevation change, wherein the elevation height of the sound column represents the audio amplitude, and the elevation speed of the sound column represents the beat value;
or displaying the audio data by using the flicker change of the light, wherein the brightness of the light represents the audio amplitude, and the flicker speed of the light represents the beat value.
6. The method for displaying song tempos according to claim 1, wherein said step of obtaining more than two beat points in said audio data; acquiring the time interval between every two adjacent beat points; the step value of the audio data obtained according to the acquisition cycle and the time interval "is specifically:
Performing wavelet transform processing on the audio data to obtain a coefficient value of the audio data;
carrying out peak value detection on the coefficient value of the audio data to obtain more than two beat points;
Acquiring the time interval of two adjacent beat points in the original audio signal;
acquiring a time interval with the maximum occurrence probability;
and acquiring a beat value according to the acquisition period and the time interval with the maximum occurrence probability.
7. the method as claimed in claim 6, wherein said step of performing wavelet transform processing on said audio data to obtain coefficient values of the audio data further comprises:
And normalizing the quantization bit number of the audio data.
8. The method for displaying song tempo of claim 6, wherein after said obtaining the tempo value, further comprising:
constructing a beat interval histogram according to the time interval between every two adjacent beat values;
smoothing the beat interval histogram;
and acquiring the time interval with the highest occurrence probability in the smoothed beat interval histogram to obtain a beat value mean value.
9. The method for displaying song tempo according to claim 8, wherein the step of obtaining the time interval with the highest occurrence probability in the smoothed beat interval histogram to obtain the average value of the beat values specifically comprises:
acquiring a time interval with the highest occurrence probability in the beat interval histogram after smoothing processing;
according to the formula: and obtaining a beat value mean value, wherein the beat value is the preset time interval with the highest acquisition cycle/probability.
10. the method for displaying song rhythm of claim 8, wherein after obtaining the mean value of tempo values, further comprising:
acquiring an initial beat point of the audio data;
And acquiring all the beat points of the audio data according to the initial beat point and the beat value mean value.
11. a song tempo display system comprising:
the preset module is used for acquiring audio data of the song according to a preset acquisition cycle;
the first acquisition module is used for acquiring the frequency amplitude of the audio data;
the second acquisition module is used for acquiring the beat value of the audio data;
the display module is used for displaying the audio data according to the beat value and the frequency amplitude, wherein the display speed of the audio amplitude is determined according to the beat value;
The second acquisition module includes:
the first acquisition unit is used for acquiring more than two beat points in the audio data;
a second acquisition unit configured to acquire a time interval between every two adjacent beat points;
and the third acquisition unit is used for acquiring the beat value of the audio data according to the acquisition cycle and the time interval.
12. the display system of song tempos of claim 11,
the first obtaining unit is specifically configured to perform wavelet transform processing on the audio data to obtain a coefficient value of the audio data; carrying out peak value detection on the detail coefficient value of the audio data to obtain more than two beat points;
The second obtaining unit is specifically configured to obtain a time interval between two adjacent beat points in the original audio signal; acquiring a time interval with the maximum occurrence probability;
And the third acquisition unit is specifically used for acquiring the beat value according to the acquisition cycle and the time interval with the maximum occurrence probability.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510827808.1A CN105513583B (en) | 2015-11-25 | 2015-11-25 | song rhythm display method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510827808.1A CN105513583B (en) | 2015-11-25 | 2015-11-25 | song rhythm display method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105513583A CN105513583A (en) | 2016-04-20 |
| CN105513583B true CN105513583B (en) | 2019-12-17 |
Family
ID=55721512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510827808.1A Active CN105513583B (en) | 2015-11-25 | 2015-11-25 | song rhythm display method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105513583B (en) |
Families Citing this family (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106373595A (en) * | 2016-08-31 | 2017-02-01 | 天脉聚源（北京）科技有限公司 | Method and device for displaying audio information |
| CN106340307A (en) * | 2016-08-31 | 2017-01-18 | 天脉聚源（北京）科技有限公司 | Method and device used for displaying audio information |
| CN106356045A (en) * | 2016-08-31 | 2017-01-25 | 天脉聚源（北京）科技有限公司 | Audio information display method and device |
| GB201620839D0 (en) * | 2016-12-07 | 2017-01-18 | Weav Music Ltd | Data format |
| CN106652981B (en) * | 2016-12-28 | 2019-09-13 | 广州酷狗计算机科技有限公司 | BPM detection method and device |
| JP6930547B2 (en) * | 2017-01-27 | 2021-09-01 | ソニーグループ株式会社 | Information processing equipment, information processing methods and their programs |
| CN107452361B (en) * | 2017-08-08 | 2020-07-07 | 腾讯音乐娱乐（深圳）有限公司 | Song sentence dividing method and device |
| CN107545883A (en) * | 2017-10-13 | 2018-01-05 | 广州酷狗计算机科技有限公司 | The method and apparatus for determining the rhythm speed grade of music |
| CN108109611A (en) * | 2017-12-11 | 2018-06-01 | 大连高马艺术设计工程有限公司 | A kind of learning aids system of visual control fundamental tone and overtone and synthetic video |
| CN108335688B (en) * | 2017-12-28 | 2021-07-06 | 广州市百果园信息技术有限公司 | Method for detecting main beat point in music, computer storage medium and terminal |
| CN108108457B (en) * | 2017-12-28 | 2020-11-03 | 广州市百果园信息技术有限公司 | Method, storage medium, and terminal for extracting large tempo information from music tempo points |
| CN108319657B (en) * | 2018-01-04 | 2022-02-01 | 广州市百果园信息技术有限公司 | Method for detecting strong rhythm point, storage medium and terminal |
| CN108320730B (en) * | 2018-01-09 | 2020-09-29 | 广州市百果园信息技术有限公司 | Music classification method, beat point detection method, storage device and computer device |
| CN108305604B (en) * | 2018-01-30 | 2020-11-24 | 浙江省公众信息产业有限公司 | Music visualization method and device and computer readable storage medium |
| CN108335703B (en) * | 2018-03-28 | 2020-10-09 | 腾讯音乐娱乐科技（深圳）有限公司 | Method and apparatus for determining accent position of audio data |
| CN108495208A (en) * | 2018-03-29 | 2018-09-04 | 云谷（固安）科技有限公司 | Flexible screen speaker and its control method |
| CA3037908A1 (en) * | 2019-03-25 | 2020-09-25 | Kemal S. Ahmed | Beat tracking visualization through textual medium |
| CN110085253A (en) * | 2019-05-09 | 2019-08-02 | 广州小鹏汽车科技有限公司 | A kind of control method, device, vehicle and the storage medium of Music Visualization information |
| CN110390943B (en) * | 2019-06-28 | 2022-07-08 | 上海元笛软件有限公司 | Audio synthesis method and device, computer equipment and storage medium |
| CN110717053A (en) * | 2019-10-17 | 2020-01-21 | 广州酷狗计算机科技有限公司 | Picture display method, device, terminal and storage medium based on songs |
| CN111128100B (en) | 2019-12-20 | 2021-04-20 | 网易（杭州）网络有限公司 | Rhythm point detection method and device and electronic equipment |
| CN112973092B (en) * | 2021-02-02 | 2022-03-25 | 随机漫步(上海)体育科技有限公司 | Training assisting method and device, storage medium, electronic equipment and bicycle |
| CN111773620A (en) * | 2020-07-01 | 2020-10-16 | 随机漫步(上海)体育科技有限公司 | Method and device for assisting bicycle training and method and device for training network model |
| CN111859015A (en) * | 2020-07-01 | 2020-10-30 | 腾讯音乐娱乐科技（深圳）有限公司 | Music response method and related equipment |
| CN113823325B (en) * | 2021-06-03 | 2024-08-16 | 腾讯科技（北京）有限公司 | Audio rhythm detection method, device, equipment and medium |
| CN114356470A (en) * | 2021-11-26 | 2022-04-15 | 惠州华阳通用电子有限公司 | Dynamic display method and system of audio playing control |
| CN114630140A (en) * | 2022-03-17 | 2022-06-14 | 阿里巴巴（中国）有限公司 | Information setting method and device based on audio data |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3743757A (en) * | 1971-10-18 | 1973-07-03 | Nippon Musical Instruments Mfg | Automatic rhythm instrument with temporary roll beat circuit |
| JPH08190394A (en) * | 1995-01-09 | 1996-07-23 | Taito Corp | Method for displaying graphics interlocked to music content on karaoke sing-along machine and device therefor |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030205124A1 (en) * | 2002-05-01 | 2003-11-06 | Foote Jonathan T. | Method and system for retrieving and sequencing music by rhythmic similarity |
| JP2006106818A (en) * | 2004-09-30 | 2006-04-20 | Toshiba Corp | Music retrieval device, music retrieval method and music retrieval program |
| CN2833663Y (en) * | 2005-10-20 | 2006-11-01 | 深圳市蔚科电子科技开发有限公司 | MP3 player with beating function |
| US7645929B2 (en) * | 2006-09-11 | 2010-01-12 | Hewlett-Packard Development Company, L.P. | Computational music-tempo estimation |
| CN101645268B (en) * | 2009-08-19 | 2012-03-14 | 李宋 | Computer real-time analysis system for singing and playing |
| CN103578478B (en) * | 2013-11-11 | 2016-08-17 | 科大讯飞股份有限公司 | Obtain the method and system of musical tempo information in real time |
-
2015
- 2015-11-25 CN CN201510827808.1A patent/CN105513583B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3743757A (en) * | 1971-10-18 | 1973-07-03 | Nippon Musical Instruments Mfg | Automatic rhythm instrument with temporary roll beat circuit |
| JPH08190394A (en) * | 1995-01-09 | 1996-07-23 | Taito Corp | Method for displaying graphics interlocked to music content on karaoke sing-along machine and device therefor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105513583A (en) | 2016-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105513583B (en) | song rhythm display method and system | |
| US8907195B1 (en) | Method and apparatus for musical training | |
| US9552741B2 (en) | Systems and methods for quantifying a sound into dynamic pitch-based graphs | |
| US8853516B2 (en) | Audio analysis apparatus | |
| KR20150016225A (en) | Automatic conversion of speech into song, rap or other audible expression having target meter or rhythm | |
| JP6759545B2 (en) | Evaluation device and program | |
| JP6627482B2 (en) | Technique determination device | |
| WO2017090720A1 (en) | Technique determining device and recording medium | |
| Müller et al. | Interactive fundamental frequency estimation with applications to ethnomusicological research | |
| CN113674723B (en) | Audio processing method, computer equipment and readable storage medium | |
| KR101221673B1 (en) | Apparatus for practicing electric guitar performance | |
| CN110751935A (en) | Method for determining musical instrument playing point and scoring rhythm | |
| US10546566B2 (en) | Systems and methods for detecting musical features in audio content | |
| CN104978329A (en) | Method for generating musical melody in line drawing form | |
| WO2022172576A1 (en) | Sound generation method using machine learning model, method of training machine learning model, sound generation device, training device, sound generation program, and training program | |
| JP5012263B2 (en) | Performance clock generating device, data reproducing device, performance clock generating method, data reproducing method and program | |
| CN115619897A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| Chudy et al. | On the relation between gesture, tone production and perception in classical cello performance | |
| EP3786942A1 (en) | Musical instrument tuning | |
| Gu | Recognition algorithm of piano playing music in intelligent background | |
| JP2009014923A (en) | Musical performance clock generating device, data reproducing device, musical performance clock generating method, data reproducing method, and program | |
| US10395669B2 (en) | Voice analysis apparatus, voice analysis method, and program | |
| WO2017164216A1 (en) | Acoustic processing method and acoustic processing device | |
| CN113488007B (en) | Information processing method, information processing device, electronic equipment and storage medium | |
| WO2016039465A1 (en) | Acoustic analysis device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |