CN108172210A - A kind of performance harmony generation method based on song rhythm - Google Patents
A kind of performance harmony generation method based on song rhythm Download PDFInfo
- Publication number
- CN108172210A CN108172210A CN201810101219.9A CN201810101219A CN108172210A CN 108172210 A CN108172210 A CN 108172210A CN 201810101219 A CN201810101219 A CN 201810101219A CN 108172210 A CN108172210 A CN 108172210A
- Authority
- CN
- China
- Prior art keywords
- frame
- song
- bpm
- harmony
- performance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H7/00—Instruments in which the tones are synthesised from a data store, e.g. computer organs
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/315—Sound category-dependent sound synthesis processes [Gensound] for musical use; Sound category-specific synthesis-controlling parameters or control means therefor
- G10H2250/455—Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis
Abstract
The present invention relates to a kind of performance harmony generation methods based on song rhythm.From the application for singing harmony, rhythm detection is carried out to the performance song of singer based on spectral flux, for the retardation for adaptively adjusting harmony part according to song rhythm to generate harmony, can simplify beat extraction process reduces time complexity, and the musical form of abundant soloist.A kind of performance harmony generation method based on song rhythm proposed by the invention, this method is simple, realizes flexibly, has stronger practicability.
Description
Technical field
The present invention relates to songs to synthesize field, particularly a kind of performance harmony generation method based on song rhythm.
Background technology
Song is analyzed it and is studied with important meaning as a kind of more complicated audio signal and artistic expression
Justice.With popularizing for music entertainment, the hot spot of research and application is treated as the sound effect of music voice, by science
Boundary and the extensive concern of industry.Although the audio effect processing technology applied relative maturity is sung for K, due to itself voice
And the limitation of ability is sung, it is difficult to the performance harmonization of oneself for a user.Therefore, how research is based on singer's sound
Sound feature generates harmony and how the harmony adapted to according to the generation of song rhythm has very important actual application value.
Invention content
The purpose of the present invention is to provide a kind of performance harmony generation methods based on song rhythm, can be according to beat
Speed adaptively generates harmony, in the form of enriching the musical of soloist.
To achieve the above object, the technical scheme is that：A kind of performance harmony generation method based on song rhythm,
It is characterized in that, it realizes in accordance with the following steps：
Step S1：The song audio signal of singing opera arias of input is pre-processed, pretreatment mode includes：Filtering, preemphasis
And normalization；
Step S2：Framing is carried out, and calculate the logarithmic spectrum of each frame to pretreated song audio x (n)
Step S3：By logarithm spectral sequenceThe spectral flux SF (n) of singing voice signals is calculated, passes it through low-pass filtering
As endpoint intensity curve F (t) after smooth, the autocorrelation sequence TG (τ) of endpoint intensity curve is then calculated, TG (τ) is made to take most
The τ being worth greatly is the beat period, thus can calculate BPM characteristic values；
Step S4：It calculates the average BPM characteristic values of whole section of input singing voice signals and is denoted as BPM, harmony part is calculated by BPM
Retardation delay；
Step S5：It replicates song audio x (n) after portion pre-processes and its pitch is promoted into tierce journey, then by prolonging
When device generation harmony part h (n)；
Step S6：It is generated performance that primary sound portion x (n) is superimposed output y (n) with harmony part h (n) linear scales
Harmony.
In an embodiment of the present invention, in the step S2, the logarithmic spectrum of each frameCalculating according to following step
It is rapid to realize：
Step S21：Hop is moved to obtaining x after song audio framing of singing opera arias according to the frame length K of each frame and each frame framei
(n)；
Step S22：To xi(n) it carries out short time discrete Fourier transform and obtains frequency domain signal Xi(k)；
Step S23：According to formulaObtain logarithm spectral sequence
In an embodiment of the present invention, the frame length K is the hits in 10ms to 30ms, and the time of each frames of K=is long
Spend * sample frequencys；The frame moves hop as the adjacent underlapped part of two frames, hop=K/3.
In an embodiment of the present invention, in the step S3, the spectral flux SF (n) is：
Wherein, n is frame number in formula, and K is frame length, and H (x) is halfwave rectifier function；
The autocorrelation sequence TG (τ) is：
TG (τ)=W (τ) ∑ F (t) F (t- τ)；
Wherein, W (τ) is gaussian weighing function；
The BPM characteristic values are：
BPM=60*fs/hop* τmax；
Wherein, fs is sample rate, and hop is moved for frame, τmaxFor the beat period.
In an embodiment of the present invention, in the step S4, it is as follows to implement step：
Step S41：The every 2 seconds BPM characteristic values for calculating one singing voice signals of extraction, the BPM characteristic values of whole section of time signal
The average value of sequence isAnd it is denoted as BPM；
Step S42：Setting postpones umber of beats D and according to formulaComputing relay amount delay.
In an embodiment of the present invention, in the step S5, the method for improving of the pitch is using the sound for stablizing tone color
High conversion method.
In an embodiment of the present invention, in the step S5, the tierce journey is the tierce journey not exclusively coordinated,
I.e. pitch is original 2^ (3/12) or 2^ (4/12) times.
In an embodiment of the present invention, in the step S6, the formula of the linear scale superposition is：
Y (n)=x (n)+k*h (n)；
Wherein, k is psychrometric ratio, takes k=0.8 that can obtain better effect.
Compared to the prior art, the invention has the advantages that：The present invention proposes a kind of based on song rhythm
Harmony generation method is sung, from the application for singing harmony, can simplify beat extraction process reduces time complexity, and energy
Harmony is adaptively generated according to the speed of beat, the musical form of soloist can be enriched.This method is simple, realizes spirit
It is living, there is stronger practicability.
Description of the drawings
Fig. 1 is the flow chart of the performance harmony generation method based on song rhythm in the present invention.
Specific embodiment
Below in conjunction with the accompanying drawings, technical scheme of the present invention is specifically described.
The present invention proposes a kind of performance harmony generation method based on song rhythm, is broadly divided into three ranks as shown in Figure 1
Section：In rhythm detection-phase, for song and human hearing characteristic proposition flux filter method is sung, calculated using spectral flux
To endpoint intensity curve, and then extract BPM characteristic values；In harmony part generation phase, propose to sing harmony for harmony is sung
Generating algorithm calculates harmony part retardation according to BPM characteristic values dynamic, is generated using the pitch transfer algorithm for stablizing tone color same
People's harmony part；In superposition synthesis phase, harmony is sung using linear scale superposition output according to retardation and psychrometric ratio.Specifically
It is as follows：
Step S1：Calculate the log spectrum of song audio signal：Entire song audio signal is filtered first, pre-add
The pretreatments such as weight, normalization.Then it is K according to frame by obtained voice signal, frame shifting is that hop is divided into the speech frame of segment and obtains
xi(n), wherein, the time span * sample frequencys of each frames of K=, hop=K/3.Each frame is handled as follows：By xi(n)
X is obtained through short time discrete Fourier transformi(k)=STFT (xi(n)), then according to formulaObtained logarithm frequency
Spectral sequence
Step S2：Calculate BPM characteristic values：By log spectrum sequenceThe spectral flux SF (n) of singing voice signals is calculated,
Then it passes it through low-pass filtering and is smoothly used as endpoint intensity curve F (t) afterwards；Calculate the autocorrelation sequence TG of endpoint intensity curve
(τ), and autocorrelation sequence is weighted using Gauss function, it is the beat period to make the τ that TG (τ) is maximized, according to
Formula BPM=60*fs/hop* τmaxBPM characteristic values are calculated.
Step S3：Calculate average tempo：The every 2 seconds BPM characteristic values for calculating one singing voice signals of extraction, whole section of time signal
The average value of BPM characteristic value sequences be
Step S4：Computing relay amount：IfAccording to formulaCalculate harmony part retardation
Otherwise delay illustrates that BPM characteristic values are not dealt with beyond process range.
Step S5：Generate harmony part：Using the pitch conversion method for stablizing tone color, replicate original signal and carry its pitch
The tierce journey not exclusively coordinated is upgraded to, i.e. pitch is original 2^ (3/12) or 2^ (4/12) times, and phase is obtained by delayer
To the harmony part signal h (n) of main part delay delay.
Step S6：Linear scale is superimposed：According to formula y (n)=x (n)+k*h (n), by primary sound portion x (n) and harmony part h
(n) linear superposition output y (n) is generated performance harmony.Wherein, k is psychrometric ratio, takes k=0.8 that can obtain preferably
Effect.
The above are preferred embodiments of the present invention, all any changes made according to the technical solution of the present invention, and generated function is made
During with range without departing from technical solution of the present invention, all belong to the scope of protection of the present invention.
Claims (8)
1. a kind of performance harmony generation method based on song rhythm, which is characterized in that realize in accordance with the following steps：
Step S1：The song audio signal of singing opera arias of input is pre-processed, pretreatment mode includes：Filtering and is returned at preemphasis
One changes；
Step S2：Framing is carried out, and calculate the logarithmic spectrum of each frame to pretreated song audio x (n)
Step S3：By logarithm spectral sequenceThe spectral flux SF (n) of singing voice signals is calculated, it is smooth to pass it through low-pass filtering
Afterwards as endpoint intensity curve F (t), the autocorrelation sequence TG (τ) of endpoint intensity curve is then calculated, is maximized TG (τ)
τ be the beat period, thus can calculate BPM characteristic values；
Step S4：It calculates the average BPM characteristic values of whole section of input singing voice signals and is denoted as BPM, the delay of harmony part is calculated by BPM
Measure delay；
Step S5：It replicates song audio x (n) after portion pre-processes and its pitch is promoted into tierce journey, then pass through delayer
Generate harmony part h (n)；
Step S6：Primary sound portion x (n) is superimposed with harmony part h (n) linear scales output y (n) be generated performance and
Sound.
2. a kind of performance harmony generation method based on song rhythm according to claim 1, which is characterized in that described
In step S2, the logarithmic spectrum of each frameCalculating realized according to following steps：
Step S21：Hop is moved to obtaining x after song audio framing of singing opera arias according to the frame length K of each frame and each frame framei(n)；
Step S22：To xi(n) it carries out short time discrete Fourier transform and obtains frequency domain signal Xi(k)；
Step S23：According to formulaObtain logarithm spectral sequence
A kind of 3. performance harmony generation method based on song rhythm according to claim 2, which is characterized in that the frame
Long K be 10ms to 30ms in hits, the time span * sample frequencys of each frames of K=；It is adjacent two frame that the frame, which moves hop,
Underlapped part, hop=K/3.
4. a kind of performance harmony generation method based on song rhythm according to claim 1, which is characterized in that described
In step S3, the spectral flux SF (n) is：
Wherein, n is frame number in formula, and K is frame length, and H (x) is halfwave rectifier function；
The autocorrelation sequence TG (τ) is：
TG (τ)=W (τ) ∑ F (t) F (t- τ)；
Wherein, W (τ) is gaussian weighing function；
The BPM characteristic values are：
BPM=60*fs/hop* τmax；
Wherein, fs is sample rate, and hop is moved for frame, τmaxFor the beat period.
5. a kind of performance harmony generation method based on song rhythm according to claim 1, which is characterized in that described
Step S4, specific implementation step are as follows：
Step S41：The every 2 seconds BPM characteristic values for calculating one singing voice signals of extraction, the BPM characteristic value sequences of whole section of time signal
Average value beAnd it is denoted as BPM；
Step S42：Setting postpones umber of beats D and according to formulaComputing relay amount delay.
6. a kind of performance harmony generation method based on song rhythm according to claim 1, which is characterized in that described
In step S5, the method for improving of the pitch is using the pitch conversion method for stablizing tone color.
7. a kind of performance harmony generation method based on song rhythm according to claim 1, which is characterized in that described
In step S5, the tierce journey is the tierce journey not exclusively coordinated, i.e., pitch is original 2^ (3/12) or 2^ (4/12)
Times.
8. a kind of performance harmony generation method based on song rhythm according to claim 1, which is characterized in that described
In step S6, the formula of the linear scale superposition is：
Y (n)=x (n)+k*h (n)；
Wherein, k is psychrometric ratio, takes k=0.8 that can obtain better effect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810101219.9A CN108172210B (en) | 2018-02-01 | 2018-02-01 | Singing harmony generation method based on singing voice rhythm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810101219.9A CN108172210B (en) | 2018-02-01 | 2018-02-01 | Singing harmony generation method based on singing voice rhythm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108172210A true CN108172210A (en) | 2018-06-15 |
| CN108172210B CN108172210B (en) | 2021-03-02 |
Family
ID=62512557
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810101219.9A Expired - Fee Related CN108172210B (en) | 2018-02-01 | 2018-02-01 | Singing harmony generation method based on singing voice rhythm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108172210B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109545176A (en) * | 2019-01-21 | 2019-03-29 | 北京小唱科技有限公司 | Dynamic echo processing method and processing device for audio |
| CN109920449A (en) * | 2019-03-18 | 2019-06-21 | 广州市百果园网络科技有限公司 | Beat analysis method, audio-frequency processing method and device, equipment, medium |
| CN110853604A (en) * | 2019-10-30 | 2020-02-28 | 西安交通大学 | Automatic generation method of Chinese folk songs with specific region style based on variational self-encoder |
| CN112908289A (en) * | 2021-03-10 | 2021-06-04 | 百果园技术(新加坡)有限公司 | Beat determining method, device, equipment and storage medium |
| CN113411663A (en) * | 2021-04-30 | 2021-09-17 | 成都东方盛行电子有限责任公司 | Music beat extraction method for non-woven engineering |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1134580A (en) * | 1995-02-02 | 1996-10-30 | 雅马哈株式会社 | Harmony chorus apparatus generating chorus sound derived from vocal sound |
| CN1153964A (en) * | 1995-02-27 | 1997-07-09 | 雅马哈株式会社 | Karaoke apparatus creating virtual harmony voice over actual singing voice |
| JP2001117578A (en) * | 1999-10-21 | 2001-04-27 | Yamaha Corp | Device and method for adding harmony sound |
| US6816833B1 (en) * | 1997-10-31 | 2004-11-09 | Yamaha Corporation | Audio signal processor with pitch and effect control |
| CN102568457A (en) * | 2011-12-23 | 2012-07-11 | 深圳市万兴软件有限公司 | Music synthesis method and device based on humming input |
| CN102568454A (en) * | 2011-12-13 | 2012-07-11 | 北京百度网讯科技有限公司 | Method and device for analyzing music BPM (Beat Per Minutes) |
| CN105070283A (en) * | 2015-08-27 | 2015-11-18 | 百度在线网络技术（北京）有限公司 | Singing voice scoring method and apparatus |
| CN105659322A (en) * | 2013-09-19 | 2016-06-08 | 微软技术许可有限责任公司 | Recommending audio sample combinations |
| CN106228973A (en) * | 2016-07-21 | 2016-12-14 | 福州大学 | Stablize the music voice modified tone method of tone color |
| CN106373580A (en) * | 2016-09-05 | 2017-02-01 | 北京百度网讯科技有限公司 | Singing synthesis method based on artificial intelligence and device |
| CN106653037A (en) * | 2015-11-03 | 2017-05-10 | 广州酷狗计算机科技有限公司 | Audio data processing method and device |
| US20170221466A1 (en) * | 2012-10-19 | 2017-08-03 | Sing Trix Llc | Vocal processing with accompaniment music input |
-
2018
- 2018-02-01 CN CN201810101219.9A patent/CN108172210B/en not_active Expired - Fee Related
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1134580A (en) * | 1995-02-02 | 1996-10-30 | 雅马哈株式会社 | Harmony chorus apparatus generating chorus sound derived from vocal sound |
| CN1153964A (en) * | 1995-02-27 | 1997-07-09 | 雅马哈株式会社 | Karaoke apparatus creating virtual harmony voice over actual singing voice |
| US6816833B1 (en) * | 1997-10-31 | 2004-11-09 | Yamaha Corporation | Audio signal processor with pitch and effect control |
| JP2001117578A (en) * | 1999-10-21 | 2001-04-27 | Yamaha Corp | Device and method for adding harmony sound |
| CN102568454A (en) * | 2011-12-13 | 2012-07-11 | 北京百度网讯科技有限公司 | Method and device for analyzing music BPM (Beat Per Minutes) |
| CN102568457A (en) * | 2011-12-23 | 2012-07-11 | 深圳市万兴软件有限公司 | Music synthesis method and device based on humming input |
| US20170221466A1 (en) * | 2012-10-19 | 2017-08-03 | Sing Trix Llc | Vocal processing with accompaniment music input |
| CN105659322A (en) * | 2013-09-19 | 2016-06-08 | 微软技术许可有限责任公司 | Recommending audio sample combinations |
| CN105070283A (en) * | 2015-08-27 | 2015-11-18 | 百度在线网络技术（北京）有限公司 | Singing voice scoring method and apparatus |
| CN106653037A (en) * | 2015-11-03 | 2017-05-10 | 广州酷狗计算机科技有限公司 | Audio data processing method and device |
| CN106228973A (en) * | 2016-07-21 | 2016-12-14 | 福州大学 | Stablize the music voice modified tone method of tone color |
| CN106373580A (en) * | 2016-09-05 | 2017-02-01 | 北京百度网讯科技有限公司 | Singing synthesis method based on artificial intelligence and device |
Non-Patent Citations (3)
| Title |
|---|
| ALONSO M.: ""tempo and beat estimation of musical signals"", 《INTERNATIONAL SYMPOSIUM ON MUSIC INFORMATION RETRIEVAL》 * |
| HAYATO KUMAGAI: ""synchronization method for improving temporal harmony of music and video clips"", 《INTERNATIONAL CONFERENCE ON APPLIED COMPUTING & INFORMATION TECHNOLOGY/INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE& INTELLIGENCE 》 * |
| 王孝欣: ""一种基于简单自相关的基音周期搜索算法"", 《工业控制计算机》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109545176A (en) * | 2019-01-21 | 2019-03-29 | 北京小唱科技有限公司 | Dynamic echo processing method and processing device for audio |
| CN109545176B (en) * | 2019-01-21 | 2022-03-04 | 北京小唱科技有限公司 | Dynamic echo processing method and device for audio |
| CN109920449A (en) * | 2019-03-18 | 2019-06-21 | 广州市百果园网络科技有限公司 | Beat analysis method, audio-frequency processing method and device, equipment, medium |
| CN110853604A (en) * | 2019-10-30 | 2020-02-28 | 西安交通大学 | Automatic generation method of Chinese folk songs with specific region style based on variational self-encoder |
| CN112908289A (en) * | 2021-03-10 | 2021-06-04 | 百果园技术(新加坡)有限公司 | Beat determining method, device, equipment and storage medium |
| CN112908289B (en) * | 2021-03-10 | 2023-11-07 | 百果园技术(新加坡)有限公司 | Beat determining method, device, equipment and storage medium |
| CN113411663A (en) * | 2021-04-30 | 2021-09-17 | 成都东方盛行电子有限责任公司 | Music beat extraction method for non-woven engineering |
| CN113411663B (en) * | 2021-04-30 | 2023-02-21 | 成都东方盛行电子有限责任公司 | Music beat extraction method for non-woven engineering |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108172210B (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108172210A (en) | A kind of performance harmony generation method based on song rhythm | |
| JP6290858B2 (en) | Computer processing method, apparatus, and computer program product for automatically converting input audio encoding of speech into output rhythmically harmonizing with target song | |
| CN102054480B (en) | Method for separating monaural overlapping speeches based on fractional Fourier transform (FrFT) | |
| US10825438B2 (en) | Electronic musical instrument, musical sound generating method of electronic musical instrument, and storage medium | |
| JPH0756587A (en) | Mark marking device of song in recorded instrumental accompaniement system | |
| CN104282316A (en) | Karaoke scoring method based on voice matching, and device thereof | |
| CN110136730B (en) | Deep learning-based piano and acoustic automatic configuration system and method | |
| CN110516102B (en) | Lyric time stamp generation method based on spectrogram recognition | |
| KR101840015B1 (en) | Music Accompaniment Extraction Method for Stereophonic Songs | |
| Kumar et al. | Musical onset detection on carnatic percussion instruments | |
| Elowsson et al. | Modelling perception of speed in music audio | |
| Benetos et al. | Auditory spectrum-based pitched instrument onset detection | |
| JP2013164584A (en) | Acoustic processor | |
| TWI419150B (en) | Singing and grading system | |
| Ellis et al. | Inharmonic speech: a tool for the study of speech perception and separation | |
| Azarov et al. | Guslar: a framework for automated singing voice correction | |
| Bonada et al. | Generation of growl-type voice qualities by spectral morphing | |
| Coyle et al. | Onset detection using comb filters | |
| CN108538309A (en) | A kind of method of song detecting | |
| Bonjyotsna et al. | Analytical study of vocal vibrato and mordent of Indian popular singers | |
| Theimer et al. | Definitions of audio features for music content description | |
| Yang et al. | Singing voice separation based on deep regression neural network | |
| Sinith et al. | Real-time swara recognition system in Indian Music using TMS320C6713 | |
| Wang et al. | Beijing opera synthesis based on straight algorithm and deep learning | |
| Li | Automatic Piano Harmony Arrangement System Based on Deep Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210302 Termination date: 20220201 |