CN104391980B - The method and apparatus for generating song - Google Patents
The method and apparatus for generating song Download PDFInfo
- Publication number
- CN104391980B CN104391980B CN201410743457.1A CN201410743457A CN104391980B CN 104391980 B CN104391980 B CN 104391980B CN 201410743457 A CN201410743457 A CN 201410743457A CN 104391980 B CN104391980 B CN 104391980B
- Authority
- CN
- China
- Prior art keywords
- text
- song
- sentence
- feature
- lyrics
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
- G06F16/61—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
- G06F16/68—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/683—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/685—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using automatically derived transcript of audio data, e.g. lyrics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of method and apparatus for generating song.This method comprises: receiving the text of user's input；Extract the text feature information of the text；Obtain the song with the text feature information matches；Speech synthesis processing is carried out to the text according to the song, obtains the new song that the lyrics are the text.This programme, after the text for receiving user's input, by the text feature information for extracting text, and the text feature information is matched with the corresponding informance of the lyrics of existing song, so as to obtain the song with the text feature information matches, by dubbing in background music the song of successful match as background music, and using the text as the new lyrics for corresponding to the background music, and by speech synthesis, realize text generation song.
Description
Technical field
The present embodiments relate to technical field of information processing more particularly to a kind of method and apparatus for generating song.
Background technique
Text is the form of expression of written language, and a text can be a sentence, a paragraph or a chapter.
It include at present two ways for the processing of text, one is be added, delete or replace to text itself；Separately
To use speech synthesis technique by text-processing for speech form in one, content expressed by speech form and text representation it is interior
Hold identical.
But only have in the case of certain text or voice expression be it is inadequate, as people are having glad or sad feelings
It can want to go to express by song when thread, but due to the music theory knowledge of shortage system, it is not known that how to set a song to music, most multipotency is write as text
This, and cannot be still at present song by text-processing to the processing mode of text.
Summary of the invention
The embodiment of the present invention provides a kind of method and apparatus for generating song, and text is automatically become song to realize.
In a first aspect, the embodiment of the invention provides a kind of methods for generating song, comprising:
Receive the text of user's input；
Extract the text feature information of the text；
Obtain the song with the text feature information matches；
Speech synthesis processing is carried out to the text according to the song, obtains the new song that the lyrics are the text.
Second aspect, the embodiment of the invention provides a kind of devices for generating song, comprising:
Received text module, for receiving the text of user's input；
Characteristic extracting module, for extracting the text feature information of the text；
Matching module, for obtaining and the song of the text feature information matches；
Voice synthetic module obtains the lyrics for carrying out speech synthesis processing to the text according to the song as institute
State the new song of text.
The method and apparatus provided in an embodiment of the present invention for generating song pass through after the text for receiving user's input
It extracts the text feature information of text, and the corresponding informance of the text feature information and the lyrics of existing song is carried out
Match, so as to obtain passing through dubbing in background music the song of successful match as back with the song of the text feature information matches
Scape music, and using the text as the new lyrics for corresponding to the background music, and by speech synthesis, it realizes text
Song is generated, the mode of text-processing is increased, meets more user demands.
Detailed description of the invention
In order to illustrate more clearly of the present invention, one will be done to attached drawing needed in the present invention below and be simply situated between
It continues, it should be apparent that, drawings in the following description are some embodiments of the invention, and those of ordinary skill in the art are come
It says, without any creative labor, is also possible to obtain other drawings based on these drawings.
Fig. 1 a is a kind of flow chart of the method for generation song that the embodiment of the present invention one provides；
Fig. 1 b is " to wish in the method provided in an embodiment of the present invention for generating song for carrying out matched song with text
Good fortune " information schematic diagram；
Fig. 1 c is the schematic diagram that the text of user's input is received in the method provided in an embodiment of the present invention for generating song；
After Fig. 1 d is receives text shown in Fig. 1 c, it is in using the method for the generation song of the offer of the embodiment of the present invention one
Existing effect picture；
Fig. 2 is to obtain in the method provided by Embodiment 2 of the present invention for generating song from the song data library lookup pre-established
To the method flow diagram of the song with the text feature information matches；
Fig. 3 is a kind of structural schematic diagram of the device for generation song that the embodiment of the present invention three provides.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to the embodiment of the present invention
In technical solution be described in further detail, it is clear that described embodiments are some of the embodiments of the present invention, rather than complete
The embodiment in portion.It is understood that described herein, the specific embodiments are only for explaining the present invention, rather than to of the invention
It limits, based on the embodiments of the present invention, those of ordinary skill in the art are obtained without creative efforts
Every other embodiment, shall fall within the protection scope of the present invention.It also should be noted that for ease of description, attached drawing
In only some but not all of the content related to the present invention is shown.
Embodiment one
Fig. 1 a is please referred to, is a kind of flow chart of the method for generation song that the embodiment of the present invention one provides.The present invention is real
The method for applying example can be executed by configuring the device of the generation song with hardware and/or software realization, which can be with
It is configured in chat software, such as wechat, QQ and footpath between fields footpath between fields, can also be configured at and be capable of providing singing service and Intelligent dialogue service
Server in, be such as capable of providing sing service and Intelligent dialogue service robot in, it is soft that all kinds of translations can also be configured at
In part, such as Baidu's translation.
This method comprises: operation 110~operation 140.
In operation 110, the text of user's input is received.
Wherein, the text of user's input can for user's creation want to go expression by song and reflection user it is glad
Or the text of sad mood, the non-user for being also possible to user's acquisition are original and reflect the glad or sad mood of user
Text, regardless of whether being that user is original, the text is all different from the lyrics of existing song.
In operation 120, the text feature information of the text is extracted.
Wherein, the text feature information for extracting the text, preferably includes:
Sentence cutting processing is carried out to the text, obtains the subordinate sentence that the text includes；
Extract the sentence characteristics for the subordinate sentence that the text includes.
Wherein, sentence cutting is mainly realized according to the punctuation marks such as fullstop, comma.
Further, the sentence characteristics for extracting the subordinate sentence that the text includes, preferably include:
Extract the character quantity of the subordinate sentence that the text includes, the simple or compound vowel of a Chinese syllable of each word and the tone of each word in the subordinate sentence
At least one of.
Text with user's input is that " the waning city of lights is brilliant, bustling with activity people boiling such as sea." for into
Row explanation.
According to the user input text in punctuation mark ", " and "." sentence cutting processing is carried out to the text, it obtains
The first subordinate sentence " the waning city of lights is brilliant " and the second subordinate sentence " bustling with activity people boiling such as sea " that the text includes, table
1 is the sentence characteristics for extracting the subordinate sentence that the text includes.
Table 1
In table 1, a sound, two sound, three sound, the four tones of standard Chinese pronunciation and softly corresponding tone uses " 1 ", " 2 ", " 3 ", " 4 " and " 0 " respectively
It indicates.
It should be noted that 3 sentence characteristics of subordinate sentence are listed in table 1, it can also be by the last character in subordinate sentence
Another sentence characteristics of simple or compound vowel of a Chinese syllable as subordinate sentence.
It should also be noted that, the text feature information of the text is other than it may include the sentence characteristics of subordinate sentence,
Can also include the affective characteristics of subordinate sentence, for example, actively, it is passive or neutral.
In operation 130, the song with the text feature information matches is obtained.
This operation is specifically, using the text feature information, to obtain and the text matches in existing song
Song.As previously mentioned, the text of user's input is different from the lyrics of existing song, therefore in this operation, specifically can will described in
Text feature information is matched with the corresponding informance of the lyrics of existing song, to obtain and the text feature information matches
Song.
In this operation, the sentence characteristics of text can be matched with the sentence characteristics of the lyrics in song, it can also be with
Other text feature information such as affective characteristics by subordinate sentence in text are matched with the character pair of the lyrics in song, can be with
The combinations matches of different text feature information.
Below to be illustrated for the sentence characteristics of the sentence characteristics of text and the lyrics.
Fig. 1 b is please referred to, there is shown the first segment lyrics of song " blessing " and the corresponding music score of Chinese operas of the first segment lyrics.
Similarly, the sentence characteristics of the lyrics can be obtained according to the operation for the sentence characteristics for extracting text.With " it not ask,
It not say, All is understood, and no word are necessary, at the moment, snuggles up to candle light, let us is quietly spent." for this part lyrics, extraction
Sentence characteristics are as shown in table 2.
Table 2
Similarly, the sentence characteristics of other available existing songs, by the way that the sentence of existing a large amount of songs is special
Sign is matched with the sentence characteristics of the text of user's input respectively, the available and text matched song of sentence characteristics.
As previously mentioned, can extract the text includes when the text feature information includes the sentence characteristics of text
The character quantity of subordinate sentence, at least one feature in the subordinate sentence in the simple or compound vowel of a Chinese syllable of each word and the tone of each word, in other words,
The present embodiment is not limited the quantity of the sentence characteristics of text, similarly, is obtaining and the text feature information matches
Song operation in, the quantity of the sentence characteristics of the lyrics in song is not limited, if get and text feature believe
Matching can be compared by ceasing corresponding song information.
It should be noted that with the corresponding informance of different text feature information and song, the effect of obtained new song
Fruit is different.For example, corresponding with song " character quantities of the lyrics " with " character quantity of subordinate sentence " in the sentence characteristics of text
It is matched, the song of obtained successful match can be good at the subordinate sentence number of words of compatible text, and timing is relatively good；For another example,
It is corresponding with song " in the lyrics each word corresponding simple or compound vowel of a Chinese syllable " with " simple or compound vowel of a Chinese syllable of each word in subordinate sentence " in the sentence characteristics of text
It is matched, the song of obtained successful match can be good at the corresponding rhythm of simple or compound vowel of a Chinese syllable of each word of compatible text；For another example,
Matched with the affective characteristics of text with affective characteristics corresponding in song, the song of obtained successful match be able to maintain with
The consistent emotion keynote of text of user's input.
In operation 140, speech synthesis processing is carried out to the text according to the song, obtaining the lyrics is the text
New song.
The technical solution of the present embodiment, after the text for receiving user's input, the text feature by extracting text is believed
Breath, and the text feature information is matched with the corresponding informance of the lyrics of existing song, so as to obtain with it is described
The song of text feature information matches is made by dubbing in background music the song of successful match as background music, and by the text
For the new lyrics corresponding to the background music, and by speech synthesis, realizes text generation song, increase at text
The mode of reason meets more user demands, and user is not necessarily to the music theory knowledge for having system, can be realized as user's input
Text automatic music composing.
On the basis of the present embodiment, when preferably carrying out speech synthesis processing, by dubbing in background music as background for the song
Music, and the voice of corresponding beat is generated according to the music score of Chinese operas information, or corresponding beat and sound are generated according to the music score of Chinese operas information
High voice, obtains the new song, and the word content of the voice is the text.It in other words, will be with the text feature
The song of information matches is dubbed in background music as background music, and controls the text using with the music score of Chinese operas of the song of the text matches
Read aloud pause duration, obtain synthesis song；Or it can be by the conduct of dubbing in background music of the song with the text feature information matches
Background music, and using the performance pause duration and pitch for controlling the text with the music score of Chinese operas of the song of the text matches, it obtains
To synthesis song.
The method can further control the section of synthesis song while being embodied as the text automatic music composing of user's input
Bat and/or pitch.
The method provided in an embodiment of the present invention for generating song can be applied to several scenes, below with five kinds of application scenarios
For be illustrated respectively.
Application scenarios one: user is sung using chat software to be explained or blesses.
In the social chat software such as wechat, QQ, footpath between fields footpath between fields, if user is ashamed of explaining, the embodiment of the present invention can be used
Method, user need to only input passage, can set a song to music automatically for the text of user's input, to help user's romance to explain,
User can play the song automatically generated by the voice play function of chat software.
For example, user inputs the text of one section of oneself creation in chat software interface as illustrated in figure 1 c, it is as follows:
The waning city of lights is brilliant
Bustling with activity people boiling such as sea
It is chilly and quiet as you I year Central China it is clear and bright white
See that fireworks illuminate the haze of the night sky
The hand produced clouds with one turn of the hand and rain with another is confounded black and white
The sea turns into mulberry fields and vice versa for the dream of illusion
It is how many pure eventually all by common customs burial
Only your warm is always difficult to forget
One two, city people
Poly- dissipate of making a return journey several times is hovered
Whether the future that you promise to undertake still remains
One one, city people
One dream thousand of success and failure, gain and loss carries
Affection as high as the heaven and hatred as deep as the sea once
It is wind volume dust already
" key generates song and sends " button in the chat software interface of user click as illustrated in figure 1 c, automatically with song
Form play back, as shown in Figure 1 d.
Application scenarios two: by the way that for providing the robot of the service of singing and Intelligent dialogue service, this, which is sentenced, to sing
For little Du robot, the text which is used to input user generates new song by processing, then save with
It plays.
The following are scenario simulations:
User A:Hi, little Du robot
Small degree: Hi
User A: a song please is compiled for me
Small degree: it is good, it please say the lyrics
User A:(says passage)
Small degree: (with graceful song sing come).
User A: it listens very well！Please this song is sung and is listened to my Amy user B
Small degree: out of question！
(user A gives user B little Du robot)
User B:Hi, little Du robot
Small degree: it could you tell me to be user B
User B: yes
Small degree: user A has a first song to give you！(singing the new song generated according to the text that user A is said).
Application scenarios three: to the pithy formula class of user's input, the text that poem class etc. is not easy to recite is write music for a song automatically, is passed through
It sings to assist specific crowd (such as student) to remember.
For example, the text of user's input is as follows, it is pithy formula class text:
For the number of jack per line two to be added, absolute value adds not reversion.
Contrary sign mutually increases reduction, and big number determines and symbol.
Opposite number is summed each other, the result is that zero must remember.
A similar song is matched to above-mentioned pithy formula class text, student is allowed to help memory faster by study song.
Application scenarios four: it can translate to increase in (TTS) in Baidu and sing function.
In Baidu's translation, user inputs a Duan Wenben, and using the text of user's input as original language, existing Baidu
Translation supports by source language translation into another language, can will be defeated with user by increasing singing function in Baidu translates
The corresponding translation processing of the text entered, using the new song generated according to translation as the corresponding pronunciation of translation, due to according to translation
Include in the new song generated with the information such as the dubbing in background music of the matched song of translation, beat or the rhythm, therefore, user translates seeing
While literary, click corresponding with singing function Action Button can by song by way of sing translation, to increase
The interest for the translation pronunciation that user hears, is conducive to attract more users.
Such as: user input Chinese happy birthday song, Baidu translation can be transcribed into as Italian and will translate
Text, which is sung, to be come.
Application scenarios five: automatic music function is realized in Baidu music.
Increase an automatic music function for Baidu music, user can save one by uploading the lyrics of oneself creation
First one's own song is simultaneously shared with good friend.
Embodiment two
The present embodiment on the basis of the above embodiments, provide obtain with the songs of the text feature information matches this
The preferred embodiment of one operation, is specifically optimized for obtaining and the text feature information from the song data library lookup pre-established
Matched song.
Referring to Fig. 2, being obtained in the method provided in this embodiment for generating song from the song data library lookup pre-established
Method to the song with the text feature information matches specifically includes: operation 210~operation 240.
In operation 210, searches and obtained with the text feature information matches extremely in the song database pre-established
Few two songs.
The technical solution of the present embodiment includes offline part and online part, wherein the groundwork of offline part is song
Bent acquisition, feature extraction and the building of song database.The groundwork of online part is: receiving the text of user's input
This, and extract the text feature information of the text；It is searched from the song database pre-established special with the text
Reference ceases most matched song；Using text as the new lyrics, and using most matched song dub in background music and beat and/or sound
The control information such as rule generates song by speech synthesis technique.
This operation is specifically to build library offline first, that is, pre-establishes song database, then On-line matching, namely preparatory
It is searched in the song database of foundation and obtains at least two songs with the text feature information matches.
In operation 220, the syllable characteristic of the text is determined respectively according to the music score of Chinese operas of at least two songs.
" the waning city of lights is brilliant, bustling with activity people's boiling for the text inputted with aforementioned Fig. 1 b and aforementioned user
Such as sea." for be illustrated.By the text of user's input, " the waning city of lights is brilliant, and bustling with activity people's boiling is such as
Sea." sentence cutting processing is carried out, obtain the first subordinate sentence " the waning city of lights is brilliant " that the text includes and second point
Sentence " bustling with activity people boiling such as sea ", so as to obtain the sentence characteristics of text as shown in Table 1.
It should be noted that if each subordinate sentence of the text character quantity that includes is fewer, it can be by the sentence of text
Subcharacter is as text feature information, to obtain the song with the text feature information matches；If each subordinate sentence of text
The character quantity for including is relatively more, may cause and uses the sentence characteristics of text as text feature information, matches less than suitable
Song obtains the complete phrase segment (example of semanteme that each subordinate sentence includes at this time, it may be necessary to carry out the cutting of phrase segment to each subordinate sentence
Such as, the first phrase segment " the waning city of lights " and the second phrase segment " brilliant " that the first subordinate sentence includes), thus according to
The similar mode of the sentence characteristics of text is extracted, the feature (as shown in table 3) of each phrase segment is extracted, for example, each phrase piece
The character quantity of section, the simple or compound vowel of a Chinese syllable of each word and tone etc. in each phrase segment are special using the feature of each phrase segment as text
Reference breath improves song successful match rate to obtain the song with the text feature information matches.It needs to illustrate
It is that the phrase segment characterizations of text are similar with the function of the sentence characteristics of text, is provided to obtain and text feature information
The song matched.
Table 3
Assuming that song shown in Fig. 1 b is and wherein one at least two songs of the characteristic matching of the phrase segment
Song.
As shown in Figure 1 b, the lyrics of the song include two layers, and first layer is not from " asking, not say, all, which are all in, does not say
In " start, terminate to " fearness be only afraid of the cunning of tear gently ", the second layer is opened from " how much worrying, how much sorrow, life is inevitably bitter and bitterly "
Begin, terminates to " cold and hot dribs and drabs is in heart ".The first layer lyrics identical music score of Chinese operas corresponding with the second layer lyrics, the specifically lyrics
The numbers and symbols of top combines.Syllable characteristic according to the determining text of the music score of Chinese operas of the song is as shown in table 4.It needs
Bright, the syllable characteristic of the text may include the character quantity of syllable, lead-in in the simple or compound vowel of a Chinese syllable and syllable of lead-in in syllable
At least one of tone.
Table 4
Table 5 is the part syllable characteristic of song " blessing ".
Table 5
In operation 230, according to the syllable characteristic of the text, the text and at least two songs are calculated separately
Match total score.
In operation 240, the highest song of total score will be matched at least two songs, believed as with the text feature
Cease matched song.
It should be noted that with the syllable characteristic of different texts and the character pair of song, obtained new song
Effect is different.For example, with " character quantity of syllable " " number of characters of syllable corresponding with song in the syllable characteristic of text
Amount " is matched, and the song of obtained successful match can be good at the subordinate sentence number of words of compatible text, and timing is relatively good；Again
Such as, it is carried out with " simple or compound vowel of a Chinese syllable of lead-in in syllable " in the syllable characteristic of text " simple or compound vowel of a Chinese syllable of lead-in in syllable " corresponding with song
Match, the song of obtained successful match can be good at the corresponding rhythm of simple or compound vowel of a Chinese syllable of the lead-in of compatible text.
The technical solution of the present embodiment, including offline part and online part, first by acquisition song and corresponding
Feature, to realize the offline building of song database；Then the text of user's input is received online, and extracts the text of the text
Eigen information；The number of songs with the text feature information matches are searched from the song database pre-established,
Determine that the syllable of text is special respectively by the text feature information realization preliminary screening of song, and by the music score of Chinese operas of each song
Sign, the matching total score of the text and each song that are obtained based on syllable characteristic, so as to further screening with it is described
The most matched song of text, by dubbing in background music most matched song as background music, and using the text as corresponding to
The new lyrics of the background music, and by speech synthesis, user is not necessarily to the music theory knowledge for having system, can be realized as user
The text automatic music composing of input, the effect of the new song further improved.
As operation 230 a kind of specific embodiment, according to aforementioned hypothesis, it is assumed that song shown in Fig. 1 b for institute
The wherein song in matched at least two songs of phrase segment characterizations of text is stated, therefore can be by text shown in table 4
This syllable characteristic matches respectively with the syllable characteristic of the lyrics in song shown in table 5 " blessing ", to obtain user's input
The matching total score of song " blessing " in text and at least two songs.Similarly, the text of available user's input
With the respective matching total score of other songs at least two songs.
Further, it according to the syllable characteristic, calculates separately the text and the matching of at least two songs is total
Point, it preferably includes:
The matching total score of the text Yu at least two songs is calculated separately according to the following equation:
Wherein, Q indicates the text；S indicates any song at least two songs；Score (Q, S) is indicated
The matching total score of the text and any song；FiThe ith feature of (Q, S) expression text and any head
The matching degree of the character pair of song；λiIndicate the corresponding weight of the ith feature；The ith feature of the text is institute
The syllable characteristic for stating text can specifically include the character quantity of syllable in text, first in the simple or compound vowel of a Chinese syllable and syllable of lead-in in syllable
At least one of tone of word.
For example, calculating the matching degree of the character quantity of syllable and the character quantity of syllable in song " blessing " in the text
F1(Q, S), in the text in syllable in the simple or compound vowel of a Chinese syllable of lead-in and song " blessing " in syllable the simple or compound vowel of a Chinese syllable of lead-in matching degree F2(Q,
S) and in the text in syllable in the tone of lead-in and song " blessing " in syllable the tone of lead-in matching degree F3(Q,
S), then pass through the corresponding weight λ of character quantity of syllable in text1, the corresponding weight λ of simple or compound vowel of a Chinese syllable of lead-in in syllable2And syllable
Corresponding weight λ in the tone of middle lead-in3, to obtain the text of user's input and the matching total score of song " blessing ".
As previously mentioned, with the syllable characteristic of different texts and the character pair of song, the effect of obtained new song
It is different.
This preferred embodiment, in of the character pair for the syllable characteristic and song that different texts is calculated
After degree, by controlling the different corresponding weights of syllable characteristic, the higher sound of weight in the new song that can reinforce
The corresponding music effect of feature is saved, while weakening the corresponding music effect of the lower syllable characteristic of weight, is conducive to fining ground
Control the effect of obtained new music.
As another specific embodiment of operation 230, according to preceding description it is found that the phrase segment characterizations of text
It is similar with the function of the sentence characteristics of text, it is provided to obtain the song with text feature information matches, therefore for user
It, can be by song " blessing " shown in the syllable characteristic of text shown in table 4 and table 5 for the text of input and song " blessing "
The syllable characteristic of the middle lyrics matches respectively, and by song " blessing " shown in the phrase segment characterizations of text shown in table 3 and table 1
The sentence characteristics of the middle lyrics match respectively, so that the text for obtaining user's input " is wished with the song at least two songs
Good fortune " matching total score.Similarly, the text of available user's input and other songs at least two songs are respective
Match total score.
The matching total score of the text Yu at least two songs is preferably calculated separately according to the following equation:
Wherein, Q indicates the text；S indicates any song at least two songs；Score (Q, S) is indicated
The matching total score of the text and any song；FiThe ith feature of (Q, S) expression text and any head
The matching degree of the character pair of song；λiIndicate the corresponding weight of the ith feature；The ith feature of the text is institute
The syllable characteristic and the corresponding text feature of the text feature information for stating text, specific in this example, i-th of the text
Feature is the syllable characteristic of the text and the phrase segment characterizations of the text.
The difference of present embodiment and above embodiment is, calculates the ith feature of the text of matching total score
Range it is different, specifically, in above embodiment, the range of the ith feature of the text is that the syllable of the text is special
It levies, in present embodiment, the range of the ith feature of the text is the syllable characteristic and text feature letter of the text
Cease corresponding text feature.Correspondingly, more feature weights be can control in the present embodiment, be conducive to further refine ground
Control the effect of obtained new music.
In above two embodiment, the character pair of the ith feature of the text and any song
Matching degree Fi(Q, S) can be determined by following formula:
Fi(Q, S)=EditDistance (Qi,Si)
Wherein, EditDistance (Qi,Si) be the text ith feature it is corresponding with any song spy
Editing distance between sign.
In embodiments of the present invention, each feature is essentially all a digital sequence, wherein rhythm auxiliary sequence can lead to
The method for crossing number is converted into Serial No..When being compared in each feature to the text and song of user's input, actually
The comparison of two Serial No.s, thus can be calculated by way of editing distance in text any feature with it is right in song
Answer the matching degree of feature.
In the present embodiment, the foundation of the song database may include:
Obtain song and corresponding song information, wherein the song information includes: the lyrics and the music score of Chinese operas, further includes song
At least one feature in the sentence characteristics of word and the syllable characteristic of the music score of Chinese operas；
The song that will acquire storage corresponding with corresponding song information, obtains the song database.
Wherein, the sentence characteristics of the lyrics may include:
In the character quantity of every lyrics, every lyrics in the tone of each corresponding simple or compound vowel of a Chinese syllable of word and each word at least one
?.
Example in embodiment one and embodiment two has been described in detail, and repeats no more.
Embodiment three
Referring to Fig. 3, a kind of device for generating song provided in this embodiment includes: that received text module 310, feature mention
Modulus block 320, matching module 330 and voice synthetic module 340.
Wherein, received text module 310 is used to receive the text of user's input；Characteristic extracting module 320 is for extracting institute
State the text feature information of text；Matching module 330 is for obtaining and the song of the text feature information matches；Speech synthesis
Module 340 is used to carry out speech synthesis processing to the text according to the song, obtains the new song that the lyrics are the text.
The technical solution of the present embodiment, after the text for receiving user's input, the text feature by extracting text is believed
Breath, and the text feature information is matched with the corresponding informance of the lyrics of existing song, so as to obtain with it is described
The song of text feature information matches is made by dubbing in background music the song of successful match as background music, and by the text
For the new lyrics corresponding to the background music, and by speech synthesis, user is not necessarily to the music theory knowledge for having system, Ji Keshi
It is now the text automatic music composing of user's input.
In the above scheme, the characteristic extracting module 320 preferably includes: sentence cutting submodule and feature extraction submodule
Block.
Wherein, sentence cutting submodule carries out at sentence cutting for the text received to the received text module
Reason, obtains the subordinate sentence that the text includes；Feature extraction submodule is used to extract the sentence characteristics for the subordinate sentence that the text includes.
Further, the feature extraction submodule specifically can be used for: extract the character for the subordinate sentence that the text includes
At least one of the simple or compound vowel of a Chinese syllable of each word and the tone of each word in quantity, the subordinate sentence.
In the above scheme, the matching module 330 specifically can be used for: obtain from the song data library lookup pre-established
To the song with the text feature information matches.
The matching module 330 can specifically include: song searches submodule, feature determines submodule, score calculates son
Module and matching determine submodule.
Wherein, song is searched submodule and is obtained and the text feature for searching in the song database pre-established
At least two songs of information matches；Feature determines submodule for according to the music score of Chinese operas determining institutes respectively of at least two songs
State the syllable characteristic of text；Score computational submodule is used for according to the syllable characteristic of the text, calculate separately the text with
The matching total score of at least two songs；It matches and determines that submodule is used to that total score highest will to be matched at least two songs
Song, as the song with the text feature information matches.
Further, the score computational submodule specifically can be used for, and calculate separately the text according to the following equation
With the matching total score of at least two songs:
Wherein, Q indicates the text；S indicates any song at least two songs；Score (Q, S) is indicated
The matching total score of the text and any song；FiThe ith feature of (Q, S) expression text and any head
The matching degree of the character pair of song；λiIndicate the corresponding weight of the ith feature；The ith feature of the text can be with
It for the syllable characteristic, or is the syllable characteristic and the corresponding text feature of the text feature information.
Further, the matching degree F of the character pair of the ith feature of the text and any songi(Q,S)
It can be determined by following formula:
Fi(Q, S)=EditDistance (Qi,Si)
Wherein, EditDistance (Qi,Si) be the text ith feature it is corresponding with any song spy
Editing distance between sign.
In the above scheme, described device can also include: that module is established in song acquisition module and library.
Wherein, song acquisition module is for obtaining song and corresponding song information, wherein the song information packet
Include: the lyrics and the music score of Chinese operas further include at least one feature in the sentence characteristics of the lyrics and the syllable characteristic of the music score of Chinese operas；Module is established in library
Song storage corresponding with corresponding song information for obtaining the song acquisition module, obtains the song database.
Wherein, the sentence characteristics of the lyrics may include: the character quantity of every lyrics, each word pair in every lyrics
At least one of in the tone of the simple or compound vowel of a Chinese syllable and each word answered.
In the above scheme, the voice synthetic module 340 specifically can be used for: when carrying out speech synthesis processing, by institute
Dubbing in background music as background music for song is stated, and generates the voice of corresponding beat according to the music score of Chinese operas information, or according to the music score of Chinese operas
Information generates the voice of corresponding beat and pitch, obtains the new song, and the word content of the voice is the text.
Generation song provided by any embodiment of the invention can be performed in the device provided in an embodiment of the present invention for generating song
Bent method, has the corresponding function module and beneficial effect of execution method.
Finally, it should be noted that the above various embodiments is only used to illustrate the technical scheme of the present invention, rather than it is limited
System；Preferred embodiment in embodiment, is not intended to limit it, to those skilled in the art, the present invention can be with
There are various modifications and changes.All any modification, equivalent replacement, improvement and so within the spirit and principles of the present invention,
It should be included within protection scope of the present invention.
Claims (20)
1. a kind of method for generating song characterized by comprising
Receive the text of user's input；
Extract the text feature information of the text；Wherein, the text feature information includes the sentence of subordinate sentence in the text
The affective characteristics of feature and the subordinate sentence, the sentence characteristics of the subordinate sentence include the rhythm of the character quantity of the subordinate sentence, each word
The tone of female, each word；
According to the affective characteristics of the sentence characteristics of subordinate sentence described in the text feature information and the subordinate sentence, obtain and the text
The song of eigen information matches；
Speech synthesis processing is carried out to the text according to the song, obtains the new song that the lyrics are the text.
2. the method according to claim 1, wherein extracting the text feature information of the text, comprising:
Sentence cutting processing is carried out to the text, obtains the subordinate sentence that the text includes；
Extract the sentence characteristics for the subordinate sentence that the text includes.
3. according to the method described in claim 2, wrapping it is characterized in that, extract the sentence characteristics for the subordinate sentence that the text includes
It includes:
Extract the character quantity of the subordinate sentence that the text includes, in the subordinate sentence in the simple or compound vowel of a Chinese syllable of each word and the tone of each word
At least one.
4. being wrapped the method according to claim 1, wherein obtaining the song with the text feature information matches
It includes:
The song with the text feature information matches is obtained from the song data library lookup pre-established.
5. according to the method described in claim 4, it is characterized in that, from the song data library lookup pre-established obtain with it is described
The song of text feature information matches, comprising:
It is searched in the song database pre-established and obtains at least two songs with the text feature information matches；
Determine the syllable characteristic of the text respectively according to the music score of Chinese operas of at least two songs；
According to the syllable characteristic of the text, the matching total score of the text Yu at least two songs is calculated separately；
The highest song of total score will be matched at least two songs, as the song with the text feature information matches.
6. according to the method described in claim 5, it is characterized in that, according to the syllable characteristic, calculate separately the text with
The matching total score of at least two songs, comprising:
The matching total score of the text Yu at least two songs is calculated separately according to the following equation:
Wherein, Q indicates the text；S indicates any song at least two songs；Described in Score (Q, S) expression
The matching total score of text and any song；Fi(Q, S) indicates the ith feature and any song of the text
Character pair matching degree；λiIndicate the corresponding weight of the ith feature；The ith feature of the text is the sound
Feature is saved, or is the syllable characteristic and the corresponding text feature of the text feature information.
7. according to the method described in claim 6, it is characterized in that, the ith feature of the text and any song
Character pair matching degree Fi(Q, S) is determined by following formula:
Fi(Q, S)=EditDistance (Qi,Si)
Wherein, EditDistance (Qi,Si) be the text ith feature and any song character pair it
Between editing distance.
8. according to the method described in claim 4, it is characterized in that, the foundation of the song database, comprising:
Obtain song and corresponding song information, wherein the song information includes: the lyrics and the music score of Chinese operas, further includes the lyrics
At least one feature in the syllable characteristic of sentence characteristics and the music score of Chinese operas；
The song that will acquire storage corresponding with corresponding song information, obtains the song database.
9. according to the method described in claim 8, it is characterized in that, the sentence characteristics of the lyrics include:
In the character quantity of every lyrics, every lyrics in the tone of each corresponding simple or compound vowel of a Chinese syllable of word and each word at least one of.
10. -9 any method according to claim 1, which is characterized in that carry out language to the text according to the song
Sound synthesis processing obtains the new song that the lyrics are the text, comprising:
When carrying out speech synthesis processing, by dubbing in background music as background music for the song, and according to the music score of Chinese operas information of the song
The voice of corresponding beat is generated, or generates the voice of corresponding beat and pitch according to the music score of Chinese operas information, obtains the new song,
The word content of the voice is the text.
11. a kind of device for generating song characterized by comprising
Received text module, for receiving the text of user's input；
Characteristic extracting module, for extracting the text feature information of the text；Wherein, the text feature information includes described
The affective characteristics of the sentence characteristics of subordinate sentence and the subordinate sentence in text, the sentence characteristics of the subordinate sentence include the character of the subordinate sentence
Quantity, the simple or compound vowel of a Chinese syllable of each word, each word tone；
Matching module, it is special for the sentence characteristics of the subordinate sentence according to the text feature information and the emotion of the subordinate sentence
Sign obtains the song with the text feature information matches；
Voice synthetic module, for carrying out speech synthesis processing to the text according to the song, obtaining the lyrics is the text
This new song.
12. device according to claim 11, which is characterized in that the characteristic extracting module includes:
Sentence cutting submodule carries out sentence cutting processing for the text received to the received text module, obtains
The subordinate sentence that the text includes；
Feature extraction submodule, for extracting the sentence characteristics for the subordinate sentence that the text includes.
13. device according to claim 12, which is characterized in that the feature extraction submodule is specifically used for: extracting institute
State the character quantity of the subordinate sentence that text includes, at least one of the simple or compound vowel of a Chinese syllable of each word and the tone of each word in the subordinate sentence.
14. device according to claim 11, which is characterized in that the matching module is specifically used for: from what is pre-established
Song data library lookup obtains the song with the text feature information matches.
15. device according to claim 14, which is characterized in that the matching module specifically includes:
Song searches submodule, obtains and the text feature information matches for searching in the song database pre-established
At least two songs；
Feature determines submodule, for determining the syllable characteristic of the text respectively according to the music score of Chinese operas of at least two songs；
Score computational submodule calculates separately the text and at least two head for the syllable characteristic according to the text
The matching total score of song；
Match and determine submodule, for will the matching highest song of total score at least two songs, as with the text
The matched song of characteristic information.
16. device according to claim 15, which is characterized in that the score computational submodule is specifically used for, under
State the matching total score that formula calculates separately the text Yu at least two songs:
Wherein, Q indicates the text；S indicates any song at least two songs；Described in Score (Q, S) expression
The matching total score of text and any song；Fi(Q, S) indicates the ith feature and any song of the text
Character pair matching degree；λiIndicate the corresponding weight of the ith feature；The ith feature of the text is the sound
Feature is saved, or is the syllable characteristic and the corresponding text feature of the text feature information.
17. device according to claim 16, which is characterized in that the ith feature of the text and any first song
The matching degree F of bent character pairi(Q, S) is determined by following formula:
Fi(Q, S)=EditDistance (Qi,Si)
Wherein, EditDistance (Qi,Si) be the text ith feature and any song character pair it
Between editing distance.
18. device according to claim 14, which is characterized in that described device further include:
Song acquisition module, for obtaining song and corresponding song information, wherein the song information include: the lyrics and
The music score of Chinese operas further includes at least one feature in the sentence characteristics of the lyrics and the syllable characteristic of the music score of Chinese operas；
Module is established in library, and the storage corresponding with corresponding song information of the song for obtaining the song acquisition module obtains
The song database.
19. device according to claim 18, which is characterized in that the sentence characteristics of the lyrics include:
In the character quantity of every lyrics, every lyrics in the tone of each corresponding simple or compound vowel of a Chinese syllable of word and each word at least one of.
20. any device of 1-19 according to claim 1, which is characterized in that the voice synthetic module is specifically used for:
When carrying out speech synthesis processing, by dubbing in background music as background music for the song, and according to the music score of Chinese operas information of the song
The voice of corresponding beat is generated, or generates the voice of corresponding beat and pitch according to the music score of Chinese operas information, obtains the new song,
The word content of the voice is the text.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410743457.1A CN104391980B (en) | 2014-12-08 | 2014-12-08 | The method and apparatus for generating song |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410743457.1A CN104391980B (en) | 2014-12-08 | 2014-12-08 | The method and apparatus for generating song |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104391980A CN104391980A (en) | 2015-03-04 |
| CN104391980B true CN104391980B (en) | 2019-03-08 |
Family
ID=52609884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410743457.1A Active CN104391980B (en) | 2014-12-08 | 2014-12-08 | The method and apparatus for generating song |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104391980B (en) |
Families Citing this family (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105096962B (en) * | 2015-05-22 | 2019-04-16 | 努比亚技术有限公司 | A kind of information processing method and terminal |
| CN105070283B (en) * | 2015-08-27 | 2019-07-09 | 百度在线网络技术（北京）有限公司 | The method and apparatus dubbed in background music for singing voice |
| CN105513607B (en) * | 2015-11-25 | 2019-05-17 | 网易传媒科技（北京）有限公司 | A kind of method and apparatus write words of setting a song to music |
| CN105740394B (en) * | 2016-01-27 | 2019-02-26 | 广州酷狗计算机科技有限公司 | Song generation method, terminal and server |
| GB2551807B (en) * | 2016-06-30 | 2022-07-13 | Lifescore Ltd | Apparatus and methods to generate music |
| CN106339152B (en) * | 2016-08-30 | 2019-10-15 | 维沃移动通信有限公司 | A kind of generation method and mobile terminal of lyrics poster |
| CN106373580B (en) * | 2016-09-05 | 2019-10-15 | 北京百度网讯科技有限公司 | The method and apparatus of synthesis song based on artificial intelligence |
| CN107799119A (en) * | 2016-09-07 | 2018-03-13 | 中兴通讯股份有限公司 | Audio preparation method, apparatus and system |
| CN106557298A (en) * | 2016-11-08 | 2017-04-05 | 北京光年无限科技有限公司 | Background towards intelligent robot matches somebody with somebody sound outputting method and device |
| CN106776517B (en) * | 2016-12-20 | 2020-07-14 | 科大讯飞股份有限公司 | Automatic poetry method, device and system |
| CN108268530B (en) * | 2016-12-30 | 2022-04-29 | 阿里巴巴集团控股有限公司 | Lyric score generation method and related device |
| CN106898341B (en) * | 2017-01-04 | 2021-03-09 | 清华大学 | Personalized music generation method and device based on common semantic space |
| CN107122493B (en) * | 2017-05-19 | 2020-04-28 | 北京金山安全软件有限公司 | Song playing method and device |
| EP3642734A1 (en) * | 2017-06-21 | 2020-04-29 | Microsoft Technology Licensing, LLC | Providing personalized songs in automated chatting |
| CN109599079B (en) * | 2017-09-30 | 2022-09-23 | 腾讯科技（深圳）有限公司 | Music generation method and device |
| CN109801618B (en) * | 2017-11-16 | 2022-09-13 | 深圳市腾讯计算机系统有限公司 | Audio information generation method and device |
| CN109979497B (en) * | 2017-12-28 | 2021-02-26 | 阿里巴巴集团控股有限公司 | Song generation method, device and system and data processing and song playing method |
| CN108428441B (en) * | 2018-02-09 | 2021-08-06 | 咪咕音乐有限公司 | Multimedia file generation method, electronic device and storage medium |
| CN108765162A (en) * | 2018-05-10 | 2018-11-06 | 阿里巴巴集团控股有限公司 | A kind of finance data output method, device and electronic equipment |
| CN108877753B (en) * | 2018-06-15 | 2020-01-21 | 百度在线网络技术（北京）有限公司 | Music synthesis method and system, terminal and computer readable storage medium |
| CN109036355B (en) * | 2018-06-29 | 2023-04-25 | 平安科技（深圳）有限公司 | Automatic composing method, device, computer equipment and storage medium |
| CN109166564B (en) * | 2018-07-19 | 2023-06-06 | 平安科技（深圳）有限公司 | Method, apparatus and computer readable storage medium for generating a musical composition for a lyric text |
| CN110852093B (en) * | 2018-07-26 | 2023-05-16 | 腾讯科技（深圳）有限公司 | Poem generation method, device, computer equipment and storage medium |
| CN109241312B (en) * | 2018-08-09 | 2021-08-31 | 广东数相智能科技有限公司 | Melody word filling method and device and terminal equipment |
| CN109522427B (en) * | 2018-09-30 | 2021-12-10 | 北京光年无限科技有限公司 | Intelligent robot-oriented story data processing method and device |
| CN109493845A (en) * | 2019-01-02 | 2019-03-19 | 百度在线网络技术（北京）有限公司 | For generating the method and device of audio |
| CN110097886B (en) * | 2019-04-29 | 2021-09-10 | 贵州小爱机器人科技有限公司 | Intention recognition method and device, storage medium and terminal |
| CN112185321B (en) * | 2019-06-14 | 2024-05-31 | 微软技术许可有限责任公司 | Song generation |
| CN110516110B (en) * | 2019-07-22 | 2023-06-23 | 平安科技（深圳）有限公司 | Song generation method, song generation device, computer equipment and storage medium |
| CN112420008A (en) * | 2019-08-22 | 2021-02-26 | 北京峰趣互联网信息服务有限公司 | Method and device for recording songs, electronic equipment and storage medium |
| CN111339352B (en) * | 2020-01-22 | 2024-04-26 | 花瓣云科技有限公司 | Audio generation method, device and storage medium |
| CN113282270B (en) * | 2021-06-25 | 2024-01-26 | 杭州网易云音乐科技有限公司 | Music gift generation method, music gift display device, medium and computing device |
| CN113793578B (en) * | 2021-08-12 | 2023-10-20 | 咪咕音乐有限公司 | Method, device and equipment for generating tune and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101901598A (en) * | 2010-06-30 | 2010-12-01 | 北京捷通华声语音技术有限公司 | Humming synthesis method and system |
| CN102053998A (en) * | 2009-11-04 | 2011-05-11 | 周明全 | Method and system device for retrieving songs based on voice modes |
| CN102193992A (en) * | 2010-03-11 | 2011-09-21 | 姜胡彬 | System and method for generating custom songs |
| CN102201233A (en) * | 2011-05-20 | 2011-09-28 | 北京捷通华声语音技术有限公司 | Mixed and matched speech synthesis method and system thereof |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002262348A (en) * | 2001-02-27 | 2002-09-13 | Matsushita Electric Ind Co Ltd | Authentication system and mobile phone with card function |
| CN1246826C (en) * | 2004-06-01 | 2006-03-22 | 安徽中科大讯飞信息科技有限公司 | Method for outputting mixed with background sound and text sound in speech synthetic system |
| US8244546B2 (en) * | 2008-05-28 | 2012-08-14 | National Institute Of Advanced Industrial Science And Technology | Singing synthesis parameter data estimation system |
| CN101694772B (en) * | 2009-10-21 | 2014-07-30 | 北京中星微电子有限公司 | Method for converting text into rap music and device thereof |
| US9620092B2 (en) * | 2012-12-21 | 2017-04-11 | The Hong Kong University Of Science And Technology | Composition using correlation between melody and lyrics |
-
2014
- 2014-12-08 CN CN201410743457.1A patent/CN104391980B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102053998A (en) * | 2009-11-04 | 2011-05-11 | 周明全 | Method and system device for retrieving songs based on voice modes |
| CN102193992A (en) * | 2010-03-11 | 2011-09-21 | 姜胡彬 | System and method for generating custom songs |
| CN101901598A (en) * | 2010-06-30 | 2010-12-01 | 北京捷通华声语音技术有限公司 | Humming synthesis method and system |
| CN102201233A (en) * | 2011-05-20 | 2011-09-28 | 北京捷通华声语音技术有限公司 | Mixed and matched speech synthesis method and system thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104391980A (en) | 2015-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104391980B (en) | The method and apparatus for generating song | |
| CN108962217B (en) | Speech synthesis method and related equipment | |
| Agawu | Music as discourse: Semiotic adventures in romantic music | |
| Halliwell | Opera and the Novel: The Case of Henry James | |
| Kramer | Interpreting music | |
| Rumph | Mozart and Enlightenment Semiotics | |
| Moisala | Kaija Saariaho | |
| Tan | Acoustic Interculturalism | |
| Wee | Phonological tone | |
| Fuller | An introduction to Chinese poetry: from the Canon of poetry to the lyrics of the Song dynasty | |
| Glaser | Modernism's Metronome: Meter and Twentieth-Century Poetics | |
| Bernard | The Musicality of Language: Redefining History in Suzan-Lori Parks's The Death of the Last Black Man in the Whole Entire World | |
| Newark et al. | Proust and music: The anxiety of competence | |
| Hunt | Composition as Commentary: Voice and Poetry in Electroacoustic Music | |
| Gunn | Discoveries from the Fortepiano: A Manual for Beginning and Seasoned Performers | |
| Neufeld | Living the Work: Meditations on a Lark | |
| Yan | The Creative Reproduction of Chinese Ancient Poetry's Phonological Beauty in English Translation. | |
| Scoditti | Kitawa oral poetry: An example from Melanesia | |
| House | Strange Flowers: Cultivating new music for gamelan on British soil | |
| Healy | Imagined Vocalities: Exploring Voice in the Practice of Instrumental Music Performance | |
| Ninoshvili | The poetics of pop polyphony: Translating georgian Song for the World | |
| Zhuo | Experiencing identity, forming poetic space: Expression and interaction in a portfolio of original compositions | |
| Choi | Conductor's Guide to Lyric Diction in Standard Chinese | |
| Lokhina | A Performance Guide to Selected Songs by Georgy Sviridov | |
| Rudig | The Music of Sylvano Bussotti and Its Interpretation: Biopolitics, Intersubjectivity, and Modernist Canon Formation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |