WO2020082573A1 - 基于长短时神经网络的多声部音乐生成方法及装置 - Google Patents

基于长短时神经网络的多声部音乐生成方法及装置 Download PDF

Info

Publication number
WO2020082573A1
WO2020082573A1 PCT/CN2018/123549 CN2018123549W WO2020082573A1 WO 2020082573 A1 WO2020082573 A1 WO 2020082573A1 CN 2018123549 W CN2018123549 W CN 2018123549W WO 2020082573 A1 WO2020082573 A1 WO 2020082573A1
Authority
WO
WIPO (PCT)
Prior art keywords
sequence
music
voice
note
notes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/123549
Other languages
English (en)
French (fr)
Inventor
刘奡智
王义文
王健宗
肖京
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020082573A1 publication Critical patent/WO2020082573A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/32—Constructional details
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/311—Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation

Definitions

  • the present application relates to the field of artificial intelligence technology, and in particular, to a multi-voice music generation method and device based on long-short-term neural networks.
  • Music is usually composed of multiple audio tracks and has its own time dynamics. The concert expands interdependently with the passage of time. The success of natural language generation and monophonic music generation is not easy to spread to multi-track music.
  • the existing music generation method is usually single-melody music, because of the complex interaction between multiple melody, it is difficult to generate multi-melody music.
  • the embodiments of the present application provide a multi-voice music generation method and device based on a long-short-term neural network to solve the problem in the prior art that it is difficult to generate coordinated music between multiple voices.
  • a multi-voice music generation method based on a long-short time neural network, the method includes: constructing a music generation model, the music generation model including a first long-short time Neural network, a second long-short-term neural network, a single hidden layer neural network and a dependent network; train the music generation model through music sample data including multiple voices to obtain a trained network of the music generation model Parameters and the probability density distribution of the notes of the multiple voices; acquiring user input feature parameters for pre-generating multi-voice music, the feature parameters including preset music duration, preset rhythm sequence, and preset sustain sequence ; Entering a random sequence of multiple note notes into the trained music generation model in order to make the music generation model according to the random note sequence, the network parameters and the note probability of the multiple voices The density distribution generates multi-voice music that matches the characteristic parameters.
  • a multi-voice music generation device based on a long-short-term neural network
  • the device includes: a construction unit for constructing a music generation model, the music generation model includes A first long-short-term neural network, a second long-short-term neural network, a single hidden layer neural network and a dependent network; a first acquisition unit for training the music generation model through music sample data including multiple voices To obtain the network parameters of the trained music generation model and the probability density distribution of the notes of the multiple voices; the second acquisition unit is used to acquire the feature parameters input by the user for pre-generating multi-voice music,
  • the characteristic parameters include a preset music duration, a preset rhythm sequence, and a preset sustain sequence; a generating unit is used to sequentially input a random sequence of notes of multiple voices into the trained music generation model, so that the The music generation model is generated based on the random sequence of the notes, the network parameters, and the probability density distribution of the notes of the multiple voices.
  • a computer non-volatile storage medium includes a stored program, and the device where the storage medium is located is controlled to execute the above-mentioned based on Steps of multi-voice music generation method of long-term neural network.
  • a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing
  • the steps of the above-mentioned multi-voice music generation method based on long-term and short-term neural networks are realized.
  • the long- and short-term neural networks are used to directly process the note sequence, and at the same time, the correlation between the sequence time before and after can be used to obtain the probability density distribution of notes;
  • the musical note sequence of each voice part generates coordinated music between multiple voice parts, thereby solving the problem that it is difficult to generate coordinated music between multiple voice parts in the prior art.
  • FIG. 1 is a flowchart of a multi-voice music generation method based on a long-short-term neural network according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of a music generation model according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a multi-voice music generating device based on a long-short-term neural network according to an embodiment of the present application
  • FIG. 4 is a schematic diagram of a computer device according to an embodiment of the present application.
  • FIG. 1 is a flowchart of a multi-voice music generation method based on a long-short-term neural network according to an embodiment of the present application. As shown in FIG. 1, the method includes:
  • Step S101 Construct a music generation model, the music generation model includes a first long-short-term neural network, a second long-short-term neural network, a single hidden layer neural network, and a dependent network;
  • Step S102 Train the music generation model through the music sample data including multiple voices, to obtain the network parameters of the trained music generation model and the probability density distribution of the notes of the multiple voices;
  • Step S103 Obtain a characteristic parameter input by the user for pre-generating multi-voice music, and the characteristic parameter includes a preset music duration, a preset rhythm sequence, and a preset sustain sequence;
  • step S104 a random sequence of multiple notes is sequentially input into the trained music generation model, so that the music generation model generates multiple matching feature parameters according to the random sequence of notes, network parameters, and the probability density distribution of multiple notes Part music.
  • the long- and short-term neural networks are used to directly process the note sequence, and at the same time, the correlation between the sequence time before and after can be used to obtain the probability density distribution of notes;
  • the musical note sequence of each voice part generates coordinated music between multiple voice parts, thereby solving the problem that it is difficult to generate coordinated music between multiple voice parts in the prior art.
  • the method before training the music generation model through music sample data including multiple voices, the method further includes: acquiring multiple music training samples, where the music training sample includes multiple voice information; extracting the voice of each voice Note sequence, rhythm sequence and sustain sequence of music training samples; where the note sequence of each voice is expressed as: t ⁇ [T], T is the duration of the music training sample, which is an integer multiple of the sixteenth note; i is the voice; It is the note at the current time t; the note sequence of multiple voices, the rhythm sequence and sustain sequence of the music training sample are used as the music sample data.
  • each song includes a sequence of notes of multiple voices, a rhythm sequence and a sustain sequence of this song.
  • the serialization of each piece of music with time is beneficial to the long-term and short-term neural network learning the dependence of notes between time scales.
  • midi data for 389 hymns, where each song contains four parts: soprano, alto, tenor and bass.
  • the alto, tenor and bass of the relatively low pitch accompany the highest pitched soprano.
  • 80% of the midi data is used for music training samples, and 20% of the midi data is used for music evaluation samples.
  • the method further includes: excluding songs that have two or more notes in the same voice part at the same time.
  • a musical note code is used to represent musical notes in the musical note sequence, for example, "C4, E5, C5", and a "-" is used to express the duration of musical notes in the musical note sequence.
  • a sustain sequence M "0" indicates that there is no sustain mark, and "1" indicates that there is a sustain mark.
  • rhythm sequence S any value of "1, 2, 3, 4" is used to indicate the position of the note in this beat.
  • training the music generation model through music sample data including multiple voices to obtain the network parameters of the trained music generation model and the probability density distribution of the notes of the multiple voices, including: inputting music samples into the music generation model Data; Get the probability density function of the notes of each voice output by the music generation model: among them, Is the note at the current time t, Remove all notes in the note sequence except the current note; Is a rhythm sequence and a sustain sequence; ⁇ i is a network-dependent parameter; training a music generation model maximizes the value of the following formula: Obtain the network parameters of the music generation model and the probability density distribution of notes of multiple voices when the value of the formula is the largest.
  • the long-term and short-term neural network is a recurrent neural network, which can use its internal memory to process any sequence of input sequences.
  • each network parameter of the music generation model can be directly initialized, for example, a large number of music sample data is randomly generated and collected to train the music generation model.
  • the stochastic gradient descent algorithm can be used to update the network parameters of the long-term and short-term neural networks, such as: layer-to-layer connection weights and neuron bias, to achieve music generation model music generation
  • the effect is constantly approaching the optimal effect.
  • the system assigns constraints to the parameter values of the long- and short-term neural network so that it can continue to meet the requirements of the neural network parameters. Therefore, through multiple iterations, adjusting the parameters of the long and short time neural network to optimize the objective function.
  • the first long-term and short-term neural network of the music generation model receives each The first note sequence of the preset duration before the current note in the note sequence of each part, and outputs the first parameter to the dependent network according to the first note sequence;
  • the second long-short neural network receives the note sequence of each voice The second note sequence of the preset duration after the note at the current moment, and output the second parameter to the dependent network according to the second note sequence;
  • the single hidden layer neural network receives the note at the current moment in the note sequence of each voice and passes it to the dependent network ;
  • the first long-short-term neural network receives the first note sequence of 16 time nodes before the current note in the note sequence of each voice
  • the second long-short-term neural network receives the current note sequence of each voice in the note sequence The second note sequence of 16 time nodes after the time note.
  • the note sequence of each voice part is first vector converted by the embedding layer and then output to the first long-short-term neural network or the second long-short-term neural network;
  • the second parameter output by the neural network and the current note output by the single hidden layer neural network are fused through the fusion layer and input into the dependent network.
  • Multi-part music including: inputting a random sequence of notes of the first part, second part, third part, and fourth part into the trained music generation model in sequence; the music generation model is based on the i-th part A random sequence of notes, network parameters, characteristic parameters, and the probability density distribution of the notes of the i-th part generate multiple notes of the i-th part, i is sequentially taken as one, two, three, and four; generated from multiple notes of the i-th part A new sequence of notes in the i-th part; a new sequence of notes in the first part, a new sequence of notes in the second part, a new sequence of notes in the third part, and a new sequence of notes in the fourth part to form a multi-voice music.
  • the preset music duration input by the user is the same as the sequence duration of the preset rhythm sequence and the preset sustain sequence, for example, the duration of 40 sixteen notes.
  • An embodiment of the present application provides a multi-voice music generation device based on a long-short-term neural network.
  • the device is used to execute the above-described multi-voice music generation method based on a long-short-term neural network.
  • the device includes: The construction unit 10, the first acquisition unit 20, the second acquisition unit 30, and the generation unit 40.
  • the construction unit 10 is used to construct a music generation model.
  • the music generation model includes a first long-short-term neural network, a second long-short-term neural network, a single hidden layer neural network, and a dependent network;
  • the first obtaining unit 20 is configured to train the music generation model through the music sample data including multiple voices, and obtain the network parameters of the trained music generation model and the probability density distribution of the notes of the multiple voices;
  • the second obtaining unit 30 is configured to obtain a feature parameter input by the user for pre-generating multi-voice music, and the feature parameter includes a preset music duration, a preset rhythm sequence, and a preset sustain sequence;
  • the generating unit 40 is used to sequentially input a random sequence of notes of multiple voices into the trained music generation model, so that the music generation model generates matching features according to the random sequence of notes, network parameters, and the probability density distribution of multiple voices Multi-part music with parameters.
  • the long- and short-term neural networks are used to directly process the note sequence, and at the same time, the correlation between the sequence time before and after can be used to obtain the probability density distribution of notes;
  • the musical note sequence of each voice part generates coordinated music between multiple voice parts, thereby solving the problem that it is difficult to generate coordinated music between multiple voice parts in the prior art.
  • the device further includes: a third acquisition unit, an extraction unit, and a processing unit.
  • the third obtaining unit is used to obtain multiple music training samples, wherein the music training samples include multiple voice information; the extracting unit is used to extract the note sequence of each voice, the rhythm sequence and the sustain sequence of the music training sample ; Where the note sequence of each voice is expressed as: t ⁇ [T], T is the duration of the music training sample, which is an integer multiple of the sixteenth note; i is the voice; It is the note at the current time t; the processing unit is used to take note sequences of multiple voices, rhythm sequences and sustain sequences of music training samples as music sample data.
  • each song includes a sequence of notes of multiple voices, a rhythm sequence and a sustain sequence of this song. Processing each piece of music with time series is helpful for the long-term and short-term neural network to learn the dependence of notes between time scales.
  • midi data for 389 hymns, where each song contains four parts: soprano, alto, tenor and bass.
  • the alto, tenor and bass of the relatively low pitch accompany the highest pitched soprano.
  • 80% of the midi data is used for music training samples, and 20% of the midi data is used for music evaluation samples.
  • a musical note code is used to represent musical notes in the musical note sequence, for example, "C4, E5, C5", and a "-" is used to express the duration of musical notes in the musical note sequence.
  • a sustain sequence M "0" indicates that there is no sustain mark, and "1" indicates that there is a sustain mark.
  • rhythm sequence S any value of "1, 2, 3, 4" is used to indicate the position of the note in this beat.
  • the first acquisition unit 20 includes an input subunit, a first acquisition subunit, a training subunit, and a second acquisition subunit.
  • the input subunit is used to input music sample data into the music generation model; the first acquisition subunit is used to acquire the note probability density function of each part output by the music generation model: among them, Is the note at the current time t, Remove all notes in the note sequence except the current note; Is a rhythm sequence and a sustain sequence; ⁇ i is a network-dependent parameter; a training subunit is used to train a music generation model to maximize the value of the following formula: The second obtaining subunit is used to obtain the network parameters of the music generation model and the note probability density distribution of multiple voices when the value of the formula is the largest.
  • the long-term and short-term neural network is a recurrent neural network, which can use its internal memory to process any sequence of input sequences.
  • each network parameter of the music generation model can be directly initialized, for example, a large number of music sample data is randomly generated and collected to train the music generation model.
  • the stochastic gradient descent algorithm can be used to update the network parameters of the long-term and short-term neural networks, such as: layer-to-layer connection weights and neuron bias, to achieve music generation model music generation
  • the effect is constantly approaching the optimal effect.
  • the system assigns constraints to the parameter values of the long- and short-term neural network so that it can continue to meet the requirements of the neural network parameters. Therefore, through multiple iterations, adjusting the parameters of the long and short time neural network to optimize the objective function.
  • the music generation model is shown in FIG. 2.
  • the first long-term and short-term neural network of the music generation model receives the notes at the current time in the note sequence of each voice The first note sequence of the preset duration before the first note sequence, and output the first parameter to the dependent network according to the first note sequence;
  • the second long-short-term neural network receives the first duration of the preset duration after the note at the current moment in the note sequence of each voice Two-note sequence, and output the second parameter to the dependent network according to the second note sequence;
  • the single hidden layer neural network receives the note at the current time in the note sequence of each voice and passes it to the dependent network; the dependent network according to the first parameter, the second The parameters and the current note output the note probability density function of each voice.
  • the first long-short-term neural network receives the first note sequence of 16 time nodes before the current note in the note sequence of each voice
  • the second long-short-term neural network receives the current note sequence of each voice in the note sequence The second note sequence of 16 time nodes after the time note.
  • the note sequence of each voice part is first vector converted by the embedding layer and then output to the first long-short-term neural network or the second long-short-term neural network; the first parameter output by the first long-short-term neural network, and the second long-short-term
  • the second parameter output by the neural network and the current note output by the single hidden layer neural network are fused through the fusion layer and input into the dependent network.
  • the generating unit 40 includes an input subunit for sequentially inputting notes of the first voice, the second voice, the third voice, and the fourth voice into the trained music generation model Random sequence; the music generation model generates multiple notes of the i-th part based on the random sequence of notes of the i-th part, network parameters, characteristic parameters and the probability density distribution of the note of the i-th part, i takes one, two, three, Four; generate a new sequence of notes of the i-th part based on a plurality of notes of the i-th part; combine the new sequence of notes of the first part, the new sequence of notes of the second part, the new sequence of notes of the third part, the first The new sequence of notes of the four voices is combined to form multi-voice music.
  • An embodiment of the present application provides a computer non-volatile storage medium, where the storage medium includes a stored program, wherein, when the program is running, the device where the storage medium is located is controlled to perform the following steps:
  • the music generation model includes a first long-short-term neural network, a second long-short-term neural network, a single hidden layer neural network, and a dependent network; train the music generation model through music sample data including multiple voices , Get the network parameters of the trained music generation model and the probability density distribution of the notes of multiple voices; obtain the user-input feature parameters for pre-generating multi-voice music, the feature parameters include preset music duration, preset rhythm sequence And preset sustain sequence; input a random sequence of multiple notes to the trained music generation model in order to make the music generation model generate a match according to the random sequence of notes, network parameters and the probability density distribution of the notes of multiple voices Multi-part music with characteristic parameters.
  • the device on which the storage medium is located also performs the following steps: acquiring multiple music training samples, where the music training samples include multiple voice information; extracting the note sequence of each voice, the music training sample Rhythm sequence and sustain sequence; where the note sequence of each voice is expressed as: t ⁇ [T], T is the duration of the music training sample, which is an integer multiple of the sixteenth note; i is the voice; It is the note at the current time t; the note sequence of multiple voices, the rhythm sequence and sustain sequence of the music training sample are used as the music sample data.
  • the device that controls the storage medium further performs the following steps: input music sample data into the music generation model; and obtain the probability density function of the notes of each part output by the music generation model: among them, Is the note at the current time t, Remove all notes in the note sequence except the current note; Is a rhythm sequence and a sustain sequence; ⁇ i is a network-dependent parameter; training a music generation model maximizes the value of the following formula: Obtain the network parameters of the music generation model and the probability density distribution of notes of multiple voices when the value of the formula is the largest.
  • the device that controls the storage medium further performs the following steps: the first long-short-term neural network of the music generation model receives the first note sequence of the preset duration before the note at the current moment in the note sequence of each voice , And output the first parameter to the dependent network according to the first note sequence; the second long-short-term neural network receives the second note sequence of the preset duration after the note at the current moment in the note sequence of each voice, and according to the second note sequence Output the second parameter to the dependent network; the single hidden layer neural network receives the notes at the current time in the note sequence of each voice part and passes them to the dependent network; the dependent network outputs each voice part according to the first parameter, the second parameter, and the current time note The probability density function of the notes.
  • the device on which the storage medium is located also performs the following steps: sequentially input the notes of the first voice, the second voice, the third voice, and the fourth voice into the trained music generation model in sequence Sequence;
  • the music generation model generates multiple notes of the i-th part based on the random sequence of notes of the i-th part, network parameters, characteristic parameters and the probability density distribution of the note of the i-th part, i takes one, two, three, four in sequence Generating a new sequence of notes of the i-th part based on a plurality of notes of the i-th part; combining a new sequence of notes of the first part, a new sequence of notes of the second part, a new sequence of notes of the third part, a fourth The new sequence of notes of the voices is combined to form multi-voice music.
  • an embodiment of the present application provides a computer device 100, including a memory 102, a processor 101, and a computer program 103 stored in the memory 102 and executable on the processor 101, the processor The following steps are realized when the computer program is executed:
  • the music generation model includes a first long-short-term neural network, a second long-short-term neural network, a single hidden layer neural network, and a dependent network; train the music generation model through music sample data including multiple voices , Get the network parameters of the trained music generation model and the probability density distribution of the notes of multiple voices; obtain the user-input feature parameters for pre-generating multi-voice music, the feature parameters include preset music duration, preset rhythm sequence And preset sustain sequence; input a random sequence of multiple notes to the trained music generation model in order to make the music generation model generate a match according to the random sequence of notes, network parameters and the probability density distribution of the notes of multiple voices Multi-part music with characteristic parameters.
  • the processor also implements the following steps when executing the computer program: acquiring multiple music training samples, where the music training samples include multiple voice information; extracting the note sequence of each voice, the rhythm sequence of the music training sample Sustain sequence; where the sequence of notes for each voice is expressed as: t ⁇ [T], T is the duration of the music training sample, which is an integer multiple of the sixteenth note; i is the voice; It is the note at the current time t; the note sequence of multiple voices, the rhythm sequence and sustain sequence of the music training sample are used as the music sample data.
  • the processor also implements the following steps when executing the computer program: input music sample data into the music generation model; and obtain the probability density function of the notes of each part output by the music generation model: among them, Is the note at the current time t, Remove all notes in the note sequence except the current note; Is a rhythm sequence and a sustain sequence; ⁇ i is a network-dependent parameter; training a music generation model maximizes the value of the following formula: Obtain the network parameters of the music generation model and the probability density distribution of notes of multiple voices when the value of the formula is the largest.
  • the processor also implements the following steps when executing the computer program:
  • the first long-short-term neural network of the music generation model receives the first note sequence of the preset duration before the note at the current moment in the note sequence of each voice, and according to The first note sequence outputs the first parameter to the dependent network;
  • the second long-short-term neural network receives the second note sequence of the preset duration after the note at the current moment in the note sequence of each voice, and outputs the second according to the second note sequence Parameters to the dependent network;
  • the single hidden layer neural network receives the notes at the current time in the note sequence of each part and passes them to the dependent network;
  • the dependent network outputs the note probability of each part according to the first parameter, the second parameter, and the notes at the current time Density function.
  • a random sequence of notes of the first voice, the second voice, the third voice, and the fourth voice is sequentially input into the trained music generation model; music The generation model generates multiple notes of the i-th part based on the random sequence of notes of the i-th part, network parameters, characteristic parameters and the probability density distribution of the note of the i-th part, i takes one, two, three, and four in order; Multiple notes in the i-part generate a new sequence of notes in the i-th part; the new sequence of notes in the first part, the new sequence of notes in the second part, the new sequence of notes in the third part, the The new sequence of notes is combined to form multi-voice music.
  • terminals involved in the embodiments of the present application may include, but are not limited to, personal computers (Personal Computers, PCs), personal digital assistants (Personal Digital Assistants, PDAs), wireless handheld devices, tablet computers (Tablet Computers), Mobile phones, MP3 players, MP4 players, etc.
  • the application may be an application program (nativeApp) installed on the terminal, or may also be a webpage program (webApp) of a browser on the terminal, which is not limited in this embodiment of the present application.
  • the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.
  • the above integrated unit implemented in the form of a software functional unit may be stored in a computer-readable storage medium.
  • the above software function unit is stored in a storage medium, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (Processor) to perform the methods described in the embodiments of the present application Partial steps.
  • the foregoing storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other media that can store program codes .

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Auxiliary Devices For Music (AREA)
  • Electrophonic Musical Instruments (AREA)

Abstract

基于长短时神经网络的多声部音乐生成方法及装置，该方法包括：构建音乐生成模型，音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络（S101）；通过包括多个声部的音乐样本数据训练音乐生成模型，得到训练好的音乐生成模型的网络参数及多个声部的音符概率密度分布（S102）；获取用户输入的用于预生成多声部音乐的特征参数，特征参数包括预设音乐时长、预设节奏序列及预设延音序列（S103）；向训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使音乐生成模型根据音符随机序列、网络参数及多个声部的音符概率密度分布生成匹配特征参数的多声部音乐（S104）。

Description

基于长短时神经网络的多声部音乐生成方法及装置
本申请要求于2018年10月26日提交中国专利局、申请号为201811257165.1、申请名称为“基于长短时神经网络的多声部音乐生成方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能技术领域，尤其涉及一种基于长短时神经网络的多声部音乐生成方法及装置。
背景技术
音乐通常由多个音轨组成，并具有各自的时间动态，音乐会随着时间的推移而相互依存地展开。自然语言生成和单音音乐生成的成功不容易普及到多音轨的音乐。现有的音乐生成方法通常是单旋律音乐，因为多个旋律之间的复杂的相互影响，很难生成多旋律的音乐。
因此，如何生成多个声部之间协调的音乐成为目前亟待解决的问题。
申请内容
有鉴于此，本申请实施例提供了一种基于长短时神经网络的多声部音乐生成方法及装置，用以解决现有技术中难以生成多个声部之间协调的音乐的问题。
为了实现上述目的，根据本申请的一个方面，提供了一种基于长短时神经网络的多声部音乐生成方法，所述方法包括：构建音乐生成模型，所述音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；通过包括多个声部的 音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布；获取用户输入的用于预生成多声部音乐的特征参数，所述特征参数包括预设音乐时长、预设节奏序列及预设延音序列；向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐。
为了实现上述目的，根据本申请的一个方面，提供了一种基于长短时神经网络的多声部音乐生成装置，所述装置包括：构建单元，用于构建音乐生成模型，所述音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；第一获取单元，用于通过包括多个声部的音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布；第二获取单元，用于获取用户输入的用于预生成多声部音乐的特征参数，所述特征参数包括预设音乐时长、预设节奏序列及预设延音序列；生成单元，用于向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐。
为了实现上述目的，根据本申请的一个方面，提供了一种计算机非易失性存储介质，所述存储介质包括存储的程序，在所述程序运行时控制所述存储介质所在设备执行上述的基于长短时神经网络的多声部音乐生成方法的步骤。
为了实现上述目的，根据本申请的一个方面，提供了一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述的基于长短时神经网络的多声部音乐生成方法的步骤。
在本方案中，通过构建包括长短时神经网络的音乐生成模型，利用长短时神经网络直接对音符序列进行处理，同时能够利用序列时间前 后之间的相关性，得到音符概率密度分布；从而调整多个声部的音符序列，生成多声部之间协调的音乐，从而解决现有技术中难以生成多个声部之间协调的音乐的问题。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其它的附图。
图1是根据本申请实施例的一种基于长短时神经网络的多声部音乐生成方法的流程图；
图2是根据本申请实施例的一种音乐生成模型的示意图；
图3是根据本申请实施例的一种基于长短时神经网络的多声部音乐生成装置的示意图；
图4是根据本申请实施例的一种计算机设备的示意图。
具体实施方式
为了更好的理解本申请的技术方案，下面结合附图对本申请实施例进行详细描述。
应当明确，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
在本申请实施例中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。
图1是根据本申请实施例的一种基于长短时神经网络的多声部音乐 生成方法的流程图，如图1所示，该方法包括：
步骤S101，构建音乐生成模型，音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；
步骤S102，通过包括多个声部的音乐样本数据训练音乐生成模型，得到训练好的音乐生成模型的网络参数及多个声部的音符概率密度分布；
步骤S103，获取用户输入的用于预生成多声部音乐的特征参数，特征参数包括预设音乐时长、预设节奏序列及预设延音序列；
步骤S104，向训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使音乐生成模型根据音符随机序列、网络参数及多个声部的音符概率密度分布生成匹配特征参数的多声部音乐。
在本方案中，通过构建包括长短时神经网络的音乐生成模型，利用长短时神经网络直接对音符序列进行处理，同时能够利用序列时间前后之间的相关性，得到音符概率密度分布；从而调整多个声部的音符序列，生成多声部之间协调的音乐，从而解决现有技术中难以生成多个声部之间协调的音乐的问题。
可选地，在通过包括多个声部的音乐样本数据训练音乐生成模型之前，方法还包括：获取多个音乐训练样本，其中，音乐训练样本包括多个声部信息；提取每个声部的音符序列、音乐训练样本的节奏序列及延音序列；其中，每个声部的音符序列表示为：
Figure PCTCN2018123549-appb-000001
t∈[T]，T为音乐训练样本的时长，是十六分音符的整数倍；i为声部；
Figure PCTCN2018123549-appb-000002
为当前时刻t的音符；将多个声部的音符序列、音乐训练样本的节奏序列及延音序列作为音乐样本数据。
可以理解地，每首曲子包括多个声部的音符序列、这个曲子的节奏序列及延音序列。将每首曲子随时间序列化处理，有利于长短时神经网络学习音符之间随时间尺度的依赖关系。
例如，收集389首众赞歌的midi数据，其中，每首曲子包含四个声部：女高音、女低音、男高音和男低音。相对音高较低的女低音、男高音和男低音给音高最高的女高音作伴奏。将其中80％的midi数据用于音乐 训练样本，其中20％的midi数据用于音乐评估样本。
可选地，获取多个音乐训练样本之后，方法还包括：剔除在一个声部内，有两个及以上的音符同时出现的曲子。
可选地，音符序列中用音符代码来表示音符，例如“C4，E5，C5”，音符序列中用“-”来表示音符的持续。延音序列M中，用“0”表示该处没有延音记号，用“1”则表示该处有延音记号。节奏序列S中，用“1，2，3，4”中的任意一个值来表示音符在这一拍中的位置。
可选地，通过包括多个声部的音乐样本数据训练音乐生成模型，得到训练好的音乐生成模型的网络参数及多个声部的音符概率密度分布，包括：向音乐生成模型中输入音乐样本数据；获取音乐生成模型输出的每个声部的音符概率密度函数：
Figure PCTCN2018123549-appb-000003
其中，
Figure PCTCN2018123549-appb-000004
为当前时刻t的音符，
Figure PCTCN2018123549-appb-000005
为音符序列中除去当前音符剩下的所有音符；
Figure PCTCN2018123549-appb-000006
为节奏序列及延音序列；θ i为依赖网络的参数；训练音乐生成模型使以下公式的值最大化：
Figure PCTCN2018123549-appb-000007
获取当公式的值最大时音乐生成模型的网络参数及多个声部的音符概率密度分布。
长短时神经网络为循环神经网络，能够利用其内部的记忆来处理任意时序的输入序列。初始地，可以直接初始化音乐生成模型的各个网络参数，例如，随机生成并采集大数量的音乐样本数据，以对音乐生成模型进行训练。其后，可以通过随机梯度下降算法，使得其中的长短时神经网络的网络参数随之更新，如：层与层之间的连接权值和神经元偏置等，以达到音乐生成模型的音乐生成效果不断逼近最优的效果。
在训练期间，系统对长短时神经网络的参数值赋予约束条件，从而使其继续满足对神经网路的参数的要求。从而通过多次迭代，调节长短时神经网络的参数的值来对目标函数进行优化。
图2是根据本申请实施例的一种音乐生成模型的示意图，如图2所示，训练过程中，向音乐生成模型中输入音乐样本数据之后，音乐生成模型的第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据第一音符序列输出第一参数至依赖网络；第二长短时神经网络接收每个声部的音符序列中当前时刻音符后的预设 时长的第二音符序列，并根据第二音符序列输出第二参数至依赖网络；单隐藏层神经网络接收每个声部的音符序列中当前时刻音符并传递至依赖网络；依赖网络根据第一参数、第二参数及当前时刻音符输出每个声部的音符概率密度函数。
可选地，第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的16个时间节点的第一音符序列，第二长短时神经网络接收每个声部的音符序列中当前时刻音符后的16个时间节点的第二音符序列。
具体地，每个声部的音符序列先通过嵌入层进行向量转换后输出至第一长短时神经网络或第二长短时神经网络；第一长短时神经网络输出的第一参数、第二长短时神经网络输出的第二参数及单隐藏层神经网络输出的当前时刻音符通过融合层进行融合后输入依赖网络中。
可选地，向训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使音乐生成模型根据音符随机序列、网络参数及多个声部的音符概率密度分布生成匹配特征参数的多声部音乐，包括：向训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；音乐生成模型基于第i声部的音符随机序列、网络参数、特征参数及第i声部的音符概率密度分布生成第i声部的多个音符，i依次取一、二、三、四；根据第i声部的多个音符生成第i声部的音符新序列；将第一声部的音符新序列、第二声部的音符新序列、第三声部的音符新序列、第四声部的音符新序列组合形成多声部音乐。
可选地，用户输入的预设音乐时长与预设节奏序列及预设延音序列的序列时长相同，例如为40个十六音符的时长。
本申请实施例提供了一种基于长短时神经网络的多声部音乐生成装置，该装置用于执行上述基于长短时神经网络的多声部音乐生成方法，如图3所示，该装置包括：构建单元10、第一获取单元20、第二获取单元30、生成单元40。
构建单元10，用于构建音乐生成模型，音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一 个依赖网络；
第一获取单元20，用于通过包括多个声部的音乐样本数据训练音乐生成模型，得到训练好的音乐生成模型的网络参数及多个声部的音符概率密度分布；
第二获取单元30，用于获取用户输入的用于预生成多声部音乐的特征参数，特征参数包括预设音乐时长、预设节奏序列及预设延音序列；
生成单元40，用于向训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使音乐生成模型根据音符随机序列、网络参数及多个声部的音符概率密度分布生成匹配特征参数的多声部音乐。
在本方案中，通过构建包括长短时神经网络的音乐生成模型，利用长短时神经网络直接对音符序列进行处理，同时能够利用序列时间前后之间的相关性，得到音符概率密度分布；从而调整多个声部的音符序列，生成多声部之间协调的音乐，从而解决现有技术中难以生成多个声部之间协调的音乐的问题。
可选地，装置还包括：第三获取单元、提取单元、处理单元。
第三获取单元，用于获取多个音乐训练样本，其中，音乐训练样本包括多个声部信息；提取单元，用于提取每个声部的音符序列、音乐训练样本的节奏序列及延音序列；其中，每个声部的音符序列表示为：
Figure PCTCN2018123549-appb-000008
t∈[T]，T为音乐训练样本的时长，是十六分音符的整数倍；i为声部；
Figure PCTCN2018123549-appb-000009
为当前时刻t的音符；处理单元，用于将多个声部的音符序列、音乐训练样本的节奏序列及延音序列作为音乐样本数据。
可以理解地，每首曲子包括多个声部的音符序列、这个曲子的节奏序列及延音序列。将每首曲子随时间序列处理，有利于长短时神经网络学习音符之间随时间尺度的依赖关系。
例如，收集389首众赞歌的midi数据，其中，每首曲子包含四个声部：女高音、女低音、男高音和男低音。相对音高较低的女低音、男高音和男低音给音高最高的女高音作伴奏。将其中80％的midi数据用于音乐训练样本，其中20％的midi数据用于音乐评估样本。
可选地，音符序列中用音符代码来表示音符，例如“C4，E5，C5”， 音符序列中用“-”来表示音符的持续。延音序列M中，用“0”表示该处没有延音记号，用“1”则表示该处有延音记号。节奏序列S中，用“1，2，3，4”中的任意一个值来表示音符在这一拍中的位置。
可选地，第一获取单元20，包括输入子单元、第一获取子单元、训练子单元、第二获取子单元。
输入子单元，用于向音乐生成模型中输入音乐样本数据；第一获取子单元，用于获取音乐生成模型输出的每个声部的音符概率密度函数：
Figure PCTCN2018123549-appb-000010
其中，
Figure PCTCN2018123549-appb-000011
为当前时刻t的音符，
Figure PCTCN2018123549-appb-000012
为音符序列中除去当前音符剩下的所有音符；
Figure PCTCN2018123549-appb-000013
为节奏序列及延音序列；θ i为依赖网络的参数；训练子单元，用于训练音乐生成模型使以下公式的值最大化：
Figure PCTCN2018123549-appb-000014
第二获取子单元，用于获取当公式的值最大时音乐生成模型的网络参数及多个声部的音符概率密度分布。
长短时神经网络为循环神经网络，能够利用其内部的记忆来处理任意时序的输入序列。初始地，可以直接初始化音乐生成模型的各个网络参数，例如，随机生成并采集大数量的音乐样本数据，以对音乐生成模型进行训练。其后，可以通过随机梯度下降算法，使得其中的长短时神经网络的网络参数随之更新，如：层与层之间的连接权值和神经元偏置等，以达到音乐生成模型的音乐生成效果不断逼近最优的效果。
在训练期间，系统对长短时神经网络的参数值赋予约束条件，从而使其继续满足对神经网路的参数的要求。从而通过多次迭代，调节长短时神经网络的参数的值来对目标函数进行优化。
可选地，音乐生成模型如图2所示，训练过程中，向音乐生成模型中输入音乐样本数据之后，音乐生成模型的第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据第一音符序列输出第一参数至依赖网络；第二长短时神经网络接收每个声部的音符序列中当前时刻音符后的预设时长的第二音符序列，并根据第二音符序列输出第二参数至依赖网络；单隐藏层神经网络接收每个声部的音符序列中当前时刻音符并传递至依赖网络；依赖网络根据第一参数、第二参数及当前时刻音符输出每个声部的音符概率密度函数。
可选地，第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的16个时间节点的第一音符序列，第二长短时神经网络接收每个声部的音符序列中当前时刻音符后的16个时间节点的第二音符序列。
具体地，每个声部的音符序列先通过嵌入层进行向量转换后输出至第一长短时神经网络或第二长短时神经网络；第一长短时神经网络输出的第一参数、第二长短时神经网络输出的第二参数及单隐藏层神经网络输出的当前时刻音符通过融合层进行融合后输入依赖网络中。
具体地，生成新音乐过程中，生成单元40包括输入子单元，用于向训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；音乐生成模型基于第i声部的音符随机序列、网络参数、特征参数及第i声部的音符概率密度分布生成第i声部的多个音符，i依次取一、二、三、四；根据第i声部的多个音符生成第i声部的音符新序列；将第一声部的音符新序列、第二声部的音符新序列、第三声部的音符新序列、第四声部的音符新序列组合形成多声部音乐。
本申请实施例提供了一种计算机非易失性存储介质，存储介质包括存储的程序，其中，在程序运行时控制存储介质所在设备执行以下步骤：
构建音乐生成模型，音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；通过包括多个声部的音乐样本数据训练音乐生成模型，得到训练好的音乐生成模型的网络参数及多个声部的音符概率密度分布；获取用户输入的用于预生成多声部音乐的特征参数，特征参数包括预设音乐时长、预设节奏序列及预设延音序列；向训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使音乐生成模型根据音符随机序列、网络参数及多个声部的音符概率密度分布生成匹配特征参数的多声部音乐。
可选地，在程序运行时控制存储介质所在设备还执行以下步骤：获取多个音乐训练样本，其中，音乐训练样本包括多个声部信息；提取每个声部的音符序列、音乐训练样本的节奏序列及延音序列；其中，每个声部的音符序列表示为：
Figure PCTCN2018123549-appb-000015
t∈[T]，T为音乐训练样本的时长，是十 六分音符的整数倍；i为声部；
Figure PCTCN2018123549-appb-000016
为当前时刻t的音符；将多个声部的音符序列、音乐训练样本的节奏序列及延音序列作为音乐样本数据。
可选地，在程序运行时控制存储介质所在设备还执行以下步骤：向音乐生成模型中输入音乐样本数据；获取音乐生成模型输出的每个声部的音符概率密度函数：
Figure PCTCN2018123549-appb-000017
其中，
Figure PCTCN2018123549-appb-000018
为当前时刻t的音符，
Figure PCTCN2018123549-appb-000019
为音符序列中除去当前音符剩下的所有音符；
Figure PCTCN2018123549-appb-000020
为节奏序列及延音序列；θ i为依赖网络的参数；训练音乐生成模型使以下公式的值最大化：
Figure PCTCN2018123549-appb-000021
获取当公式的值最大时音乐生成模型的网络参数及多个声部的音符概率密度分布。
可选地，在程序运行时控制存储介质所在设备还执行以下步骤：音乐生成模型的第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据第一音符序列输出第一参数至依赖网络；第二长短时神经网络接收每个声部的音符序列中当前时刻音符后的预设时长的第二音符序列，并根据第二音符序列输出第二参数至依赖网络；单隐藏层神经网络接收每个声部的音符序列中当前时刻音符并传递至依赖网络；依赖网络根据第一参数、第二参数及当前时刻音符输出每个声部的音符概率密度函数。
可选地，在程序运行时控制存储介质所在设备还执行以下步骤：向训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；音乐生成模型基于第i声部的音符随机序列、网络参数、特征参数及第i声部的音符概率密度分布生成第i声部的多个音符，i依次取一、二、三、四；根据第i声部的多个音符生成第i声部的音符新序列；将第一声部的音符新序列、第二声部的音符新序列、第三声部的音符新序列、第四声部的音符新序列组合形成多声部音乐。
如图4所示，本申请实施例提供了一种计算机设备100，包括存储器102、处理器101以及存储在所述存储器102中并可在所述处理器101上运行的计算机程序103，处理器执行计算机程序时实现以下步骤：
构建音乐生成模型，音乐生成模型包括一个第一长短时神经网络、一 个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；通过包括多个声部的音乐样本数据训练音乐生成模型，得到训练好的音乐生成模型的网络参数及多个声部的音符概率密度分布；获取用户输入的用于预生成多声部音乐的特征参数，特征参数包括预设音乐时长、预设节奏序列及预设延音序列；向训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使音乐生成模型根据音符随机序列、网络参数及多个声部的音符概率密度分布生成匹配特征参数的多声部音乐。
可选地，处理器执行计算机程序时还实现以下步骤：获取多个音乐训练样本，其中，音乐训练样本包括多个声部信息；提取每个声部的音符序列、音乐训练样本的节奏序列及延音序列；其中，每个声部的音符序列表示为：
Figure PCTCN2018123549-appb-000022
t∈[T]，T为音乐训练样本的时长，是十六分音符的整数倍；i为声部；
Figure PCTCN2018123549-appb-000023
为当前时刻t的音符；将多个声部的音符序列、音乐训练样本的节奏序列及延音序列作为音乐样本数据。
可选地，处理器执行计算机程序时还实现以下步骤：向音乐生成模型中输入音乐样本数据；获取音乐生成模型输出的每个声部的音符概率密度函数：
Figure PCTCN2018123549-appb-000024
其中，
Figure PCTCN2018123549-appb-000025
为当前时刻t的音符，
Figure PCTCN2018123549-appb-000026
为音符序列中除去当前音符剩下的所有音符；
Figure PCTCN2018123549-appb-000027
为节奏序列及延音序列；θ i为依赖网络的参数；训练音乐生成模型使以下公式的值最大化：
Figure PCTCN2018123549-appb-000028
获取当公式的值最大时音乐生成模型的网络参数及多个声部的音符概率密度分布。
可选地，处理器执行计算机程序时还实现以下步骤：音乐生成模型的第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据第一音符序列输出第一参数至依赖网络；第二长短时神经网络接收每个声部的音符序列中当前时刻音符后的预设时长的第二音符序列，并根据第二音符序列输出第二参数至依赖网络；单隐藏层神经网络接收每个声部的音符序列中当前时刻音符并传递至依赖网络；依赖网络根据第一参数、第二参数及当前时刻音符输出每个声部的音符概率密度函数。
可选地，处理器执行计算机程序时还实现以下步骤：向训练好的音乐 生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；音乐生成模型基于第i声部的音符随机序列、网络参数、特征参数及第i声部的音符概率密度分布生成第i声部的多个音符，i依次取一、二、三、四；根据第i声部的多个音符生成第i声部的音符新序列；将第一声部的音符新序列、第二声部的音符新序列、第三声部的音符新序列、第四声部的音符新序列组合形成多声部音乐。
需要说明的是，本申请实施例中所涉及的终端可以包括但不限于个人计算机(Personal Computer，PC)、个人数字助理(Personal Digital Assistant，PDA)、无线手持设备、平板电脑(Tablet Computer)、手机、MP3播放器、MP4播放器等。
可以理解的是，所述应用可以是安装在终端上的应用程序(nativeApp)，或者还可以是终端上的浏览器的一个网页程序(webApp)，本申请实施例对此不进行限定。
在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机装置(可以是个人计算机，服务器，或者网络装置等)或处理器(Processor)执行本申请各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (20)

  1. 一种基于长短时神经网络的多声部音乐生成方法，其特征在于，所述方法包括：
    构建音乐生成模型，所述音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；
    通过包括多个声部的音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布；
    获取用户输入的用于预生成多声部音乐的特征参数，所述特征参数包括预设音乐时长、预设节奏序列及预设延音序列；
    向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐。
  2. 根据权利要求1所述的方法，其特征在于，在所述通过包括多个声部的音乐样本数据训练所述音乐生成模型之前，所述方法还包括：
    获取多个音乐训练样本，其中，所述音乐训练样本包括多个声部信息；
    提取每个声部的音符序列、所述音乐训练样本的节奏序列及延音序列；其中，所述每个声部的音符序列表示为：
    Figure PCTCN2018123549-appb-100001
    t∈[T]，T为所述音乐训练样本的时长，是十六分音符的整数倍；i为声部；
    Figure PCTCN2018123549-appb-100002
    为当前时刻t的音符；
    将所述多个声部的音符序列、所述音乐训练样本的节奏序列及延音序列作为所述音乐样本数据。
  3. 根据权利要求2所述的方法，其特征在于，所述通过包括多个声部的音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布，包括：
    向所述音乐生成模型中输入所述音乐样本数据；
    获取所述音乐生成模型输出的每个声部的音符概率密度函数：
    Figure PCTCN2018123549-appb-100003
    其中，
    Figure PCTCN2018123549-appb-100004
    为当前时刻t的音符，
    Figure PCTCN2018123549-appb-100005
    为音符序列中除去当前音符剩下的所有音符；
    Figure PCTCN2018123549-appb-100006
    为所述节奏序列及延音序列；θ i为所述依赖网络的参数；
    训练所述音乐生成模型使以下公式的值最大化：
    Figure PCTCN2018123549-appb-100007
    获取当所述公式的值最大时所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布。
  4. 根据权利要求3所述的方法，其特征在于：
    所述向所述音乐生成模型中输入所述音乐样本数据之后，所述音乐生成模型的所述第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据所述第一音符序列输出第一参数至所述依赖网络；
    所述第二长短时神经网络接收每个声部的音符序列中所述当前时刻音符后的预设时长的第二音符序列，并根据所述第二音符序列输出第二参数至所述依赖网络；
    所述单隐藏层神经网络接收每个声部的音符序列中所述当前时刻音符并传递至所述依赖网络；
    所述依赖网络根据所述第一参数、所述第二参数及所述当前时刻音符输出所述每个声部的音符概率密度函数。
  5. 根据权利要求1所述的方法，其特征在于，所述向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐，包括：
    向所述训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；
    所述音乐生成模型基于第i声部的音符随机序列、所述网络参数、所述特征参数及所述第i声部的音符概率密度分布生成所述第i声部的多个音符，i依次取一、二、三、四；
    根据所述第i声部的多个音符生成所述第i声部的音符新序列；
    将所述第一声部的音符新序列、所述第二声部的音符新序列、所述第三声部的音符新序列、所述第四声部的音符新序列组合形成所述多声部音 乐。
  6. 一种基于长短时神经网络的多声部音乐生成装置，其特征在于，所述装置包括：
    构建单元，用于构建音乐生成模型，所述音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；
    第一获取单元，用于通过包括多个声部的音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布；
    第二获取单元，用于获取用户输入的用于预生成多声部音乐的特征参数，所述特征参数包括预设音乐时长、预设节奏序列及预设延音序列；
    生成单元，用于向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐。
  7. 根据权利要求6所述的装置，其特征在于，所述装置还包括：
    第三获取单元，用于获取多个音乐训练样本，其中，所述音乐训练样本包括多个声部信息；
    提取单元，用于提取每个声部的音符序列、所述音乐训练样本的节奏序列及延音序列；其中，所述每个声部的音符序列表示为：
    Figure PCTCN2018123549-appb-100008
    t∈[T]，T为所述音乐训练样本的时长，是十六分音符的整数倍；i为声部；
    Figure PCTCN2018123549-appb-100009
    为当前时刻t的音符；
    处理单元，用于将所述多个声部的音符序列、所述音乐训练样本的节奏序列及延音序列作为所述音乐样本数据。
  8. 根据权利要求7所述的装置，其特征在于，所述第一获取单元，包括：
    输入子单元，用于向所述音乐生成模型中输入所述音乐样本数据；
    第一获取子单元，用于获取所述音乐生成模型输出的每个声部的音符概率密度函数：
    Figure PCTCN2018123549-appb-100010
    其中，
    Figure PCTCN2018123549-appb-100011
    为当前时刻t的音符，
    Figure PCTCN2018123549-appb-100012
    为音符序列中除去当前音符剩下的所有音符；
    Figure PCTCN2018123549-appb-100013
    为所述节奏序列及延音序列；θ i为所述依赖网络的参数；
    训练子单元，用于训练所述音乐生成模型使以下公式的值最大化：
    Figure PCTCN2018123549-appb-100014
    第二获取子单元，用于获取当所述公式的值最大时所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布。
  9. 根据权利要求8所述的装置，其特征在于：所述音乐生成模型的所述第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据所述第一音符序列输出第一参数至所述依赖网络；
    所述第二长短时神经网络接收每个声部的音符序列中所述当前时刻音符后的预设时长的第二音符序列，并根据所述第二音符序列输出第二参数至所述依赖网络；
    所述单隐藏层神经网络接收每个声部的音符序列中所述当前时刻音符并传递至所述依赖网络；
    所述依赖网络根据所述第一参数、所述第二参数及所述当前时刻音符输出所述每个声部的音符概率密度函数。
  10. 根据权利要求6所述的装置，其特征在于，所述生成单元包括：
    输入子单元，用于向所述训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；
    所述音乐生成模型基于第i声部的音符随机序列、所述网络参数、所述特征参数及所述第i声部的音符概率密度分布生成所述第i声部的多个音符，i依次取一、二、三、四；
    根据所述第i声部的多个音符生成所述第i声部的音符新序列；
    将所述第一声部的音符新序列、所述第二声部的音符新序列、所述第三声部的音符新序列、所述第四声部的音符新序列组合形成所述多声部音乐。
  11. 一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现以下步骤：
    构建音乐生成模型，所述音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；
    通过包括多个声部的音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布；
    获取用户输入的用于预生成多声部音乐的特征参数，所述特征参数包括预设音乐时长、预设节奏序列及预设延音序列；
    向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐。
  12. 根据权利要求11所述的计算机设备，其特征在于，所述处理器执行所述计算机程序时还实现以下步骤：
    获取多个音乐训练样本，其中，所述音乐训练样本包括多个声部信息；
    提取每个声部的音符序列、所述音乐训练样本的节奏序列及延音序列；其中，所述每个声部的音符序列表示为：
    Figure PCTCN2018123549-appb-100015
    t∈[T]，T为所述音乐训练样本的时长，是十六分音符的整数倍；i为声部；
    Figure PCTCN2018123549-appb-100016
    为当前时刻t的音符；
    将所述多个声部的音符序列、所述音乐训练样本的节奏序列及延音序列作为所述音乐样本数据。
  13. 根据权利要求12所述的计算机设备，其特征在于，所述处理器执行所述计算机程序时还实现以下步骤：
    向所述音乐生成模型中输入所述音乐样本数据；
    获取所述音乐生成模型输出的每个声部的音符概率密度函数：
    Figure PCTCN2018123549-appb-100017
    其中，
    Figure PCTCN2018123549-appb-100018
    为当前时刻t的音符，
    Figure PCTCN2018123549-appb-100019
    为音符序列中除去当前音符剩下的所有音符；
    Figure PCTCN2018123549-appb-100020
    为所述节奏序列及延音序列；θ i为所述依赖网络的参数；
    训练所述音乐生成模型使以下公式的值最大化：
    Figure PCTCN2018123549-appb-100021
    获取当所述公式的值最大时所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布。
  14. 根据权利要求13所述的计算机设备，其特征在于，所述处理器执行所述计算机程序时还实现以下步骤：
    所述音乐生成模型的所述第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据所述第一音符序列输出第一参数至所述依赖网络；
    所述第二长短时神经网络接收每个声部的音符序列中所述当前时刻音符后的预设时长的第二音符序列，并根据所述第二音符序列输出第二参数至所述依赖网络；
    所述单隐藏层神经网络接收每个声部的音符序列中所述当前时刻音符并传递至所述依赖网络；
    所述依赖网络根据所述第一参数、所述第二参数及所述当前时刻音符输出所述每个声部的音符概率密度函数。
  15. 根据权利要求11所述的计算机设备，其特征在于，所述处理器执行所述计算机程序时还实现以下步骤：
    向所述训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；
    所述音乐生成模型基于第i声部的音符随机序列、所述网络参数、所述特征参数及所述第i声部的音符概率密度分布生成所述第i声部的多个音符，i依次取一、二、三、四；
    根据所述第i声部的多个音符生成所述第i声部的音符新序列；
    将所述第一声部的音符新序列、所述第二声部的音符新序列、所述第三声部的音符新序列、所述第四声部的音符新序列组合形成所述多声部音乐。
  16. 一种计算机非易失性可读存储介质，所述存储介质包括存储的程序，其特征在于，在所述程序运行时控制所述存储介质所在设备执行以下 步骤：
    构建音乐生成模型，所述音乐生成模型包括一个第一长短时神经网络、一个第二长短时神经网络、一个单隐藏层神经网络及一个依赖网络；
    通过包括多个声部的音乐样本数据训练所述音乐生成模型，得到训练好的所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布；
    获取用户输入的用于预生成多声部音乐的特征参数，所述特征参数包括预设音乐时长、预设节奏序列及预设延音序列；
    向所述训练好的音乐生成模型中依次输入多个声部的音符随机序列，以使所述音乐生成模型根据所述音符随机序列、所述网络参数及所述多个声部的音符概率密度分布生成匹配所述特征参数的多声部音乐。
  17. 根据权利要求16所述的计算机非易失性可读存储介质，其特征在于，在所述程序运行时控制所述存储介质所在设备执行以下步骤：
    获取多个音乐训练样本，其中，所述音乐训练样本包括多个声部信息；
    提取每个声部的音符序列、所述音乐训练样本的节奏序列及延音序列；其中，所述每个声部的音符序列表示为：
    Figure PCTCN2018123549-appb-100022
    t∈[T]，T为所述音乐训练样本的时长，是十六分音符的整数倍；i为声部；
    Figure PCTCN2018123549-appb-100023
    为当前时刻t的音符；
    将所述多个声部的音符序列、所述音乐训练样本的节奏序列及延音序列作为所述音乐样本数据。
  18. 根据权利要求17所述的计算机非易失性可读存储介质，其特征在于，在所述程序运行时控制所述存储介质所在设备执行以下步骤：
    向所述音乐生成模型中输入所述音乐样本数据；
    获取所述音乐生成模型输出的每个声部的音符概率密度函数：
    Figure PCTCN2018123549-appb-100024
    其中，
    Figure PCTCN2018123549-appb-100025
    为当前时刻t的音符，
    Figure PCTCN2018123549-appb-100026
    为音符序列中除去当前音符剩下的所有音符；
    Figure PCTCN2018123549-appb-100027
    为所述节奏序列及延音序列；θ i为所述依赖网络的参数；
    训练所述音乐生成模型使以下公式的值最大化：
    Figure PCTCN2018123549-appb-100028
    获取当所述公式的值最大时所述音乐生成模型的网络参数及所述多个声部的音符概率密度分布。
  19. 根据权利要求18所述的计算机非易失性可读存储介质，其特征在于，在所述程序运行时控制所述存储介质所在设备执行以下步骤：
    所述音乐生成模型的所述第一长短时神经网络接收每个声部的音符序列中当前时刻音符前的预设时长的第一音符序列，并根据所述第一音符序列输出第一参数至所述依赖网络；
    所述第二长短时神经网络接收每个声部的音符序列中所述当前时刻音符后的预设时长的第二音符序列，并根据所述第二音符序列输出第二参数至所述依赖网络；
    所述单隐藏层神经网络接收每个声部的音符序列中所述当前时刻音符并传递至所述依赖网络；
    所述依赖网络根据所述第一参数、所述第二参数及所述当前时刻音符输出所述每个声部的音符概率密度函数。
  20. 根据权利要求16所述的计算机非易失性可读存储介质，其特征在于，在所述程序运行时控制所述存储介质所在设备执行以下步骤：
    向所述训练好的音乐生成模型中依次输入第一声部、第二声部、第三声部、第四声部的音符随机序列；
    所述音乐生成模型基于第i声部的音符随机序列、所述网络参数、所述特征参数及所述第i声部的音符概率密度分布生成所述第i声部的多个音符，i依次取一、二、三、四；
    根据所述第i声部的多个音符生成所述第i声部的音符新序列；
    将所述第一声部的音符新序列、所述第二声部的音符新序列、所述第三声部的音符新序列、所述第四声部的音符新序列组合形成所述多声部音乐。
PCT/CN2018/123549 2018-10-26 2018-12-25 基于长短时神经网络的多声部音乐生成方法及装置 Ceased WO2020082573A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811257165.1A CN109346045B (zh) 2018-10-26 2018-10-26 基于长短时神经网络的多声部音乐生成方法及装置
CN201811257165.1 2018-10-26

Publications (1)

Publication Number Publication Date
WO2020082573A1 true WO2020082573A1 (zh) 2020-04-30

Family

ID=65311898

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/123549 Ceased WO2020082573A1 (zh) 2018-10-26 2018-12-25 基于长短时神经网络的多声部音乐生成方法及装置

Country Status (2)

Country Link
CN (1) CN109346045B (zh)
WO (1) WO2020082573A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115035884A (zh) * 2022-05-27 2022-09-09 北京有竹居网络技术有限公司 音乐织体生成方法、装置、电子设备及存储介质
CN115346502A (zh) * 2022-08-11 2022-11-15 陕西师范大学 基于密集网络和多头注意力机制的音乐自动生成方法

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110008373B (zh) * 2019-03-14 2021-02-26 浙江大学 基于消息传递网络的音乐图结构信息生成模型的构建方法
CN110136730B (zh) * 2019-04-08 2021-07-20 华南理工大学 一种基于深度学习的钢琴和声自动编配系统及方法
CN110222226B (zh) * 2019-04-17 2024-03-12 平安科技（深圳）有限公司 基于神经网络的以词生成节奏的方法、装置及存储介质
CN110162659B (zh) * 2019-05-27 2020-09-15 湖南工程学院 一种基于音高网络的定节奏新曲生成方法
CN110517699B (zh) * 2019-08-23 2023-05-26 平安科技（深圳）有限公司 信息隐写方法、装置、设备及存储介质
CN114556465B (zh) * 2019-10-17 2025-09-23 雅马哈株式会社 演奏解析方法、演奏解析装置以及计算机程序产品
CN112017621B (zh) * 2020-08-04 2024-05-28 河海大学常州校区 基于对位和声关系的lstm多轨音乐生成方法
CN112885315B (zh) * 2020-12-24 2024-01-02 携程旅游信息技术（上海）有限公司 模型的生成方法、音乐合成的方法、系统、设备及介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107123415A (zh) * 2017-05-04 2017-09-01 吴振国 一种自动编曲方法及系统
CN107644630A (zh) * 2017-09-28 2018-01-30 清华大学 基于神经网络的旋律生成方法及装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9792889B1 (en) * 2016-11-03 2017-10-17 International Business Machines Corporation Music modeling

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107123415A (zh) * 2017-05-04 2017-09-01 吴振国 一种自动编曲方法及系统
CN107644630A (zh) * 2017-09-28 2018-01-30 清华大学 基于神经网络的旋律生成方法及装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BRUNNER, G. ET AL.: "JamBot: Music Theory Aware Chord Based Generation of Polyphonic Music with LSTMs,", 2017 INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE, 8 November 2017 (2017-11-08), pages 519 - 526, XP033353354, ISSN: 2375-0197 *
王程等 (WANG, CHENG ET AL.): "面向自动音乐生成的深度递归神经网络方法 (Recurrent Neural Network Method for Automatic Generation of Music)", 小型微型计算机系统 (JOURNAL OF CHINESE COMPUTER SYSTEMS), vol. 38, no. 10, 31 October 2017 (2017-10-31) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115035884A (zh) * 2022-05-27 2022-09-09 北京有竹居网络技术有限公司 音乐织体生成方法、装置、电子设备及存储介质
CN115346502A (zh) * 2022-08-11 2022-11-15 陕西师范大学 基于密集网络和多头注意力机制的音乐自动生成方法

Also Published As

Publication number Publication date
CN109346045B (zh) 2023-09-19
CN109346045A (zh) 2019-02-15

Similar Documents

Publication Publication Date Title
WO2020082573A1 (zh) 基于长短时神经网络的多声部音乐生成方法及装置
WO2020082574A1 (zh) 一种基于生成对抗网络的音乐生成方法及装置
CN103902642A (zh) 利用歌词与旋律之间的相关性的音乐作曲系统
Samsekai Manjabhat et al. Raga and tonic identification in carnatic music
CN103824565A (zh) 一种基于音符和时值建模的哼唱识谱方法及系统
CN113870818B (zh) 歌曲和弦编配模型的训练方法、装置、介质和计算设备
CN109326270A (zh) 音频文件的生成方法、终端设备及介质
JP2020003535A (ja) プログラム、情報処理方法、電子機器、及び学習済みモデル
CN113744721A (zh) 模型训练方法、音频处理方法、设备及可读存储介质
Huang et al. Piano music teaching under the background of artificial intelligence
CN114882859B (zh) 一种旋律与歌词对齐方法、装置、电子设备及存储介质
WO2026045686A1 (zh) 一种音乐识别模型的训练方法、音乐识别方法及相关设备
CN115273806A (zh) 歌曲合成模型的训练方法和装置、歌曲合成方法和装置
CN107993636A (zh) 基于递归神经网络的乐谱建模与生成方法
Zhang [Retracted] Research on Music Classification Technology Based on Deep Learning
CN112420002A (zh) 乐曲生成方法、装置、电子设备及计算机可读存储介质
Iskandar et al. Syllabic level automatic synchronization of music signals and text lyrics
Chang The integration of artificial intelligence and ethnic music cultural inheritance under deep learning
Wei et al. Optimization of piano performance teaching mode using network big data analysis technology
CN114117096B (zh) 多媒体数据处理方法及相关设备
Ding Application of big data mining technology in the digital construction of vocal music teaching resource library
Bairwa et al. MGU-V: a deep learning approach for lo-fi music generation using variational autoencoders with state-of-the-art performance on combined MIDI datasets
Wang et al. Construction of an Art Education Teaching System Assisted by Artificial Intelligence.
CN120045741A (zh) 歌曲生成方法、装置、电子设备及介质
CN116129934B (zh) 音频分析方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18938021

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18938021

Country of ref document: EP

Kind code of ref document: A1