WO2024093798A1 - 音乐创作方法、装置、电子设备及可读存储介质 - Google Patents

音乐创作方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2024093798A1
WO2024093798A1 PCT/CN2023/126882 CN2023126882W WO2024093798A1 WO 2024093798 A1 WO2024093798 A1 WO 2024093798A1 CN 2023126882 W CN2023126882 W CN 2023126882W WO 2024093798 A1 WO2024093798 A1 WO 2024093798A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
track
segments
music
timeline
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/126882
Other languages
English (en)
French (fr)
Inventor
彭浩翔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Priority to US18/876,601 priority Critical patent/US20250372068A1/en
Publication of WO2024093798A1 publication Critical patent/WO2024093798A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H7/00—Instruments in which the tones are synthesised from a data store, e.g. computer organs
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G10H1/0025—Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/101—Music Composition or musical creation; Tools or processes therefor
    • G10H2210/111—Automatic composing, i.e. using predefined musical rules
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/471—General musical sound synthesis principles, i.e. sound category-independent synthesis methods

Definitions

  • the present invention relates to a music creation method, device, electronic device and readable storage medium.
  • the present disclosure provides a music creation method, device, electronic device and readable storage medium.
  • the present disclosure provides a music creation method, comprising:
  • each of the first audio tracks is divided into a plurality of candidate track segments according to a timeline; each of the candidate track segments corresponds to an audio segment;
  • the one or more selected candidate track segments are determined as target track segments and an audio segment corresponding to the target track segment is determined to be added to the first audio track where the target track segment is located at a timeline position corresponding to the target track segment; wherein the audio segments added to the plurality of target track segments belonging to the same first audio track are the same; and the audio segments added to the plurality of target track segments belonging to different first audio tracks are the same.
  • the audio clip added to the target track clip of an audio track is different;
  • the audio clips added to the multiple first audio tracks are mixed, synthesized and played according to the timeline.
  • the method before presenting the plurality of first audio tracks, the method further includes:
  • the audio segments corresponding to the multiple candidate track segments on the corresponding first audio tracks are the audio segments of the musical instruments corresponding to the first audio tracks.
  • it also includes: adjusting the position range covered by the target track segments included in the multiple first audio tracks on the timeline, and adjusting the speed of the corresponding audio segment based on the position range covered by the adjusted target track segments on the timeline, so that the duration of the audio segment matches the position range covered by the adjusted target track segments on the timeline.
  • the method further includes: in response to a trigger operation on a newly added track control, generating and displaying a newly added first audio track, and determining audio segments corresponding to a plurality of candidate track segments of the newly added first audio track.
  • the method further includes: in response to a trigger operation for a delete track control, deleting the first audio track corresponding to the delete track control.
  • the method further includes: in response to an export instruction, exporting and storing the mixed data obtained by mixing and synthesizing the audio clips on the multiple first audio tracks as an audio file in a specified format.
  • the method further includes: adding the audio material imported by the user to the second audio track for mixing with the audio clip added to the first audio track; wherein the starting time position of the position interval covered by the audio material on the timeline is aligned with the starting time position of the timeline;
  • the audio material on the second audio track is mixed, synthesized and played with the audio clips on the plurality of first audio tracks according to the timeline.
  • the method further includes: performing audio processing on the audio material on the second audio track, where the audio processing includes: One or more of cutting, speed change, pitch change, and voice change.
  • it also includes: in the process of playing the mixed data obtained by mixing the audio clips on the multiple first audio tracks, responding to the trigger operation for the custom audio clip at the first moment, synthesizing the custom audio clip with the mixed data played after the playback moment corresponding to the trigger operation and playing the synthesized audio data.
  • the method further includes: obtaining recorded audio, and mixing the recorded audio with the audio clips on the multiple first audio tracks according to the timeline to obtain mixed data, and then synthesizing and playing the synthesized audio data.
  • the method further includes: acquiring video material, and synthesizing the mixed data obtained by mixing the video material with the audio clips on the multiple first audio tracks according to the timeline, and playing the obtained video data.
  • the method further includes: performing image processing on the video material to obtain video material with target image effects.
  • the present disclosure provides a music creation device, comprising:
  • a display module used for displaying a plurality of first audio tracks, wherein each of the first audio tracks is divided into a plurality of candidate track segments according to a timeline; each of the candidate track segments corresponds to an audio segment;
  • an audio track processing module configured to respond to a selection operation on one or more candidate track segments among the candidate track segments of the plurality of first audio tracks, determine the selected one or more candidate track segments as target track segments, and determine that an audio segment corresponding to the target track segment is added to the first audio track where the target track segment is located at a timeline position corresponding to the target track segment; wherein the audio segments added to the plurality of target track segments belonging to the same first audio track are the same; and the audio segments added to the target track segments of different first audio tracks are different;
  • a synthesis module configured to respond to a mixing instruction and perform mixing synthesis on the audio clips added to the plurality of first audio tracks according to a timeline;
  • the playing module is used to play the mixing data generated by the mixing synthesis.
  • the present disclosure provides an electronic device, comprising: a memory and a processor;
  • the memory is configured to store computer program instructions
  • the processor is configured to execute the computer program instructions so that the electronic device implements The music composition method described above.
  • the present disclosure provides a readable storage medium, including: computer program instructions, at least one processor of an electronic device executes the computer program instructions, so that the electronic device implements the above-mentioned music creation method.
  • the present disclosure provides a computer program product, and an electronic device runs the computer program product, so that the electronic device implements the above-mentioned music creation method.
  • the present disclosure provides a music creation method, device, electronic device and readable storage medium, wherein the method divides a first audio track into multiple candidate track segments according to a timeline, each candidate track segment corresponds to a music beat, and in addition, by pre-establishing a correspondence between multiple candidate track segments and audio segments on the first audio track, a user can add an audio segment to the corresponding music beat by performing a simple selection operation on one or more of the candidate track segments, which is convenient for the user to understand and create music; thereafter, in response to a mixing instruction, the audio segments added to the multiple first audio tracks are mixed, synthesized and played according to the timeline, so that the user can preview the created audio.
  • creation can be performed with only a mobile device, breaking the existing restrictions on hardware devices for music creation.
  • FIG1 is a flow chart of a music creation method provided by an embodiment of the present disclosure.
  • FIG2 is a flow chart of a music composition method provided by another embodiment of the present disclosure.
  • FIG3 is a flow chart of a music creation method provided by another embodiment of the present disclosure.
  • FIGS. 4A to 4E are schematic diagrams of interactive interfaces provided by an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of the structure of a music creation device provided in an embodiment of the present disclosure.
  • the present disclosure provides a music creation method, device, electronic device, readable storage medium and computer program product, wherein the present disclosure provides a music creation tool that reduces the threshold for users to create and edit music by using an abstract music data model and a digital creation link, and only a mobile device is needed to completely create a piece of music.
  • the music creation tool adds rhythm, style and timbre selection on the basis of providing users with atomic creation capabilities. Users can create according to their own preferences, and at the same time migrate the hardware device capabilities to the software, breaking away from expensive and heavy hardware devices, completely simulating the immersive music experience brought by the former, and users can create anytime, anywhere.
  • the music creation tool also provides music re-creation (remix) capabilities, and performs secondary creation based on existing works to meet the user's music creation needs.
  • the music creation tool can automatically initialize the corresponding instrument track to match the music style according to the music style selected by the user, further facilitate the user to get started, reduce user creation barriers, and provide original capabilities for user creation to a greater extent.
  • the music creation tool has also established a complete music creation chain, connecting the entire process from 0 to 1 in the creation process, including various nodes such as music creation, voice input (recording audio), real-time video (recording video), special effects rendering, work preview and work saving. It can more comprehensively meet the various needs of users in the creation process, greatly enhance the user's interest in creation, and make comprehensive music creation possible.
  • the music creation tool provided by the present disclosure provides a first audio track corresponding to the musical instrument and a second audio track corresponding to the music re-creation.
  • the music creation tool provides functions such as the ability to add free sound effects, audio recording, audio processing, video recording, and image processing, for realizing at least the following creation capabilities:
  • Atomic creation capability A piece of music is divided into instrument tracks, timelines, rhythms, and fragments. Users can choose instruments, rhythms, music styles, etc., and create through simple operations, which greatly reduces the threshold for creation and stimulates users' interest in creation. At the same time, it provides real-time improvisation, such as electronic music, Vocal effects can also be added directly and conveniently during real-time recording.
  • Music re-creation refers to re-creating existing music.
  • the imported audio material can also provide audio processing capabilities such as speed change, voice change, and pitch change. Re-creation not only retains the original music style, but also can add the user's understanding and ideas of the work, greatly stimulating the user's creative potential.
  • Real-time audio and video recording It provides complete audio and video creation tools, opening up the complete link from music production, original sound input to audio recording and saving music videos (MV). It also adds video recording and image processing, such as filters, special effects rendering, etc., to provide a one-stop solution for video and audio creation.
  • the music creation method disclosed in the present invention is performed by an electronic device.
  • the electronic device may be a tablet computer, a mobile phone (such as a folding screen mobile phone, a large screen mobile phone, etc.), a wearable device, a vehicle-mounted device, an augmented reality (AR)/virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like.
  • the present invention does not impose any restrictions on the specific type of the electronic device.
  • Figure 1 is a schematic diagram of the process of the music creation method provided by the embodiment of the present disclosure.
  • the music creation method provided by the present disclosure may include:
  • each first audio track is divided into multiple candidate track segments according to a timeline, and each candidate track segment corresponds to an audio segment.
  • a music creation tool can be installed in an electronic device, and the music creation tool can provide multiple types of audio tracks.
  • the electronic device can add audio clips for mixing and synthesis on the audio tracks.
  • Each type can correspond to one or more audio tracks, and the number of audio tracks of different types can be the same or different. In the creation process, the number of audio tracks corresponding to some types can be adjusted by the user.
  • the various types of audio tracks provided by the music creation tool may include but are not limited to: The corresponding first audio track, the second audio track corresponding to the music re-creation (remix capability), etc.
  • Each type may include one or more audio tracks, and the present disclosure does not limit the types of audio tracks provided by the music creation tool and the number of audio tracks included in each type.
  • the music creation tool can provide a first audio track corresponding to the instrument, and the first audio track can be divided into multiple candidate track segments according to the timeline, and the position intervals covered by the candidate track segments on the multiple first audio tracks on the timeline can be the same, or, it can also be understood that the lengths of the candidate track segments of the multiple first audio tracks are consistent.
  • Multiple first audio tracks can be divided according to the set music rhythm, and each candidate track segment corresponds to a beat, wherein the slower the set music rhythm, the longer the interval covered by the candidate track segment on the timeline, and the faster the set music rhythm, the shorter the interval covered by the candidate track segment on the timeline.
  • the music rhythm supports user adjustment.
  • the audio segments corresponding to the multiple candidate track segments belonging to the same first audio track are the same, that is, they correspond to the audio segments of the same instrument; the audio segments corresponding to the candidate track segments on different first audio tracks are different, that is, they correspond to the audio segments of different instruments.
  • the time range corresponding to the candidate track segment on the timeline hereinafter referred to as the duration corresponding to the candidate track segment
  • the duration of the audio segment corresponding to the candidate track segment may be consistent or inconsistent.
  • the audio segments corresponding to the candidate track segments on each first audio track may be pre-recorded and processed using corresponding musical instruments and then stored in the storage space of the electronic device. Based on the user's selection operation on the candidate track segment, the corresponding audio segment may be read from the storage space of the electronic device and added to the corresponding position of the corresponding candidate track segment on the timeline of the first audio track where the currently operated candidate track segment is located.
  • the music creation tool can display multiple first audio tracks and multiple candidate track segments included in each first audio track through an electronic device.
  • each first audio track can correspond to a display area
  • each candidate track segment corresponds to a display area in the display area corresponding to the first audio track.
  • the display areas corresponding to the multiple candidate track segments included in the first audio track can be arranged in sequence according to the sequence of the positions of the multiple candidate track segments on the timeline, such as from left to right, from top to bottom, etc.
  • the display areas corresponding to the multiple candidate track segments belonging to the same first audio track do not overlap with each other, so that users can clearly distinguish multiple candidate track segments and perform selection operations.
  • the plurality of first audio tracks may be generated and set manually one by one by the user, and the user sets the audio segments corresponding to the candidate track segments on each first audio track by setting the instruments corresponding to the first audio tracks.
  • the music creation tool can automatically match the instrument combination according to the music style selected by the user, generate multiple first audio tracks corresponding to the multiple instruments included in the instrument combination, and each first audio track is divided into multiple candidate tracks according to the timeline; and based on the instruments corresponding to each first audio track, the audio clips corresponding to the multiple candidate track clips on each first audio track are respectively determined.
  • multiple music styles can be set in the music creation tool in advance, and each music style corresponds to a musical instrument combination.
  • the music style information input by the user is obtained, and the correspondence between the pre-set music style and the musical instrument combination can be queried to determine the musical instrument combination corresponding to the music style specified by the music style information input by the user, and a corresponding first audio track can be established for each instrument in the musical instrument combination. Therefore, in this embodiment, the number of first audio tracks can be one or more, and the number is related to the music style (the number of instruments included in the musical instrument combination corresponding to the music style).
  • each musical instrument may correspond to multiple audio clips, and the duration, pitch, volume, etc. of different audio clips may be different.
  • the user's selection operation may be responded to to automatically select an appropriate audio clip from the multiple audio clips corresponding to the musical instrument and add it to the position of the corresponding candidate track clip on the timeline.
  • the strategies mentioned here may be, but are not limited to, selecting based on the duration of the candidate track clip, selecting a duration close to the duration of the candidate track clip as much as possible, etc.
  • a user wants to create Chinese-style music
  • he or she inputs music style information into the music creation tool to indicate that the music style to be created is Chinese-style.
  • the music creation tool can then match three traditional Chinese musical instruments, erhu, guzheng and pipa, for the user, and establish first audio tracks corresponding to the erhu, guzheng and pipa respectively.
  • the user can add the rhythm audio clip corresponding to the erhu to the first audio track corresponding to the erhu, add the rhythm audio clip corresponding to the guzheng to the first audio track corresponding to the guzheng, and add the rhythm audio clip corresponding to the pipa to the first audio track corresponding to the pipa.
  • the first audio track can be understood as an audio track that supports pre-editing.
  • the audio clips on each rhythm point i.e. the target track clip
  • the first audio track can be added or deleted at will.
  • first audio tracks may also be determined in other ways, and are not limited to the implementation methods of the above examples.
  • the one or more selected candidate track segments are determined as target track segments and an audio segment corresponding to the target track segment is added to the first audio track where the target track segment is located at a timeline position corresponding to the target track segment.
  • the selection operation for the candidate track segment may be, but is not limited to, click, double-click, long press, slide, etc.
  • the selection operation for the candidate track segments on different first audio tracks may be the same or different types of operations, which is not limited in the present disclosure.
  • the selected candidate track segment is the target track segment.
  • the audio segment corresponding to the selected candidate track segment is the audio segment added to the first audio track and can participate in the mixing synthesis.
  • the audio segment corresponding to the unselected candidate track segment can be understood as an audio segment not added to the first audio track and cannot participate in the mixing synthesis.
  • the selected candidate track fragment and the unselected candidate track fragment may adopt different display styles, and the selected candidate track fragments on different first audio tracks may adopt different display styles to facilitate user distinction, for example, using different colors to fill the display area corresponding to the candidate track fragment in the user interface.
  • the duration corresponding to the target track segment and the duration of the corresponding audio segment may be consistent or inconsistent. If the duration corresponding to the target track segment and the duration of the corresponding audio segment are consistent, the audio segment will be added to the position interval of the target track segment on the timeline, and the starting time of the audio segment will be consistent with the starting time of the target track segment on the timeline. For example, if the user selects the first candidate track segment on a first audio track, audio segment 1 will be added to the position interval of the first track segment on the timeline, that is, audio segment 1 occupies one track segment.
  • the audio segment will be added to the position interval of the selected candidate track segment and one or more adjacent candidate track segments on the timeline, and the starting time of the audio segment will be consistent with the starting time of the selected candidate track segment on the timeline. For example, if the user selects the first track segment on a first audio track, The duration of audio segment 2 is 1.5 times the duration of the selected track segment, so audio segment 2 is added to the position interval of the first track segment and the second track segment on the timeline, that is, the audio segment occupies 2 track segments. It can also be understood that the target track segment includes multiple candidate track segments.
  • the music creation tool may obtain the mixing instruction input by the user, and in response to the mixing instruction, mix and synthesize the audio clips added to the multiple first audio tracks according to the timeline and play them.
  • the mixing instruction may be, but is not limited to, triggered by the user operating one or more buttons on the interactive interface provided by the music creation tool.
  • the music composition tool may provide a start mixing control and an end mixing control.
  • the music composition tool may automatically start mixing and synthesizing the audio clips on each first audio track until the user operates the end mixing control to stop the mixing and synthesis.
  • the music creation tool may provide a start mixing control, an end mixing control, and a play button for controlling the synchronous playback and pause of multiple first audio tracks.
  • the music creation tool starts to mix and synthesize the audio clips on each first audio track until the user operates the end mixing control. It should be noted that since there is a time sequence between the user's operation to start the mixing control and the play button, there is no mixing input data in this time period. In the exported audio file, the audio clip corresponding to this time period can be understood as a silent clip.
  • the starting positions of the first audio tracks are aligned on the timeline, and in response to the mixing instruction, the audio clips on the first audio tracks can be mixed and synthesized starting from the starting time position of the timeline, and the synthesized audio data can be played.
  • the synthesized audio data can be played.
  • the synthesized time position the audio data of the audio clips whose position intervals on the first audio tracks cover the time position at the corresponding time position are mixed.
  • mixing synthesis When performing mixing synthesis, you can perform mixing synthesis based on the relationship between the audio clips on the timeline to obtain mixed data, and then input the mixed data to the sound card for conversion and playback; you can also input the audio clips on each first audio track into the sound card through different channels for playback, and record the sound output by the sound card Thus, the mixing data is obtained.
  • the method provided in this embodiment divides the first audio track into multiple candidate track segments according to the timeline, each candidate track segment corresponds to a beat.
  • the user can add audio segments to the corresponding beat by performing a simple selection operation on one or more of the candidate track segments, which is convenient for the user to understand and create music; then, in response to the play instructions for the multiple first audio tracks, the audio segments added to the multiple first audio tracks are mixed and synthesized and played according to the timeline, so that the user can preview the created synthesized audio.
  • a music creation tool that uses an abstract music data model and a digital creation link to reduce the threshold for users to create and edit music, a complete piece of music can be created with only a mobile device, breaking the existing restrictions on hardware devices for music creation.
  • the music creation tool may also respond to an export instruction to export and store the mixed data obtained by mixing and synthesizing the audio clips on the multiple first audio tracks according to the timeline as an audio file in a finger format.
  • the duration of the audio file may be determined according to the length of the first audio track, or may be a preset duration, or may be determined according to the time from when the user controls to start mixing to when the mixing ends.
  • FIG. 2 is a flow chart of a music creation method provided by another embodiment of the present disclosure. As shown in FIG. 2, the method of this embodiment may include:
  • each first audio track is divided into multiple candidate track segments according to a timeline, and each candidate track segment corresponds to an audio segment.
  • the one or more selected candidate track segments are determined as target track segments and an audio segment corresponding to the target track segment is added to the first audio track where the target track segment is located at a timeline position corresponding to the target track segment.
  • Step S201 and step S202 can refer to the detailed description of steps S101 and S102 in the embodiment shown in FIG. 1 , and for the sake of brevity, they are not repeated here.
  • S203 Acquire the audio material imported by the user, and add the audio material to the second audio track.
  • the audio material can be an existing song or a song imported by the user, and can be trimmed, speed-changed.
  • the audio can be processed by pitch change and voice change as part of the mix, or it can be pure existing human voice, such as freestyle rap or a cappella.
  • the second audio track is convenient for users to create secondary works based on existing works.
  • the music creation tool can display to the user an entry for importing audio materials for mixing through an electronic device, through which the user can enter an audio material selection page for selection, wherein the audio materials available for selection by the user can be displayed in thumbnails and aggregates on the audio material selection page.
  • the music creation tool can also provide the user with controls or function panels corresponding to user audio processing through an electronic device, so that the user can perform audio processing on the selected audio materials.
  • the second audio track can also be understood as an audio track that supports pre-editing.
  • the playback can be triggered at the required time, and the audio material on the second audio track can be deleted, replaced and processed at any time.
  • the operation on the first audio track and the operation on the second audio track may be performed in any order and may be performed repeatedly.
  • the music creation tool may obtain the mixing instruction input by the user, and in response to the mixing instruction, mix and synthesize the audio clips added to the plurality of first audio tracks and the audio materials on the second audio track and play them.
  • the mixing instruction may be, but is not limited to, triggered by the user operating one or more buttons on the interactive interface provided by the music creation tool.
  • the music creation tool may provide a start mixing control and an end mixing control.
  • the music creation tool may automatically start mixing and synthesizing the audio clips on each first audio track and the audio material on the second audio track until the user stops the mixing and synthesizing. In this way, it can be understood that the first audio track and the second audio track are aligned on the timeline, and the audio material on the second audio track starts from the start time of the timeline.
  • the music creation tool may provide a start mixing control, an end mixing control, a play button 1 for controlling the synchronous playback and pause of multiple first audio tracks, and a play button 2 for controlling the playback and pause of the second audio track; when the user sequentially operates the start mixing control, the play button 1, and the play button 2, the music creation tool may provide a start mixing control, an end mixing control, a play button 1 for controlling the synchronous playback and pause of multiple first audio tracks, and a play button 2 for controlling the playback and pause of the second audio track.
  • Play button 2 wherein the order of operating play button 1 and play button 2 is not limited, and the music creation tool mixes and synthesizes the audio clips on the corresponding audio track in the order of user operation until the user ends the mixing control.
  • play button 1 and play button 2 since there is a time sequence between the user's operation to start the mixing control, play button 1 and play button 2, there is no mixing input data in the time period from the user's operation to start the mixing control to the operation of the first play button.
  • the audio corresponding to this time period can be understood as a silent clip.
  • the audio corresponding to this time period is a mixture of the audio clips on the audio track corresponding to the first play button.
  • mixing synthesis can be performed based on the relationship between the audio clips on each first audio track and the audio materials on the second audio track on the timeline to obtain mixed data, and then the mixed data can be input into the sound card for conversion and playback; or the audio clips on each first audio track and the audio materials on the second audio track can be input into the sound card through different channels for playback, and the sound output by the sound card can be recorded to obtain mixed data.
  • the music creation tool can also respond to the export instruction to export and store the mixed data obtained by mixing and synthesizing the audio clips on the multiple first audio tracks and the audio materials on the second audio track according to the timeline as an audio file in a finger format.
  • FIG3 is a flow chart of a music creation method provided by another embodiment of the present disclosure. Referring to FIG3 , the method of this embodiment includes:
  • S301 Display multiple first audio tracks, where each first audio track is divided into multiple candidate track segments according to a timeline, and each candidate track segment corresponds to an audio segment.
  • the one or more selected candidate track segments are determined as target track segments and an audio segment corresponding to the target track segment is added to the first audio track where the target track segment is located at a timeline position corresponding to the target track segment.
  • S304 in the process of playing the mixed data obtained by mixing the audio clips on the plurality of first audio tracks, in response to a trigger operation for the custom audio clip, obtaining the custom audio clip
  • the segment is used to synthesize the mixed audio data played after the playing time corresponding to the trigger operation.
  • the user can simultaneously add a custom audio clip to add a free sound effect.
  • the addition of the free sound effect i.e., the custom audio clip
  • the addition of the free sound effect is not limited by the minimum unit time, that is, it is not limited by the candidate track clips included in the first audio track, and can be triggered by the user in real time at any time node. And the user can add different custom audio clips at different playback times.
  • Customized audio clips may include but are not limited to some private labels and personal logos of human voices, electronic sounds, special effects sounds, etc.
  • the music creation tool can display icons corresponding to different customized audio clips to the user through the electronic device, and the user can add customized audio clips by operating the icons, and the operation can be but not limited to single click, double click, long press, etc.
  • the custom audio clip can be synthesized with the mixed data after the playback time corresponding to the triggering operation and the synthesized audio data can be played.
  • the mixed data here is obtained by mixing and synthesizing multiple audio clips on the first audio track and the audio material on the second audio track.
  • the music creation tool can also respond to export instructions, export and store the audio data obtained by mixing and synthesizing multiple audio clips on the first audio track and the custom audio clips according to the timeline as an audio file in a finger format.
  • the method further includes:
  • S305 Obtain recorded audio, where the recorded audio is used for mixing and synthesizing with mixed data obtained by mixing and synthesizing the audio clips on the multiple first audio tracks.
  • a voice pickup module (such as a microphone) can be turned on to synchronously record and obtain recorded audio in real time to add vocal effects to the synthesized music.
  • the mixed data synthesized by multiple first audio tracks is the background music presentation of the recorded audio.
  • users can turn audio recording on or off at any time.
  • Step S305 can realize the link between music creation and original sound input, so as to meet the user's creative needs.
  • the music creation tool can also respond to the export instruction, export the audio data obtained by mixing and synthesizing the audio clips on the multiple first audio tracks and the recorded audio according to the timeline, and store them as an audio file in a finger format.
  • the method further includes:
  • S306 Obtain video material, where the video material is used to be mixed with audio clips on multiple first audio tracks to obtain mixed data to obtain video data.
  • the video material may be an existing video in an electronic device that can be imported by the user, or may be recorded in real time by starting the camera of the electronic device during playback, or may be a combination of the two, and the present disclosure does not limit this. If the video material imported by the user is included, and the real-time video recording is also started, during the real-time video recording, the existing video imported by the user from the electronic device can be played as a recorded picture-in-picture, or it can completely replace the recorded video, that is, the picture of the picture-in-picture occupies the entire video screen. In addition, if the existing video material imported by the user is included, it can be imported before starting the mixing, that is, before the user enters the mixing instruction.
  • the mixed sound data can be integrated with the video material as background sound.
  • This method can facilitate users to create music videos (MVs) and meet the creative needs of users.
  • the video material can also be processed, and the image processing methods include but are not limited to: filters, special effects, picture enhancement, rotation, etc. If the video material is recorded in real time, each frame of the video image captured by the camera can be processed during the recording process and synthesized synchronously with the audio track; if the video material is imported by the user, the video frame image can be processed frame by frame during the synthesis process and synthesized synchronously with the audio track.
  • the method of this embodiment provides a video recording function through a music creation tool, completely opening up the complete link from music creation, original sound input to audio recording, and also adds the functions of video recording, special effects rendering and MV saving, providing a one-stop solution for video and audio creation.
  • the music creation tool can also respond to the export instruction to export and store the video data obtained by synthesizing the audio clips and video materials on multiple first audio tracks according to the timeline as a video file in a finger format.
  • the above steps S304 to S306 may be performed in parallel.
  • the music creation tool can be deployed on a mobile device, and by realizing the music creator capability on the mobile terminal, music creation is unrestricted, and users can express their inspiration anytime, anywhere.
  • the above-mentioned multiple capabilities of the music creation tool can realize the professionalism of the creation and simplify the music creation process, thereby stimulating the public's interest and making comprehensive creation possible.
  • Figures 4A to 4E take the electronic device as a mobile phone, the mobile phone is installed with a music creation tool, and music creation is performed through application 1 as an example.
  • FIG. 4A to FIG. 4E are schematic diagrams of human-computer interaction interfaces provided by embodiments of the present disclosure.
  • the music creation tool is started, a music style is selected, and audio clips of corresponding instruments are added to the track clips on the first audio track and audio materials are added to the second audio track.
  • the music creation tool can display a user interface 11 as shown in FIG. 4A on the mobile phone, wherein the user interface 11 includes: area 101, area 102, area 103, and area 104.
  • area 101 can be understood as an atomic audio creation area.
  • users can select a music style and automatically determine and display the instrument combination and the corresponding first audio track based on the music style. They can also add or delete the first audio track, change the timeline length to increase or decrease the beat, adjust the rhythm speed, and so on.
  • area 101 includes: a label 101a and an area 101b , wherein label 101a is used to trigger display of a music style list, and area 101b is used to display a first audio track corresponding to a currently selected music style and components or information related to the first audio track.
  • the music creation tool can display the user interface 12 shown in FIG. 4B in response to the trigger operation on the tag 101a.
  • the user interface 12 includes a music style list, which includes a variety of music style options for the user to choose from.
  • the user can view more music style options by sliding up and down or in other ways to switch music styles, and one or more first audio tracks of the instrument combination corresponding to the selected music style option are displayed in area 101b.
  • the music creation tool When the editing tool is started, multiple first audio tracks corresponding to the instrument combination corresponding to the specified music style can be displayed by default, and the candidate track segments on each first audio track are all unselected, and the timeline length of the first audio track can also be displayed according to the default length, such as 10 time units by default.
  • the currently selected music style can be displayed as selected, and the others can be displayed as unselected.
  • the music creation tool can exemplarily display the user interface 13 shown in FIG4C on the mobile phone, and the area 101b displays the instruments corresponding to the rock style, which are the first audio tracks corresponding to the bass, guitar, drum set, and keyboard, respectively.
  • the music style list can be exited by clicking any other position outside the music style list.
  • the music style list may also include a custom style option, such as the “Custom 1” option shown in FIG. 4B .
  • the custom style option may include a combination of instruments that the user has previously defined.
  • multiple first audio tracks corresponding to the corresponding instrument combination may be displayed; in other embodiments, no first audio track may be displayed in area 101b, but the user generates a custom style option by triggering the addition of a custom style, and adds a first audio track to the generated custom style option and sets the associated instrument type by adding a new audio track, and saves the instrument combination information of the custom style option to the music style list for the user to use again.
  • Different custom style options can be displayed through the music style name area, and the music style name can be edited by the user.
  • area 101b may include a display area corresponding to each first audio track, and the display area corresponding to the first audio track may include a label s1 for setting the volume of the audio clip, a label s2 for modifying the instrument type, a track s3, and a deletion label s4.
  • Track s3 is divided into multiple track segments according to time.
  • multiple square areas are displayed in an arrangement from left to right. Each square area represents a track segment.
  • the user can select a track segment by operating the square area, and add a corresponding audio segment to the corresponding position of the track on the timeline on the first audio track.
  • the display style of the selected track segment can be different from that of other unselected track segments. For example, as shown in FIG4A , the square area corresponding to the selected track segment is gray, and the square area corresponding to the unselected track segment is white.
  • the display styles of the selected track segments on different first audio tracks can be different, for example, different colors; The unselected track segments on the first audio track may adopt the same display style, for example, all are white.
  • the gray area on each first audio track in the user interface 11 shown in FIG4A is the selected track segment. Since the time lengths of the audio segments corresponding to different instruments may be different, one or more track segments may need to be occupied. When multiple track segments need to be occupied, the square areas of multiple track segments may be merged in response to the user's selection operation. For example, the 1st to 3rd time units on the first audio track corresponding to the piano in the last row are merged, and the 8th to 10th time units are merged, and the audio segment corresponding to the piano corresponds to the time range corresponding to the three track segments on the timeline.
  • the user can operate multiple times (such as continuously clicking) the square area corresponding to the same track segment to adjust the pitch of the audio segment.
  • it can be distinguished by but not limited to color brightness. The brighter the color, the higher the pitch, and the darker the color, the lower the pitch.
  • the area 101 may further include a label s5 for adding a new audio track.
  • the first audio track may be added by operating the label s5.
  • the newly added first audio track may be added in the last row according to the arrangement order of the audio tracks.
  • the instrument corresponding to the newly added first audio track may be set by operating the label s2 for modifying the instrument type corresponding to the newly added first audio track.
  • the music creation tool can respond to the user's trigger operation (such as clicking) on the label s2, and display the user interface 14 shown in Figure 4D on the mobile phone, and the user interface 14 displays a list of musical instruments, and the user can select the desired musical instrument from the list of musical instruments.
  • the musical instrument list can be exited by triggering any position outside the list area in the display screen.
  • the various musical instruments in the musical instrument list can be displayed in sequence according to the set order, or can also be displayed according to the category of musical instruments, and the name of each category is displayed in the musical instrument list. This disclosure does not limit this, and Figure 4D shows the former situation.
  • area 101 also includes area 101c, which is used to display the timeline corresponding to the atomic creation area.
  • the time units included in the current timeline in area 101c are arranged in sequence.
  • the user can increase the time unit or delete the time unit by operating the label s6 for increasing the beat and the label s7 for decreasing the beat to change the length of the timeline.
  • you can operate the labels s6 and s7 multiple times in succession (such as continuously clicking).
  • area 101 also includes area 101d, which is used to display the speed adjustment axis, which can also be understood as the music rhythm speed adjustment axis or the music beat speed adjustment axis.
  • the user can adjust the music rhythm by dragging the adjustment button on the speed adjustment axis.
  • the current speed value can be displayed in area 101d.
  • the speed value displayed in area 101d changes synchronously with the adjustment.
  • the display style of the time unit identifiers displayed in area 101b and area 101c of the user interface may remain unchanged (such as the size of the square area representing the time unit and the candidate track segment remains unchanged), or may change (such as the size of the square area representing the time unit and the candidate track segment becomes longer as the rhythm slows down or becomes shorter as the rhythm slows down).
  • the speed of the audio segment corresponding to the selected candidate track segment also needs to be adjusted so that the speed of the rhythm audio segment is consistent with the adjusted music rhythm, thereby adapting to the duration of the adjusted time unit.
  • area 101 also includes: a play button 101e, by operating the play button 101e, the electronic device can be controlled to play the audio clips on the multiple first audio tracks in area 101 for the user to preview the mixing effect.
  • a play button 101e by operating the play button 101e, the electronic device can be controlled to play the audio clips on the multiple first audio tracks in area 101 for the user to preview the mixing effect.
  • it can be played according to the timeline.
  • the user interface it can be understood as playing in columns from left to right in area 101b.
  • the square area corresponding to a column of track clips corresponding to the playing position can be highlighted, for example, the position and size of the square area corresponding to this column of track clips can change.
  • Area 102 can be understood as a local audio BGM creation area. Through the operation area 102, users can upload local audio files for secondary creation. The audio files uploaded through the operation area 102 are added to the second audio track. Through the operation area 102, the uploaded audio files can also be speed-changed, voice-changed, cropped, pitch-changed, volume-set, etc. Domain x5. Among them, label x1 is used to enter the audio file selection page, through which the audio material to be imported for secondary creation can be selected and added to the second audio track. Timeline x2 can display the total duration and playback progress of the audio material added by the user. The volume setting button x3 can increase or decrease the volume of the audio material on the second audio track during synthesis.
  • area 102 can also include: label x6, used to enter the audio processing function panel, the audio processing function panel can provide buttons or components corresponding to one or more audio processing functions such as cropping, speed change, voice change, and pitch change, and the audio material on the second audio track can be cropped, speed changed, voice changed, etc. by triggering the corresponding button or component.
  • the audio processing function panel can also provide a download function to download the audio material obtained after audio processing.
  • labels corresponding to some audio processing functions may be set in area 102.
  • buttons or components corresponding to cropping, speed change, voice change, and pitch change may be set in area 102 (such as below the black frame in the area where labels x2, x3, x4, and x6 are located) for user convenience.
  • Label x6 may not be set, and a download button may be set in area 102 to facilitate users in downloading audio materials obtained through audio processing.
  • the first audio track and the second audio track can be pre-edited through the area 101 and the area 102.
  • Area 103 is a free sound effect creation area. By pressing the keys on the keyboard provided in the operation area 103, free sound effects corresponding to the keys can be added at any time point on the third audio track.
  • the free sound effects corresponding to the keys on the keyboard support user customization. Users can bind the custom audio clips of the favorite sound effects to the keys according to their needs and use them when creating.
  • area 103 includes area 103a and multiple keys 103b.
  • Area 103a is used to display the theme content of area 103. For example, area 103a displays the area name "free sound effect creation area” and the detailed introduction of the area "press the corresponding keys on the keyboard to provide more free creation capabilities on the time track"; in addition, multiple keys 130b can correspond to different brands of music respectively.
  • multiple keys 103b correspond to FUHH, UFO, STRIKE, LONDON, MOON, WIPE, TIMER, FLASH, and ORDER brand music in turn, and correspond to the identifiers A, S, D, F, G, H, J, and K in turn.
  • the present disclosure does not limit the number of keys in the keyboard, and users can add and delete keys as needed.
  • the key shape can be round, the color can be colorful, and the logo can also be in other fonts and sizes.
  • the audio clips on the first audio track and the second audio track can be played by triggering the play button in area 101 and the play button in area 102.
  • free sound effects can be added to any timeline on the timeline by pressing the keys on the keyboard in operation area 103.
  • Area 104 is an audio and video creation area. Users can realize real-time recording of ambient audio and video by operating buttons in area 104, and can also import existing video materials from electronic devices by operating buttons in area 104. And the recorded video/imported video can be previewed through the preview window; in addition, the video material can be processed through image processing related buttons. Exemplarily, as shown in FIG4A, area 104 includes area y1, preview window y2, start preview label y3, end preview label y4, start recording label y5, end recording label y6, download material label y7, special effect label y8, rotation component y9 and movement component y10.
  • the user in area y1 displays the theme and detailed introduction of area 104, such as the text content "audio and video creation area” and “providing video track + microphone track + mixing track collection fusion capability and video special effects editing".
  • area 104 such as the text content "audio and video creation area” and “providing video track + microphone track + mixing track collection fusion capability and video special effects editing”.
  • other content can also be displayed, which is not limited in the present disclosure.
  • the preview window y2 can display the real-time recorded video screen, and can also be used to preview the video data synthesized by playing the video material and other audio tracks after the recording is finished.
  • the present disclosure does not limit the size and display style of the preview window y2.
  • the start preview tag y3 is used to trigger the playback of video data synthesized from the video material and other audio tracks in the preview window y2; similarly, the end preview tag y4 is used to trigger the end of the playback of the video synthesized from the video material and other audio tracks in the preview window y2.
  • the start recording tag y5 is used to trigger the recording of audio and/or video material and to trigger the start of mixed recording.
  • an option to enable the microphone to record audio and an option to enable the camera to record video can also be set.
  • the user can choose to enable the microphone alone or start the camera recording alone, or they can also choose both at the same time, which is more flexible.
  • buttons to disable the microphone and camera can be set separately. When the disable option is not selected, the microphone and camera are enabled by default for mixed recording. When they need to be disabled, they are set based on demand.
  • End Recording Tag y6 User triggered stop recording of audio and/or video material and end mix recording.
  • the user triggers the start recording tag y5 to input the mixing instruction, triggering the start of mixing recording and synchronously starting the recording of video and audio, and clicks tag 101e and tag x4 to play the audio clip on the first audio track.
  • free sound effects can also be added during the playback of the recorded mixed data.
  • the user triggers the start recording tag y6 to stop the mixed recording and stop recording audio and video. And jump to the preview interface to preview the final mixed data/video data.
  • the download material tag y7 is used to export the final mixed audio data/video data into an audio file/video file of a specified format.
  • the special effects label y8 is used to enter the special effects list.
  • the music creation tool responds to the trigger operation of the special effects label y8 by the user 1, and can display the user interface 15 as shown in FIG4E.
  • the special effects list is displayed in the user interface 15, and each special effect is displayed by the special effect name in the special effects list.
  • the user can choose to use the special effects.
  • the user can view more special effects by sliding the screen up and down or rolling the mouse wheel, but is not limited to.
  • the special effects to be used can be selected before starting the recording, or the special effects can be selected during the recording process.
  • the special effects are selected during the recording process, the special effects are applied to the video images recorded after the recording moment when the special effects are triggered, and the special effects are not applied to the video images recorded before.
  • the music creation tool can respond to the user's operation to apply the special effects to the video frame images displayed in the current preview window, and display the video frame images with the special effects added in the preview window for the user to preview the effects.
  • the rotation component y9 is used to rotate the video frame image of the video material.
  • the rotation can be clockwise or counterclockwise, and the rotation direction is not limited.
  • the rotation angle range is 0-360 degrees.
  • the rotation component y9 can be triggered in real time to rotate the video screen.
  • the moving component y10 is used to move the video frame image, and the moving component may include a component for moving the video frame image along the X axis and a component for moving the video frame image along the Y axis. Since moving the video frame image will cause part of the image area to move out of the preview window, there will be a part of the preview window that is not covered by the video frame image, and the uncovered window area may display a preset background color, such as black, gray, etc. During the video recording process, the moving component y10 may be triggered in real time to move the video screen horizontally or vertically.
  • area 104 may also include: playback controls for controlling preview playback and pausing preview playback, a timeline, a volume button, a full-screen button, and an entrance to a video-related function panel, etc.
  • the function panel may include controls corresponding to the download function, components for setting picture-in-picture, etc.
  • the present disclosure provides a music creation tool that reduces the threshold for users to create and edit music by using an abstract music data model and a digital creation link. You can create a complete piece of music with just a mobile device.
  • the music creation tool provides users with atomic creation capabilities and adds rhythm, style and timbre selection. Users can create according to their preferences. At the same time, the hardware device capabilities are migrated to the software, which is free from expensive and heavy hardware devices, and completely simulates the immersive music experience brought by the former. Users can create anytime and anywhere.
  • the music creation tool also provides music re-creation (remix) capabilities, which can be used to perform secondary creation based on existing works to meet the user's music creation needs.
  • the corresponding instrument track is automatically initialized to match the style, which is further convenient to get started, reduces user creation barriers, and provides users with original capabilities to a greater extent.
  • the music creation tool has established a complete music creation link, opening up the entire process from 0 to 1 in the creation process, including music creation, voice input, real-time video, special effects rendering, and work preservation.
  • Various nodes have greatly increased the user's interest in creation and made comprehensive music creation possible.
  • FIG5 is a schematic diagram of the structure of a music creation device provided in an embodiment of the present disclosure.
  • the music creation device 500 provided in this embodiment includes:
  • the display module 501 is used to display multiple first audio tracks, wherein each of the first audio tracks is divided into multiple candidate track segments according to a timeline; each of the candidate track segments corresponds to an audio segment.
  • the audio track processing module 502 is used to respond to the selection operation of one or more candidate track segments among the candidate track segments of the multiple first audio tracks, determine the one or more selected candidate track segments as target track segments and determine that the audio segment corresponding to the target track segment is added to the first audio track where the target track segment is located at the timeline position corresponding to the target track segment; wherein the audio segments added to multiple track segments belonging to the same first audio track are the same; and the audio segments added to the track segments of different first audio tracks are different.
  • the synthesis module 503 is used to respond to the mixing instruction and perform mixing synthesis on the audio clips added to the multiple first audio tracks according to the timeline.
  • the playing module 504 is used to play the mixed audio data generated by the mixed audio synthesis.
  • the audio track processing module 502 is further used to obtain a music style specified by a user, and determine an instrument combination matching the music style based on the music style specified by the user; generate the first audio tracks corresponding to the instruments included in the instrument combination, and determine the audio segments corresponding to the multiple candidate track segments on the first audio tracks; wherein the The audio segments corresponding to the multiple track segments on the first audio track are audio segments of musical instruments corresponding to the first audio track.
  • the audio track processing module 502 is further used to adjust the position range covered by the target track segments included in the multiple first audio tracks on the timeline, and adjust the speed of the corresponding audio segment based on the position range covered by the adjusted target track segments on the timeline, so that the duration of the audio segment matches the position range covered by the adjusted target track segments on the timeline.
  • the audio track processing module 502 is further used to respond to a trigger operation on a newly added track control, generate and display a newly added first audio track, and determine audio segments corresponding to multiple candidate track segments of the newly added first audio track.
  • the audio track processing module 502 is further configured to respond to a trigger operation for a delete track control and delete the first audio track corresponding to the delete track control.
  • the method further includes: an export module 505 for responding to an export instruction, exporting and storing the mixed data obtained by mixing and synthesizing the audio clips on the multiple first audio tracks as an audio file in a specified format.
  • the audio track processing module 502 is further used to add the audio material imported by the user to the second audio track for mixing with the audio clip added to the first audio track; wherein the starting time position of the position interval covered by the audio material on the timeline is aligned with the starting time position of the timeline;
  • the synthesis module 503 is used to respond to the mixing instruction and mix the audio material on the second audio track with the audio clips on the multiple first audio tracks according to the timeline to obtain mixed data; the playing module 504 is used to play the corresponding mixed data.
  • the audio track processing module 502 is further used to perform audio processing on the audio material on the second audio track, and the audio processing includes: one or more of: cropping, speed change, pitch change, and voice change.
  • the synthesis module 503 is also used to respond to a trigger operation on a custom audio clip during the process of playing the mixed data obtained by mixing the audio clips on the multiple first audio tracks, and synthesize the custom audio clip with the mixed data played after the playback time corresponding to the trigger operation; the playback module 504 is used to play the synthesized audio data.
  • the apparatus 500 further includes: an audio recording module 506 for obtaining recorded audio.
  • the synthesis module 503 is further configured to synthesize the mixed audio data obtained by mixing the recorded audio with the audio clips on the plurality of first audio tracks according to the timeline and play the synthesized audio data.
  • the apparatus 500 further includes: a video processing module 507 for acquiring video material.
  • a synthesis module 503 is further configured to synthesize the mixed audio data obtained by mixing the video material with the audio clips on the plurality of first audio tracks according to the timeline, and play the obtained video data.
  • the video processing module 507 is further used to perform image processing on the video material to obtain video material with target image effects.
  • the device of this embodiment can be used to execute the technical solution of any of the aforementioned method embodiments. Its implementation principle and technical effects are similar. Please refer to the detailed description of the aforementioned method embodiments. For the sake of brevity, they will not be repeated here.
  • the present disclosure provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; when the one or more processors execute the one or more computer programs, the electronic device implements the music creation method of the previous embodiment.
  • the present disclosure provides a chip system, which is applied to an electronic device including a memory and a sensor; the chip system includes: a processor; when the processor executes the music creation method of the above embodiment.
  • the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor in an electronic device, the music composition method of the foregoing embodiment is implemented.
  • the present disclosure provides a computer program product, which, when executed on a computer, enables the computer to execute the music composition method of the foregoing embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)

Abstract

一种音乐创作方法、装置、电子设备及可读存储介质，该方法包括：通过将第一音频轨道按照时间线划分为多个候选轨道片段，每个候选轨道片段对应一节拍；建立第一音频轨道上的多个候选轨道片段与音频片段之间的对应关系，用户通过针对其中一个或多个候选轨道片段执行简单的选中操作便可在相应音乐节拍中添加音频片段，方便用户理解和创作音乐；响应混音指令，按照时间线将多个第一音频轨道上添加的音频片段进行混音合成并播放。该方法能够降低用户创作音乐和编辑音乐门槛的难度。

Description

音乐创作方法、装置、电子设备及可读存储介质
本申请要求于2022年10月31日递交的中国专利申请第202211348849.9号的优先权，在此全文引用上述中国专利申请公开的内容以作为本申请的一部分。
技术领域
本公开涉及一种音乐创作方法、装置、电子设备及可读存储介质。
背景技术
目前用户对短视频发布的背景音仍然是从专业歌手的曲库中选取，曲库本身的存量歌曲有限。用户即使时而有迸发创作热情和灵感，面对需要组织乐队，乐器采购和培训，反复排练，录制，后期混音等一系列环节，由于链路长成本高使大多数用户最终也望而却步，无从下手。音乐创作和编辑的门槛高，成本高，链路长且繁琐，不利于用户灵感的发挥，因此，如何降低音乐创作门槛，帮助用户从零开始创作一首音乐是当前亟待解决的问题。
发明内容
为了解决上述技术问题，本公开提供了一种音乐创作方法、装置、电子设备及可读存储介质。
本公开提供了一种音乐创作方法，包括：
展示多个第一音频轨道，其中，各所述第一音频轨道按照时间线划分为多个候选轨道片段；每个所述候选轨道片段与一个音频片段相对应；
响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置；其中，属于同一所述第一音频轨道的多个目标轨道片段添加的音频片段相同；不同所述第 一音频轨道的目标轨道片段上添加的音频片段不同；
响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成并播放。
在一些实施例中，所述展示多个第一音频轨道之前，还包括：
获取用户指定的音乐风格，并基于所述用户指定的音乐风格确定与所述音乐风格匹配的乐器组合；
生成所述乐器组合包括的各乐器分别对应的所述第一音频轨道，并确定各所述分别对应的第一音频轨道上的多个候选轨道片段分别对应的音频片段；其中，所述第一音频轨道上的多个轨道片段分别对应的音频片段为所述第一音频轨道对应的乐器的音频片段。
在一些实施例中，还包括：调整所述多个第一音频轨道包括的目标轨道片段在时间线上覆盖的位置范围，且基于调整后所述目标轨道片段在时间线上覆盖的位置范围调整相应音频片段的速度，使得所述音频片段的时长与调整后的目标轨道片段在时间线上覆盖的位置范围相匹配。
在一些实施例中，还包括：响应针对新增轨道控件的触发操作，生成并展示新增第一音频轨道，并确定与所述新增第一音频轨道的多个候选轨道片段分别对应的音频片段。
在一些实施例中，还包括：响应针对删除轨道控件的触发操作，删除与所述删除轨道控件对应的所述第一音频轨道。
在一些实施例中，还包括：响应导出指令，将所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据导出并存储为指定格式的音频文件。
在一些实施例中，还包括：将用户导入的音频素材添加在第二音频轨道上，用于与所述第一音频轨道上添加的音频片段进行混音；其中，所述音频素材在所述时间线上覆盖的位置区间的起始时刻位置与所述时间线的起始时刻位置对齐；
响应所述混音指令，按照所述时间线将所述第二音频轨道上的音频素材与所述多个第一音频轨道上的音频片段进行混音合成并播放。
在一些实施例中，所述将用户导入的音频素材添加在第二音频轨道上之后，还包括：对所述第二音频轨道上的音频素材进行音频处理，音频处理包括： 裁剪、变速、变调、变声中的一项或多项。
在一些实施例中，还包括：在播放所述多个第一音频轨道上的音频片段通过混音合成得到的混音数据的过程中，响应第一时刻针对自定义音频片段的触发操作，将所述自定义音频片段与所述触发操作对应的播放时刻之后播放的混音数据进行合成并播放合成得到的音频数据。
在一些实施例中，还包括：获取录制音频，并按照时间线将所述录制音频与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据再进行合成并播放合成得到的音频数据。
在一些实施例中，还包括：获取视频素材，并按照时间线将所述视频素材与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据进行合成，并播放得到的视频数据。
在一些实施例中，还包括：对所述视频素材进行图像处理得到具有目标图像效果的视频素材。
本公开提供了一种音乐创作装置，包括：
展示模块，用于展示多个第一音频轨道，其中，各所述第一音频轨道按照时间线划分为多个候选轨道片段；每个所述候选轨道片段与一个音频片段相对应；
音轨处理模块，用于响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置；其中，属于同一所述第一音频轨道的多个目标轨道片段添加的音频片段相同；不同所述第一音频轨道的目标轨道片段上添加的音频片段不同；
合成模块，用于响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成；
播放模块，用于播放混音合成生成的混音数据。
本公开提供了一种电子设备，包括：存储器和处理器；
所述存储器被配置为存储计算机程序指令；
所述处理器被配置为执行所述计算机程序指令，使得所述电子设备实现 上述所述的音乐创作方法。
本公开提供了一种可读存储介质，包括：计算机程序指令，电子设备的至少一个处理器执行所述计算机程序指令，使得所述电子设备实现上述所述的音乐创作方法。
本公开提供了一种计算机程序产品，电子设备运行所述计算机程序产品，使得所述电子设备实现上述所述的音乐创作方法。
本公开提供一种音乐创作方法、装置、电子设备及可读存储介质，其中，该方法通过将第一音频轨道按照时间线划分为多个候选轨道片段，每个候选轨道片段对应一个音乐节拍，此外，通过预先建立第一音频轨道上的多个候选轨道片段与音频片段之间的对应关系，用户通过针对其中一个或多个候选轨道片段执行简单的选中操作便可在相应音乐节拍中添加音频片段，方便用户理解和创作音乐；之后，响应混音指令，按照时间线将多个第一音频轨道上添加的音频片段进行混音合成并播放，供用户预览创作的音频。且通过提供利用抽象音乐数据模型和数字化创作链路来降低用户创作音乐和编辑音乐门槛的音乐创作工具，只需要移动端设备就可以进行创作，打破了现有的音乐创作对于硬件设备限制。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开一实施例提供的音乐创作方法的流程图；
图2为本公开另一实施例提供的音乐创作方法的流程图；
图3为本公开另一实施例提供的音乐创作方法的流程图；
图4A至图4E为本公开一实施例提供的交互界面示意图；
图5为本公开一实施例提供的音乐创作装置的结构示意图。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例，而不是全部的实施例。
示例性地，本公开提供一种音乐创作方法、装置、电子设备、可读存储介质及计算机程序产品，其中，本公开通过提供利用抽象音乐数据模型和数字化创作链路来降低用户创作音乐和编辑音乐门槛的音乐创作工具，只需要移动端设备就可以完整创作一首音乐。此外，该音乐创作工具在为用户提供了原子创作能力的基础上加入了节奏、风格以及音色的选择，用户可根据自己的喜好进行创作，同时将硬件设备能力迁移至软件，脱离了昂贵繁重的硬件设备，完全模拟前者所带来的沉浸式音乐体验，用户可随时随地进行创作。此外，该音乐创作工具还提供了音乐再创作(remix)能力，在已有作品的基础上进行二次创作，满足用户的音乐创作需求。此外，该音乐创工具能够根据用户选择的音乐风格自动初始化相应乐器轨道匹配音乐风格，进一步方便用户上手，减少用户创作障碍，更大程度地为用户创作提供原始能力。且该音乐创作工具还建立了完整的音乐创作链路，打通创作过程中从0到1的整个流程，包含音乐创作、语音输入(录制音频)、实时视频(录制视频)、特效渲染、作品预览以及作品保存等各种节点，能够较为全面地满足用户在创作过程中的各种需求，极大提高了用户创作兴趣，使音乐创作全面化成为可能。
其中，本公开提供的音乐创作工具提供了乐器对应的第一音频轨道、音乐再创作对应的第二音频轨道，音乐创作工具提供了添加自由音效能力、音频录制、音频处理、视频录制、图像处理等功能，用于实现至少以下几种创作能力：
1、原子创作能力：把一首创作乐曲分为乐器轨，时间线，节奏，片段调。用户可自行选择乐器，节奏，音乐风格等，通过简单的操作即可进行创作，极大降低了创作门槛，激发了用户创作兴趣。同时提供实时即兴演奏，如电子音， 人声特效，在实时录制的时候也可以直接方便的加入。
2、根据所选音乐风格自动生成乐器组合：一键解决用户乐器匹配问题，例如，用户想要创作中国风的音乐，可以为用户直接匹配二胡、古筝以及琵琶等中国传统乐器，确保用户能够创作出符合预期的音乐风格的音乐。
3、音乐再创作(remix)：音乐再创作指对现有的音乐再次进行创作，此外，对导入的音频素材还可以提供变速、变声、变调等音频处理能力。再次创作既保留了原有的音乐风格，又能够加入用户对作品的理解和想法，极大地激发了用户创作潜能。
4、实时录音、录像：提供完整的音视频创作工具，打通从音乐制作、原声输入到音频录制和保存音乐短片(MV)的完整链路，同时还加入了视频录制以及图像处理，如滤镜、特效渲染等，提供视频音频创作一站式解决方案。
其中，本公开的音乐创作方法由电子设备来执行。电子设备可以是平板电脑、手机(如折叠屏手机、大屏手机等)、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)、等等设备，本公开对电子设备的具体类型不作任何限制。
基于前述描述，本公开以实施例将以电子设备为例，结合附图和应用场景，对本公开提供的音乐创作方法进行详细阐述。
请参阅图1，图1为本公开实施例提供的音乐创作方法的流程示意图。如图1所示，本公开提供的音乐创作方法可以包括：
S101、展示多个第一音频轨道，其中，各第一音频轨道按照时间线划分为多个候选轨道片段，每个候选轨道片段与一个音频片段对应。
电子设备中可以安装音乐创作工具，音乐创作工具能够提供多种类型的音频轨道，电子设备响应用户的操作可以在音频轨道上添加用于混音合成的音频片段。其中，每种类型可以对应一个或多个音频轨道，且不同类型的音频轨道数量可以相同也可以不同，且在创作过程中一些类型对应的音频轨道数量支持用户调整。
其中，音乐创作工具提供的多种类型的音频轨道可以包括但不限于：乐器 对应的第一音频轨道、音乐再创作(remix能力)对应的第二音频轨道等等。其中，每种类型可以包括一个或多个音频轨道，本公开对于音乐创作工具提供的音频轨道的类型以及各类型包括的音频轨道数量不作限定。
其中，音乐创作工具可以提供乐器对应的第一音频轨道，第一音频轨道可以按照时间线被划分为多个候选轨道片段，且多个第一音频轨道上的候选轨道片段在时间线上覆盖的位置区间可以相同，或者，也可以理解为多个第一音频轨道的候选轨道片段的长度一致。可以按照设定的音乐节奏对多个第一音频轨道进行划分，每个候选轨道片段及即对应一个节拍，其中，设定的音乐节奏越慢，候选轨道片段在时间线上覆盖的区间较长，设定的音乐节奏越快，候选轨道片段在时间线上覆盖的区间较短。其中，音乐节奏支持用户调整。
其中，属于同一第一音频轨道的多个候选轨道片段对应的音频片段相同，即对应同一乐器的音频片段；属于不同第一音频轨道上的候选轨道片段对应的音频片段不同，即对应不同乐器的音频片段。需要说明的是，候选轨道片段在时间线上对应的时间范围(以下称为候选轨道片段对应的时长)、候选轨道片段对应的音频片段的时长，两者可以一致，也可以不一致。
各第一音频轨道上的候选轨道片段分别对应的音频片段可以是预先采用相应的乐器演奏进行录制并处理再存储至电子设备的存储空间中。基于用户对候选轨道片段的选中操作可以从电子设备的存储空间中读取相应的音频片段并添加至当前操作候选轨道片段所在的第一音频轨道上相应候选轨道片段在时间线上对应的位置。
音乐创作工具可以通过电子设备展示多个第一音频轨道以及各第一音频轨道包括的多个候选轨道片段。其中，本公开对于展示样式不做限定。例如，在电子设备展示的用户界面中，每个第一音频轨道可以对应一展示区域，在第一音频轨道对应的展示区域内每个候选轨道片段分别对应一展示区域，该第一音频轨道包括的多个候选轨道片段分别对应的展示区域可以按照多个候选轨道片段在时间线上的位置的先后顺序依次排列，如由左向右依次排列、由上至下依次排列等等，属于同一第一音频轨道的多个候选轨道片段分别对应的展示区域相互之间不存在重叠，方便用户能够清楚区分多个候选轨道片段以及方便用户执行选中操作。
在一些实施例中，多个第一音频轨道可以是用户手动一一触发生成并设置的，用户通过设置第一音频轨道对应的乐器设置各第一音频轨道上的各候选轨道片段对应的音频片段。
另一些实施例中，音乐创作工具可以根据用户所选的音乐风格自动匹配乐器组合，生成与乐器组合包括的多个乐器分别对应的多个第一音频轨道，并且各第一音频轨道均按时间线划分为多个候选轨道；并基于各第一音频轨道对应的乐器分别确定每个第一音频轨道上的多个候选轨道片段对应的音频片段。
其中，可以预先在音乐创作工具中设置多种音乐风格，每种音乐风格对应一乐器组合，进行音乐创作时，得到用户输入的音乐风格信息，可以查询预先设置的音乐风格与乐器组合之间的对应关系，确定用户输入的音乐风格信息所指定的音乐风格对应的乐器组合，并针对乐器组合中的各乐器分别建立相对应的第一音频轨道，因此，本实施例中，第一音频轨道的数量可以为一个或者多个，其数量与音乐风格(音乐风格对应的乐器组合所包括的乐器数量)相关。
需要说明的是，每种乐器可以对应多个音频片段，不同音频片段的时长、音调、音量大小等等可以不同，可基于不同的策略响应用户的选中操作自动从乐器对应的多个音频片段中选择适配的音频片段添加至相应的候选轨道片段在时间线上所在的位置。此处提及的策略可以但不限于基于候选轨道片段的时长选择，尽可能选择时长与候选轨道片段的时长接近的等等。
例如，用户想要创作中国风的音乐，向音乐创作工具输入音乐风格信息指示要创作的音乐风格为中国风，则音乐创作工具可以为用户匹配二胡、古筝以及琵琶三种中国传统乐器，且建立二胡、古筝以及琵琶分别对应的第一音频轨道，用户在二胡对应的第一音频轨道上可以添加二胡对应的节奏音频片段，在古筝对应的第一音频轨道上可以添加古筝对应的节奏音频片段，在琵琶对应的第一音频轨道上可以添加琵琶对应的节奏音频片段。
通过音乐风格可以一键解决用户乐器匹配问题，降低对于用户乐器以及音乐风格之间的理解能力的要求。
需要说明的是，第一音频轨道可以理解为是支持预编辑的音频轨道，在时 间线上，可以在所需要时间触发播放，每个节奏点(即目标轨道片段)上的音频片段都可以增加，删除，修改。且第一音频轨道可以任意增加、删除。
当然，还可以通过其他方式确定多个第一音频轨道，并不限于上述示例的实现方式。
S102、响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置。
针对候选轨道片段的选中操作可以但不限于是点击、双击、长按、滑动等类型的操作。且针对不同第一音频轨道上的候选轨道片段的选中操作可以是相同或者不同类型的操作，本公开对此不做限定。
其中，被选中的候选轨道片段为目标轨道片段。被选中的候选轨道片段对应的音频片段为添加在第一音频轨道上的音频片段，能够参与混音合成。未被选中的候选轨道片段对应的音频片段可以理解为不是添加在第一音频轨道上的音频片段，无法参与混音合成。
响应针对候选轨道片段的选中操作，被选中的候选轨道片段和未被选中的候选轨道片段可以采用不同的显示样式，且不同第一音频轨道上被选中的候选轨道片段可以采用不同的显示样式，以便用户区分，例如，采用不同颜色填充用户界面中候选轨道片段对应的展示区域。
结合步骤S101所述，目标轨道片段对应的时长与相应音频片段的时长两者可以一致，也可以不一致。若目标轨道片段对应的时长与相应音频片段的时长两者一致，则音频片段将添加在目标轨道片段在时间线上的位置区间内，且音频片段的起始时刻与目标轨道片段在时间线上的起始时刻一致，例如，用户选中某个第一音频轨道上的第一个候选轨道片段，则音频片段1添加在第一个轨道片段在时间线上的位置区间内，即音频片段1占据一个轨道片段。若目标轨道片段对应的时长与相应音频片段的时长两者不一致，则音频片段将添加在被选中的候选轨道片段以及相邻的一个或多个候选轨道片段在时间线上的位置区间内，且音频片段的起始时刻与被选中的候选轨道片段在时间线上的起始时刻一致，例如，用户选中某个第一音频轨道上的第一个轨道片段， 则音频片段2的时长为选轨道片段对应的时长的1.5倍，则将音频片段2添加在第一个轨道片段和第二个轨道片段在时间线上的位置区间内，即音频片段占据2个轨道片段，也可以理解为目标轨道片段包括多个候选轨道片段。
S103、响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成并播放。
音乐创作工具可以获取用户输入的混音指令，并响应混音指令按照时间线将多个第一音频轨道上添加的音频片段进行混音合成并播放。其中，混音指令可以但不限于是用户通过操作音乐创作工具提供的交互界面上的一个或多个按钮触发的。
一些实施例中，音乐创作工具可以提供启动混音控件和结束混音控件，当用户操作启动混音控件，音乐创作工具可以自动开始对各第一音频轨道上的音频片段进行混音合成，直至用户操作结束混音控件时停止混音合成。
另一些实施例中，音乐创作工具可以提供启动混音控件、结束混音控件以及控制多个第一音频轨道同步播放以及暂停的播放按钮，当用户依次操作启动混音控件和播放按钮，音乐创作工具开始对各第一音频轨道上的音频片段进行混音合成，直至用户操作结束混音控件时停止混音合成。需要说明的是，由于用户操作启动混音控件和播放按钮之间会存在时间先后顺序，这一时间段无混音输入数据，在导出的音频文件中，该时间段对应的音频片段可以理解为静音片段。
其中，各第一音频轨道的起始位置在时间线上是对齐的，响应混音指令，可以从时间线的起始时刻位置开始对各第一音频轨道上的音频片段进行混音合成，并播放合成得到的音频数据。在合成时，针对合成的时刻位置，将各第一音频轨道上位置区间覆盖该时刻位置的音频片段在相应时刻位置的音频数据进行混合。
应理解，还可以通过其他方式触发混音，并不限于上述示例示出的实现方式。
在进行混音合成时，可以基于各音频片段在时间线上的关系进行混音合成得到混音数据再将混音数据输入至声卡转换并播放；也可以将各第一音频轨道上的音频片段以不同通道输入至声卡进行播放，并录制声卡输出的声音 从而得到混音数据。
本实施例提供的方法，通过将第一音频轨道按照时间线划分为多个候选轨道片段，每个候选轨道片段对应一个节拍，此外，通过预先建立第一音频轨道上的多个候选轨道片段与音频片段之间的对应关系，用户通过针对其中一个或多个候选轨道片段执行简单的选中操作便可在相应节拍中添加音频片段，方便用户理解和创作音乐；之后，响应针对所述多个第一音频轨道的播放指令，按照时间线将多个第一音频轨道上添加的音频片段进行混音合成播放，供用户预览创作的合成音频。且通过提供利用抽象音乐数据模型和数字化创作链路来降低用户创作音乐和编辑音乐门槛的音乐创作工具，只需要移动端设备就可以完整创作一首音乐，打破了现有的音乐创作对于硬件设备限制。
在图1所示实施例的基础上，音乐创作工具还可以响应导出指令，将按照时间线对多个第一音频轨道上的音频片段进行混音合成得到的混音数据导出并存储为指格式的音频文件。
其中，音频文件的时长可以根据第一音频轨道的长度决定，或者，也可以为预设时长，或者，还可以根据用户控制启动混音的时刻至结束混音的时刻而定。
请参阅图2，图2为本公开另一实施例提供的音乐创作方法的流程图。如图2所示，本实施例的方法可以包括：
S201、展示多个第一音频轨道，其中，各第一音频轨道按照时间线划分为多个候选轨道片段，每个候选轨道片段与一个音频片段对应。
S202、响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置。
步骤S201与步骤S202可参照图1所示实施例中步骤S101、S102的详细描述，简明起见，此处不再赘述。
S203、获取用户导入的音频素材，将所述音频素材添加在第二音频轨道上。
音频素材可以是已有的、通过用户导入的歌曲，且可以经过剪裁、变速、 变调、变音等音频处理后作为混音组合的一部分，也可以是纯已有的人声，例如freestyle的说唱，清唱。第二音频轨道便于用户对已有作品做二次创作。
在一些实施例中，音乐创作工具可以通过电子设备向用户展示导入用于混音的音频素材的入口，通过该入口可以进入音频素材选择页面进行选择，其中，音频素材选择页面中可以缩略聚合展示可供用户选择的音频素材。且音乐创作工具可以通过电子设备向用户提供用户音频处理对应的控件或者功能面板，方便用户对选择的音频素材进行音频处理。当然，也可以不做音频处理，直接将用户导入原始的音频素材与多个第一音频轨道上添加的音频素材进行混音合成。
需要说明的是，第二音频轨道也可以理解为是支持预编辑的音频轨道，在时间线上，可以在所需要时间触发播放，且第二音频轨道上的音频素材可以随时进行删除、替换以及音频处理。
其中，对第一音频轨道的操作和对第二音频轨道的操作可以不分先后顺序，且可以反复执行。
S204、响应混音指令，按照时间线将多个第一音频轨道上添加的音频片段与第二音频轨道上添加的音频素材进行混音合成并播放。
音乐创作工具可以获取用户输入的混音指令，并响应混音指令对多个第一音频轨道上添加的音频片段和第二音频轨道上的音频素材进行混音合成并播放。其中，混音指令可以但不限于是用户通过操作音乐创作工具提供的交互界面上的一个或多个按钮触发的。
一些实施例中，音乐创作工具可以提供启动混音控件和结束混音控件，当用户操作启动混音控件，音乐创作工具可以自动开始对各第一音频轨道上的音频片段和第二音频轨道上的音频素材进行混音合成，直至用户操作结束混音控件时停止混音合成。采用该方式，可以理解为第一音频轨道和第二音频轨道在时间线上是对齐的，且第二音频轨道上的音频素材是从时间线的起始时刻位置开始。
另一些实施例中，音乐创作工具可以提供启动混音控件、结束混音控件、控制多个第一音频轨道同步播放以及暂停的播放按钮1、可控制第二音频轨道播放以及暂停的播放按钮2；当用户依次操作启动混音控件、播放按钮1和播 放按钮2，其中，操作播放按钮1和播放按钮2的先后顺序不限，音乐创作工具按照用户操作顺序对相应音频轨道上的音频片段进行混音合成，直至用户操作结束混音控件时停止混音合成。需要说明的是，由于用户操作启动混音控件、播放按钮1和播放按钮2之间会存在时间先后顺序，从用户操作启动混音控件至操作第一个播放按钮时这一时间段无混音输入数据，在导出的音频文件中，该时间段对应的音频可以理解为静音片段。在用户操作第一个播放按钮至操作第二个播放按钮这一时间段内只有第一个播放按钮对应的音频轨道参与混音，因此，在导出的音频文件中，该时间段对应的音频为第一个播放按钮对应的音频轨道上的音频片段混音而成。
在进行混音合成时，可以基于各第一音频轨道上的音频片段以及第二音频轨道上的音频素材在时间线上的关系进行混音合成得到混音数据再将混音数据输入至声卡转换并播放；也可以将各第一音频轨道上的音频片段以及第二音频轨道上的音频素材通过不同通道输入至声卡进行播放，并录制声卡输出的声音从而得到混音数据。
在图2所示实施例的基础上，音乐创作工具还可以响应导出指令，将按照时间线对多个第一音频轨道上的音频片段以及第二音频轨道上的音频素材进行混音合成得到的混音数据导出并存储为指格式的音频文件。
图3为本公开另一实施例提供的音乐创作方法的流程示意图。请参阅图3所示，本实施例的方法包括：
S301、展示多个第一音频轨道，其中，各第一音频轨道按照时间线划分为多个候选轨道片段，每个候选轨道片段与一个音频片段对应。
S302、响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置。
S303、响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成并播放。
S304、在播放所述多个第一音频轨道上的音频片段通过混音合成得到的混音数据的过程中，响应针对自定义音频片段的触发操作，获取自定义音频片 段，用于与所述触发操作对应的播放时刻之后播放的混音数据进行合成。
在播放多个第一音频轨道上的音频片段通过混音合成得到的混音数据的过程中，用户可以同步添加自定义音频片段，以增加自由音效。其中，自由音效(即自定义音频片段)的添加不受最小单位时间的限制，即不受第一音频轨道上包括的候选轨道片段的限制，可以在任意时间节点，通过用户实时触发。且用户可以在不同播放时刻添加不同的自定义音频片段。
其中，自定义音频片段可以包括但不限于一些私有厂牌和个人标识的人声，电子音，特效音等。音乐创作工具可以通过电子设备向用户展示不同自定义音频片段对应的图标，用户通过操作图标添加自定义音频片段，操作可以但不限于为单击、双击、长按等等。
结合图2所示实施例，若第二音频轨道上也添加了音频素材，则在播放多个第一音频轨道上的音频片段与第二音频轨道上的音频素材进行混音合成得到的混音数据的过程中，若用户触发添加自定义音频片段，则可以将自定义音频片段与触发操作对应的播放时刻之后的混音数据进行合成并播放合成得到的音频数据，此处的混音数据为多个第一音频轨道上的音频片段与第二音频轨道上的音频素材进行混音合成得到。
通过提供实时即兴演奏，如电子音，人声特效，在混音的时候也可以直接方便的加入，能够提高用户创作兴趣，也能够保证创作的音频满足用户预期。此外，音乐创作工具还可以响应导出指令，将按照时间线对多个第一音频轨道上的音频片段以及自定义音频片段进行混音合成得到的音频数据导出并存储为指格式的音频文件。
可选地，在图3所示实施例的基础上，还包括：
S305、获取录制音频，录制音频用于与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据进行混音合成。
一些实施例中，播放多个第一音频轨道上的音频片段混音合成得到的混音数据时，可以打开语音拾取模块(如麦克风)同步实时录制获取录制音频，以为合成的音乐增加人声效果，或者，也可以理解为多个第一音频轨道合成的混音数据是录制音频的背景音乐呈现。
此外，在混音的过程中，用户可以随时打开或者关闭音频录制。
通过步骤S305可以实现音乐创作以及原声输入之间的链路打通，满足用户创作需求。此外，音乐创作工具还可以响应导出指令，将按照时间线对多个第一音频轨道上的音频片段以及录制音频进行混音合成得到的音频数据导出并存储为指格式的音频文件。
可选地，在图3所示实施例的基础上，还包括：
S306、获取视频素材，视频素材用于与多个第一音频轨道上的音频片段进行混音合成得到的混音数据进行合成得到视频数据。
其中，视频素材可以是用户可以导入的电子设备中已有视频，也可以是在播放的过程中通过启动电子设备的摄像头实时录制的，也可以是两者结合得到的，本公开对此不作限定。若即包含用户导入的视频素材，也启动实时录制视频，在实时录制视频，用户从电子设备中导入的已有视频可以作为录制的画中画播放，也可以完全取代录制的视频，即画中画的画幅占整个视频画面。此外，若包含用户导入的已有的视频素材，则可以在启动混音，即用户输入混音指令之前导入。
其中，将视频素材与混音数据合成时，混音数据可以作为背景音与视频素材融合。通过还方式能够方便用户创作音乐短片(MV)，满足用户的创作需求。在图3所示实施例的基础上，合成之前，还可以对视频素材进行图像处理，图像处理的方式包括但不限于：滤镜、特效、画面增强、旋转等等。若视频素材是实时录制的，则可以在录制的过程中对摄像头采集的每一帧视频图像进行处理并同步与音频轨道合成；若视频素材是用户导入的，则可以在合成的过程中逐帧对视频帧图像进行图像处理并同步与音频轨道合成。
本实施例的方法，再通过音乐创作工具提供视频录制功能，完全打通从音乐创作、原声输入到音频录制的完整链路，同时还加入了视频录制、特效渲染以及保存MV的功能，提供视频音频创作一站式解决方案。
此外，音乐创作工具还可以响应导出指令，将按照时间线对多个第一音频轨道上的音频片段以及视频素材进行合成得到的视频数据导出并存储为指格式的视频文件。
在图3所示实施例的基础上，在播放多个第一音频轨道上的音频片段进行混音合成得到的混音数据的过程中，可以并行执行上述步骤S304至S306。
结合前述图1至图3所示实施例，通过使用该音乐创作工具提供的功能进行音乐创作，能够实现低门槛的音乐创作，丰富原创音乐资源，提高创作乐趣以及提高用户创作兴趣，激发用户创作潜能，丰富音乐创作形式。此外，该音乐创作工具可以部署在移动端设备，通过在移动端实现音乐创作器能力，使得音乐创作不受限制，用户可以随时随地表达灵感。此外，音乐创作工具具有的上述多种能力能够实现创作的专业性以及简化音乐创作流程，从而激发大众兴趣，使创作全面化成为可能。
基于前述描述，将结合图4A至图4E示出的交互界面示意图，对本公开提供的音乐创作方法进行详细阐述。为了便于说明，图4A至图4E中，以电子设备为手机，手机中安装音乐创作工具，通过应用1进行音乐创作为例进行举例说明。
请参阅图4A至图4E，图4A至图4E为本公开实施例提供的人机交互界面示意图。
启动音乐创作工具，选择了音乐风格并且在第一音频轨道上的轨道片段上添加了相应乐器的音频片段和第二音频轨道上添加了音频素材，音乐创作工具可以在手机上显示如图4A所示的用户界面11，其中，用户界面11包括：区域101、区域102、区域103以及区域104。
其中，区域101可以理解为是原子音频创作区，在区域101中用户可以选择的音乐风格并基于音乐风格自动确定并显示乐器组合以及对应的第一音频轨道、且可新增或者删除第一音频轨道、改变时间线长度增加或者减少节拍、调整节奏快慢等等。
示例性地，参阅图4A所示，区域101中包括：标签101a和区域101b，标签101a用于触发显示音乐风格列表，区域101b用于显示与当前选择的音乐风格对应的第一音频轨道以及与第一音频轨道相关的组件或者信息。
示例性地，音乐创作工具通过响应针对标签101a的触发操作可以显示如图4B所示的用户界面12，用户界面12中包含音乐风格列表，音乐风格列表中包括多种音乐风格选项供用户选择，用户可以通过上下滑动或者其他方式查看更多音乐风格选项以切换音乐风格，并在区域101b中显示与所选择的音乐风格选项对应的乐器组合的一个或多个第一音频轨道。一些情况下，音乐创 作工具启动时，可以默认显示指定音乐风格对应的乐器组合所对应的多个第一音频轨道，且各第一音频轨道上的候选轨道片段均为未选中状态，且第一音频轨道的时间线长度也可以是按照默认长度显示，如默认显示10个时间单元。此外，在进入音乐风格列表时，可以显示当前选中的音乐风格为选中状态，其他为未选中状态。
在图4B所示实施例基础上，假设音乐创作工具相应用户针对音乐风格列表中摇滚选项的触发操作(如点击)，则音乐创作工具可以在手机上示例性地显示如图4C所示的用户界面13，区域101b中显示摇滚风格对应的乐器，分别为贝斯、吉他、架子鼓、键盘分别对应的第一音频轨道。之后，可以通过点击音乐风格列表之外的其他任意位置退出音乐风格列表。
参照图4B所示，音乐风格列表中还可以包括自定义风格选项，如图4B所示的“自定义1”选项。一些实施例中，自定义风格选项可以包括用户之前已定义好的乐器组合，当用户选择自定义风格选项，可以显示相应乐器组合对应的多个第一音频轨道；另一些实施例中，区域101b中可以不显示任何第一音频轨道，而是由用户通过触发添加自定义风格生成自定义风格选项，并采用新增音频轨道的方式向生成的自定义风格选项中添加第一音频轨道并设置关联的乐器类型，并保存自定义风格选项的乐器组合信息至音乐风格列表，方便用户再次使用。不同自定义风格选项可以通过音乐风格名称区域，音乐风格名称可以由用户编辑。
请继续参阅图4A所示，区域101b中可以包括每个第一音频轨道对应的显示区域，第一音频轨道对应的显示区域中可以包括设置音频片段音量的标签s1、修改乐器类型的标签s2、轨道s3以及删除标签s4。
其中，轨道s3按照时间划分为多个轨道片段，如图4A中所示的按照从左到右排列显示多个方形区域，每个方形区域表示一轨道片段，用户可以通过操作方形区域选中轨道片段，并向第一音频轨道上该轨道在时间线上对应的位置添加相应的音频片段，且被选中的轨道片段的显示样式可以与其他未被选中的轨道片段的显示样式不同，例如，图4A所示，被选中的轨道片段对应的方形区域为灰色，未被选中的轨道片段对应的方形区域为白色。此外，不同第一音频轨道上被选中的轨道片段的显示样式可以不同，例如，颜色不同；不 同第一音频轨道上未被选中的轨道片段可以采用相同的显示样式，例如，均为白色。
示例性地，如图4A中所示的用户界面11中各第一音频轨道上灰色区域为被选中的轨道片段。由于不同乐器对应的音频片段的时间长度可能不同，可能需要占用一个或多个轨道片段，当需要占用多个轨道片段时，响应用户的选中操作可以将多个轨道片段的方形区域合并，例如，最后一行钢琴对应的第一音频轨道上第1至第3个时间单元合并，第8至第10个时间单元合并，钢琴对应的音频片段在时间线上的对应三个轨道片段对应的时间范围。
此外，用户可以多次操作(如连续点击)同一轨道片段对应的方形区域，调整音频片段的音调，在用户界面中可以但不限于通过颜色亮度区分，颜色越亮，音调越高，颜色越暗，音调越低。
此外，区域101中还可以包括新增音频轨道的标签s5。通过操作标签s5可以增加第一音频轨道，在一些实施例中，新增第一音频轨道可以按照音频轨道的排列顺序添加在最后一行，之后，可以通过操作新增第一音频轨道对应的修改乐器类型的标签s2设置该轨道对应的乐器。
其中，针对任一第一音频轨道对应的标签s2，音乐创作工具可以响应用户针对标签s2的触发操作(如点击)，在手机上显示如图4D所示的用户界面14，用户界面14中显示乐器列表，用户可从乐器列表选择想要的乐器。选择之后，可以通过触发显示屏幕中列表区域之外的任意位置退出乐器列表。其中，乐器列表中各乐器可以按照设定的顺序依次显示，或者也可以按照乐器类别分类显示，且在乐器列表中显示每个类别的名称，本公开对此不作限定，图4D中示出的是前者情形。
此外，区域101中还包括区域101c，区域101c中用于显示原子创作区域对应的时间线，区域101c中当前的时间线包括的时间单元按照顺序排列，用户可以通过操作其中增加节拍的标签s6和减少节拍的标签s7增加时间单元或者删除时间单元从而改变时间线长度，若要增加或者删除多个时间单元，可以连续多次操作(如连续点击)标签s6和s7即可。
需要说明的是，用户针对区域101c中时间线的修改会使用户界面11中区域101b中各第一音频轨道的长度发生变化，区域101c中的第一音频轨道 上同步增加或者删除相应数量的候选轨道片段。
此外，区域101中还包括区域101d，区域101d用于展示速度调节轴，也可以理解为音乐节奏快慢调节轴或者音乐节拍快慢调节轴，用户可以通过拖动速度调节轴上的调节按钮调整音乐节奏，区域101d中可以显示当前速度值，速度值越大表示音乐节奏越快。例如，图4A中所示，当前速度为：120，向左拖动调节按钮可以降低速度，向右拖动调节按钮可以加大速度，在调节的过程中，区域101d中显示的速度值随调节而同步变化。
需要说明的是，当调整速度之后，时间线上每个时间单元的长度会发生变化，各第一音频轨道上的候选轨道片段在时间线上覆盖的区间也会发生变化；速度值越高，时间单元的时长越短，候选轨道片段在时间线上覆盖的区间越小；速度值越低，时间单元的时长越长，候选轨道片段在时间线上覆盖的区间越大。调整速度之后，用户界面的区域101b和区域101c中所显示的时间单元的标识的显示样式可以不变(如表示时间单元和候选轨道片段的方形区域的大小不变)，也可以发生变化(如表示时间单元和候选轨道片段的方形区域的大小随节奏变慢而变长或者随节奏变慢而变短)。
此外，由于调整了整体音乐节奏，被选中的候选轨道片段对应的音频片段的速度也需要进行调整，使得节奏音频片段的速度与调整后的音乐节奏一致，从而适配调整后的时间单元的时长。
此外，区域101中还包括：播放按钮101e，通过操作播放按钮101e可以控制电子设备播放区域101中的多个第一音频轨道上的音频片段供用户预览混音效果。在预览播放时，可以按照时间线播放，在用户界面中，可以理解为按照区域101b中按照从左到右的顺序按列并播放。且随着播放位置可以突出显示播放位置对应的一列轨道片段对应的方形区域，例如，这列轨道片段对应的方形区域的位置以及大小可以发生变化。
区域102可以理解为本地音频BGM创作区，通过操作区域102用户可以上传本地音频文件进行二次创作。通过操作区域102上传的音频文件添加在第二音频轨道上。通过操作区域102还可以对上传的音频文件进行变速、变声、裁剪、变调、音量设置等等。示例性地，如图4A所示，区域102中包括：标签x1、时间轴x2、音量设置按钮x3、播放按钮x4以及展示特效的区 域x5。其中，标签x1用于进入音频文件选择页，通过音频文件选择页可以选择要导入的用于二次创作的音频素材，并添加在第二音频轨道。时间轴x2可以展示用户添加的音频素材的总时长以及播放进度。通过音量设置按钮x3可以增加或者降低合成时第二音频轨道上的音频素材的音量。此外，区域102中还可以包括：标签x6，用于进入音频处理功能面板，音频处理功能面板可以提供裁剪、变速、变声、变调等一项或多项音频处理功能对应的按钮或者组件，通过触发相应按钮或者组件对第二音频轨道上的音频素材进行裁剪、变速、变声等处理。音频处理功能面板中还可以提供下载功能，以下载经过音频处理所得到的音频素材。在一些实施例中，或者，也可以将一些音频处理功能对应的标签设置在区域102中，例如，将裁剪、变速、变声、变调分别对应的按钮或者组件设置在区域102中(如标签x2、x3、x4以及x6所在区域的黑色框的下方)，方便用户使用，则可以不设置标签x6，且在区域102中设置下载按钮，方便用户下载经过也音频处理得到的音频素材。
通过区域101和区域102可以实现对第一音频轨道以及第二音频轨道进行预编辑。
区域103为自由音效创作区域，通过操作区域103提供的按键盘中的按键，在第三音频轨道上的任意时间点添加与按键相对应的自由音效。且按键盘中各按键对应的自由音效支持用户自定义，用户可根据需求将喜欢的音效的自定义音频片段与按键绑定，在创作时使用。示例性地，区域103中包括区域103a以及多个按键103b，区域103a用于展示区域103的主题内容，例如，区域103a中显示区域名称“自由音效创作区”和区域的详情介绍“按键盘对应按键，提供时间轨道上更为自由的创作能力”；此外，多个按键130b可以分别对应不同的厂牌音乐，例如，图4A所示，按照由左向右的顺序，多个按键103b依次对应FUHH、UFO、STRIKE、LONDON、MOON、WIPE、TIMER、FLASH、ORDER厂牌音乐，且依次对应标识A、S、D、F、G、H、J、K。本公开对于按键盘中的按键数量不作限定，用户可以根据需求自定义进行增加和删除，且按键盘的显示样式不做限定，除图4A所示的方式之外，还可以采用其他显示样式，例如按键形状可以为圆形、颜色可以为彩色，其中的标识也可以为其他字体和字号。
其中，可以通过触发区域101中的播放按钮以及区域102中的播放按钮播放第一音频轨道和第二音频轨道上的音频片段，在播放的过程中，通过操作区域103中的按键盘中的按键在时间线上的任意时间线添加自由音效。
区域104为音视频创作区域，用户通过操作区域104中的按钮可以实现实时录制环境音频和视频，也可以通过操作区域104中的按钮从电子设备中导入已存在的视频素材。且可以通过预览窗口预览录制的视频/导入的视频；此外，还可以通过图像处理相关按钮对视频素材进行处理。示例性地，如图4A所示，区域104包括区域y1、预览窗口y2、开始预览标签y3、结束预览标签y4、开始录制标签y5、结束录制标签y6、下载素材标签y7、特效标签y8、旋转组件y9以及移动组件y10。
其中，区域y1中用户显示区域104的主题以及详情介绍，例如显示文字内容“音视频创作区”和“提供视频轨道+麦克风轨道+混音轨道集合融合能力和视频特效编辑”，当然也可以显示其他内容，本公开对不作限定。
预览窗口y2可以显示实时录制的视频画面，也可以用于在结束录制之后预览播放视频素材与其他音频轨道合成得到的视频数据。本公开对于预览窗口y2的尺寸以及显示样式不作限定。
开始预览标签y3用于触发在预览窗口y2中播放视频素材与其他音频轨道合成得到的视频数据；类似地，结束预览标签y4用于触发在预览窗口y2中结束播放视频素材与其他音频轨道合成得到的视频。
开始录制标签y5用于触发录制音频和/或视频素材以及触发开始混音录制。在一些实施例中，还可以设置启用麦克风录制音频的选项和启用摄像头录制视频的选项，用户可以选择单独启用麦克风或者单独启动摄像头录制，或者，也可以同时选择，如此设置灵活性更高。或者，还可以分别设置禁用麦克风和摄像头的按钮，在未选择禁用时，默认同时启用麦克风和摄像头进行混音录制，需要禁用时基于需求进行设置。
结束录制标签y6用户触发停止录制音频和/或视频素材以及结束混音录制。
用户触发开始录制标签y5输入混音指令，触发开始混音录制并同步启动录制视频和音频，点击标签101e和标签x4播放第一音频轨道上的音频片段 以及用户导入的音频素材，在播放录制的混音数据的过程中，还可以加入自由音效。之后，用户触发开始录制标签y6停止混音录制以及停止录制音频和视频。并跳转至预览界面，预览最终的混音数据/视频数据。
下载素材标签y7用于将最终得到的混音数据/视频数据导出为指定格式的音频文件/视频文件。
特效标签y8用于进入特效列表。示例性地，音乐创作工具相应用户1针对特效标签y8的触发操作，可显示如图4E所示的用户界面15，用户界面15中显示特效列表，在特效列表中以特效名称展示各特效，用户可以选择使用特效，其中，在用户界面15中，用户可以但不限于通过上下滑动屏幕、滚动鼠标滚轮的方式查看更多特效。其中，可以在开始录制之前选定要使用的特效，也可以在录制的过程中选择使用特效，若在录制的过程中选择使用特效，触发选择使用特效的录制时刻之后录制的视频图像应用特效，之前录制的视频图像未应用特效。且用户在选择特效之后，音乐创作工具可以响应用户的操作可以为当前预览窗口中显示的视频帧图像应用特效，并将添加了特效的视频帧图像显示在预览窗口中，供用户预览效果。
旋转组件y9用于旋转视频素材的视频帧图像，旋转时可以顺时针旋转，也可以逆时针旋转，旋转方向不作限定。旋转角度范围为0-360度。在录制视频的过程中，可以实时触发旋转组件y9，以将视频画面旋转。
移动组件y10用于移动视频帧图像，移动组件可以包括用于沿X轴移动视频帧图像的组件和沿Y轴移动视频帧图像的组件。由于移动视频帧图像会使部分图像区域移出预览窗口，预览窗口中会存在一部分区域未被视频帧图像覆盖，该部分未被覆盖的窗口区域可以显示预先设定的背景颜色，如黑色、灰色等等。在录制视频的过程中，可以实时触发移动组件y10，以将视频画面水平或者竖直移动。
此外，区域104中还可以包括：用于控制预览播放以及暂停预览播放的播放控件、时间轴、音量按钮、全屏按钮以及进入视频相关的功能面板入口等等，功能面板中可以包括下载功能对应的控件、设置画中画的组件等等。
结合图4A至图4E所实施例，本公开通过提供利用抽象音乐数据模型和数字化创作链路来降低用户创作音乐和编辑音乐门槛的音乐创作工具，只需 要移动端设备就可以完整创作一首音乐。此外，该音乐创作工具在为用户提供了原子创作能力的基础上加入了，节奏、风格以及音色的选择，用户可根据自己的喜好进行创作，同时将硬件设备能力迁移至软件，脱离了昂贵繁重的硬件设备，完全模拟前者所带来的沉浸式音乐体验，用户可随时随地进行创作。此外，该音乐创作工具还提供了音乐再创作(remix)能力，在已有作品的基础上进行二次创作，满足用户的音乐创作需求。同时根据用户选择风格，自动初始化相应乐器轨道匹配风格，进一步方便上手，减少用户创作障碍，更大程度地为用户创作提供原始能力。且该音乐创作工具建立了完整的音乐创作链路，打通创作过程中从0到1的整个流程，包含音乐创作、语音输入、实时视频、特效渲染以及作品保存等各种节点，极大提高了用户创作兴趣，使音乐创作全面化成为可能。
图5为本公开一实施例提供的音乐创作装置的结构示意图。请参阅图5所示，本实施例提供的音乐创作装置500包括：
展示模块501，用于展示多个第一音频轨道，其中，各所述第一音频轨道按照时间线划分为多个候选轨道片段；每个所述候选轨道片段与一个音频片段相对应。
音轨处理模块502，用于响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置；其中，属于同一所述第一音频轨道的多个轨道片段添加的音频片段相同；不同所述第一音频轨道的轨道片段上添加的音频片段不同。
合成模块503，用于响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成.
播放模块504，用于播放混音合成生成的混音数据。
在一些实施例中，音轨处理模块502，还用于获取用户指定的音乐风格，并基于所述用户指定的音乐风格确定与所述音乐风格匹配的乐器组合；生成所述乐器组合包括的各乐器分别对应的所述第一音频轨道，并确定各所述分别对应的第一音频轨道上的多个候选轨道片段分别对应的音频片段；其中，所 述第一音频轨道上的多个轨道片段分别对应的音频片段为所述第一音频轨道对应的乐器的音频片段。
在一些实施例中，音轨处理模块502，还用于调整所述多个第一音频轨道包括的目标轨道片段在时间线上覆盖的位置范围，且基于调整后所述目标轨道片段在时间线上覆盖的位置范围调整相应音频片段的速度，使得所述音频片段的时长与调整后的目标轨道片段在时间线上覆盖的位置范围相匹配。
在一些实施例中，音轨处理模块502，还用于响应针对新增轨道控件的触发操作，生成并展示新增第一音频轨道，并确定与所述新增第一音频轨道的多个候选轨道片段分别对应的音频片段。
在一些实施例中，音轨处理模块502，还用于响应针对删除轨道控件的触发操作，删除与所述删除轨道控件对应的所述第一音频轨道。
在一些实施例中，还包括：导出模块505，用于响应导出指令，将所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据导出并存储为指定格式的音频文件。
在一些实施例中，音轨处理模块502，还用于将用户导入的音频素材添加在第二音频轨道上，用于与所述第一音频轨道上添加的音频片段进行混音；其中，所述音频素材在所述时间线上覆盖的位置区间的起始时刻位置与所述时间线的起始时刻位置对齐；
合成模块503，用于响应混音指令，按照所述时间线将所述第二音频轨道上的音频素材与所述多个第一音频轨道上的音频片段进行混音合成得到混音数据；播放模块504，用于播放相应的混音数据。
在一些实施例中，音轨处理模块502，还用于对所述第二音频轨道上的音频素材进行音频处理，音频处理包括：裁剪、变速、变调、变声中的一项或多项。
在一些实施例中，合成模块503，还用于在播放所述多个第一音频轨道上的音频片段通过混音合成得到的混音数据的过程中，响应针对自定义音频片段的触发操作，将所述自定义音频片段与所述触发操作对应的播放时刻之后播放的混音数据进行合成；播放模块504，用于播放合成得到的音频数据。
在一些实施例中，装置500还包括：音频录制模块506，用于获取录制音 频。合成模块503，还用于按照时间线将所述录制音频与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据再进行合成并播放合成得到的音频数据。
在一些实施例中，装置500还包括：视频处理模块507，用于获取视频素材。合成模块503，还用于按照时间线将所述视频素材与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据进行合成，并播放得到的视频数据。
在一些实施例中，视频处理模块507，还用于对所述视频素材进行图像处理得到具有目标图像效果的视频素材。
本实施例的装置可以用于执行前述任一方法实施例的技术方案，其实现原理以及技术效果类似，可参照前述方法实施例的详细描述，简明起见，此处不再赘述。
示例性地，本公开提供一种电子设备，包括：一个或多个处理器；存储器；以及一个或多个计算机程序；其中一个或多个计算机程序被存储在存储器中；一个或多个处理器在执行一个或多个计算机程序时，使得电子设备实现前文实施例的音乐创作方法。
示例性地，本公开提供一种芯片系统，芯片系统应用于包括存储器和传感器的电子设备；芯片系统包括：处理器；当处理器执行前文实施例的音乐创作方法。
示例性地，本公开提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器使得电子设备执行时实现前文实施例的音乐创作方法。
示例性地，本公开提供一种计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机执行前文实施例的音乐创作方法。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括 没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所述的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (16)

  1. 一种音乐创作方法，包括：
    展示多个第一音频轨道，其中，各所述第一音频轨道按照时间线划分为多个候选轨道片段；每个所述候选轨道片段与一个音频片段相对应；
    响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段，并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置；其中，属于同一所述第一音频轨道的多个目标轨道片段添加的音频片段相同；不同所述第一音频轨道的目标轨道片段上添加的音频片段不同；
    响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成并播放。
  2. 根据权利要求1所述的方法，其中，所述展示多个第一音频轨道之前，所述方法还包括：
    获取用户指定的音乐风格，并基于所述用户指定的音乐风格确定与所述音乐风格匹配的乐器组合；
    生成所述乐器组合包括的各乐器分别对应的所述第一音频轨道，并确定各所述分别对应的第一音频轨道上的多个候选轨道片段分别对应的音频片段；其中，所述第一音频轨道上的多个轨道片段分别对应的音频片段为所述第一音频轨道对应的乐器的音频片段。
  3. 根据权利要求1或2所述的方法，还包括：
    调整所述多个第一音频轨道包括的目标轨道片段在时间线上覆盖的位置范围，且基于调整后所述目标轨道片段在时间线上覆盖的位置范围调整相应音频片段的速度，使得所述音频片段的时长与调整后的目标轨道片段在时间线上覆盖的位置范围相匹配。
  4. 根据权利要求1至3任一项所述的方法，还包括：
    响应针对新增轨道控件的触发操作，生成并展示新增第一音频轨道，并确定与所述新增第一音频轨道的多个候选轨道片段分别对应的音频片段。
  5. 根据权利要求1至4任一项所述的方法，还包括：
    响应针对删除轨道控件的触发操作，删除与所述删除轨道控件对应的所述第一音频轨道。
  6. 根据权利要求1至5任一项所述的方法，还包括：
    响应导出指令，将所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据导出并存储为指定格式的音频文件。
  7. 根据权利要求1至6任一项所述的方法，还包括：
    将用户导入的音频素材添加在第二音频轨道上，用于与所述第一音频轨道上添加的音频片段进行混音；其中，所述音频素材在所述时间线上覆盖的位置区间的起始时刻位置与所述时间线的起始时刻位置对齐；
    响应所述混音指令，按照所述时间线将所述第二音频轨道上的音频素材与所述多个第一音频轨道上的音频片段进行混音合成并播放。
  8. 根据权利要求7所述的方法，其中，所述将用户导入的音频素材添加在第二音频轨道上之后，所述方法还包括：
    对所述第二音频轨道上的音频素材进行音频处理，音频处理包括：裁剪、变速、变调、变声中的一项或多项。
  9. 根据权利要求1至8任一项所述的方法，还包括：
    在播放所述多个第一音频轨道上的音频片段通过混音合成得到的混音数据的过程中，响应针对自定义音频片段的触发操作，将所述自定义音频片段与所述触发操作对应的播放时刻之后播放的混音数据进行合成并播放合成得到的音频数据。
  10. 根据权利要求1至9任一项所述的方法，还包括：
    获取录制音频，并按照时间线将所述录制音频与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据再进行合成并播放合成得到的音频数据。
  11. 根据权利要求1至10任一项所述的方法，还包括：
    获取视频素材，并按照时间线将所述视频素材与所述多个第一音频轨道上的音频片段进行混音合成得到的混音数据进行合成，并播放得到的视频数据。
  12. 根据权利要求11所述的方法，还包括：
    对所述视频素材进行图像处理得到具有目标图像效果的视频素材。
  13. 一种音乐创作装置，包括：
    展示模块，用于展示多个第一音频轨道，其中，各所述第一音频轨道按照时间线划分为多个候选轨道片段；每个所述候选轨道片段与一个音频片段相对应；
    音轨处理模块，用于响应针对在所述多个第一音频轨道的候选轨道片段中一个或多个候选轨道片段的选中操作，将被选中的所述一个或多个候选轨道片段确定为目标轨道片段并确定所述目标轨道片段对应的音频片段被添加在所述目标轨道片段所在的第一音频轨道上与所述目标轨道片段对应的时间线位置；其中，属于同一所述第一音频轨道的多个轨道片段添加的音频片段相同；不同所述第一音频轨道的轨道片段上添加的音频片段不同；
    合成模块，用于响应混音指令，按照时间线对所述多个第一音频轨道上添加的音频片段进行混音合成；
    播放模块，用于播放混音合成生成的混音数据。
  14. 一种电子设备，包括：存储器和处理器；
    所述存储器被配置为存储计算机程序指令；
    所述处理器被配置为执行所述计算机程序指令，使得所述电子设备实现如权利要求1至12任一项所述的音乐创作方法。
  15. 一种可读存储介质，包括：计算机程序指令；
    电子设备执行所述计算机程序指令，使得所述电子设备实现如权利要求1至12任一项所述的音乐创作方法。
  16. 一种计算机程序产品，电子设备运行所述计算机程序产品，使得所述电子设备实现如权利要求1至12任一项所述的音乐创作方法。
PCT/CN2023/126882 2022-10-31 2023-10-26 音乐创作方法、装置、电子设备及可读存储介质 Ceased WO2024093798A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/876,601 US20250372068A1 (en) 2022-10-31 2023-10-26 Music composition method and apparatus, and electronic device and readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211348849.9 2022-10-31
CN202211348849.9A CN117995146A (zh) 2022-10-31 2022-10-31 音乐创作方法、装置、电子设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2024093798A1 true WO2024093798A1 (zh) 2024-05-10

Family

ID=90893099

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/126882 Ceased WO2024093798A1 (zh) 2022-10-31 2023-10-26 音乐创作方法、装置、电子设备及可读存储介质

Country Status (3)

Country Link
US (1) US20250372068A1 (zh)
CN (1) CN117995146A (zh)
WO (1) WO2024093798A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121034258A (zh) * 2024-08-30 2025-11-28 深圳引望智能技术有限公司 一种音乐生成方法及相关装置

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102576524A (zh) * 2009-06-01 2012-07-11 音乐策划公司 接收、分析并编辑音频来创建音乐作品的系统和方法
US8244103B1 (en) * 2011-03-29 2012-08-14 Capshore, Llc User interface for method for creating a custom track
CN108369799A (zh) * 2015-09-29 2018-08-03 安泊音乐有限公司 采用基于语言学和/或基于图形图标的音乐体验描述符的自动音乐合成和生成的机器、系统和过程
US20180267772A1 (en) * 2017-03-20 2018-09-20 Chung Shan Lee Electronic device and processing method for instantly editing multiple tracks
CN112927665A (zh) * 2021-01-22 2021-06-08 咪咕音乐有限公司 创作方法、电子设备和计算机可读存储介质
CN113178182A (zh) * 2021-04-25 2021-07-27 北京灵动音科技有限公司 信息处理方法、装置、电子设备及存储介质
CN113838444A (zh) * 2021-10-13 2021-12-24 广州酷狗计算机科技有限公司 生成编曲的方法、装置、设备、介质及计算机程序
CN114067827A (zh) * 2021-12-20 2022-02-18 Oppo广东移动通信有限公司 一种音频处理方法、装置及存储介质
CN114299899A (zh) * 2021-12-03 2022-04-08 特赞（上海）信息科技有限公司 目标音乐的生成方法、装置、终端及存储介质

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102576524A (zh) * 2009-06-01 2012-07-11 音乐策划公司 接收、分析并编辑音频来创建音乐作品的系统和方法
US8244103B1 (en) * 2011-03-29 2012-08-14 Capshore, Llc User interface for method for creating a custom track
CN108369799A (zh) * 2015-09-29 2018-08-03 安泊音乐有限公司 采用基于语言学和/或基于图形图标的音乐体验描述符的自动音乐合成和生成的机器、系统和过程
US20180267772A1 (en) * 2017-03-20 2018-09-20 Chung Shan Lee Electronic device and processing method for instantly editing multiple tracks
CN112927665A (zh) * 2021-01-22 2021-06-08 咪咕音乐有限公司 创作方法、电子设备和计算机可读存储介质
CN113178182A (zh) * 2021-04-25 2021-07-27 北京灵动音科技有限公司 信息处理方法、装置、电子设备及存储介质
CN113838444A (zh) * 2021-10-13 2021-12-24 广州酷狗计算机科技有限公司 生成编曲的方法、装置、设备、介质及计算机程序
CN114299899A (zh) * 2021-12-03 2022-04-08 特赞（上海）信息科技有限公司 目标音乐的生成方法、装置、终端及存储介质
CN114067827A (zh) * 2021-12-20 2022-02-18 Oppo广东移动通信有限公司 一种音频处理方法、装置及存储介质

Also Published As

Publication number Publication date
US20250372068A1 (en) 2025-12-04
CN117995146A (zh) 2024-05-07

Similar Documents

Publication Publication Date Title
CN113365134B (zh) 音频分享方法、装置、设备及介质
US7999167B2 (en) Music composition reproduction device and composite device including the same
US20050231513A1 (en) Stop motion capture tool using image cutouts
JPH11341350A (ja) マルチメディア情報編集および再生装置、マルチメディア情報再生プログラムを記録した記録媒体ならびにシーケンス情報を記録した記録媒体
US10062367B1 (en) Vocal effects control system
US20120014673A1 (en) Video and audio content system
CN117998163B (zh) 视频编辑方法、装置、电子设备以及存储介质
CN115022674A (zh) 生成虚拟人物播报视频的方法、系统及可读存储介质
WO2024093798A1 (zh) 音乐创作方法、装置、电子设备及可读存储介质
JP5110706B2 (ja) 絵本画像再生装置、絵本画像再生方法、絵本画像再生プログラム及び記録媒体
KR20130123820A (ko) 실시간 영상합성 장치 및 그 방법
JP5044503B2 (ja) 演出画像再生装置、演出画像再生方法、演出画像再生プログラム及び記録媒体
CN113535289A (zh) 页面呈现、移动端交互及音频编辑的方法与装置
CN109327731B (zh) 一种基于卡拉ok的diy视频实时合成方法及系统
CN107610725A (zh) 一种视频制作方法及终端
WO2025067501A1 (zh) 视频编辑方法、装置、设备及介质
CN117198249A (zh) 音频处理方法、装置、计算机设备及存储介质
JP4136606B2 (ja) 音楽再生装置、プログラム及び記録媒体
JP4226563B2 (ja) 歌曲の演奏に同期して歌詞を表示して色変えする歌詞表示方法、歌詞表示プログラム、携帯情報端末
JP2005099264A (ja) 音楽演奏プログラム
CN115038020B (zh) 音频播放方法、装置、电子设备和存储介质
CN112988018A (zh) 多媒体文件输出方法、装置、设备及计算机可读存储介质
JPH09218690A (ja) 振付け教示機能付き楽音再生装置
JP2000083194A (ja) 映像デ―タ編集方法
JP2003141859A (ja) 画像音声再生システム、プログラム、及び記録媒体

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23884730

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18876601

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 18876601

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 23884730

Country of ref document: EP

Kind code of ref document: A1