WO2025256503A1 - 用于确定音频的卡点的方法、装置、设备和产品 - Google Patents

用于确定音频的卡点的方法、装置、设备和产品

Info

Publication number
WO2025256503A1
WO2025256503A1 PCT/CN2025/100055 CN2025100055W WO2025256503A1 WO 2025256503 A1 WO2025256503 A1 WO 2025256503A1 CN 2025100055 W CN2025100055 W CN 2025100055W WO 2025256503 A1 WO2025256503 A1 WO 2025256503A1
Authority
WO
WIPO (PCT)
Prior art keywords
candidate
audio
checkpoints
points
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/100055
Other languages
English (en)
French (fr)
Inventor
李仕炜
马忠平
韦金林
吴文鉴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Lemon Inc Cayman Island
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Lemon Inc Cayman Island
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd, Lemon Inc Cayman Island filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025256503A1 publication Critical patent/WO2025256503A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • This disclosure generally relates to the field of computers, and more specifically to methods, apparatus, electronic devices, and program products for determining audio timing.
  • Video editing is a non-linear editing process involving the remixing of images, background music, special effects, scenes, and other materials with the original video.
  • the timing and rhythm of the background music play a crucial role in video editing, enhancing the visual appeal of the final product.
  • Music-based video editing refers to selecting a specific musical rhythm during video production and matching the video visuals with that rhythm to enhance the visual impact and viewing experience. For example, in travelogues, video diaries, and film/TV show reviews, music-based video editing can quickly draw viewers into the story and enhance the viewing experience.
  • Embodiments of this disclosure provide a method, apparatus, electronic device, and program product for determining audio timing.
  • a method for determining audio beats includes acquiring audio for creating a video.
  • the method includes determining a plurality of candidate beats in the audio and a plurality of energies for the plurality of candidate beats.
  • the method includes determining a plurality of target beats in the audio based on the plurality of candidate beats and the plurality of energies, wherein the plurality of target beats are used to match frames in the video.
  • an apparatus for determining audio timings includes an audio acquisition module configured to acquire audio for creating a video.
  • the apparatus also includes a candidate timing determination module configured to determine a plurality of candidate timings of the audio and a plurality of energies for the plurality of candidate timings.
  • the apparatus includes a target timing determination module configured to determine a plurality of target timings of the audio based on the plurality of candidate timings and the plurality of energies, wherein the plurality of target timings are used to match video frames.
  • an electronic device in a third aspect of this disclosure, includes a processor and a memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to the first aspect.
  • a computer program product stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the method of the first aspect.
  • Figure 1 shows a schematic diagram of an example environment in which some embodiments of the present disclosure may be implemented
  • Figure 2 shows a flowchart of a method for determining audio timing according to some embodiments of this disclosure
  • Figure 3 shows a schematic diagram of audio timings in some embodiments of this disclosure
  • Figure 4 illustrates a schematic diagram of some embodiments of the present disclosure for determining a target audio point among multiple candidate points
  • Figure 5 shows a schematic diagram of determining audio timings using a sliding window in some embodiments of this disclosure
  • Figure 6A shows a schematic diagram of some embodiments of the present disclosure for importing a video to be edited
  • Figure 6B shows a schematic diagram of some embodiments of the present disclosure for importing audio
  • Figure 7 shows a block diagram of an apparatus for determining audio timings according to some embodiments of the present disclosure.
  • Figure 8 shows a block diagram of an electronic device according to some embodiments of the present disclosure.
  • a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information.
  • This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
  • sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format.
  • the pop-up window can also include a selection control allowing the user to choose "agree” or “disagree” to provide personal information to the electronic device.
  • the term “comprising” and similar terms should be understood as open-ended inclusion, i.e., “including but not limited to”.
  • the term “based on” should be understood as “at least partially based on”.
  • the term “one embodiment” or “the embodiment” should be understood as “at least one embodiment”.
  • the terms “first”, “second”, etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.
  • rhythm points of the music are usually used as the rhythm points.
  • this method cannot meet the needs of users who require videos to perfectly match the music. This is because, when there is no change in tempo within each measure of music, the beat point, as the basic unit of time in music, is evenly distributed along the time dimension. Therefore, simply using the beat points as the rhythm points does not meet the creator's need for perfect timing.
  • using only the beat points as the rhythm points in the video may filter out obvious rhythmic changes within a single beat, such as several consecutive bass notes within a beat.
  • key rhythm points (target beats) in the audio can be accurately determined. These target beats can be used for precise matching with video footage.
  • This method of determining audio rhythm based on audio energy analysis can identify multiple influential candidate beats in the audio, including but not limited to beat points. Then, based on the energy of these candidate beats, the influential and important target beats in the audio are determined. This not only ensures the effective preservation of key beats but also meets the user's need to apply video transitions based on beats, thus improving the user experience.
  • FIG. 1 illustrates a schematic diagram of an example environment 100 in which some embodiments of this disclosure may be implemented.
  • the display interface 102 is an interface for editing the timing of audio 110 in a video 104 to be edited.
  • the video 104 to be edited is displayed on the display interface 102.
  • the playback toolbar 106 the user can pause, forward, or rewind the video 104 to be edited.
  • the user can also use the full-screen play button to display the video 104 to be edited in full screen.
  • the playback toolbar 106 also displays the total duration of the video 104 to be edited and the current positioning timestamp.
  • audio 110 is the audio selected by the user to provide background music for the video 104 to be edited.
  • the user can be shown the beats of the audio 110. This allows the user to easily apply transitions when the music changes significantly during video editing, ensuring harmony between the audio and video.
  • the beats 112 of the current audio 110 can be displayed below the audio 110.
  • These beats 112 are not limited to the evenly distributed beats of each measure of audio 110; they can also include drum beats or the starting points of bass instruments such as guitar.
  • point 112-1 in point 112 can represent three consecutive drumbeats within a single beat.
  • a measure is a rhythmic unit of fixed duration, usually separated by bar lines. Each measure typically contains one or more beats.
  • the beat point is the basic unit of time in music, and beat points are evenly distributed throughout a piece of music unless there are specific tempo changes (such as crescendo, ritardando, etc.).
  • users can also manually edit the generated beats 112 using the "Add Beat 114" button. For example, users can add or delete multiple candidate beats, including at least the beat point. After the user's actions, they can save or cancel the audio and video through the toolbar 116.
  • This method of identifying a large number of diverse candidate beats, not limited to beat points, for the user's selected audio can meet the user's need for synchronized visual changes when the music changes significantly, thus improving the user experience.
  • Figure 2 shows a flowchart of a method 200 for determining audio timings according to some embodiments of the present disclosure.
  • Method 200 can be performed by a device for determining audio timings.
  • This device can be, for example, a server, a system configured on a server, or a standalone device or system.
  • the device can be implemented in software and/or hardware.
  • the method 200 will now be illustrated schematically using a device for determining audio timings as an example. Referring to Figure 2, method 200 may include blocks 202, 204, and 206.
  • audio for creating the video is obtained.
  • these videos can be short or long videos, and they can be obtained from the cloud or locally.
  • the audio used to create the video can be instrumental music or audio containing human voices, etc., and this audio can be obtained locally or from the cloud.
  • multiple candidate beats and multiple energies for the audio are determined. These candidate beats can be obtained using onset detection methods. These candidate beats can take various forms, such as beat points in the audio, drum beats of a particular instrument like a drum, or the starting point of a bass instrument like a guitar. For example, they could also be drum beats with several consecutive bass lines within a single beat. Each beat has its own energy, which in some embodiments represents the intensity of notes and/or changing musical events in the audio.
  • multiple target timing points are determined for the audio, where these target timing points are used to match video frames. For example, after determining multiple candidate timing points, considering that the number of determined candidate timing points may be too dense and thus affect the transition effect of the user's timing, target timing points suitable for the user to perform video frame-to-frame transitions can be determined from the multiple candidate timing points based on their energies. In this way, the user can achieve synchronized changes between the video frame and the audio rhythm points according to the determined target timing points.
  • multiple candidate beats and their energy are detected in the audio, and a target beat in the audio is determined based on these candidate beats and their energy. This allows for matching of the target beat with the video frame.
  • This method of determining the rhythm of audio based on its energy can identify more influential candidate beats in the audio, including but not limited to beat points. Furthermore, by effectively retaining the target beats that are influential and important in the audio based on the energy of these candidate beats, it not only ensures the preservation of key beats but also meets the user's need to apply video transitions based on beats, thus improving the user experience.
  • Figure 3 shows a schematic diagram of the audio timing points 300 in some embodiments of this disclosure. To clearly illustrate the location of these timing points in some embodiments of this disclosure, the following explanation will be provided in conjunction with Figure 3.
  • the process of a sound from beginning to end can be broadly divided into three stages: onset, attack, and decay.
  • the onset point 310 marks the sudden increase in sound energy and the beginning of the sound event.
  • the amplitude and energy of the sound can rise significantly from zero or a very low level. This is the starting point where the sound can be identified and the beginning of subsequent stages.
  • the onset point 310 shown in Figure 3 is the timing point in the audio.
  • the transient phase 320 follows the initial stage of the sound.
  • Transient phase 320 describes the rapid rise of the sound from its initial state to its maximum volume; that is, the sound intensifies rapidly within a very short time.
  • the amplitude envelope of the sound increases sharply, and the volume rises rapidly to its peak. This phase is usually very short, but it determines the clarity and impact of the sound.
  • the explosive sound 330 typically overlaps with the transient changes at the beginning of a note; it can be described as a short-duration, high-amplitude noise-like sound component that appears at the beginning of a musical note or sound event.
  • the signal changes rapidly and unpredictably in the explosive sound 330 region. For example, in piano playing, explosive sound 330 corresponds to the initial stage when a key is struck and the strings begin to vibrate.
  • the process of the sound gradually decreasing in volume until it disappears after reaching its peak is the decay phase 340.
  • the amplitude of the sound gradually decreases, and the energy gradually dissipates.
  • decay may be rapid, while in others it may be relatively slow.
  • Figure 4 illustrates a schematic diagram of some embodiments of this disclosure for determining a target audio beat 400 among multiple candidate beats.
  • candidate audio beats are detected.
  • an onset detection method can be used to determine the candidate audio beats.
  • this onset detection method can be based on audio energy, or on the spectrum, or a combination of spectrum and energy.
  • the detected candidate beats can be beat points of the audio or the starting point of a sound or musical event. Onset detection not only yields a larger number and more diverse range of candidate beats, but also provides the timestamps and energy values of the candidate beats.
  • the candidate checkpoints are sorted in descending order of energy value. Assume the following candidate checkpoints are detected (Table 1). (Time unit: seconds)
  • This array records the number of candidate checkpoints placed in the time window, using the step offset of the time window relative to the starting time as the index. For example, assuming the time series data starts from time T0, using a 10-second time window, the offset of the first 10-second window is 0 (because it starts from the starting point), the offset of the second 10-second window is 1 (because it starts from T0+1), and so on. This simplifies the management and access of the time window.
  • FIG. 5 will be used to illustrate some schematic diagrams of determining audio timing points 500 using a sliding window, as per embodiments of this disclosure. Assume that the upper limit for candidate timing points within each time window is 20, the time window size is 5 steps, and the step size for each movement is 1.
  • region 510 shows the timestamps of candidate checkpoints. Assuming a candidate checkpoint is between [n, n+1], this candidate checkpoint between [n, n+1] can influence time windows [n-4, n+1], [n-3, n+2], [n-2, n+3], [n-1, n+4], and [n, n+5] in region 520. It is understood that negative time points are not considered here, so the starting time is at least 0.
  • candidate checkpoints may not occur exactly at the beginning or end of a time window, one candidate checkpoint may affect multiple adjacent time windows. To ensure that the number of candidate checkpoints is less than 20 within any 5-step time window, the count values of all affected time windows need to be checked and updated. At 444, it is determined whether the number of existing candidate checkpoints in the time window is greater than or equal to the upper limit of 20. If not, at 448, the candidate checkpoint is added to the result list, and the count of candidate checkpoints already included in each window is updated.
  • the counter values recorded in the array in region 530 are all less than the upper limit of 20, indicating that the candidate checkpoint between [n, n+1] can be added to the result list, and the array value is updated by incrementing the count value of the corresponding window in the array by 1.
  • step 444 it is determined whether the number of existing candidate points in the time window is greater than or equal to the upper limit of 20. If so, at step 446, this candidate point is skipped and the next candidate point is re-evaluated. Referring to Figure 5, for example, if the value in the time window [n+1, n+6] is equal to 20, then the candidate point corresponding to the time window [n+1, n+6] will be skipped, and the candidate points following this candidate point will be evaluated. This process is repeated for each candidate point on the timestamp, adding the qualified candidate points to the result list. Then, at step 450, the candidate points in the result list are sorted in ascending order according to their timestamps. Finally, at step 460, the candidate points in the ascending-sorted result list are used as the target audio points.
  • each element here represents a counter for candidate checkpoints in a time window, and the subscript represents the start time of the time window (in seconds).
  • a sliding window will be used to traverse the sorted candidate checkpoints.
  • the affected time windows are [0-10], [1-11], and [2-12].
  • the counters for these time windows are all 0, indicating that the upper limit has not been reached. Therefore, the 2.5-second candidate checkpoints are added to the result list, and the count array is updated to [1,1,1,0,0,0,0,0].
  • the affected time windows are [0-10], [1-11], [2-12], and [3-13].
  • the counters for these time windows are checked; the first three time windows have a count of 1, indicating they haven't reached their upper limit.
  • the counter for time window [3-13] is 0. Therefore, the candidate checkpoint at 3.3 seconds is added to the results list, and the count array is updated to [2,2,2,1,0,0,0,0,0].
  • the affected time windows are [0-10], [1-11], [2-12], and [3-13]. If the counters for these time windows have not reached their upper limit, the 3.5-second candidate checkpoints are added to the results list, and the count array is updated to [3,3,3,2,0,0,0,0,0].
  • the time windows affected are [0-10], [1-11], ..., [8-18]. If the counters for these time windows have not reached their upper limit, the candidate checkpoints at 8.0 seconds are added to the results list, and the count array is updated to [4,4,4,3,1,1,1,1,1].
  • the time windows affected are [0-10], [1-11], ..., [8-18]. If the counters for these time windows have not reached their upper limit, the candidate checkpoints with an 8.5-second time limit are added to the results list, and the count array is updated to [5,5,5,4,2,2,2,2,2].
  • the candidate cut-off points shown in Table 4 are the target cut-off points for the audio.
  • candidate checkpoints For example, suppose there is a series of candidate checkpoints sorted in descending order of energy, starting from 0 seconds, with one candidate checkpoint appearing every 0.1 seconds, all with the same energy value.
  • the count of the time window [0-10] increases to 10.
  • the count of the time window [0-10] increases to 20.
  • the time window has reached its set upper limit, and the 1.9-second candidate checkpoint will be skipped, and the next candidate checkpoint at 2.0 seconds will be considered.
  • Subsequent candidate checkpoints (such as 2.0 seconds, 2.1 seconds, etc.) will also affect the [0-10] time window, so these candidate checkpoints will not be added to the result list. It is understandable that in practical applications, candidate checkpoints may not appear regularly, but the above helps to understand the process of filtering candidate checkpoints based on a sliding window.
  • Figure 6A shows a schematic diagram of 600A for importing videos to be edited, according to some embodiments of the present disclosure.
  • a "Start Creating" button 604A for importing videos to be edited is prominently displayed on the display interface 602A of the editing function 610A. Clicking this button allows the user to proceed to the interface for importing videos or photos.
  • a template area 606A is displayed for user use, facilitating quick entry into the creation state.
  • the lower-middle area 608A of the display page 602A previously imported but unfinished video drafts are displayed, allowing the user to continue editing at any time.
  • Figure 6B illustrates a schematic diagram of some embodiments of the present disclosure for importing audio 600B.
  • a user clicks the "Start Creating" button 604A and imports the video 604B to be edited, they can be redirected to a display interface 602B.
  • the user can add background music to the video 604B and adjust the video's visuals accordingly.
  • the user can click the "Add Audio" button 608B to access the audio addition page, where they can add the audio they need, which can be saved locally or retrieved from the cloud.
  • the user can turn off the original sound of the video using the "Turn Off Original Sound” button in area 606B, allowing the user to focus on the matching between the imported audio and the video 604B.
  • the user can click the "Export” button 610B to save and share their work. The user can also view and adjust the resolution of their work.
  • Figure 7 shows a block diagram of an apparatus 700 for determining audio timings according to some embodiments of the present disclosure.
  • the apparatus 700 includes an audio acquisition module 702 configured to acquire audio for video production.
  • the apparatus 700 also includes a candidate timing determination module 704 configured to determine a plurality of candidate timings of the audio and a plurality of energies of the plurality of candidate timings.
  • the apparatus 700 includes a target timing determination module 706 configured to determine a plurality of target timings of the audio based on the plurality of candidate timings and the plurality of energies, wherein the plurality of target timings are used to match video frames.
  • FIG 8 shows a block diagram of an electronic device 800 according to some embodiments of the present disclosure.
  • Device 800 may be the device or apparatus described in the embodiments of the present disclosure.
  • device 800 includes a central processing unit (CPU) and/or a graphics processing unit (GPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803.
  • ROM read-only memory
  • RAM random access memory
  • Various programs and data required for the operation of device 800 may also be stored in RAM 803.
  • CPU/GPU 801, ROM 802, and RAM 803 are interconnected via bus 804.
  • I/O interface 805 is also connected to bus 804.
  • device 800 may also include a coprocessor.
  • I/O interface 805 Multiple components in device 800 are connected to I/O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.
  • input unit 806 such as keyboard, mouse, etc.
  • output unit 807 such as various types of monitors, speakers, etc.
  • storage unit 808, such as disk, optical disk, etc.
  • communication unit 809 such as network card, modem, wireless transceiver, etc.
  • Communication unit 809 allows device 800 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.
  • the various methods or processes described above can be executed by CPU/GPU 801.
  • the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808.
  • part or all of the computer program can be loaded and/or installed on device 800 via ROM 802 and/or communication unit 809.
  • the computer program is loaded into RAM 803 and executed by CPU/GPU 801, one or more steps or actions in the methods or processes described above can be performed.
  • the methods and processes described above can be implemented as a computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.
  • Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device.
  • Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disc read-only memory
  • DVD digital multifunction disc
  • the computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
  • the computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing/processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and/or a wireless network, to an external computer or external storage device.
  • the network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing/processing device.
  • Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages and conventional procedural programming languages.
  • the computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • electronic circuitry such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.
  • These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions/actions specified in one or more blocks of the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and/or other device to operate in a particular manner.
  • the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions/actions specified in one or more blocks of the flowchart and/or block diagram.
  • Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions/actions specified in one or more boxes of a flowchart and/or block diagram.
  • each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function.
  • the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.
  • each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
  • Example 1 A method for determining audio timing, comprising:
  • multiple target checkpoints of the audio are determined, and the multiple target checkpoints are used to match the video frames.
  • Example 2 According to the method of Example 1, determining a plurality of candidate cadences of the audio and a plurality of energies of the plurality of candidate cadences includes:
  • each of the multiple candidate checkpoints is sorted in descending order.
  • Example 3 The method according to any one of Examples 1-2, wherein determining the plurality of target points of the audio based on the plurality of candidate points and the plurality of energies includes:
  • the multiple target checkpoints of the audio are determined by traversing the multiple candidate checkpoints arranged in descending order.
  • Example 4 The method according to any one of Examples 1-3, wherein determining the plurality of target points of the audio by traversing the plurality of candidate points in descending order based on an upper limit within a predetermined unit time includes:
  • Multiple time windows are determined based on a sliding time step on the multiple candidate checkpoints arranged in descending order, and the time sliding window indicates the predetermined unit time;
  • Example 5 The method according to any one of Examples 1-4, wherein determining the number of said candidate checkpoints within each time window based on each candidate checkpoint in descending order and the timestamp of each candidate checkpoint includes:
  • a quantity array is defined, which is used to record the number of candidate checkpoints in each time window.
  • Example 6 The method according to any one of Examples 1-5, wherein determining whether each candidate checkpoint among the plurality of candidate checkpoints is the target checkpoint based on the quantity includes:
  • the given candidate checkpoint is added to the result list, and the number array is updated;
  • the given candidate beats added to the result list are determined as the target beats of the audio.
  • Example 7 The method according to any one of Examples 1-6, wherein determining the given candidate beats added to the result list as the target beats of the audio includes:
  • the multiple candidate points are sorted in ascending order, and the candidate points in the result list after ascending order are the target points of the audio.
  • Example 8 Obtaining the audio according to any one of Examples 1-7 includes:
  • a first page In response to detecting a user click on a first button, a first page is displayed, the first page being used to display at least one of local video footage and cloud-based video footage;
  • a second page is displayed, which displays a second button for importing the audio.
  • Example 9 The method according to any one of Examples 1-8 further includes:
  • a third page is displayed, the third page being used to display at least one of local audio content and cloud-based audio content;
  • the edited video is exported.
  • Example 10 An apparatus for determining audio timing, comprising:
  • the audio acquisition module is configured to acquire audio for video production.
  • a candidate timing point determination module is configured to determine multiple candidate timing points of the audio and multiple energies of the multiple candidate timing points
  • the target checkpoint determination module is configured to determine multiple target checkpoints of the audio based on the multiple candidate checkpoints and the multiple energies, wherein the multiple target checkpoints are used to match the video frames.
  • Example 11 The apparatus according to Example 10, wherein the candidate checkpoint determination module includes:
  • An energy value determination module is configured to determine the timestamp and energy value of each of the plurality of candidate beats in the audio, the energy value indicating the intensity of a note and/or musical event in the audio;
  • the descending order module is configured to sort each of the plurality of candidate checkpoints in descending order based on the energy value.
  • Example 12 The apparatus according to any one of Examples 10-11, wherein the target checkpoint determination module comprises:
  • the traversal module is configured to determine the multiple target points of the audio by traversing the multiple candidate points arranged in descending order, based on the upper limit of the points within a predetermined unit of time.
  • Example 13 The apparatus according to any one of Examples 10-12, wherein the traversal module comprises:
  • the time window determination module is configured to determine multiple time windows based on a sliding time step on the multiple candidate checkpoints arranged in descending order, wherein the time sliding window indicates the predetermined unit time.
  • the quantity determination module is configured to determine the number of candidate checkpoints within each time window based on each candidate checkpoint sorted in descending order and the timestamp of each candidate checkpoint;
  • the judgment module is configured to determine, based on the number, whether each of the plurality of candidate checkpoints is the target checkpoint.
  • Example 14 The apparatus according to any one of Examples 10-13, wherein the quantity determining module comprises:
  • the array definition module is configured to define a quantity array based on the sliding time step, the quantity array being used to record the number of candidate checkpoints in each time window.
  • Example 15 The apparatus according to any one of Examples 10-14, wherein the determining module comprises:
  • the skip module is configured to skip a given candidate checkpoint in response to the number of candidate checkpoints within each time window being greater than or equal to the upper limit;
  • the addition module is configured to add a given candidate checkpoint to the results list and update the quantity array in response to the number of candidate checkpoints within each time window being less than the upper limit;
  • the determination module is configured to determine the given candidate cut-off point added to the result list as the target cut-off point of the audio.
  • Example 16 The apparatus according to any one of Examples 10-15, wherein the determining module comprises:
  • the ascending order module is configured to sort the multiple candidate points in the result list in ascending order based on the timestamps of the added candidate points, and the candidate points in the result list after ascending order are the target points of the audio.
  • Example 17 The apparatus according to any one of Examples 10-16, wherein the audio acquisition module comprises:
  • the first page implementation module is configured to display a first page in response to detecting a user click on a first button.
  • the first page is used to display at least one of local video footage and cloud-based video footage.
  • the second page display module is configured to display a second page in response to the video being imported.
  • the second page is used to display a second button for importing the audio.
  • Example 18 The apparatus according to any one of Examples 10-17 further includes:
  • a third page display module is configured to display a third page in response to detecting a user's click on the second button, the third page being used to display at least one of local audio materials and cloud-based audio materials;
  • the export module is configured to export the edited video in response to detecting a user's click on a third button on the second page.
  • Example 19 An electronic device comprising:
  • multiple target checkpoints of the audio are determined, and the multiple target checkpoints are used to match the video frames.
  • Example 20 The electronic device according to Example 19, wherein determining a plurality of candidate stops for the audio and a plurality of energies for the plurality of candidate stops includes:
  • each of the multiple candidate checkpoints is sorted in descending order.
  • Example 21 An electronic device according to any one of Examples 19-20, wherein determining a plurality of target points for the audio based on the plurality of candidate points and the plurality of energies includes:
  • the multiple target checkpoints of the audio are determined by traversing the multiple candidate checkpoints arranged in descending order.
  • Example 22 An electronic device according to any one of Examples 19-21, wherein determining the plurality of target points of the audio by traversing the plurality of candidate points in descending order, based on an upper limit within a predetermined unit time, comprises:
  • Multiple time windows are determined based on a sliding time step on the multiple candidate checkpoints arranged in descending order, and the time sliding window indicates the predetermined unit time;
  • Example 23 An electronic device according to any one of Examples 19-22, wherein determining the number of said candidate checkpoints within each time window based on each candidate checkpoint arranged in descending order and the timestamp of each candidate checkpoint includes:
  • a quantity array is defined, which is used to record the number of candidate checkpoints in each time window.
  • Example 24 An electronic device according to any one of Examples 19-23, wherein determining whether each candidate checkpoint among the plurality of candidate checkpoints is the target checkpoint based on the quantity includes:
  • the given candidate checkpoint is added to the result list, and the number array is updated;
  • the given candidate beats added to the result list are determined as the target beats of the audio.
  • Example 25 An electronic device according to any one of Examples 19-24, wherein determining a given candidate beat added to the result list as the target beat of the audio comprises:
  • the multiple candidate points are sorted in ascending order, and the candidate points in the result list after ascending order are the target points of the audio.
  • Example 26 An electronic device according to any one of Examples 19-25, wherein acquiring the audio comprises:
  • a first page In response to detecting a user click on a first button, a first page is displayed, the first page being used to display at least one of local video footage and cloud-based video footage;
  • a second page is displayed, which displays a second button for importing the audio.
  • Example 27 The electronic device according to any one of Examples 19-26 further includes:
  • a third page is displayed, the third page being used to display at least one of local audio materials and cloud-based audio materials;
  • the edited video is exported.
  • Example 28 A computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of Examples 1 to 9.
  • Example 29 A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of Examples 1 to 9.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)

Abstract

本公开的实施例涉及用于确定音频的卡点的方法、装置、设备和产品。该方法包括获取用于制作视频的音频，并确定音频的多个候选卡点以及多个候选卡点的多个能量。此外，该方法还包括基于多个候选卡点以及多个能量，确定音频的多个目标卡点，其中多个目标卡点用于匹配视频的画面。

Description

用于确定音频的卡点的方法、装置、设备和产品
本申请要求于2024年06月14日提交的，申请号为202410773416.0、发明名称为“用于确定音频的卡点的方法、装置、设备和产品”的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开总体涉及计算机领域，并且更具体地，涉及用于确定音频的卡点的方法、装置、电子设备和程序产品。
背景技术
视频剪辑是对一个视频源进行非线性编辑的过程，这涉及将图片、背景音乐、特效、场景等素材与原始视频进行重混合。在这个过程中，背景音乐的音乐卡点在视频剪辑中发挥着重要作用，它能够使得视频作品更具观赏性。
音乐卡点是指在制作视频时，选择一段特定的音乐节奏，使得视频的画面与这段音乐的节奏相匹配，以达到增强画面感染力和观看体验的效果。例如，在旅行游记、视频生活日志、影视剧解说等类型的视频中，通过音乐卡点可以迅速将观众带入剧情，增强观看体验。
发明内容
本公开的实施例提供了一种用于确定音频的卡点的方法、装置、电子设备和程序产品。
根据本公开的第一方面，提供了一种用于确定音频的卡点的方法。该方法包括获取用于制作视频的音频。该方法包括确定音频的多个候选卡点以及多个候选卡点的多个能量。此外，该方法还包括基于多个候选卡点以及多个能量，确定音频的多个目标卡点，其中多个目标卡点用于匹配视频的画面。
在本公开的第二方面中，提供了一种的用于确定音频的卡点的装置。该装置包括音频获取模块，被配置为获取用于制作视频的音频。该装置还包括候选卡点确定模块，被配置为确定音频的多个候选卡点以及多个候选卡点的多个能量。此外，该装置还包括目标卡点确定模块，被配置为基于多个候选卡点以及多个能量，确定音频的多个目标卡点，其中多个目标卡点用于匹配视频的画面。
在本公开的第三方面中，提供了一种电子设备。该电子设备包括处理器以及与处理器耦合的存储器，存储器具有存储于其中的指令，指令在被处理器执行时，使得电子设备执行根据第一方面的方法。
在本公开的第四个方面，提供了一种计算机程序产品，其上存储有包括计算机可执行指令，其中所述计算机可执行指令被处理器执行以实现第一方面的方法。
发明内容部分是为了以简化的形式来介绍对概念的选择，它们在下文的具体实施方式中将被进一步描述。发明内容部分无意标识要求保护的主题的关键特征或主要特征，也无意限制要求保护的主题的范围。
附图说明
结合附图并参考以下详细说明，本公开各实施例的上述和其它特征、优点及方面将变得更加明显。在附图中，相同或相似的附图标记表示相同或相似的元素，其中：
图1示出了本公开的一些实施例可以在其中实现的示例环境的示意图；
图2示出了本公开的一些实施例的用于确定音频的卡点的方法的流程图；
图3示出了本公开的一些实施例的音频的卡点的示意图；
图4示出了本公开的一些实施例的用于在多个候选卡点中确定音频的目标卡点的示意图；
图5示出了本公开的一些实施例的通过滑动窗口来确定音频的卡点的示意图；
图6A示出了本公开的一些实施例的用于导入待剪辑的视频的示意图；
图6B示出了本公开的一些实施例的用于导入音频的示意图；
图7示出了本公开的一些实施例的用于确定音频的卡点的装置的框图；以及
图8示出了本公开的一些实施例的电子设备的框图。
在所有附图中，相同或相似参考数字表示相同或相似元素。
具体实施方式
可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的一些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
在本公开的实施例的描述中，术语“包括”及其类似用语应当理解为开放性包括，即“包括但不限于”。术语“基于”应当理解为“至少部分地基于”。术语“一个实施例”或“该实施例”应当理解为“至少一个实施例”。术语“第一”、“第二”等等可以指代不同的或相同的对象，除非明确说明。下文还可能包括其它明确的和隐含的定义。
如前所述，将视频的画面节奏与音乐的节奏相匹配在视频剪辑中是一种重要的技巧。在相关技术中，通常是将音乐的节拍点(beat)作为音乐的节奏点，这种方法不能满足用户对视频卡点音乐的需求。因为在音乐的每个小节(measure)没有发生速度(tempo)变化的情况下，节拍点作为音乐中的基本时间单位，其在音乐的时间维度上是均匀分布的。此时，仅将节拍点作为音乐的节奏点，并不能契合视频的创作者的卡点需求。此外，仅使用音乐的节拍点(beat)作为视频的节奏点，还可能过滤掉一个节拍之内的明显的节奏变化的点，比如一个节拍之内的连续的几个重低音。
本公开的实施例中，通过检测音频中的多个更大范围的潜在的节奏点(候选卡点)以及这些候选卡点的能量，能够精确地确定出音频中关键的节奏点(目标卡点)，这些目标卡点可以被用于与视频画面的精准匹配。这种根据音频的能量分析来确定音频的节奏的方法，能够确定出包括但不限于节拍点在内的更多的在音频中具有影响力的多个候选卡点，进而再根据这些候选卡点的能量确定在音频中具有影响力和重要性的目标卡点，不仅能够确保关键卡点的有效保留，还能够满足用户对根据卡点来应用视频转场的需求，提升了用户体验。
图1示出了本公开的一些实施例可以在其中实现的示例环境100的示意图。如图1所示，显示界面102是为待编辑的视频104编辑音频110的卡点的界面。在显示界面102上，显示有待编辑的视频104，通过播放工具栏106，用户可以对待编辑的视频104暂停，前进或者后退，还可以通过全屏播放按钮来实现待编辑视频104的全屏展示，播放工具栏106上还显示有待编辑视频104的总时间长度以及当前的定位时间戳。
继续参考图1，音频110是用户选择的为待编辑的视频104进行配乐的音频。为了使得待编辑视频104的节奏与音频110的节奏相匹配，可以为用户呈现其使用的音频110的卡点，这样可以方便用户在编辑视频时，能够在音乐发生明显变化的时候应用转场，使得音视频和谐一致。例如，当用户打开自动卡点108的按钮，可以在音频110的下方显示当前音频110的卡点112，这些卡点112不局限于音频110的每个小节的均匀分布的节拍，还可以包括有音频110的鼓点或者是低音乐器(base)如吉他开始时的点位。
图1所示的卡点112中的点位112-1可以是一节拍之内的连续3个鼓点。在音乐中，小节是一段固定时值的节奏单位，通常以小节线分隔。每个小节通常包含一个或多个节拍。节拍点是音乐中的基本时间单位，节拍点在乐曲中是均匀分布的，除非有特定的速度变化标记(如渐快、渐慢等)。
参考图1，为了实现友好的用户交互和提升用户的体验，除了为用户自动识别卡点外，用户也可以通过添加卡点114按钮来实现手动对已经生成的卡点112的编辑，例如可以自行添加卡点或者删除已生成的至少包括节拍点在内的多个候选卡点。当用户操作后，可以通过工具栏116来实现对音视频的保存或者取消。通过这种为用户选定的音频确定大量的多样化的不局限于节拍点的多个候选卡点的方法，能够满足用户希望在音乐发生明显变化时来同步实现画面变化的需求，提升了用户体验感。
下面将结合图2至图8详细描述根据本公开实施例的过程。为了便于理解，在下文描述中提及的具体数据均是示例性的，并不用于限定本公开的保护范围。可以理解，以下描述的实施例还可以包括未示出的附加动作和/或可以省略所示出的动作，本公开的范围在此方面不受限制。
图2示出了本公开的一些实施例的用于确定音频的卡点的方法200的流程图。方法200可以由用于确定音频的卡点的装置执行。该装置例如可以是服务器，也可以是配置于服务器的系统，也可以是独立的装置或系统。该装置可以通过软件和/或硬件的方式实现。接下来以用于确定音频的卡点的装置为执行主体为例，对方法200进行示意性地说明。参考图2，方法200可以包括框202、框204以及框206。
在框202，获取用于制作视频的音频。在一些实施例中，这些视频可以是短视频也可以是长视频，这些视频可以从云端获取或者从本地获取。用于制作视频的音频可以是纯音乐音频或者是包含人声的音频等，这些音频可以从本地获取或者从云端获取。
在框204，确定音频的多个候选卡点以及多个候选卡点的多个能量。这些候选点可以通过起始点检测(onset detection)的方法而得到，这些候选卡点是多种形式的卡点，比如可以是音频的节拍点，还可以是音频的某个乐器如鼓的鼓点，又或者是某个低音乐器如吉他的开始点位。例如还可以是一个节拍之内有好几个连续的重低音的鼓点。每个点位都有其各自的能量，在一些实施例中，这些能量代表音频中音符和/或变化的音乐事件的强度。
在框206，基于多个候选卡点以及多个能量，确定音频的多个目标卡点，其中多个目标卡点用于匹配视频的画面。例如，当确定了多个候选卡点后，考虑到确定的候选卡点的数量可能过于密集从而影响了用户卡点的转场效果，可以在根据多个候选卡点的能量来从多个候选卡点中确定出适用于用户进行视频画面踩点转场的目标卡点，这样用户即可以根据确定出来的目标卡点来实现视频画面与音频节奏点的同步变化。
本公开的实施例中，检测音频的多个候选卡点以及多个候选卡点的能量，并根据这些候选卡点以及候选卡点的能量来确定音频的目标卡点，从而可以根据目标卡点来匹配视频画面。这种根据音频的能量来确定音频的节奏的方法，能够确定出更多的包括但不限于节拍点在内的在音频中具有影响力的多个候选卡点，进而再根据这些候选卡点的能量有效保留在音频中具有影响力和重要性的目标卡点，这样不仅能够确保关键卡点的保留，还能够满足用户对根据卡点来应用视频转场的需求，提升了用户体验。
图3示出了本公开的一些实施例的音频的卡点300的示意图。为了清晰地说明本公开的一些实施例的卡点的点位，下面将结合图3来说明。当一个声音从开始到结束的过程可以大致分为起始(onset)、瞬态(attack)、衰减(decay)这三个过程。在声音的开始阶段，声音的起始点310标志着声音能量的突然增加和声音事件的开始，声音的振幅和能量可以从零或非常低的水平开始显著上升，这是声音可被识别的起点，也是后续阶段的开端。图3所示的起始点310即为音频中的卡点。
继续参考图3，在声音的起始阶段后紧跟着瞬态阶段320，瞬态阶段320描述了声音从起始到最大音量的快速上升过程，即声音在极短的时间内迅速增强。在瞬态阶段320，声音的振幅包络急剧增加，音量快速上升并达到峰值，这个阶段通常很短，但它决定了声音的清晰度和冲击力。爆发性声音330通常与音符开始时的瞬态变化重叠，其可描述为在乐音或声音事件开始时出现的短持续时间和高振幅的类噪声声音分量。爆发性声音330区域信号变化快速且不可预测。例如，在钢琴演奏中，爆发性声音330对应于琴键被敲击、琴弦开始振动的初始阶段。
如图3所示，当声音达到峰值后，音量逐渐减小直至声音消失的过程是衰减阶段340。在这个阶段，声音的振幅逐渐降低，能量逐渐消散。在一些乐器中，衰减可能是迅速的，而在其它情形下，衰减可能是较为缓慢的。
图4示出了本公开的一些实施例的用于在多个候选卡点中确定音频的目标卡点400的示意图。参考图4，在410，检测音频的候选卡点。在一些实施例中，可以采用onset检测的方法来确定出音频的候选卡点。在一些实施例中，这种onset检测的方法可以是基于音频能量的，还可以是基于频谱的，或者是可以基于频谱和能量的结合的方法的。在一些实施例中，这些被检测出来的候选卡点可以是音频的节拍点或者某个声音或者音乐事件的起始点。通过onset检测不仅能得到数量更多，类型更全的候选卡点，还能得到候选卡点的时间戳以及候选卡点的能量值。
如图4所示，在420，将候选卡点按照能量值降序排序。假设检测到有以下候选卡点的表1。(时间单位：秒)
经过根据能量降序排序后，可以得到表2。
见图4，在430，初始化数量数组。该数组用于记录时间滑窗中已放入的候选卡点的数量，并且使用时间滑窗相对于时间起点的步长偏移量作为下标。例如，假设时间序列的数据是从T0时刻开始的，使用10秒的时间滑窗，从T0时刻开始，第一个10秒的窗口的偏移量是0(因为它从起点开始)，第二个10秒的窗口的偏移量是1(因为它从T0+1开始)，依此类推。这样，能够简化时间滑窗的管理和访问方式。
继续参考图4，在440，通过滑动窗口来遍历排序过的候选卡点。下面将结合图5来说明本公开的实施例的一些通过滑动窗口来确定音频的卡点500的示意图。假设设定的每个时间窗口内的候选卡点的上限为20个，时间窗口大小为5个步长，每次移动的步长为1。
如图4所示，在442，根据目标卡点的时间戳计算所有包含候选卡点的时间窗口。在图5中，区域510示出了候选卡点的时间戳。假设有一个候选卡点在[n,n+1]之间，那么在[n,n+1]之间的这个候选卡点可以影响区域520中的时间窗口[n-4,n+1]、时间窗口[n-3,n+2]、时间窗口[n-2,n+3]、时间窗口[n-1,n+4]以及时间窗口[n,n+5]。可以理解的是，在这里不考虑负的时间点，所以起始时间至少为0。
继续参考图4，由于候选卡点可能不是恰好在时间窗口的起始或结束时发生，一个候选卡点可能会影响多个相邻的时间窗口。为了确保在任何5个步长时间窗口内候选卡点的数量小于20个，需要检查并更新所有受影响的时间窗口的计数值。在444，判断时间窗口中已有的候选卡点的数量是否大于等于上限即20个，如果不是的话，则在448，将该候选卡点加入结果列表中，并更新用于记录每个窗口内已放入的候选卡点的数量。结合图5，在区域530中的数组中的计数器记录的值都小于上限20，表示可以将在[n,n+1]之间的这个候选卡点加入结果列表之中，并且更新数组的值即对数组中的对应的窗口的计数值均加1。
继续参考图4，在444，判断时间窗口中已有的候选卡点的数量是否大于等于上限即20，如果是的话，则在446，跳过这个候选卡点并重新判断下一个候选卡点。结合图5，例如时间窗口[n+1,n+6]中的值等于20，那么与时间窗口[n+1,n+6]对应的候选卡点将被跳过，并判断这个候选卡点后的候选卡点。如此，针对时间戳上的每个候选卡点都进行如上操作，可以将满足条件的候选卡点加入至结果列表。进而，在450，对结果列表中的候选卡点按照时间戳的顺序升序排序，从而在460，将升序排序后的结果列表中的候选卡点作为音频的目标卡点。
为了更清晰地说明本公开的实施例的通过滑动窗口来确定音频的卡点的过程，下面将举例说明，假设在窗口大小为10秒，步长为1秒的任意时间窗口内，设定窗口的候选卡点的数量小于20个，并优先保留能量最高的候选卡点。假设降序排序后的多个候选卡点的时间戳及其能量值如表3所示：
接下来，可以定义的数量数组来记录每个时间窗口内的候选卡点的数量，时间窗口的大小是10秒，步长是1秒，可以为每个可能的起始时间点(从0秒开始，每1秒1个点)来初始化一个计数器。假设初始化数组为[0,0,0,0,0,0,0,0,0]，这里的每个元素代表一个时间窗口的候选卡点的计数器，下标代表时间窗口的起始时间(以秒为单位)。
接着，将通过滑动窗口来遍历排序过的候选卡点。针对2.5秒的候选卡点，其影响的时间窗口有[0-10],[1-11]以及[2-12]。判断这些时间窗口的计数器，均为0，未达到上限，将2.5秒的候选卡点加入结果列表，并更新数量数组为[1,1,1,0,0,0,0,0,0]。
针对3.3秒的候选卡点，其影响的时间窗口有[0-10],[1-11],[2-12]以及[3-13]。判断这些时间窗口的计数器，前三个时间窗口计数为1，未达到上限，[3-13]的时间窗口的计数器为0，将3.3秒的候选卡点加入结果列表，并更新数量数组为[2,2,2,1,0,0,0,0,0]。
针对3.5秒的候选卡点，其影响的时间窗口有[0-10],[1-11],[2-12]以及[3-13]。判断这些时间窗口的计数器，未达到上限，将3.5秒的候选卡点加入结果列表，并更新数量数组为[3,3,3,2,0,0,0,0,0]。
针对8.0秒的候选卡点，其影响的时间窗口有[0-10],[1-11]……[8-18]。判断这些时间窗口的计数器，未达到上限，将8.0秒的候选卡点加入结果列表，并更新数量数组为[4,4,4,3,1,1,1,1,1]。
针对8.5秒的候选卡点，其影响的时间窗口有[0-10]，[1-11]……[8-18]。判断这些时间窗口的计数器，未达到上限，将8.5秒的候选卡点加入结果列表，并更新数量数组为[5,5,5,4,2,2,2,2,2]。
接着对结果列表中的候选卡点，按照时间戳的顺序升序排序，可以得到表4，如下：
表4所示的候选卡点即为音频的目标卡点。
又例如，假设有一系列的经过能量降序排序后候选卡点，时间从0秒开始，每隔0.1秒就有一个候选卡点，且能量值相同。当处理到1.0秒的候选卡点时，时间窗口[0-10]的计数就会增加到10，当处理到1.9秒的候选卡点时，时间窗口[0-10]的计数就会增加到20，此时该时间窗口已经达到了设定的上限，该1.9秒的候选卡点将被跳过，并判断后面的2.0秒的候选卡点。接下来的候选卡点(如2.0秒、2.1秒等)也会影响到[0-10]的时间窗口，所以这些候选卡点也不会被加入至结果列表中。可以理解的是，实际应用中候选卡点可能不是规律地出现，但以上有助于理解根据滑动窗口来筛选候选卡点的过程。
图6A示出了本公开的一些实施例的用于导入待剪辑的视频的600A的示意图。如图6A所示，在剪辑功能610A的显示界面602A的显眼位置显示有用于导入待剪辑视频的“开始创作”按钮604A，用户点击后即可以进度导入视频或者照片的界面。在按钮604A的下方，显示有用户可以使用的模板区域606A，这样能够方便用户快速进入创作状态。在显示页面602A的中下部区域608A，展示了用户之前导入的但未完成编辑的视频草稿，这样能够方便用户随时继续编辑。
图6B示出了本公开的一些实施例的用于导入音频600B的示意图。结合图6A，当用户点击“开始创作”按钮604A，并导入了待编辑的视频604B后，可以跳转到显示界面602B。在显示界面602B中，用户可以为待编辑的视频604B配乐，并根据该配乐来调整视频604B的画面。具体而言，用户可以通过点击添加音频按钮608B来跳转到添加音频的页面，从而可以添加其需要的音频，这些音频可以保存于本地也可以从云端获取。当导入了音频后，可以通过区域606B的关闭原声按钮来关闭视频的原声，这样能够方便用户聚焦于导入的音频与视频604B的匹配度上。当用户完成对视频已经音频的编辑后，可以点击导出按钮610B来保存和分享他们的作品。用户还可以看到他们作品的分辨率，还可以调整他们的作品的分辨率。
图7示出了本公开的一些实施例的用于确定音频的卡点的装置700的框图。如图7所示，装置700包括音频获取模块702，被配置为获取用于制作视频的音频。装置700还包括候选卡点确定模块704，被配置为确定音频的多个候选卡点以及多个候选卡点的多个能量。此外，装置700还包括目标卡点确定模块706，被配置基于多个候选卡点以及多个能量，确定音频的多个目标卡点，其中多个目标卡点用于匹配视频的画面。
图8示出了本公开的一些实施例的电子设备800的框图，设备800可以是本公开的实施例所描述的设备或装置。如图8所示，设备800包括中央处理单元(CPU)和/或图形处理单元(GPU)801，其可以根据存储在只读存储器(ROM)802中的计算机程序指令或者从存储单元808加载到随机访问存储器(RAM)803中的计算机程序指令，来执行各种适当的动作和处理。在RAM 803中，还可以存储设备800操作所需的各种程序和数据。CPU/GPU 801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。虽然未在图8中示出，设备800还可以包括协处理器。
设备800中的多个部件连接至I/O接口805，包括：输入单元806，例如键盘、鼠标等；输出单元807，例如各种类型的显示器、扬声器等；存储单元808，例如磁盘、光盘等；以及通信单元809，例如网卡、调制解调器、无线通信收发机等。通信单元809允许设备800通过诸如互联网的计算机网络和/或各种电信网络与其他设备交换信息/数据。
上文所描述的各个方法或过程可以由CPU/GPU 801来执行。例如，在一些实施例中，方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元808。在一些实施例中，计算机程序的部分或者全部可以经由ROM 802和/或通信单元809而被载入和/或安装到设备800上。当计算机程序被加载到RAM 803并由CPU/GPU 801执行时，可以执行上文描述的方法或过程中的一个或多个步骤或动作。
在一些实施例中，以上所描述的方法和过程可以被实现为计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于执行本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
本文所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如互联网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，编程语言包括面向对象的编程语言，以及常规的过程式编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用互联网服务提供商来通过互联网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理单元，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理单元执行时，产生了实现流程图和/或框图中的一个或多个框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的设备、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这取决于所涉及的功能。也要注意的是，框图和/或流程图中的每个框、以及框图和/或流程图中的框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所公开的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中技术的技术改进，或者使得本技术领域的其它普通技术人员能理解本文公开的各实施例。
下列出了本公开的一些示例实现。
示例1.一种用于确定音频的卡点的方法，包括：
获取用于制作视频的音频；
确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量；以及
基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点，所述多个目标卡点用于匹配视频的画面。
示例2.根据示例1所述的方法，其中确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量包括：
确定所述多个候选卡点中的每个候选卡点在所述音频中的时间戳以及能量值，所述能量值指示所述音频的音符和/或音乐事件的强度；以及
基于所述能量值，对所述多个候选卡点中的每个候选卡点降序排序。
示例3.根据示例1-2中任一项所述的方法，其中基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点包括：
基于预定单位时间内的卡点的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点。
示例4.根据示例1-3中任一项所述的方法，其中基于预定单位时间内的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点包括：
由时间滑动窗口基于滑动时间步长在经降序排列的所述多个候选卡点上确定多个时间窗口，所述时间滑动窗口指示所述预定单位时间；
基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量；以及
基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点。
示例5.根据示例1-4中任一项所述的方法，其中基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量包括：
基于所述滑动时间步长，定义数量数组，所述数量数组用于记录每个时间窗口中的候选卡点的数量。
示例6.根据示例1-5中任一项所述的方法，其中基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点包括：
响应于每个时间窗口内的所述候选卡点的所述数量大于所述上限，跳过给定候选卡点；或者
响应于每个时间窗口内的所述候选卡点的所述数量小于等于所述上限，将给定候选卡点加入结果列表，并更新所述数量数组；以及
将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点。
示例7.根据示例1-6中任一项所述的方法，其中将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点包括：
基于所述结果列表中的所加入的多个候选卡点的时间戳，对所述多个候选卡点升序排序，经升序排序的所述结果列表中的所述候选卡点为所述音频的所述目标卡点。
示例8.根据示例1-7中任一项所述的方法，获取所述音频包括：
响应于检测到用户对第一按钮的点击，显示第一页面，所述第一页面用于显示本地视频素材以及云端视频素材中的至少一项；以及
响应于所述视频已被导入，显示第二页面，所述第二页面用于显示第二按钮，所述第二按钮用于导入所述音频。
示例9.根据示例1-8中任一项所述的方法，还包括：
响应于检测到所述用户对所述第二按钮的点击，显示第三页面，所述第三页面用于显示本地音频素材以及云端音频素材中的至少一项；以及
响应于检测到所述用户对所述第二页面上的第三按钮的点击，导出所编辑的视频。
示例10.一种用于确定音频的卡点的装置，包括：
音频获取模块，被配置为获取用于制作视频的音频；
候选卡点确定模块，被配置为确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量；以及
目标卡点确定模块，被配置为基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点，所述多个目标卡点用于匹配视频的画面。
示例11.根据示例10所述的装置，其中候选卡点确定模块包括：
能量值确定模块，被配置为确定所述多个候选卡点中的每个候选卡点在所述音频中的时间戳以及能量值，所述能量值指示所述音频的音符和/或音乐事件的强度；以及
降序模块，被配置为基于所述能量值，对所述多个候选卡点中的每个候选卡点降序排序。
示例12.根据示例10-11中任一项所述的装置，其中目标卡点确定模块包括：
遍历模块，被配置为基于预定单位时间内的卡点的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点。
示例13.根据示例10-12中任一项所述的装置，其中遍历模块包括：
时间窗口确定模块，被配置为由时间滑动窗口基于滑动时间步长在经降序排列的所述多个候选卡点上确定多个时间窗口，所述时间滑动窗口指示所述预定单位时间；
数量确定模块，被配置为基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量；以及
判断模块，被配置为基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点。
示例14.根据示例10-13中任一项所述的装置，其中数量确定模块包括：
数组定义模块，被配置为基于所述滑动时间步长，定义数量数组，所述数量数组用于记录每个时间窗口中的候选卡点的数量。
示例15.根据示例10-14中任一项所述的装置，其中判断模块包括：
跳过模块，被配置为响应于每个时间窗口内的所述候选卡点的所述数量大于等于所述上限，跳过给定候选卡点；或者
加入模块，被配置为响应于每个时间窗口内的所述候选卡点的所述数量小于所述上限，将给定候选卡点加入结果列表，并更新所述数量数组；以及
确定模块，被配置为将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点。
示例16.根据示例10-15中任一项所述的装置，其中确定模块包括：
升序模块，被配置为基于所述结果列表中的所加入的多个候选卡点的时间戳，对所述多个候选卡点升序排序，经升序排序的所述结果列表中的所述候选卡点为所述音频的所述目标卡点。
示例17.根据示例10-16中任一项所述的装置，其中音频获取模块包括：
第一页面实现模块，被配置为响应于检测到用户对第一按钮的点击，显示第一页面，所述第一页面用于显示本地视频素材以及云端视频素材中的至少一项；以及
第二页面显示模块，被配置为响应于所述视频已被导入，显示第二页面，所述第二页面用于显示第二按钮，所述第二按钮用于导入所述音频。
示例18.根据示例10-17中任一项所述的装置，还包括：
第三页面显示模块，被配置为响应于检测到所述用户对所述第二按钮的点击，显示第三页面，所述第三页面用于显示本地音频素材以及云端音频素材中的至少一项；以及
导出模块，被配置为响应于检测到所述用户对所述第二页面上的第三按钮的点击，导出所编辑的视频。
示例19.一种电子设备，包括：
处理器；以及
与所述处理器耦合的存储器，所述存储器具有存储于其中的指令，所述指令在被处理器执行时，使得所述电子设备执行动作，所述动作包括：
获取用于制作视频的音频；
确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量；以及
基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点，所述多个目标卡点用于匹配视频的画面。
示例20.根据示例19所述的电子设备，其中确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量包括：
确定所述多个候选卡点中的每个候选卡点在所述音频中的时间戳以及能量值，所述能量值指示所述音频的音符和/或音乐事件的强度；以及
基于所述能量值，对所述多个候选卡点中的每个候选卡点降序排序。
示例21.根据示例19-20中任一项所述的电子设备，其中基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点包括：
基于预定单位时间内的卡点的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点。
示例22.根据示例19-21中任一项所述的电子设备，其中基于预定单位时间内的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点包括：
由时间滑动窗口基于滑动时间步长在经降序排列的所述多个候选卡点上确定多个时间窗口，所述时间滑动窗口指示所述预定单位时间；
基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量；以及
基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点。
示例23.根据示例19-22中任一项所述的电子设备，其中基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量包括：
基于所述滑动时间步长，定义数量数组，所述数量数组用于记录每个时间窗口中的候选卡点的数量。
示例24.根据示例19-23中任一项所述的电子设备，其中基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点包括：
响应于每个时间窗口内的所述候选卡点的所述数量大于等于所述上限，跳过给定候选卡点；或者
响应于每个时间窗口内的所述候选卡点的所述数量小于所述上限，将给定候选卡点加入结果列表，并更新所述数量数组；以及
将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点。
示例25.根据示例19-24中任一项所述的电子设备，其中将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点包括：
基于所述结果列表中的所加入的多个候选卡点的时间戳，对所述多个候选卡点升序排序，经升序排序的所述结果列表中的所述候选卡点为所述音频的所述目标卡点。
示例26.根据示例19-25中任一项所述的电子设备，其中获取所述音频包括：
响应于检测到用户对第一按钮的点击，显示第一页面，所述第一页面用于显示本地视频素材以及云端视频素材中的至少一项；以及
响应于所述视频已被导入，显示第二页面，所述第二页面用于显示第二按钮，所述第二按钮用于导入所述音频。
示例27.根据示例19-26中任一项所述的电子设备，还包括：
响应于检测到所述用户对所述第二按钮的点击，显示第三页面，所述第三页面用于显示本地音频素材以及云端音频素材中的至少一项；以及
响应于检测到所述用户对所述第二页面上的第三按钮的点击，导出所编辑的视频。
示例28.一种计算机可读存储介质，其上存储有计算机可执行指令，其中所述计算机可执行指令被处理器执行以实现根据示例1至9中任一项所述的方法。
示例29.一种计算机程序产品，所述计算机程序产品被有形地存储在计算机可读介质上并且包括计算机可执行指令，所述计算机可执行指令在由设备执行时使所述设备执行根据示例1至9中任一项所述的方法。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本公开，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (12)

  1. 一种用于确定音频的卡点的方法，包括：
    获取用于制作视频的音频；
    确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量；以及
    基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点，所述多个目标卡点用于匹配视频的画面。
  2. 根据权利要求1所述的方法，其中确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量包括：
    确定所述多个候选卡点中的每个候选卡点在所述音频中的时间戳以及能量值，所述能量值指示所述音频的音符和/或音乐事件的强度；以及
    基于所述能量值，对所述多个候选卡点中的每个候选卡点降序排序。
  3. 根据权利要求2所述的方法，其中基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点包括：
    基于预定单位时间内的卡点的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点。
  4. 根据权利要求3所述的方法，其中基于预定单位时间内的上限，通过遍历经降序排列的所述多个候选卡点，来确定所述音频的所述多个目标卡点包括：
    由时间滑动窗口基于滑动时间步长在经降序排列的所述多个候选卡点上确定多个时间窗口，所述时间滑动窗口指示所述预定单位时间；
    基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量；以及
    基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点。
  5. 根据权利要求4所述的方法，其中基于经降序排列的每个候选卡点以及每个候选卡点的时间戳，确定每个时间窗口内的所述候选卡点的数量包括：
    基于所述滑动时间步长，定义数量数组，所述数量数组用于记录每个时间窗口中的候选卡点的数量。
  6. 根据权利要求5所述的方法，其中基于所述数量，确定所述多个候选卡点中的每个候选卡点是否为所述目标卡点包括：
    响应于每个时间窗口内的所述候选卡点的所述数量大于等于所述上限，跳过给定候选卡点；或者
    响应于每个时间窗口内的所述候选卡点的所述数量小于所述上限，将给定候选卡点加入结果列表，并更新所述数量数组；以及
    将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点。
  7. 根据权利要求6所述的方法，其中将所述结果列表中的所加入的给定候选卡点确定为所述音频的所述目标卡点包括：
    基于所述结果列表中的所加入的多个候选卡点的时间戳，对所述多个候选卡点升序排序，经升序排序的所述结果列表中的所述候选卡点为所述音频的所述目标卡点。
  8. 根据权利要求1所述的方法，其中获取所述音频包括：
    响应于检测到用户对第一按钮的点击，显示第一页面，所述第一页面用于显示本地视频素材以及云端视频素材中的至少一项；以及
    响应于所述视频已被导入，显示第二页面，所述第二页面用于显示第二按钮，所述第二按钮用于导入所述音频。
  9. 根据权利要求8所述的方法，还包括：
    响应于检测到所述用户对所述第二按钮的点击，显示第三页面，所述第三页面用于显示本地音频素材以及云端音频素材中的至少一项；以及
    响应于检测到所述用户对所述第二页面上的第三按钮的点击，导出所编辑的视频。
  10. 一种的用于确定音频的卡点的装置，包括：
    音频获取模块，被配置为获取用于制作视频的音频；
    候选卡点确定模块，被配置为确定所述音频的多个候选卡点以及所述多个候选卡点的多个能量；以及
    目标卡点确定模块，被配置为基于所述多个候选卡点以及所述多个能量，确定所述音频的多个目标卡点，所述多个目标卡点用于匹配视频的画面。
  11. 一种电子设备，包括：
    处理器；以及
    与所述处理器耦合的存储器，所述存储器具有存储于其中的指令，所述指令在被处理器执行时，使得所述电子设备执行根据权利要求1至9中任一项所述的方法。
  12. 一种计算机程序产品，其包括计算机可执行指令，其中所述计算机可执行指令被处理器执行以实现根据权利要求1至9中任一项所述的方法。
PCT/CN2025/100055 2024-06-14 2025-06-10 用于确定音频的卡点的方法、装置、设备和产品 Pending WO2025256503A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410773416.0 2024-06-14
CN202410773416.0A CN118488262A (zh) 2024-06-14 2024-06-14 用于确定音频的卡点的方法、装置、设备和产品

Publications (1)

Publication Number Publication Date
WO2025256503A1 true WO2025256503A1 (zh) 2025-12-18

Family

ID=92193074

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/100055 Pending WO2025256503A1 (zh) 2024-06-14 2025-06-10 用于确定音频的卡点的方法、装置、设备和产品

Country Status (2)

Country Link
CN (1) CN118488262A (zh)
WO (1) WO2025256503A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118488262A (zh) * 2024-06-14 2024-08-13 北京字跳网络技术有限公司 用于确定音频的卡点的方法、装置、设备和产品

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110933487A (zh) * 2019-12-18 2020-03-27 北京百度网讯科技有限公司 卡点视频的生成方法、装置、设备及存储介质
CN112235631A (zh) * 2019-07-15 2021-01-15 北京字节跳动网络技术有限公司 视频处理方法、装置、电子设备及存储介质
CN116189708A (zh) * 2023-02-20 2023-05-30 北京达佳互联信息技术有限公司 转场点的确定方法、装置、设备及存储介质
WO2023174066A1 (zh) * 2022-03-18 2023-09-21 北京字跳网络技术有限公司 视频生成方法、装置、电子设备和存储介质
CN118488262A (zh) * 2024-06-14 2024-08-13 北京字跳网络技术有限公司 用于确定音频的卡点的方法、装置、设备和产品

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112235631A (zh) * 2019-07-15 2021-01-15 北京字节跳动网络技术有限公司 视频处理方法、装置、电子设备及存储介质
CN110933487A (zh) * 2019-12-18 2020-03-27 北京百度网讯科技有限公司 卡点视频的生成方法、装置、设备及存储介质
WO2023174066A1 (zh) * 2022-03-18 2023-09-21 北京字跳网络技术有限公司 视频生成方法、装置、电子设备和存储介质
CN116189708A (zh) * 2023-02-20 2023-05-30 北京达佳互联信息技术有限公司 转场点的确定方法、装置、设备及存储介质
CN118488262A (zh) * 2024-06-14 2024-08-13 北京字跳网络技术有限公司 用于确定音频的卡点的方法、装置、设备和产品

Also Published As

Publication number Publication date
CN118488262A (zh) 2024-08-13

Similar Documents

Publication Publication Date Title
CN113365134B (zh) 音频分享方法、装置、设备及介质
US8346540B2 (en) Deep tag cloud associated with streaming media
US12176008B2 (en) Method and apparatus for matching music with video, computer device, and storage medium
CN111095939B (zh) 识别媒体项目的先前流传输部分以避免重复重放
WO2020093883A1 (zh) 获取视频片段的方法、装置、服务器和存储介质
CN108833787B (zh) 用于生成短视频的方法和装置
WO2025256503A1 (zh) 用于确定音频的卡点的方法、装置、设备和产品
CN112866776B (zh) 视频生成方法和装置
WO2020259130A1 (zh) 精选片段处理方法、装置、电子设备及可读介质
US20230129509A1 (en) Previewing method and apparatus for effect application, and device, and storage medium
CN114339076A (zh) 视频拍摄方法、装置、电子设备以及存储介质
WO2022017451A1 (zh) 视频配音方法、装置、设备及存储介质
CN107450874A (zh) 一种多媒体数据双屏播放方法及系统
WO2023109103A1 (zh) 视频编辑方法、装置、电子设备、介质
CN115083432A (zh) 音频可视化方法、装置、电子设备及存储介质
US10460712B1 (en) Synchronizing playback of a digital musical score with an audio recording
CN116049490A (zh) 素材搜索方法、装置和电子设备
US20250239275A1 (en) Video processing method, apparatus, electronic device, and storage medium
WO2025044745A1 (zh) 用于缩放时间轴的方法、装置、电子设备和介质
US12040000B2 (en) Special effect processing method and apparatus
US9293124B2 (en) Tempo-adaptive pattern velocity synthesis
US20250315200A1 (en) Lyric-based information prompting method and apparatus, device, medium and product
US20260037107A1 (en) Display videos associated with audio on profile page
WO2023160713A1 (zh) 音乐生成方法、装置、设备、存储介质及程序
CN113766325B (zh) 视频播放方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25821214

Country of ref document: EP

Kind code of ref document: A1