WO2023169258A1 - 音频检测方法、装置、存储介质及电子设备 - Google Patents

音频检测方法、装置、存储介质及电子设备 Download PDF

Info

Publication number
WO2023169258A1
WO2023169258A1 PCT/CN2023/078752 CN2023078752W WO2023169258A1 WO 2023169258 A1 WO2023169258 A1 WO 2023169258A1 CN 2023078752 W CN2023078752 W CN 2023078752W WO 2023169258 A1 WO2023169258 A1 WO 2023169258A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
music
events
event
metadata information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/078752
Other languages
English (en)
French (fr)
Inventor
王乔木
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Priority to US18/844,887 priority Critical patent/US20250218418A1/en
Publication of WO2023169258A1 publication Critical patent/WO2023169258A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0033—Recording/reproducing or transmission of music for electrophonic musical instruments
    • G10H1/0041—Recording/reproducing or transmission of music for electrophonic musical instruments in coded form
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/051—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction or detection of onsets of musical sounds or notes, i.e. note attack timings
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/075—Musical metadata derived from musical analysis or for use in electrophonic musical instruments
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/121—Musical libraries, i.e. musical databases indexed by musical parameters, wavetables, indexing schemes using musical parameters, musical rule bases or knowledge bases, e.g. for automatic composing methods
    • G10H2240/131—Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
    • G10H2240/141—Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process

Definitions

  • the present disclosure relates to the field of audio processing technology, such as audio detection methods, devices, storage media and electronic equipment.
  • the audio detection method cannot collect music-related statistical data in the audio to be detected (such as music duration, music playback start and end time, etc.).
  • the present disclosure provides audio detection methods, devices, storage media and electronic equipment to achieve accurate acquisition of statistical data in audio to be detected.
  • an audio detection method including:
  • Metadata information matching the music event is determined, and statistics in the detected audio are determined based on the metadata information.
  • an audio detection device including:
  • a music event identification module configured to obtain audio segments in the detected audio and identify music events in the audio segments
  • a statistical data determination module is configured to determine metadata information matching the music event, and determine statistical data in the detected audio based on the metadata information.
  • the present disclosure also provides an electronic device, which includes:
  • processors one or more processors
  • a storage device configured to store one or more programs
  • the one or more A processor implements the above audio detection method.
  • the present disclosure also provides a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform the above audio detection method.
  • the present disclosure also provides a computer program product, including a computer program carried on a non-transitory computer-readable medium, where the computer program includes program code for executing the above audio detection method.
  • Figure 1 is a schematic flow chart of an audio detection method provided by an embodiment of the present disclosure
  • Figure 2 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • Figure 3 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • Figure 4 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • Figure 5 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • Figure 6 is a schematic structural diagram of an audio detection device provided by an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the term “include” and its variations are open-ended, ie, “including but not limited to.”
  • the term “based on” means “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • Figure 1 is a schematic flow chart of an audio detection method provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure is adapted to automatically obtain statistical data of music events in audio.
  • This method can be performed by an audio detection device provided by an embodiment of the present disclosure.
  • the audio detection device can be implemented in the form of software and/or hardware, and implemented through electronic equipment.
  • the electronic equipment can be a mobile terminal or a personal computer (Personal Computer, PC) terminal, etc.
  • the method in this embodiment includes:
  • the electronic device may be any electronic device with audio and video playback functions and/or audio and video processing functions, and may include but is not limited to smart phones, wearable devices, computers, servers, and other devices.
  • the above-mentioned electronic device can obtain the detected audio in a variety of ways.
  • the detected audio can be collected in real time through an audio collection device, or the detected audio can be retrieved from a preset storage location or other devices.
  • the embodiment of the present disclosure does not limit the method of obtaining the detected audio.
  • Detected audio refers to audio that requires statistical data detection, which can include but is not limited to audio in live videos, audio in videos, broadcast audio, etc., and is not limited to this.
  • obtaining the detected audio may be to extract audio data from a video (such as a real-time live video or an offline video) as the detected audio.
  • the detected audio is divided into multiple audio segments, and recognition processing is performed on each audio segment.
  • the detected audio is real-time data
  • the audio collected in real time is divided into audio segments in sequence, and the obtained audio segments are recognized and processed in real time;
  • the audio segments can be divided according to the timing of the audio segments.
  • Each audio segment is identified and processed in turn.
  • the obtained multiple audio segments may be processed in parallel to improve processing efficiency.
  • the audio segment may be audio data with a preset time length, and the audio segment may include one or more of music, environmental sounds, speech, noise and other events.
  • the duration of the audio segment may be preset, for example, determined based on the recognition accuracy, and is not limited to this.
  • the duration of the audio segment may be 20 seconds.
  • Music events may refer to sound events characterized by one or more of elements such as rhythm (such as beat, tempo, and articulation), pitch (such as melody and harmony), dynamics (such as the volume of a sound or note), and may include But it is not limited to events such as background music, a cappella singing, etc.
  • At least one sound feature can be extracted through any feature extraction method (such as Mel cepstrum coefficient extraction method, linear prediction coefficient extraction method, etc.), Compare the extracted sound features with music features in a music database, and determine whether the audio segment contains music events based on the comparison results, where the music database may refer to a database containing multiple music features.
  • the audio segment can be recognized through a music recognition model, and whether the audio segment contains a music event is determined based on the recognition result.
  • the music recognition model can use music, chat sounds, noise, etc.
  • Audio data including music are used as positive samples, and samples excluding music, such as chat sounds and noise audio data, are used as negative samples.
  • the music recognition model is trained based on the above sample data. When the training end conditions are met, a model with music event recognition function is obtained. This embodiment does not limit the method of identifying music events.
  • the metadata information is the description information of the music metadata including the music characteristics in the music event.
  • the metadata information may be a tag formed by multiple description information of the music metadata, wherein the description information of the music metadata It may include but is not limited to music spectrum information, music name, music type, singer, composer and other information, which is not limited.
  • the metadata information may be in the form of music name-singer/performer.
  • Music metadata is characterized by metadata information. Metadata information is unique and can uniquely represent music metadata. Using metadata information as a statistical dimension can improve the reliability of music event statistics in audio, and then use metadata information to Statistics of music events in multiple audio segments can improve the accuracy of statistical data corresponding to music events.
  • determining the metadata information matching the music event may be by extracting music features in the music event, matching the music features with music features corresponding to multiple music metadata, and matching the successfully matched music elements.
  • the metadata information of the data is determined as metadata information matching the music event.
  • music features include but are not limited to feature information such as pitch, beat, lyrics, etc.
  • the above feature information is extracted for the music event, and the extracted feature information is matched in the preset metadata database to obtain Metadata information matching the music event, wherein the preset metadata database may contain multiple metadata information and feature information corresponding to the metadata information.
  • the music signature may be an audio fingerprint signature.
  • the audio fingerprint features of the audio segment are extracted, matched in the fingerprint feature database based on the audio fingerprint features, and metadata information matching the music event is determined, where the fingerprint feature database includes music metadata and corresponding fingerprint features.
  • the music metadata can correspond to multiple fingerprint features, and the music metadata is divided into multiple music sub-data, and between the multiple music sub-data There may be some overlap of data, and the fingerprint characteristics corresponding to each music sub-data are determined separately.
  • matching the audio fingerprint characteristics of the audio segment with the fingerprint characteristics of the music metadata may be to match the audio fingerprint characteristics of the audio segment with the fingerprint characteristics of multiple music subdata in the music metadata respectively.
  • the audio fingerprint feature of the music sub-data is successfully matched with the fingerprint feature of any music sub-data, then it is determined that the music sub-data
  • the metadata information corresponding to the music event in the audio segment is determined according to the metadata information of the music metadata to which it belongs.
  • Statistical data is the result of counting music events in multiple audio segments, which is music statistics.
  • the statistical data may include but is not limited to multiple music playback durations in the detected audio, music playback start time and music playback stop time, multiple receiving users during the music playback process (such as audio listening users or viewing users of the video to which the audio belongs).
  • Quantity and other information, the type of statistical data in statistical data can be determined according to business needs, and there is no limit to this.
  • the audio duration corresponding to all music events corresponding to each metadata information in the detected audio can be counted based on the metadata information corresponding to each music event; it can also be based on the metadata information corresponding to the music event. and the timestamp of the music event, determine the continuous music events corresponding to the metadata information, and the number of continuous music events corresponding to each metadata information in the detected audio, to obtain the application status of music in the detected audio; it can also be statistics
  • the audio interval corresponding to each metadata information in the detected audio, as well as the number of receiving users for each audio interval, are used to evaluate the traffic-draining ability of the music corresponding to each metadata information.
  • the music event after obtaining the statistical data, it may also include: obtaining the music metadata corresponding to the music event according to the metadata information corresponding to each music event, and comparing the music metadata in the audio segment according to the music metadata corresponding to the music event.
  • the music event is repaired and the repaired music event is obtained to avoid the noise included in the detected audio interfering with the music event and causing the music event to be unclear.
  • repairing the music event in the audio segment according to the music metadata corresponding to the music event may intercept the music sub-data corresponding to the music event in the music metadata, and replace the audio data of the music event based on the music sub-data.
  • the statistical data may also include: performing operations such as cropping and splicing the audio segments according to the metadata information corresponding to the music events in each audio segment to obtain one or more new audio segments. For example, the Audio data corresponding to the same metadata information in the detected audio is trimmed and spliced.
  • the audio detection method achieves preliminary identification of music events in multiple audio segments by acquiring audio segments in the detected audio and identifying music events in the audio segments; and determines elements matching the music events.
  • Data information realizes the matching and acquisition of reference data, providing a reference basis for obtaining statistical data; statistics of music events in multiple audio segments are performed based on the metadata information obtained by matching, and statistical data in the detected audio is obtained, realizing the Audio identifies and counts music dimensions, which facilitates subsequent analysis of the detected audio based on statistical data, and enables accurate acquisition of statistical data on music events.
  • FIG. 2 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • the method of this embodiment can be combined with multiple solutions of the audio detection method provided in the above embodiments.
  • identifying music events in the audio segment includes: The audio segment is input into a pre-trained music recognition model to obtain a music event recognition result output by the music recognition model, wherein the music recognition model is trained based on audio samples and event tags corresponding to the audio samples.
  • the method in this embodiment includes:
  • the music recognition model has the ability to identify music events in audio data. For an input audio segment, it can identify whether the audio segment includes a music event.
  • the training process of the music recognition model may include: obtaining audio samples and event tags corresponding to the audio samples, where the audio samples may include a variety of different sound events, such as music, laughter, chat, noise and other events, correspondingly , the event tag corresponding to the audio sample can be an event identifier, such as a music identifier, a laughter identifier, a noise identifier, etc. Audio samples including music events are regarded as positive samples, and audio samples including laughter, chat, noise and other events are regarded as negative samples.
  • the event labels corresponding to the positive and negative samples can be positive and negative respectively.
  • the initial training model is trained based on the audio samples corresponding to the positive and negative samples and the event labels corresponding to the audio samples to obtain the music recognition model.
  • the initial training model may include but is not limited to long short-term memory network model, support vector machine model, etc., which are not limited here.
  • the audio segments can be input into the pre-trained music recognition model to classify or identify the sound events in the audio segments.
  • the music recognition model can quickly output the music event recognition results.
  • the pre-trained music recognition model can be used in online applications of audio detection devices without complex calculations, and can quickly obtain music event recognition results, thus improving the speed of audio detection.
  • the music recognition model may also output the start and end timestamps of the recognized music events in the audio segment.
  • the training samples of the music recognition model also include the start and end timestamps corresponding to the music event tags in the audio samples. The music recognition model trained through the above training samples can identify whether the input audio segment includes music events, and the location of the music events. Start and end timestamps.
  • the method further includes: determining whether the duration of the music event in the audio segment is greater than a first preset duration, and if the duration of the music event in the audio segment is not greater than For the first preset duration, the music event is unmarked.
  • the duration of the music event may be determined based on the start and end timestamps of the music event.
  • the music event recognition result includes a music event
  • the audio segment in the detected audio can It can include events such as playing music or singing, but it may also be caused by interfering sounds.
  • the interfering sounds can be short text message alerts or mobile phone ringtones.
  • the interfering sounds may also include music. This situation indicates that the audio The music events in the segment are not real music events, and the music events need to be unmarked to avoid misjudgment of music events.
  • the duration of the music event in the audio segment is judged. If the duration of the music event in the audio segment is greater than the first preset duration, it indicates that the music event meets the music standard, and the music event mark of the audio segment remains unchanged; if the audio If the duration of the music event in the segment is less than or equal to the first preset duration, it indicates that the music event does not meet the music standards, and the music event mark of the audio segment is cancelled.
  • the first preset time period may be set based on historical experience. For example, the first preset time period may be 6 seconds.
  • the audio detection method inputs audio segments into a pre-trained music recognition model, classifies or identifies sound events in the audio segments, and obtains music event recognition results. Remove music events that are less than or equal to the first preset duration, reduce the interference of misidentified music events, and reduce the increase in statistical workload caused by short-term music events.
  • FIG. 3 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • the method of this embodiment can be combined with multiple solutions of the audio detection method provided in the above embodiments.
  • determining metadata information matching the music event includes: for an audio segment containing a music event, extracting audio fingerprint features of the audio segment; based on the audio fingerprint The features are matched in a fingerprint feature database to determine metadata information matching the music event, where the fingerprint feature database includes music metadata and corresponding fingerprint features.
  • the method in this embodiment includes:
  • S330 Perform matching in a fingerprint feature database based on the audio fingerprint features to determine metadata information matching the music event, where the fingerprint feature database includes music metadata and corresponding fingerprint features.
  • the audio fingerprint feature refers to the digital feature of the music event, which is the music fingerprint feature and is unique. Audio fingerprint features can be extracted from the audio segment through audio fingerprint technology, which includes but is not limited to the Philips algorithm or the Shazam algorithm.
  • the fingerprint feature database refers to a database containing music metadata and fingerprint features, which can pre-store multiple music metadata and fingerprint features corresponding to music metadata.
  • the fingerprint features corresponding to the music metadata can be used to match the audio fingerprint features. If the match is successful, the metadata matching the music event will be obtained.
  • the fingerprint features may include but are not limited to frequency parameters and time parameters corresponding to the frequency spectrum of the music metadata.
  • extracting the audio fingerprint features of the audio segment includes: intercepting the audio segment according to the start and end timestamps of the music events in the audio segment to obtain the intercepted audio segment, and extracting the intercepted audio segment. Audio fingerprint characteristics of the audio segment.
  • the identification result of the music event includes the start and end timestamps of the music events, and the start timestamp and end timestamp of the music events in the audio segment are obtained; and the corresponding audio is obtained based on the start timestamps and end timestamps of the music events. Intercept the segment and extract the audio data corresponding to the music event in the audio segment.
  • the audio data corresponding to the intercepted music event is determined to determine the audio fingerprint characteristics, avoiding the audio data of the non-music event. It interferes with audio fingerprint features and at the same time reduces the amount of audio data required to determine audio fingerprint features, which is beneficial to the rapid extraction of audio fingerprint features.
  • extracting the audio fingerprint characteristics of the audio segment includes: extracting audio data of the track where the music event is located in the audio segment, based on the audio data of the track where the music event is located.
  • Data extraction audio fingerprint features include: extracting audio data of the track where the music event is located in the audio segment, based on the audio data of the track where the music event is located.
  • Data extraction audio fingerprint features include: extracting audio data of the track where the music event is located in the audio segment, based on the audio data of the track where the music event is located.
  • Data extraction audio fingerprint features includes: extracting audio data of the track where the music event is located in the audio segment, based on the audio data of the track where the music event is located.
  • Data extraction audio fingerprint features includes: extracting audio data of the track where the music event is located in the audio segment, based on the audio data of the track where the music event is located.
  • Data extraction audio fingerprint features includes: extracting audio data of the track where the music event is located in the audio segment, based on the audio data of the track
  • Different audio tracks can include music events at the same time, or one or more audio tracks can include music events independently.
  • the audio detection method provided by the embodiment of the present disclosure extracts the audio fingerprint features of the audio segment from the audio segment containing the music event, and performs matching in the fingerprint feature library based on the extracted audio fingerprint features to determine the metadata that matches the music event.
  • Information, metadata information corresponding to music events is obtained through fingerprint feature database matching, and the processing speed is fast, which can save time in audio detection.
  • FIG 4 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure.
  • the method of this embodiment can be combined with multiple solutions of the audio detection method provided in the above embodiments.
  • determining the statistical data in the detected audio based on the metadata information includes: according to the start and end timestamps of the music events in each audio segment, corresponding to the same metadata information Music events are merged to obtain statistical data in the detected audio.
  • the method in this embodiment includes:
  • the start and end timestamps of music events refer to the start timestamp and end timestamp of music events. If the metadata information of the music events is the same, it means that the above-mentioned multiple music events are part of the same piece of music or song. Music events with the same metadata information can be merged, and the statistical data in the detected audio is determined based on the merged music events. , avoid recognition errors caused by dividing the detected audio data into audio segments, and improve the accuracy of statistical data.
  • music events corresponding to the same metadata information are merged to obtain statistical data in the detected audio, including: for adjacent music event, if the metadata information corresponding to the adjacent music events is the same, and the interval duration of the adjacent music events is less than the second preset duration, then the adjacent music events will be merged; if the adjacent music events If the metadata information corresponding to the events is different, or the metadata information corresponding to the adjacent music events is the same, and the interval duration of the adjacent music events is greater than or equal to the second preset duration, then the adjacent music events will not be processed. Merge.
  • the adjacent music event may be a music event in an adjacent audio segment or an adjacent music event within an audio segment, which is not limited here.
  • adjacent music events if the metadata information corresponding to the adjacent music events is the same, and the interval duration of the adjacent music events is less than the second preset duration, it indicates that the adjacent music events belong to the same song and the two music events The intervals between are normal singing or playback pauses, or recognition errors due to audio segment division, then adjacent music events can be merged to calibrate the recognized music events; if the metadata information corresponding to adjacent music events are different, indicating that the adjacent music events do not belong to the same song, the adjacent music events will not be merged to distinguish statistics between different songs; if the metadata information corresponding to the adjacent music events is the same, and the interval between adjacent music events is longer than Or equal to the second preset duration, indicating that adjacent music events belong to the same song but have a long pause time. For example, if the same song is played twice, adjacent music events will not be merged to avoid long playback intervals. The statistics of the same song are entered into the same statistics.
  • the audio detection method provided by the embodiment of the present disclosure merges the music events corresponding to the same metadata information according to the start and end timestamps of the music events in each audio segment, so that the audio segment containing the merged music events can be obtained accurately, so as to accurately obtain the Detect music events in audio to improve statistical accuracy.
  • the detected audio is the audio in the live video; the method It also includes: determining the viewing data of the live broadcast interval corresponding to each metadata information in the statistical data.
  • the live video can be a live video collected in real time or a historical live video. Extract audio from the live video to obtain the detected audio. By identifying and counting music events on audio extracted from live videos, the usage of music metadata in live videos can be obtained.
  • the viewing data of the live broadcast interval refers to the viewing statistics of the live broadcast room within the preset time period, which can include but is not limited to the total number of views, the number of independent visits, the average viewing time and other data.
  • Statistical data can be used as viewing data matching conditions. According to the matching conditions, the viewing data of the live broadcast interval is matched in the live broadcast database to achieve accurate acquisition of viewing data.
  • the live broadcast database can include but is not limited to real-time statistical video viewing data. Through the statistical data of music metadata in live videos and the video viewing data corresponding to the statistical data, it is used to evaluate the role of music metadata in attracting traffic in live videos, or to predict the development trend of music metadata.
  • Figure 5 is a schematic flow chart of another audio detection method provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment provides an example to illustrate the audio detection method in the above embodiment.
  • the method in this embodiment includes:
  • the audio in the live stream is segmented to obtain multiple audio stream slices (i.e., the above-mentioned audio segments). Multiple audio stream slices can be processed in parallel;
  • Recognizing music events on audio stream slices includes: extracting short-term features and long-term features from each audio stream slice, and reducing the dimensionality of the extracted short-term features and long-term features through a dimensionality reduction algorithm to remove short-term features and Redundant information of long-term features to obtain main features.
  • the dimensionality of the reduced features is greatly reduced, and the performance will be improved to a certain extent.
  • SVM Support Vector Machine
  • short-term features include at least one of the following features: Perceptual Linear Predictive Coefficients (PLP), Linear Predictive Cepstrum Coefficients (LPCC), Linear Frequency Cepstral Coefficients (Linear Frequency Cepstral) Coefficients (LFCC), Pitch, Short-time Energy (STE), Sub-Band Energy Distribution (SBED), Brightness (BR) and Bandwidth (BW).
  • Long-term characteristics include at least one of the following characteristics: Spectrum Flux (SF), Long-Term Average Spectrum (LTAS), and LPC entropy (LPC entropy).
  • the recognition result is a music event, continue to determine whether the duration of the current music event is greater than the first preset duration; if the recognition result is not a music event, then cancel the marking of the music event. If the duration of the current music event is greater than the first preset duration, continue to extract the audio fingerprint features of the music event; if the duration of the current music event If the duration of the event is not greater than the first preset duration, the music event is unmarked.
  • the audio fingerprint features are extracted from music events through the audio fingerprint extraction algorithm, and the audio fingerprint features are matched in the fingerprint feature library to obtain metadata information. If the metadata information of adjacent music events is the same, that is, the adjacent music events are the same song, and the interval between adjacent music events is less than the second preset duration, it indicates that the two belong to the same song and there is just a normal singing or playback pause in between. , then merge adjacent music events; if the metadata information is the same and the interval between adjacent music events is not less than the second preset duration, it indicates that although they belong to the same song, the pause time is long and they are not suitable for merging. processing, adjacent music events will not be merged. If the metadata information is not the same, that is, the adjacent music events are not the same song, the adjacent music events will not be merged.
  • the method further includes: obtaining statistical data of the merged music events, such as the playback start time, playback end time and other data of the music events. Statistics can be used for music rights billing.
  • FIG. 6 is a schematic structural diagram of an audio detection device provided by an embodiment of the present disclosure. As shown in Figure 6, the device includes:
  • the music event identification module 610 is configured to obtain the audio segment in the detected audio and identify the music event in the audio segment; the statistical data determination module 620 is configured to determine the metadata information matching the music event, based on the The metadata information determines statistics in the detected audio.
  • the music event identification module 610 may also be configured to:
  • the audio segment is input into a pre-trained music recognition model to obtain a music event recognition result output by the music recognition model, wherein the music recognition model is trained based on audio samples and event tags corresponding to the audio samples.
  • the device may also be configured to:
  • the statistical data determination module 620 may also include:
  • the fingerprint feature extraction unit is configured to extract the audio fingerprint features of the audio segment for the audio segment containing the music event; the metadata matching unit is configured to perform matching in the fingerprint feature library based on the audio fingerprint features and determine the Metadata information matching the music event, wherein the fingerprint feature database includes music metadata and corresponding fingerprint features.
  • the fingerprint feature extraction unit may also be configured to:
  • the audio segment is intercepted to obtain the interception Audio segment, extract the audio fingerprint feature of the intercepted audio segment; or, extract the audio data of the audio track where the music event is located in the audio segment, and extract the audio fingerprint feature based on the audio data of the audio track where the music event is located.
  • the statistical data determination module 620 may also include:
  • the data merging unit is configured to merge music events corresponding to the same metadata information according to the start and end timestamps of the music events in each audio segment to obtain statistical data in the detected audio.
  • the data merging unit may also be configured to:
  • the adjacent music events For adjacent music events, if the metadata information corresponding to the adjacent music events is the same, and the interval duration of the adjacent music events is less than the second preset duration, the adjacent music events will be merged; if If the metadata information corresponding to the adjacent music events is different, or the metadata information corresponding to the adjacent music events is the same, and the interval duration of the adjacent music events is greater than or equal to the second preset duration, then the metadata information corresponding to the adjacent music events is not the same. Adjacent music events are merged.
  • the detected audio is the audio in the live video; the device may also be configured to: determine the viewing data of the live broadcast interval corresponding to each metadata information in the statistical data .
  • the audio detection device provided by the embodiment of the present disclosure can execute the audio detection method provided by any embodiment of the present disclosure, and has corresponding functional modules and effects for executing the audio detection method.
  • the multiple units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned divisions, as long as they can achieve the corresponding functions; in addition, the names of the multiple functional units are only for the convenience of distinguishing each other. , are not used to limit the protection scope of the embodiments of the present disclosure.
  • Terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (Portable Android Device, PAD), portable multimedia players Mobile terminals such as (Portable Media Player, PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital television (TV), desktop computers, etc.
  • PDA Personal Digital Assistant
  • PAD Portable Multimedia Players Mobile terminals
  • PMP Portable Multimedia Player
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital television (TV), desktop computers, etc.
  • TV digital television
  • the electronic device 400 shown in FIG. 7 is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present disclosure.
  • the electronic device 400 may include a processing device (such as a central processing unit, a graphics processor, etc.) 401, which may process data according to a program stored in a read-only memory (Read-Only Memory, ROM) 402 or from a storage device. 408 loads the program in the random access memory (Random Access Memory, RAM) 403 to perform various appropriate actions and processes. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored.
  • Processing device 401, ROM 402 and RAM 403 They are connected to each other via bus 404.
  • An input/output (I/O) interface 405 is also connected to bus 404.
  • the following devices can be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a Liquid Crystal Display (LCD) , an output device 407 such as a speaker, a vibrator, etc.; a storage device 408 including a magnetic tape, a hard disk, etc.; and a communication device 409.
  • the communication device 409 may allow the electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data.
  • FIG. 7 illustrates electronic device 400 with various means, implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via communication device 409, or from storage device 408, or from ROM 402.
  • the processing device 401 When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
  • the electronic device provided by the embodiment of the present disclosure belongs to the same concept as the audio detection method provided by the above embodiment.
  • Technical details that are not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same effect as the above embodiment. .
  • Embodiments of the present disclosure provide a computer storage medium on which a computer program is stored.
  • the program is executed by a processor, the audio detection method provided by the above embodiments is implemented.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof.
  • Examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard drives, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM) or flash memory), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • the computer-readable signal medium may also be any computer-readable medium other than computer-readable storage media that can transmit, Propagate or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
  • Program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (Radio Frequency, RF), etc., or any suitable combination of the above.
  • the client and server can communicate using any currently known or future developed network protocol, such as HyperText Transfer Protocol (HTTP), and can communicate with digital data in any form or medium.
  • HTTP HyperText Transfer Protocol
  • Communications e.g., communications network
  • Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any current network for knowledge or future research and development.
  • LANs Local Area Networks
  • WANs Wide Area Networks
  • the Internet e.g., the Internet
  • end-to-end networks e.g., ad hoc end-to-end networks
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs.
  • the electronic device executes the above-mentioned one or more programs.
  • Obtain audio segments in the detected audio identify music events in the audio segments; determine metadata information matching the music events, and determine statistical data in the detected audio based on the metadata information.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C" or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user computer through any kind of network, including a LAN or WAN, or may be connected to an external computer (eg, through the Internet using an Internet service provider).
  • each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block in the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration may It can be implemented with a dedicated hardware-based system that performs the specified function or operation, or it can be implemented with a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of the unit/module does not constitute a limitation on the unit itself.
  • exemplary types of hardware logic components include: field programmable gate array (Field Programmable Gate Array, FPGA), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), application specific standard product (Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programming Logic Device (CPLD), etc.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. Examples of machine-readable storage media would include an electrical connection based on one or more wires, a portable computer disk, a hard drive, RAM, ROM, EPROM or flash memory, optical fiber, CD-ROM, optical storage device, magnetic storage device, or Any suitable combination of the above.
  • Example 1 provides an audio detection method, which includes:
  • Metadata information matching the music event is determined, and statistics in the detected audio are determined based on the metadata information.
  • Example 2 provides an audio detection method, further including:
  • the identifying music events in the audio segment includes:
  • the audio segment is input into a pre-trained music recognition model to obtain a music event recognition result output by the music recognition model, wherein the music recognition model is trained based on audio samples and event tags corresponding to the audio samples.
  • Example 3 provides an audio detection method, Also includes:
  • the method further includes:
  • Example 4 provides an audio detection method, further including:
  • Determining metadata information matching the music event includes:
  • Matching is performed in a fingerprint feature database based on the audio fingerprint features to determine metadata information matching the music event, where the fingerprint feature database includes music metadata and corresponding fingerprint features.
  • Example 5 provides an audio detection method, further including:
  • the extraction of audio fingerprint features of the audio segment includes:
  • Audio data of the audio track where the music event is located is extracted from the audio segment, and audio fingerprint features are extracted based on the audio data of the audio track where the music event is located.
  • Example 6 provides an audio detection method, further including:
  • Determining statistical data in the detected audio based on the metadata information includes:
  • music events corresponding to the same metadata information are merged to obtain statistical data in the detected audio.
  • Example 7 provides an audio detection method, further including:
  • music events corresponding to the same metadata information are merged to obtain statistical data in the detected audio, including:
  • the adjacent music events will not be merged.
  • Example 8 provides an audio detection method, further including:
  • the detected audio is the audio in the live video
  • the method also includes:
  • the viewing data of the live broadcast interval corresponding to each metadata information in the statistical data is determined.
  • Example 9 provides an audio detection device, which includes:
  • a music event identification module configured to obtain audio segments in the detected audio and identify music events in the audio segments
  • a statistical data determination module is configured to determine metadata information matching the music event, and determine statistical data in the detected audio based on the metadata information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

音频检测方法、装置、存储介质及电子设备（400）。该音频检测方法包括：获取被检测音频中的音频段（S210），识别音频段中的音乐事件（S110，S310，S410）；确定与音乐事件相匹配的元数据信息（S420），基于元数据信息确定被检测音频中的统计数据（S120，S230，S340）。

Description

音频检测方法、装置、存储介质及电子设备
本申请要求在2022年03月08日提交中国专利局、申请号为202210220184.7的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开涉及音频处理技术领域，例如涉及音频检测方法、装置、存储介质及电子设备。
背景技术
随着互联网技术普及，以及音视频的迅速流行，用户可以通过手机、电脑等电子设备播放音视频，例如直播节目、歌曲、有声小说等。
相关技术中至少存在以下技术问题：音频检测方法，无法统计待检测音频中的与音乐相关的统计数据(例如音乐时长、音乐播放起止时间等)。
发明内容
本公开提供了音频检测方法、装置、存储介质及电子设备，以实现对待检测音频中的统计数据的准确获取。
第一方面，本公开提供了一种音频检测方法，包括：
获取被检测音频中的音频段，识别所述音频段中的音乐事件；
确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
第二方面，本公开还提供了一种音频检测装置，包括：
音乐事件识别模块，设置为获取被检测音频中的音频段，识别所述音频段中的音乐事件；
统计数据确定模块，设置为确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
第三方面，本公开还提供了一种电子设备，所述电子设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多 个处理器实现上述的音频检测方法。
第四方面，本公开还提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行上述的音频检测方法。
第五方面，本公开还提供了一种计算机程序产品，包括承载在非暂态计算机可读介质上的计算机程序，所述计算机程序包含用于执行上述的音频检测方法的程序代码。
附图说明
图1为本公开实施例提供的一种音频检测方法流程示意图；
图2为本公开实施例提供的另一种音频检测方法流程示意图；
图3为本公开实施例提供的另一种音频检测方法流程示意图；
图4为本公开实施例提供的另一种音频检测方法流程示意图；
图5为本公开实施例提供的另一种音频检测方法流程示意图；
图6为本公开实施例提供的一种音频检测装置结构示意图；
图7为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
下面将参照附图描述本公开的实施例。虽然附图中显示了本公开的一些实施例，然而本公开可以通过多种形式来实现，提供这些实施例是为了理解本公开。本公开的附图及实施例仅用于示例性作用。
本公开的方法实施方式中记载的多个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有指出，否则应该理解为“一个或多个”。
图1为本公开实施例所提供的一种音频检测方法流程示意图，本公开实施例适应于在音频中自动获取音乐事件的统计数据的情况，该方法可以由本公开实施例提供的音频检测装置来执行，该音频检测装置可以通过软件和/或硬件的形式实现，通过电子设备来实现，该电子设备可以是移动终端或个人电脑(Personal Computer，PC)端等。如图1，本实施例的方法包括：
S110、获取被检测音频中的音频段，识别所述音频段中的音乐事件。
S120、确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
在本公开实施例中，电子设备可为具有音视频播放功能和/或音视频处理功能的任何电子设备，可以包括但不限于智能手机、可穿戴设备、计算机、服务器等设备。上述电子设备可以通过多种方式获取被检测音频。例如，可以通过音频采集装置实时采集被检测音频，也可以从预设存储位置或者其他设备调取被检测音频，本公开实施例对被检测音频的获取方法不进行限定。
被检测音频指的是需要进行统计数据检测的音频，可以包括但不限于直播视频中的音频、视频中的音频、广播音频等，对此不作限定。相应的，在一些实施例中，获取被检测音频可以是从视频(例如实时直播视频或者离线视频)中抽取音频数据，作为被检测音频。
为了提高音频的识别精度和识别效率，将被检测音频划分为多个音频段，对每一个音频段进行识别处理。在被检测音频为实时数据的情况下，对实时采集的音频依次划分音频段，并对得到的音频段进行实时识别处理；在被检测音频为离线数据的情况下，可根据划分音频段的时序依次对每个音频段进行识别处理。在一些实施例中，对于离线音频数据，可以是对得到的多个音频段进行并行数据，以提高处理效率。
本实施例中，音频段可以是具有预设时间长度的音频数据，该音频段可包含音乐、环境声音、语音、噪声等事件中的一项或多项。其中，音频段的时长可预先设置，例如根据识别精度确定，对此不作限定，示例性的，音频段的时长可以是20s。音乐事件指的可以是由节奏(例如拍子、节拍和发音)、音调(例如旋律以及和声)、力度(例如声音或音符的音量)等元素中的一个或多个表征的声音事件，可以包括但不限于背景音乐、清唱等的事件。
为了识别出音频段中的音乐事件，需要对音频段进行特征分析，以判断该音频段中是否包含音乐事件。在一些实施例中，可以通过任何特征提取方式(例如梅尔倒谱系数提取方式、线性预测系数提取方式等)提取至少一个声音特征， 将提取的声音特征与音乐数据库中的音乐特征进行比较，根据比较结果确定该音频段中是否包含音乐事件，其中，音乐数据库指的可以是包含多种音乐特征的数据库。在一些实施例中，可以通过音乐识别模型对音频段进行识别，根据识别结果确定该音频段中是否包含音乐事件，其中，音乐识别模型可以将音乐、聊天声音、噪音等作为训练样本，其中，包括音乐的音频数据作为正样本，不包括音乐的样本，例如聊天声音、噪音音频数据作为负样本。基于上述样本数据对音乐识别模型进行训练，在满足训练结束条件的情况下，得到具有音乐事件识别功能的模型。本实施例对识别音乐事件的方法不作限定。
在上述实施例的基础上，对于存在音乐事件的音频段，分别确定与音乐事件相匹配的元数据信息，以得到被检测音频的统计数据，对于不包括音乐事件的音频段，无需对该音频段进行元数据信息的匹配，避免无效处理导致的计算资源浪费。元数据信息是包括音乐事件中音乐特征的音乐元数据的描述信息，在一些实施例中，元数据信息可以是由音乐元数据的多个描述信息形成的标签，其中，音乐元数据的描述信息可以包括但不限于音乐的频谱信息、音乐名称、音乐类型、演唱者、作曲人等信息，对此不作限定，示例性的，元数据信息可以是音乐名称-演唱者/演奏者的形式。通过元数据信息表征音乐元数据，元数据信息具有唯一性，可以对音乐元数据进行唯一表示，将元数据信息作为统计维度，可以提高音频中音乐事件统计的可靠性，进而通过元数据信息对多个音频段中的音乐事件进行统计，可以提高音乐事件对应统计数据的准确度。
在一些实施例中，确定音乐事件相匹配的元数据信息，可以是通过提取音乐事件中的音乐特征，将该音乐特征与多个音乐元数据对应的音乐特征相匹配，将匹配成功的音乐元数据的元数据信息确定为与音乐事件相匹配的元数据信息。在一些实施例中，音乐特征包括但不限于音调、节拍、歌词等的特征信息，相应的，对音乐事件进行上述特征信息提取，将提取的特征信息在预设元数据库中进行匹配，得到与音乐事件相匹配的元数据信息，其中，预设元数据库中可以包含多个元数据信息以及元数据信息对应的特征信息。在一些实施例中，音乐特征可以是音频指纹特征。相应的，提取音频段的音频指纹特征，基于音频指纹特征在指纹特征库中进行匹配，确定与音乐事件相匹配的元数据信息，其中，指纹特征库中包括音乐元数据和对应的指纹特征。对于音频指纹特征与确定音频指纹特征的音频段一一对应，本实施例中音乐元数据可以是对应多个指纹特征，将音乐元数据划分为多个音乐子数据，多个音乐子数据之间可存在部分数据的重叠，分别确定每一音乐子数据对应的指纹特征。相应的，将音频段的音频指纹特征与音乐元数据的指纹特征相匹配，可以是将音频段的音频指纹特征分别与音乐元数据中多个音乐子数据的指纹特征分别进行匹配，若音频段的音频指纹特征与任一音乐子数据的指纹特征匹配成功，则确定将该音乐子数 据所属音乐元数据的元数据信息确定为音频段中音乐事件对应的元数据信息。
统计数据是对多个音频段中音乐事件进行统计的结果，即为音乐统计数据。其中，统计数据可以包括但不限于被检测音频中多个音乐播放时长、音乐播放开始时间和音乐播放停止时间、多个音乐播放过程中接收用户(例如音频收听用户或者音频所属视频的观看用户)数量等信息，统计数据中的统计数据类型可根据业务需求确定，对此不作限定。
在一些实施例中，可以是基于每个音乐事件对应的元数据信息，统计被检测音频中每个元数据信息对应的所有音乐事件对应的音频时长；还可以是根据音乐事件对应的元数据信息和音乐事件的时间戳，确定元数据信息对应的连续音乐事件，以及被检测音频中每个元数据信息对应的连续音乐事件的数量，以得到被检测音频中音乐的应用情况；还可以是统计被检测音频中每个元数据信息对应的音频区间，以及每个音频区间的接收用户数量，以评估每个元数据信息对应音乐的引流能力等。
在上述实施例的基础上，在得到统计数据之后，还可以包括：根据每个音乐事件对应的元数据信息获取音乐事件对应的音乐元数据，根据音乐事件对应的音乐元数据对音频段中的音乐事件进行修复，得到修复后的音乐事件，以避免被检测音频中包括的噪声干扰导致音乐事件，导致音乐事件不清晰的情况。其中，根据音乐事件对应的音乐元数据对音频段中的音乐事件进行修复，可以是截取音乐事件在音乐元数据中对应的音乐子数据，基于音乐子数据替代音乐事件的音频数据。或者，在得到统计数据之后，还可以包括：根据每个音频段中的音乐事件对应的元数据信息，对音频段进行裁剪、拼接等操作，得到一个或多个新的音频段，例如可以将被检测音频中同一元数据信息对应的音频数据进行剪裁和拼接。
本公开实施例提供的音频检测方法，通过获取被检测音频中的音频段，识别音频段中的音乐事件，实现了对多个音频段中音乐事件的初步识别；确定与音乐事件相匹配的元数据信息，实现了参考数据的匹配获取，为得到统计数据提供了参考基础；根据匹配得到的元数据信息对多个音频段中音乐事件进行统计，得到被检测音频中的统计数据，实现了对音频进行音乐维度的识别和统计，便于基于统计数据对被检测音频进行后续分析，实现了音乐事件的统计数据的准确获取。
参考图2，图2为本公开实施例提供的另一种音频检测方法流程示意图，本实施例的方法与上述实施例中提供的音频检测方法中多个方案可以结合。本实施例提供的音频检测方法中，所述识别所述音频段中的音乐事件，包括：将所 述音频段输入至预先训练的音乐识别模型中，得到所述音乐识别模型输出的音乐事件识别结果，其中，所述音乐识别模型基于音频样本与所述音频样本对应的事件标签训练得到。
如图2，本实施例的方法包括：
S210、获取被检测音频中的音频段。
S220、将所述音频段输入至预先训练的音乐识别模型中，得到所述音乐识别模型输出的音乐事件识别结果，其中，所述音乐识别模型基于音频样本与所述音频样本对应的事件标签训练得到。
S230、确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
本实施例中，音乐识别模型具有在音频数据中识别音乐事件的能力，对于输入的音频段，可识别该音频段中是否包括音乐事件。相应的，音乐识别模型的训练过程可以包括：获取音频样本和音频样本对应的事件标签，其中，音频样本可以包括多种不同的声音事件，例如音乐、笑声、聊天、噪音等事件，相应的，音频样本对应的事件标签可以是事件标识，例如音乐标识、笑声标识、噪声标识等。将包括音乐事件的音频样本作为正样本，将包括笑声、聊天、噪音等事件的音频样本作为负样本，相应的，正负样本分别对应的事件标签可以是正和负。基于正负样本对应的音频样本和音频样本对应的事件标签对初始训练模型进行训练，得到音乐识别模型。其中，初始训练模型可以包括但不限于长短期记忆网络模型、支持向量机模型等，在此不做限定。在音乐识别模型训练完成后，可以将音频段输入至预先训练的音乐识别模型中，对音频段中的声音事件进行分类或识别，音乐识别模型可以快速输出音乐事件识别结果。预先训练的音乐识别模型在音频检测装置线上应用中，无需复杂计算，可以快速得到的音乐事件识别结果，从而提高音频检测的速度。
在上述实施例的基础上，音乐识别模型还可以是输出识别出的音乐事件在音频段中的起止时间戳。相应的，音乐识别模型的训练样本中，还包括音频样本中音乐事件标签对应的起止时间戳，通过上述训练样本训练得到的音乐识别模型可识别输入音频段中是否包括音乐事件，以及音乐事件的起止时间戳。
在识别所述音频段中的音乐事件之后，所述方法还包括：确定所述音频段中的音乐事件的时长是否大于第一预设时长，若所述音频段中的音乐事件的时长不大于第一预设时长，则取消标记所述音乐事件。其中，音乐事件的时长可以是基于音乐事件的起止时间戳确定。
当音乐事件识别结果中包括音乐事件时，表明该被检测音频中的音频段可 能包括播放音乐或唱歌等事件，但也可能是由干扰声音产生的，例如干扰声音可以为时长较短的短信提示音或手机铃声等，干扰声音中也可能包含音乐，此种情况表明该音频段中的音乐事件并非真正的音乐事件，需要取消标记音乐事件，避免音乐事件误判的情况发生。
对音频段中的音乐事件的时长进行判断，若音频段中的音乐事件的时长大于第一预设时长，表明该音乐事件符合音乐的标准，则音频段的音乐事件标记保持不变；若音频段中的音乐事件的时长小于或等于第一预设时长，表明该音乐事件不符合音乐的标准，则取消对该音频段的音乐事件标记。第一预设时长可以根据历史经验设定，例如，第一预设时长可以为6s。
本公开实施例提供的音频检测方法，通过将音频段输入至预先训练的音乐识别模型中，对音频段中的声音事件进行分类或识别，得到音乐事件识别结果。去除小于或等于第一预设时长的音乐事件，减少误识别音乐事件的干扰，以及减少短时间音乐事件对统计工作量的增加。
参考图3，图3为本公开实施例提供的另一种音频检测方法流程示意图，本实施例的方法与上述实施例中提供的音频检测方法中多个方案可以结合。本实施例提供的音频检测方法中，所述确定与所述音乐事件相匹配的元数据信息，包括：对于包含音乐事件的音频段，提取所述音频段的音频指纹特征；基于所述音频指纹特征在指纹特征库中进行匹配，确定与所述音乐事件相匹配的元数据信息，其中，所述指纹特征库中包括音乐元数据和对应的指纹特征。如图3，本实施例的方法包括：
S310、获取被检测音频中的音频段，识别所述音频段中的音乐事件。
S320、对于包含音乐事件的音频段，提取所述音频段的音频指纹特征。
S330、基于所述音频指纹特征在指纹特征库中进行匹配，确定与所述音乐事件相匹配的元数据信息，其中，所述指纹特征库中包括音乐元数据和对应的指纹特征。
S340、基于所述元数据信息确定所述被检测音频中的统计数据。
本实施例中，音频指纹特征指的是音乐事件的数字特征，即为音乐指纹特征，具有唯一性。可以通过音频指纹技术对音频段进行音频指纹特征的提取，音频指纹技术包括但不限于Philips算法或Shazam算法等。
指纹特征库指的是包含音乐元数据和指纹特征的数据库，可以预先存储有多个音乐元数据和音乐元数据对应的指纹特征。音乐元数据对应的指纹特征可以用于与音频指纹特征进行匹配，若匹配成功，则得到音乐事件相匹配的元数 据信息，指纹特征可以包括但不限于音乐元数据的频谱对应的频率参数和时间参数。
在上述实施例的基础上，所述提取所述音频段的音频指纹特征，包括：根据音频段中音乐事件的起止时间戳，对所述音频段进行截取，得到截取音频段，提取所述截取音频段的音频指纹特征。在一些实施例中，对音乐事件的识别结果中包括音乐事件的起止时间戳，获取音频段中音乐事件的开始时间戳和结束时间戳；根据音乐事件的开始时间戳和结束时间戳对所在音频段进行截取，提取音频段中与音乐事件对应的音频数据，通过剔除非音乐事件的部分音频数据，仅对截取的音乐事件对应的音频数据确定音频指纹特征，避免了非音乐事件部分的音频数据对音频指纹特征的干扰，同时，减少了确定音频指纹特征的音频数据量，有利于音频指纹特征的快速提取。
在上述实施例的基础上，所述提取所述音频段的音频指纹特征，包括：在所述音频段中提取所述音乐事件所在音轨的音频数据，基于所述音乐事件所在音轨的音频数据提取音频指纹特征。在一些实施例中，被检测音频可以包括多个音轨，即每一音频段包括多个音轨，示例性的，被检测音频中可以包括背景采集音轨和语音采集音轨，在任一音频段中，背景采集音轨中的音频数据可以是背景音乐，语音采集音轨中的音频数据可以是主持人的对话语音数据；示例性的，背景采集音轨中的音频数据可以是噪声，语音采集音轨中的音频数据可以是主持人的唱歌语音。不同音轨中可以是同时包括音乐事件，也可以其中一个或多个音轨中独立包括音乐事件，通过提取音乐事件所在音轨的音频数据，剔除非音乐事件所在音轨的音乐数据，减少了非音乐事件的干扰，有利于提高后续音频指纹特征的提取的准确性。
本公开实施例提供的音频检测方法，通过对包含音乐事件的音频段，提取音频段的音频指纹特征，基于提取的音频指纹特征在指纹特征库中进行匹配，确定与音乐事件相匹配的元数据信息，通过指纹特征库匹配得到音乐事件对应的元数据信息，处理速度快，可以节省音频检测的时间。
参考图4，图4为本公开实施例提供的另一种音频检测方法流程示意图，本实施例的方法与上述实施例中提供的音频检测方法中多个方案可以结合。本实施例提供的音频检测方法中，所述基于所述元数据信息确定所述被检测音频中的统计数据，包括：根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到被检测音频中的统计数据。
如图4，本实施例的方法包括：
S410、获取被检测音频中的音频段，识别所述音频段中的音乐事件。
S420、确定与所述音乐事件相匹配的元数据信息。
S430、根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到被检测音频中的统计数据。
音乐事件的起止时间戳指的是音乐事件的开始时间戳和结束时间戳。若音乐事件的元数据信息相同，则表明上述多个音乐事件为同一首音乐或歌曲的一部分，可以将为同一元数据信息的音乐事件进行合并，基于合并音乐事件确定被检测音频中的统计数据，避免对被检测音频数据进行音频段的划分导致的识别误差，提高统计数据的准确性。
在上述实施例的基础上，所述根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到被检测音频中的统计数据，包括：对于相邻音乐事件，若所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长小于第二预设时长，则将所述相邻音乐事件进行合并；若所述相邻音乐事件对应的元数据信息不同，或者，所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长大于或等于第二预设时长，则不对所述相邻音乐事件进行合并。
相邻音乐事件可以是相邻音频段中的音乐事件，也可以是音频段内的相邻音乐事件，在此不做限定。
示例性的，对于相邻音乐事件，若相邻音乐事件对应的元数据信息相同，且相邻音乐事件的间隔时长小于第二预设时长，表明相邻音乐事件属于同一歌曲并且两音乐事件之间的间隔为正常歌唱或播放停顿，或者由于音频段划分导致的识别误差，则可以将相邻音乐事件进行合并，以便对识别出的音乐事件进行校准；若相邻音乐事件对应的元数据信息不同，表明相邻音乐事件不属于同一歌曲，则不对相邻音乐事件进行合并，以对不同歌曲进行区分统计；若相邻音乐事件对应的元数据信息相同，且相邻音乐事件的间隔时长大于或等于第二预设时长，表明相邻音乐事件属于同一歌曲但中间停顿时间较长，例如同一首歌曲播放两次的情况，则不对相邻音乐事件进行合并，以避免将播放间隔时间较长的同一歌曲统计进同一统计数据。
本公开实施例提供的音频检测方法，通过根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，可以得到包含合并音乐事件的音频段，以便准确获取被检测音频中的音乐事件，提升统计数据的准确度。
在上述实施例的基础上，所述被检测音频为直播视频中的音频；所述方法 还包括：确定所述统计数据中每个元数据信息所对应的直播区间的观看数据。
直播视频可以是实时采集的直播视频，还可以是历史直播视频。从直播视频中进行音频抽取，得到被检测音频。通过对从直播视频中抽取的音频进行音乐事件的识别与统计，以得到直播视频中音乐元数据的使用情况。
直播区间的观看数据指的是预设时间段内直播间的观看统计数据，可以包括但不限于总观看数、独立访问数和平均观看时间等数据。可以将统计数据作为观看数据匹配条件，根据匹配条件，在直播数据库中匹配直播区间的观看数据，实现观看数据的准确获取，其中，直播数据库中可以包括但不限于实时统计的视频观看数据。通过直播视频中音乐元数据的统计数据，以及统计数据对应的视频观看数据，用于评估音乐元数据在直播视频中的引流作用，或者，对音乐元数据进行发展趋势的预测。
参考图5，图5为本公开实施例提供的另一种音频检测方法流程示意图，本实施例在上述实施例的基础上，提供了一个示例，对上述实施例音频检测方法进行说明。
如图5，本实施例的方法包括：
以视频直播为例，对直播流中的音频进行切分，得到多个音频流切片(即上述音频段)，多个音频流切片可以并行处理；
对音频流切片进行音乐事件识别，包括：对每个音频流切片提取短时特征和长时特征，通过降维算法对提取的短时特征和长时特征进行降维，以去除短时特征和长时特征的冗余信息，从而得到主要特征。经过降维的特征的维数得到大大减少，性能还会得到一定程度的提高。将主要特征输入至支持向量机(Support Vector Machine，SVM)分类器，得到识别结果。其中，短时特征至少包括以下特征中的一项：知觉线性预测系数(Perceptual Linear Predictive Coefficients，PLP)、线性预测倒谱系数(Linear Predictive Cepstrum Coefficients，LPCC)、线性频率倒谱系数(Linear Frequency Cepstral Coefficients，LFCC)、Pitch、短时能量(Short-time Energy，STE)、子带能量分布(Sub-Band Energy Distribution，SBED)、亮度(Brightness，BR)和带宽(Bandwidth，BW)。长时特征至少包括以下特征中的一项：谱通量(Spectrum Flux，SF)、长时平均谱(Long-Term Average Spectrum，LTAS)和LPC熵(LPC entropy)。
若识别结果为音乐事件，则继续判断当前音乐事件的时长是否大于第一预设时长；若识别结果不为音乐事件，则取消标记音乐事件。若当前音乐事件的时长大于第一预设时长，则继续提取该音乐事件的音频指纹特征；若当前音乐 事件的时长不大于第一预设时长，则取消标记音乐事件。
通过音频指纹提取算法对音乐事件提取音频指纹特征，将音频指纹特征在指纹特征库进行匹配，得到元数据信息。若相邻音乐事件的元数据信息相同，即相邻音乐事件为同一歌曲，并且相邻的音乐事件的间隔时长小于第二预设时长，表明两者属于同一歌曲并且中间只是正常歌唱或播放停顿，则将相邻音乐事件进行合并；若元数据信息相同，并且相邻的音乐事件的间隔时长不小于第二预设时长，表明两者虽然属于同一歌曲但停顿时间较长，不适合进行合并处理，则不将相邻音乐事件进行合并。若元数据信息不相同，即相邻音乐事件不为同一歌曲，则不将相邻音乐事件进行合并。
在相邻音乐事件进行合并之后，所述方法还包括：获取合并后音乐事件的统计数据，例如音乐事件的播放起始时间、播放结束时间等数据。统计数据可以用于音乐版权计费。
图6为本公开实施例所提供的一种音频检测装置的结构示意图。如图6所示，所述装置包括：
音乐事件识别模块610，设置为获取被检测音频中的音频段，识别所述音频段中的音乐事件；统计数据确定模块620，设置为确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
在本公开实施例的一些实现方式中，音乐事件识别模块610还可以设置为：
将所述音频段输入至预先训练的音乐识别模型中，得到所述音乐识别模型输出的音乐事件识别结果，其中，所述音乐识别模型基于音频样本与所述音频样本对应的事件标签训练得到。
在本公开实施例的一些实现方式中，所述装置还可以设置为：
确定所述音频段中的音乐事件的时长是否大于第一预设时长，响应于所述音频段中的音乐事件的时长不大于所述第一预设时长，取消标记所述音乐事件。
在本公开实施例的一些实现方式中，统计数据确定模块620还可以包括：
指纹特征提取单元，设置为对于包含音乐事件的音频段，提取所述音频段的音频指纹特征；元数据匹配单元，设置为基于所述音频指纹特征在指纹特征库中进行匹配，确定与所述音乐事件相匹配的元数据信息，其中，所述指纹特征库中包括音乐元数据和对应的指纹特征。
在本公开实施例的一些实现方式中，所述指纹特征提取单元还可以设置为：
根据音频段中音乐事件的起止时间戳，对所述音频段进行截取，得到截取 音频段，提取所述截取音频段的音频指纹特征；或者，在所述音频段中提取所述音乐事件所在音轨的音频数据，基于所述音乐事件所在音轨的音频数据提取音频指纹特征。
在本公开实施例的一些实现方式中，所述统计数据确定模块620还可以包括：
数据合并单元，设置为根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到被检测音频中的统计数据。
在本公开实施例的一些实现方式中，所述数据合并单元还可以设置为：
对于相邻音乐事件，若所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长小于第二预设时长，则将所述相邻音乐事件进行合并；若所述相邻音乐事件对应的元数据信息不同，或者，所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长大于或等于第二预设时长，则不对所述相邻音乐事件进行合并。
在本公开实施例的一些实现方式中，所述被检测音频为直播视频中的音频；所述装置还可以设置为：确定所述统计数据中每个元数据信息所对应的直播区间的观看数据。
本公开实施例所提供的音频检测装置可执行本公开任意实施例所提供的音频检测方法，具备执行音频检测方法相应的功能模块和效果。
上述装置所包括的多个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，多个功能单元的名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。
下面参考图7，其示出了适于用来实现本公开实施例的电子设备(例如图7中的终端设备或服务器)400的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字电视(Television，TV)、台式计算机等等的固定终端。图7示出的电子设备400仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图7所示，电子设备400可以包括处理装置(例如中央处理器、图形处理器等)401，其可以根据存储在只读存储器(Read-Only Memory，ROM)402中的程序或者从存储装置408加载到随机访问存储器(Random Access Memory，RAM)403中的程序而执行多种适当的动作和处理。在RAM403中，还存储有电子设备400操作所需的多种程序和数据。处理装置401、ROM 402以及RAM 403 通过总线404彼此相连。输入/输出(Input/Output，I/O)接口405也连接至总线404。
通常，以下装置可以连接至I/O接口405：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置406；包括例如液晶显示器(Liquid Crystal Display，LCD)、扬声器、振动器等的输出装置407；包括例如磁带、硬盘等的存储装置408；以及通信装置409。通信装置409可以允许电子设备400与其他设备进行无线或有线通信以交换数据。虽然图7示出了具有多种装置的电子设备400，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置409从网络上被下载和安装，或者从存储装置408被安装，或者从ROM 402被安装。在该计算机程序被处理装置401执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施例提供的电子设备与上述实施例提供的音频检测方法属于同一构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的效果。
本公开实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的音频检测方法。
本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、RAM、ROM、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、 传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，RF)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如超文本传输协议(HyperText Transfer Protocol，HTTP)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(Local Area Network，LAN)，广域网(Wide Area Network，WAN)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：
获取被检测音频中的音频段，识别所述音频段中的音乐事件；确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括LAN或WAN—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开多种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可 以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元/模块的名称在一种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(Field Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、片上系统(System on Chip，SOC)、复杂可编程逻辑设备(Complex Programming Logic Device，CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、RAM、ROM、EPROM或快闪存储器、光纤、CD-ROM、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，【示例一】提供了一种音频检测方法，该方法包括：
获取被检测音频中的音频段，识别所述音频段中的音乐事件；
确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
根据本公开的一个或多个实施例，【示例二】提供了一种音频检测方法，还包括：
所述识别所述音频段中的音乐事件，包括：
将所述音频段输入至预先训练的音乐识别模型中，得到所述音乐识别模型输出的音乐事件识别结果，其中，所述音乐识别模型基于音频样本与所述音频样本对应的事件标签训练得到。
根据本公开的一个或多个实施例，【示例三】提供了一种音频检测方法， 还包括：
在识别所述音频段中的音乐事件之后，所述方法还包括：
确定所述音频段中的音乐事件的时长是否大于第一预设时长，响应于所述音频段中的音乐事件的时长不大于所述第一预设时长，取消标记所述音乐事件。
根据本公开的一个或多个实施例，【示例四】提供了一种音频检测方法，还包括：
所述确定与所述音乐事件相匹配的元数据信息，包括：
对于包含音乐事件的音频段，提取所述音频段的音频指纹特征；
基于所述音频指纹特征在指纹特征库中进行匹配，确定与所述音乐事件相匹配的元数据信息，其中，所述指纹特征库中包括音乐元数据和对应的指纹特征。
根据本公开的一个或多个实施例，【示例五】提供了一种音频检测方法，还包括：
所述提取所述音频段的音频指纹特征，包括：
根据音频段中音乐事件的起止时间戳，对所述音频段进行截取，得到截取音频段，提取所述截取音频段的音频指纹特征；或者，
在所述音频段中提取所述音乐事件所在音轨的音频数据，基于所述音乐事件所在音轨的音频数据提取音频指纹特征。
根据本公开的一个或多个实施例，【示例六】提供了一种音频检测方法，还包括：
所述基于所述元数据信息确定所述被检测音频中的统计数据，包括：
根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到被检测音频中的统计数据。
根据本公开的一个或多个实施例，【示例七】提供了一种音频检测方法，还包括：
所述根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到被检测音频中的统计数据，包括：
对于相邻音乐事件，若所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长小于第二预设时长，则将所述相邻音乐事件进行合并；
若所述相邻音乐事件对应的元数据信息不同，或者，所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长大于或等于第二预设时 长，则不对所述相邻音乐事件进行合并。
根据本公开的一个或多个实施例，【示例八】提供了一种音频检测方法，还包括：
所述被检测音频为直播视频中的音频；
所述方法还包括：
确定所述统计数据中每个元数据信息所对应的直播区间的观看数据。
根据本公开的一个或多个实施例，【示例九】提供了一种音频检测装置，该装置包括：
音乐事件识别模块，设置为获取被检测音频中的音频段，识别所述音频段中的音乐事件；
统计数据确定模块，设置为确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
此外，虽然采用特定次序描绘了多个操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了多个实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的一些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的多种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。

Claims (12)

  1. 一种音频检测方法，包括：
    获取被检测音频中的音频段，识别所述音频段中的音乐事件；
    确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
  2. 根据权利要求1所述的方法，其中，所述识别所述音频段中的音乐事件，包括：
    将所述音频段输入至预先训练的音乐识别模型中，得到所述音乐识别模型输出的音乐事件识别结果，其中，所述音乐识别模型基于音频样本与所述音频样本对应的事件标签训练得到。
  3. 根据权利要求1所述的方法，在所述识别所述音频段中的音乐事件之后，还包括：
    确定所述音频段中的音乐事件的时长是否大于第一预设时长，响应于所述音频段中的音乐事件的时长不大于所述第一预设时长，取消标记所述音乐事件。
  4. 根据权利要求1所述的方法，其中，所述确定与所述音乐事件相匹配的元数据信息，包括：
    对于包含所述音乐事件的音频段，提取所述音频段的音频指纹特征；
    基于所述音频指纹特征在指纹特征库中进行匹配，确定与所述音乐事件相匹配的元数据信息，其中，所述指纹特征库中包括音乐元数据和对应的指纹特征。
  5. 根据权利要求4所述的方法，其中，所述提取所述音频段的音频指纹特征，包括：
    根据所述音频段中所述音乐事件的起止时间戳，对所述音频段进行截取，得到截取音频段，提取所述截取音频段的音频指纹特征；或者，
    在所述音频段中提取所述音乐事件所在音轨的音频数据，基于所述音乐事件所在音轨的音频数据提取音频指纹特征。
  6. 根据权利要求1所述的方法，其中，所述基于所述元数据信息确定所述被检测音频中的统计数据，包括：
    根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到所述被检测音频中的统计数据。
  7. 根据权利要求6所述的方法，其中，所述根据每个音频段中音乐事件的起止时间戳，对相同元数据信息对应的音乐事件进行合并，得到所述被检测音 频中的统计数据，包括：
    对于相邻音乐事件，在所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长小于第二预设时长的情况下，将所述相邻音乐事件进行合并；
    在所述相邻音乐事件对应的元数据信息不同，或者，所述相邻音乐事件对应的元数据信息相同，且所述相邻音乐事件的间隔时长大于或等于第二预设时长的情况下，不对所述相邻音乐事件进行合并。
  8. 根据权利要求1所述的方法，其中，所述被检测音频为直播视频中的音频；
    所述方法还包括：
    确定所述统计数据中每个元数据信息所对应的直播区间的观看数据。
  9. 一种音频检测装置，包括：
    音乐事件识别模块，设置为获取被检测音频中的音频段，识别所述音频段中的音乐事件；
    统计数据确定模块，设置为确定与所述音乐事件相匹配的元数据信息，基于所述元数据信息确定所述被检测音频中的统计数据。
  10. 一种电子设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序；
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-8中任一所述的音频检测方法。
  11. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-8中任一所述的音频检测方法。
  12. 一种计算机程序产品，包括承载在非暂态计算机可读介质上的计算机程序，所述计算机程序包含用于执行如权利要求1-8中任一所述的音频检测方法的程序代码。
PCT/CN2023/078752 2022-03-08 2023-02-28 音频检测方法、装置、存储介质及电子设备 Ceased WO2023169258A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/844,887 US20250218418A1 (en) 2022-03-08 2023-02-28 Audio detection method and apparatus, storage medium and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210220184.7A CN114596878B (zh) 2022-03-08 2022-03-08 一种音频检测方法、装置、存储介质及电子设备
CN202210220184.7 2022-03-08

Publications (1)

Publication Number Publication Date
WO2023169258A1 true WO2023169258A1 (zh) 2023-09-14

Family

ID=81807399

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/078752 Ceased WO2023169258A1 (zh) 2022-03-08 2023-02-28 音频检测方法、装置、存储介质及电子设备

Country Status (3)

Country Link
US (1) US20250218418A1 (zh)
CN (1) CN114596878B (zh)
WO (1) WO2023169258A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596878B (zh) * 2022-03-08 2026-01-02 北京字跳网络技术有限公司 一种音频检测方法、装置、存储介质及电子设备
CN115472148B (zh) * 2022-08-31 2025-06-27 海尔优家智能科技（北京）有限公司 测试结果的确定方法、装置、存储介质及电子装置
CN117746894A (zh) * 2022-09-13 2024-03-22 广州视源电子科技股份有限公司 朗读事件识别方法、装置、教学设备和存储介质
CN115866279B (zh) * 2022-09-20 2024-11-29 北京奇艺世纪科技有限公司 直播视频处理方法、装置、电子设备及可读存储介质
CN116072147A (zh) * 2023-01-09 2023-05-05 北京达佳互联信息技术有限公司 音乐检测模型训练方法、装置、电子设备及存储介质
EP4728513A1 (en) * 2023-06-14 2026-04-22 Qualcomm Incorporated Knowledge-based audio scene graph

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010078984A (ja) * 2008-09-26 2010-04-08 Sanyo Electric Co Ltd 楽曲抽出装置および楽曲記録装置
CN105874732A (zh) * 2014-01-07 2016-08-17 高通股份有限公司 用于识别音频流中的一首音乐的方法和装置
WO2020176057A1 (en) * 2019-02-25 2020-09-03 Ahmet Aksoy Music analysis system and method for public spaces
CN113032616A (zh) * 2021-03-19 2021-06-25 腾讯音乐娱乐科技（深圳）有限公司 音频推荐的方法、装置、计算机设备和存储介质
CN113987258A (zh) * 2021-11-10 2022-01-28 北京有竹居网络技术有限公司 音频的识别方法、装置、可读介质和电子设备
CN114596878A (zh) * 2022-03-08 2022-06-07 北京字跳网络技术有限公司 一种音频检测方法、装置、存储介质及电子设备

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7336890B2 (en) * 2003-02-19 2008-02-26 Microsoft Corporation Automatic detection and segmentation of music videos in an audio/video stream
CN102956230B (zh) * 2011-08-19 2017-03-01 杜比实验室特许公司 对音频信号进行歌曲检测的方法和设备
GB2507551A (en) * 2012-11-04 2014-05-07 Julian Andrew John Fells Copyright protection by comparing identifiers of first and second electronic content
CN103440330A (zh) * 2013-09-03 2013-12-11 网易（杭州）网络有限公司 一种音乐节目信息获取方法和设备
CN104091596B (zh) * 2014-01-20 2016-05-04 腾讯科技（深圳）有限公司 一种乐曲识别方法、系统和装置
US9620105B2 (en) * 2014-05-15 2017-04-11 Apple Inc. Analyzing audio input for efficient speech and music recognition
US10089987B2 (en) * 2015-12-21 2018-10-02 Invensense, Inc. Music detection and identification
CN105657535B (zh) * 2015-12-29 2018-10-30 北京搜狗科技发展有限公司 一种音频识别方法和装置
US10296638B1 (en) * 2017-08-31 2019-05-21 Snap Inc. Generating a probability of music using machine learning technology
JP7143327B2 (ja) * 2017-10-03 2022-09-28 グーグル エルエルシー コンピューティング装置によって実施される方法、コンピュータシステム、コンピューティングシステム、およびプログラム
CN111723235B (zh) * 2019-03-19 2023-09-26 百度在线网络技术（北京）有限公司 音乐内容识别方法、装置及设备
US10796684B1 (en) * 2019-04-30 2020-10-06 Dialpad, Inc. Chroma detection among music, speech, and noise
CN113724736A (zh) * 2021-08-06 2021-11-30 杭州网易智企科技有限公司 一种音频处理方法、装置、介质和电子设备
CN114023289B (zh) * 2021-11-09 2025-10-10 北京百度网讯科技有限公司 音乐识别方法、音乐特征提取模型的训练方法及装置
GB2612994A (en) * 2021-11-18 2023-05-24 Audoo Ltd Media identification system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010078984A (ja) * 2008-09-26 2010-04-08 Sanyo Electric Co Ltd 楽曲抽出装置および楽曲記録装置
CN105874732A (zh) * 2014-01-07 2016-08-17 高通股份有限公司 用于识别音频流中的一首音乐的方法和装置
WO2020176057A1 (en) * 2019-02-25 2020-09-03 Ahmet Aksoy Music analysis system and method for public spaces
CN113032616A (zh) * 2021-03-19 2021-06-25 腾讯音乐娱乐科技（深圳）有限公司 音频推荐的方法、装置、计算机设备和存储介质
CN113987258A (zh) * 2021-11-10 2022-01-28 北京有竹居网络技术有限公司 音频的识别方法、装置、可读介质和电子设备
CN114596878A (zh) * 2022-03-08 2022-06-07 北京字跳网络技术有限公司 一种音频检测方法、装置、存储介质及电子设备

Also Published As

Publication number Publication date
US20250218418A1 (en) 2025-07-03
CN114596878B (zh) 2026-01-02
CN114596878A (zh) 2022-06-07

Similar Documents

Publication Publication Date Title
WO2023169258A1 (zh) 音频检测方法、装置、存储介质及电子设备
CN110503961B (zh) 音频识别方法、装置、存储介质及电子设备
EP3945435A1 (en) Dynamic identification of unknown media
CN113596579B (zh) 视频生成方法、装置、介质及电子设备
CN108877779B (zh) 用于检测语音尾点的方法和装置
US20160196812A1 (en) Music information retrieval
US9224385B1 (en) Unified recognition of speech and music
JP7567028B2 (ja) ターゲットビデオを生成するための方法、装置、サーバ及び媒体
EP3092734A1 (en) Method and device for identifying a piece of music in an audio stream
EP3468205A1 (en) Temporal fraction with use of content identification
CN111859008B (zh) 一种推荐音乐的方法及终端
CN109949798A (zh) 基于音频的广告检测方法以及装置
CN107679196A (zh) 一种多媒体识别方法、电子设备及存储介质
CN112071287A (zh) 用于生成歌谱的方法、装置、电子设备和计算机可读介质
WO2022160603A1 (zh) 歌曲的推荐方法、装置、电子设备及存储介质
CN114999454A (zh) 语音交互设备的性能测试方法、装置、设备及可读介质
US20240404548A1 (en) Method, apparatus, device and storage medium for video recording
US11609948B2 (en) Music streaming, playlist creation and streaming architecture
CN116072147A (zh) 音乐检测模型训练方法、装置、电子设备及存储介质
CN111798853A (zh) 语音识别的方法、装置、设备和计算机可读介质
WO2026040959A1 (zh) 音频副本检测方法及装置、电子设备、计算机存储介质以及计算机程序产品
CN119274119B (zh) 一种视频内容风险检测方法、装置、介质及设备
CN114595361B (zh) 一种音乐热度的预测方法、装置、存储介质及电子设备
CN114329042A (zh) 数据处理方法、装置、设备、存储介质及计算机程序产品
CN115565508B (zh) 歌曲匹配方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23765836

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18844887

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.12.2024)

WWP Wipo information: published in national office

Ref document number: 18844887

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 23765836

Country of ref document: EP

Kind code of ref document: A1