WO2023221237A1 - 自适应层次聚合的弱监督声音事件检测方法及系统 - Google Patents

自适应层次聚合的弱监督声音事件检测方法及系统 Download PDF

Info

Publication number
WO2023221237A1
WO2023221237A1 PCT/CN2022/101361 CN2022101361W WO2023221237A1 WO 2023221237 A1 WO2023221237 A1 WO 2023221237A1 CN 2022101361 W CN2022101361 W CN 2022101361W WO 2023221237 A1 WO2023221237 A1 WO 2023221237A1
Authority
WO
WIPO (PCT)
Prior art keywords
sound event
aggregation
optimal
acoustic model
probability
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/101361
Other languages
English (en)
French (fr)
Inventor
毛启容
高利剑
沈雅馨
任庆桦
詹永照
成科扬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu University
Original Assignee
Jiangsu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu University filed Critical Jiangsu University
Priority to US18/035,934 priority Critical patent/US12080319B2/en
Priority to GB2307040.2A priority patent/GB2616363B/en
Publication of WO2023221237A1 publication Critical patent/WO2023221237A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention relates to the technical field of sound event detection in artificial intelligence technology, and specifically relates to an adaptive hierarchical aggregation weakly supervised sound event detection method and system.
  • One of the most important tasks in weakly supervised sound event detection is to design an aggregation function.
  • the role of the aggregation function is to infer the sentence-level probability from the frame-level probability sequence predicted by the model, that is, to infer the "category information" of the event from the predicted "location information", thereby effectively modeling weakly labeled audio samples.
  • the current mainstream aggregation functions can be roughly divided into two categories: maximum aggregation and weighted average aggregation. Maximum aggregation captures the most salient information in the signal, resulting in excellent performance for audio classification.
  • Weighted average aggregation performs a weighted average of all frame-level probabilities to obtain sentence-level predictions. This aggregation method that considers all frame-level probabilities instead of only focusing on salient information can recall more positive sample frames (i.e., event activation frames) , which can achieve better performance in the audio positioning subtask.
  • weighted average aggregation also takes event-irrelevant information into account, bringing interference information to audio classification, resulting in suboptimal audio classification performance.
  • no single aggregation method can provide an optimal strategy for all types of events. For example, weighted average aggregation is more suitable for longer duration continuous events (such as music), while for some shorter events (such as dog barking), you should consider using maximum aggregation to focus on the most significant audio frames.
  • the design of aggregation strategies should be adaptive to the natural characteristics of sound events.
  • the present invention provides an adaptive hierarchical aggregation weakly supervised sound event detection method and system, which uses hierarchical structure and continuous relaxation method to automatically learn the optimal aggregation strategy for each type of event, which can capture The salient information of multiple segments can retain complete positioning boundaries, thereby simultaneously improving the performance of audio classification and audio positioning in weakly supervised sound event detection.
  • the present invention achieves the above technical objectives through the following technical means.
  • Adaptive hierarchical aggregation weakly supervised sound event detection method specifically:
  • the formulation of the optimal aggregation strategy is specifically: using Calculate the probability ⁇ * of selecting different R under the optimal relaxation parameters.
  • the R corresponding to the maximum selection probability in is the optimal number of sub-packets of the current category.
  • ⁇ is the probability set
  • R is the number of sub-packets
  • ⁇ k is the k-th dimension relaxation parameter
  • softmax() is the operator.
  • the sentence-level prediction probability of the k-th type of sound event is expressed as:
  • ⁇ hi is the adaptive hierarchical aggregation algorithm
  • F w represents the acoustic model
  • ⁇ avg represents the mean aggregation
  • ⁇ max represents the maximum aggregation
  • b r is the sub-package set B r elements.
  • sentence-level predicted probability of the k-th type of sound event is for:
  • N + is the set of all optional sub-package numbers.
  • the average prediction error of all sound event categories in L W and ⁇ are the model parameters and relaxation parameters respectively
  • X and Y are the Mel spectrum features and sentence level labels of the model input respectively
  • Y k is the k-th type of sound event Sentence-level labels
  • BCELoss represents the binary cross-entropy function
  • K is the total number of categories of sound events.
  • the acoustic model is any mainstream deep learning model
  • the benchmark model of the acoustic model is a convolutional recurrent neural network model.
  • the extracted features are Mel spectrum features.
  • the acoustic model training and verification uses the DCASE2017 data set.
  • the audio signal is downsampled to 16kHz, the frame length and frame shift are set to 1024 and 664 respectively. After framing, each signal obtains 240 frame samples, and the Mel spectrum feature is 64 dimensions.
  • An adaptive hierarchical aggregation weakly supervised sound event detection system including a sequentially connected acoustic model and an adaptive hierarchical aggregation algorithm module.
  • the acoustic model inputs preprocessed and feature extracted audio signals, and the acoustic model predicts frame levels. Prediction probability, the adaptive hierarchical aggregation algorithm module aggregates the frame level prediction probability to obtain the sentence level prediction probability.
  • the adaptive hierarchical aggregation algorithm of the present invention first obtains significant information in multiple audio segments, breaking the limitation that the maximum aggregation method can only capture the most significant segments in the signal, and expands the sensing area during positioning; secondly, only The weighted average of the salient information of multiple segments is used to obtain the final prediction, which solves the noise problem caused by the weighted average aggregation considering all frame signals; therefore, the adaptive hierarchical aggregation algorithm has the ability to capture the salient information of multiple segments while ensuring complete positioning boundaries. Making it suitable for both subtasks of weakly supervised sound event detection - audio classification and audio localization.
  • the adaptive hierarchical aggregation of the present invention uses the continuous relaxation method to jointly learn the optimal weight of the model and the optimal aggregation strategy for each type of sound event; shorter sound events (such as "gunshots") usually only last for a short period of time, which In this case, maximum aggregation is often better than weighted average aggregation.
  • adaptive hierarchical aggregation can automatically learn a smaller number of sub-packets, that is, most signal frames belong to the same sub-packet, increasing the proportion of maximum aggregation; compared with Long duration or periodic sound time (such as "music” or “alarm sound”), where there are fewer noise segments and event information is distributed throughout the long sequence.
  • weighted average aggregation is often better than maximum aggregation.
  • adaptive hierarchical aggregation can automatically allocate a larger number of sub-packets, that is, one sub-packet contains less frame information, increasing the proportion of weighted average aggregation; adaptive aggregation realizes the most customized based on the natural attributes of sound events.
  • Optimal aggregation strategy making it suitable for more complex acoustic scenes.
  • the adaptive hierarchical aggregation algorithm of the present invention is lightweight in design, relies only on a set of learnable parameters for implementation, and is easy to efficiently embed into any acoustic model to complete weakly supervised sound event detection tasks.
  • Figure 1 is a framework diagram of the weakly supervised sound event detection system based on adaptive hierarchical aggregation according to the present invention
  • Figure 2 is a flow chart of the adaptive hierarchical aggregation algorithm according to the present invention.
  • Figure 3(a) is a comparison chart of the visualization results of weakly supervised sound event detection according to the present invention.
  • Figure 3(b) is a comparison chart 2 of the visualization results of weakly supervised sound event detection according to the present invention.
  • the weakly supervised sound event detection system based on adaptive hierarchical aggregation of the present invention includes sequentially connected acoustic models and adaptive hierarchical aggregation algorithm modules 5.
  • the sequence is signal preprocessing ⁇ acoustics Feature extraction ⁇ acoustic model ⁇ hierarchical aggregation.
  • the process of signal preprocessing 2 is to process the high-dimensional and complex original audio signal 1 into a short-term, continuous signal frame sequence of lower dimensions that is convenient for subsequent processing.
  • the acoustic feature extraction process extracts Mel spectrum features 3 that are consistent with the characteristics of the human ear for each frame sample, preliminarily filters out redundant information, and improves the efficiency of acoustic model modeling.
  • the acoustic model can be any mainstream deep learning model.
  • the benchmark model of the acoustic model is the convolutional recurrent neural network model.
  • the convolutional recurrent neural network (CRNN) 4 is selected.
  • the convolutional neural network 4 consists of 6 convolutions. It consists of blocks and a 3-layer long short-term memory network (LSTM). Each convolution block contains a convolution layer 7, a normalization layer 8, and a ReLU activation layer 9.
  • the extracted Mel spectrum feature 3 sequence is sent to the convolutional recurrent neural network 4 to obtain the frame level prediction probability sequence, that is, the positioning information of the sound event, and then the sentence level prediction probability is calculated through the adaptive hierarchical aggregation algorithm module 5 , that is, the classification information of sound events.
  • Figure 2 shows the process of gradually aggregating frame-level prediction probabilities to obtain sentence-level prediction probabilities from bottom to top, and shows the gradient propagation path during backpropagation from top to bottom, where the grayscale represents the size of the gradient.
  • Signal preprocessing 2 first resamples the original signal according to a specific sampling rate. After sampling, it first performs pre-emphasis processing to compensate for the energy of high-frequency components. Then it divides the frames according to the specified frame length to obtain several consecutive shorter frame samples. Finally Add window processing to each frame of sample to smooth the frame signal to prevent energy leakage, obtain a short-term continuous signal frame sequence, and complete the signal preprocessing process.
  • the specific process is as follows: Select the signal s from the large-scale weakly labeled sound event data set proposed by the DCASE2017 challenge. The sampling rate of the original signal s is 22.5kHz. It is down-sampled to 16kHz to reduce complexity.
  • the signal s Before receiving by the convolutional recurrent neural network 4, The signal s needs to be preprocessed to increase high frequency resolution.
  • the data duration in the DCASE17 data set is all 10 seconds, that is, the above signal s has a total of 160,000 sampling points.
  • frame processing is required to reduce the computational complexity.
  • the frame length is set to 1024 sampling points (64 milliseconds), and the frame shift is 664 sampling points (41.5 milliseconds), that is, an overlap of 22.5 milliseconds is reserved before and after each frame to ensure the smoothness of the frame signal.
  • window processing is added to each frame of sample to complete the signal preprocessing process.
  • the short-time Fourier transform is used to convert the time domain signal of each frame into frequency
  • 64 Mel filters are used to filter the signal of each frame to obtain the 64-dimensional Mel spectrum feature 3, that is, for each signal.
  • the input feature dimension of the convolutional recurrent neural network 4 is 240*64.
  • F w represent the convolutional recurrent neural network 4.
  • the frame-level prediction probability can be obtained:
  • the prediction error (formula (6)) is constructed and the model is trained.
  • the convolutional recurrent neural network 4 test phase performs forward calculation based on the determined optimal model weights and optimal aggregation strategy to complete sound event detection of unknown data. The specific process is as follows:
  • the continuous relaxation method is used to convert the discrete search space of the optional number of sub-packets into an optimizeable continuous search space, which can be jointly optimized with the convolutional recurrent neural network 4 to automatically select the optimal number of sub-packets for each type of sound event. , that is, adaptively customize the optimal aggregation strategy for specific events;
  • ⁇ k is the k-th dimension relaxation parameter
  • softmax() is the operator
  • b r is the r-th element in the sub-package set B;
  • the activation probability of the k-th type of sound event (i.e., sentence-level prediction probability) can be expressed as:
  • the number of sub-packages R determines the aggregation strategy.
  • the proportion of maximum aggregation is smaller, and the proportion of mean aggregation is larger, that is, more attention is paid to global information;
  • maximum aggregation is The proportion of effects is large, and the proportion of mean aggregation is small, that is, more attention is paid to local salient information; adaptive hierarchical aggregation automatically learns the number of sub-packets for each type of sound event, realizing a customized aggregation strategy for specific events;
  • W and ⁇ are model parameters and relaxation parameters respectively;
  • the optimal model weight W * and the optimal relaxation parameter ⁇ * can be obtained.
  • the probability of selecting different R under the optimal relaxation parameter can be calculated using formula (1) ⁇ * ;
  • the R corresponding to the maximum selection probability in is the optimal number of sub-packets of the current category.
  • the adaptive hierarchical aggregation algorithm has completed customizing an optimal aggregation strategy for each type of sound event.
  • the continuous relaxation method for joint optimization only a set of low-dimensional relaxation parameters need to be introduced to efficiently complete the joint optimization; compared with model parameters, the number of relaxation parameters is very small, and optimization only needs to focus on the natural attributes of sound events, such as duration , period and other global characteristics without paying attention to high-dimensional signal content, so it is easy to converge, thereby guiding the model parameters to quickly converge to the global optimum under the optimal aggregation strategy.
  • Manually selecting the number of sub-packets and repeatedly training the convolutional recurrent neural network can also find the optimal aggregation strategy, but its computational complexity is as high as O(N K ), where N represents the number of optional sub-packets in the search space, and K represents the sound event Number of categories.
  • the present invention uses the continuous relaxation method for joint optimization and only needs to introduce a set of low-dimensional relaxation parameters to reduce the computational complexity to O(N).
  • Adaptive hierarchical aggregation is an independent module controlled by a set of independent, learnable parameters.
  • the input of the module is the audio positioning result predicted by the model, and the output is the audio classification result.
  • the adaptive hierarchical aggregation method can be conveniently and effectively embedded in any Weakly supervised sound event detection is implemented in the acoustic model; using the convolutional recurrent neural network as the benchmark model can simultaneously learn multi-scale acoustic features of spatial and temporal context features, which is the mainstream model framework in the field of sound event detection.
  • the above process completes the joint optimization of the convolutional recurrent neural network 4 and the adaptive hierarchical aggregation algorithm.
  • an unknown sound signal is given, and after preprocessing and feature extraction, it is sent to the trained Convolutional Recurrent Neural Network , obtain the positioning output (frame-level prediction probability) of all target events to be detected, implement the audio positioning task, and use the optimal aggregation strategy specific to each type of event Get the activation probabilities of all categories (sentence-level prediction probabilities) to implement the audio classification task.
  • Figure 3(a) and (b) are visualizations of the sound event positioning results of two typical audio signals.
  • the comparison methods are maximum aggregation and weighted average aggregation. Among them, maximum aggregation can only capture significant areas and cause incomplete positioning boundaries, especially when detecting long sound events (such as "train sound"), while weighted average aggregation always brings more false detections, especially when detecting long sound events. When detecting short or multi-segment sound events (such as "screams" and "honks”).
  • the positioning effects of three typical sound events in Figure 3(a) and (b) confirm that adaptive hierarchical aggregation can not only capture the salient information of multiple segments and discard redundant information, but also reduce the false detection rate while ensuring the positioning boundary. complete to achieve optimal sound event detection performance.
  • this application also provides an electronic device.
  • the electronic device includes one or more processors and one or more memories, and a computer is stored in the memory.
  • Computer-readable code wherein the computer-readable code, when executed by one or more processors, performs an implementation of the weakly supervised sound event detection method based on adaptive hierarchical aggregation of the present invention.
  • the memory may include non-volatile storage media and internal memory; the non-volatile storage media may store operating systems and computer-readable codes.
  • the computer-readable code includes program instructions, which when executed, can cause the processor to perform any weakly supervised sound event detection method based on adaptive hierarchical aggregation.
  • the processor is used to provide computing and control capabilities to support the operation of the entire electronic device.
  • the memory provides an environment for the execution of computer-readable code in the non-volatile storage medium.
  • the computer-readable code When executed by the processor, it can cause the processor to execute any weakly supervised sound event detection method based on adaptive hierarchical aggregation. .
  • the processor can be a central processing unit (Central Processing Unit, CPU), and the processor can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general processor may be a microprocessor or the processor may be any conventional processor.
  • the computer-readable storage medium may be an internal storage unit of the electronic device described in the previous embodiment, such as a hard disk or memory of the computer device.
  • the computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), or a secure digital (SD) equipped on the electronic device. card, flash card, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

一种自适应层次聚合的弱监督声音事件检测方法及系统，该系统包括声学模型和自适应层次聚合算法模块(5)，声学模型输入预处理和特征提取的音频信号，声学模型预测得到帧级别预测概率，自适应层次聚合算法模块(5)将帧级别预测概率聚合得到句级别预测概率；联合优化声学模型和松弛化参数，得到最优模型权重和最优松弛化参数，根据最优松弛化参数为每类声音事件制定最优聚合策略；输入预处理和特征提取的未知音频信号，得到所有目标声音事件的帧级别预测概率，并根据每类目标声音事件的最优聚合策略，得到所有目标声音事件类别的句级别预测概率。适用于复杂的声学场景，同时适用于弱监督声音事件检测中的音频分类和定位，具有良好的通用性。

Description

自适应层次聚合的弱监督声音事件检测方法及系统 技术领域
本发明涉及人工智能技术中的声音事件检测技术领域，具体涉及一种自适应层次聚合的弱监督声音事件检测方法及系统。
背景技术
弱监督声音事件检测中，最重要的任务之一是设计聚合函数。聚合函数的作用是从模型预测的帧级别概率序列中推断句级别概率，即从预测的“定位信息”推断事件的“类别信息”，从而有效建模弱标注的音频样本。当前主流的聚合函数大致可以分为两类：最大值聚合和加权平均聚合。最大值聚合捕捉信号中最显著的信息，从而为音频分类带来出色的性能。然而，由于最大值聚合检测事件的定位边界总是不完整的，造成较差的定位性能，体现在较多的漏检测；为了解决最大值聚合在音频定位任务上的缺陷，各种类型的加权平均聚合算法发展了起来。加权平均聚合对所有帧级别的概率进行加权平均，以获得句级别的预测，这种考虑所有帧级别概率而不是仅关注显著信息的聚合方式能够召回更多的正样本帧(即事件激活帧)，在音频定位子任务中能够取得较好的性能。但与此同时，加权平均聚合也将事件无关的信息考虑进来，给音频分类带了干扰信息，造成次优的音频分类性能。实际上，没有任何单一的聚合方法可以为所有类型的事件提供最优策略。例如，加权平均聚合更适合于持续时间较长的连续事件(如音乐)，而对于一些较短的事件(如狗叫)，应该考虑使用最大值聚合来关注最显著的音频帧。显然，聚合策略的设计应该自适应声音事件的自然特性。
近年来，研究自适应聚合的方法逐渐被提出，如McFee等人提出的自动聚合及Zhang等人提出的阶乘聚合都采用自适应的加权Softmax聚合方法，即在Softmax聚合中将可学习参数乘以帧级别概率，其中不同类别的事件权重不同。然而，这两类自适应方法本质上利用不同的权重调和最大值聚合和加权平均聚合，无法同时有效兼顾音频分类和音频定位任务，且无法高效地为每类事件自适应学习定制的聚合策略，从而导致弱监督声音事件检测性能欠佳。
发明内容
针对现有技术中存在不足，本发明提供了一种自适应层次聚合的弱监督声音事件检测方法及系统，利用分层结构和连续松弛法自动为每类事件学习最优聚合策略，既能够捕捉多片段的显著信息又能保留完善的定位边界，实现同时提高弱监督声音事件检测中音频分类和音频定位的性能。
本发明是通过以下技术手段实现上述技术目的的。
自适应层次聚合的弱监督声音事件检测方法，具体为：
提取预处理音频信号的声学特征，并输入声学模型，将声学模型预测的帧级别预测概率序列分成若干个连续的子包，利用最大值聚合计算每个子包的显著信息，得到子包级预测集合，利用均值聚合取子包级预测集合的平均概率作为句级别预测概率；
联合优化声学模型和松弛化参数，直至收敛，得到最优模型权重和最优松弛化参数，根据最优松弛化参数为每类声音事件制定最优聚合策略；
给定未知的音频信号，进行预处理和特征提取，送入训练后的声学模型，得到所有目标声音事件的帧级别预测概率，实现音频定位任务，并根据每类目标声音事件的最优聚合策略，得到所有目标声音事件类别的句级别预测概率，实现音频分类任务。
进一步，所述制定最优聚合策略具体为：利用
Figure PCTCN2022101361-appb-000001
计算最优松弛化参数下选择不同R的概率λ *，对于第k类声音事件，
Figure PCTCN2022101361-appb-000002
中最大选择概率对应的R即为当前类别最优子包数量
Figure PCTCN2022101361-appb-000003
其中：λ为概率集合，R为子包数量，α k为第k维松弛化参数，softmax()为运算符。
更进一步，所述当子包数量为R时，第k类声音事件的句级别预测概率表示为：
Figure PCTCN2022101361-appb-000004
其中：
Figure PCTCN2022101361-appb-000005
为第k类声音事件的句级别预测概率，φ hi为自适应层次聚合算法，F w表示声学模型，φ avg表示均值聚合，φ max表示最大值聚合，b r为子包集合B中的第r个元素。
更进一步，所述第k类声音事件的句级别预测期望概率
Figure PCTCN2022101361-appb-000006
为：
Figure PCTCN2022101361-appb-000007
其中：
Figure PCTCN2022101361-appb-000008
表示第k类事件选择R个子包数量的概率，N +为所有可选的子包数量的集合。
更进一步，所述联合优化声学模型和松弛化参数采用反向传播进行的：
Figure PCTCN2022101361-appb-000009
其中：L所有声音事件类别的平均预测误差，W、α分别为模型参数和松弛化参数，X、Y分别为模型输入的梅尔频谱特征和句级别标签，Y k为第k类声音事件的句级别标签，BCELoss表示二进制交叉熵函数，K为声音事件的类别总数。
进一步，所述声学模型为任意主流的深度学习模型，声学模型的基准模型为卷积循环神经网络模型。
进一步，提取的特征为梅尔频谱特征。
更进一步，所述声学模型训练和验证采用DCASE2017数据集。
更进一步，所述音频信号下采样至16kHz，帧长和帧移分别设置为1024、664，分帧后每条信号得到240帧样本，梅尔频谱特征为64维。
一种自适应层次聚合的弱监督声音事件检测系统，包括依次相连的声学模型和自适应层次聚合算法模块，所述声学模型输入预处理和特征提取的音频信号，所述声学模型预测得到帧级别预测概率，所述自适应层次聚合算法模块将帧级别预测概率聚合得到句级别预测概率。
本发明的有益效果为：
(1)本发明自适应层次聚合算法首先获取多个音频片段中的显著信息，打破了最大值聚合方法只能够捕捉信号中最显著的片段的局限，扩大的定位时的感知区域；其次，仅对多片段的显著信息加权平均得到最终预测，解决了加权平均聚合考虑所有帧信号而带来噪声问题；因此，自适应层次聚合算法具备捕捉多片段显著信息的同时保证完整的定位边界的能力，使其同时适用于弱监督声音事件检测的两个子任务-音频分类和音频定位。
(2)本发明自适应层次聚合利用连续松弛法联合学习模型最优权重及每类声音事件最优聚合策略；较短的声音事件(如“枪声”)通常仅持续一个短时片段，这种情况下最大值聚合往往优于加权平均聚合，此时自适应层次聚合能够自动学习较小的子包数量，即大部分信号帧属于同一个子包，增加最大值聚合的作用比例；而相对于持续时间较长或周期性声音时间(如“音乐”或“警报声”)，此时噪声片段较少、事件信息分布于整个长序列，这种情况下加权平均聚合往往优于最大值聚合，此时自适应层次聚合能够自动分配较多的子包数量，即一个子包包含较少的帧信息，增加加权平均聚合的作用比例；自适应聚合实现了根据声音事件的自然属性来定制的最优聚合策略，从而适用于更加复杂的声学场景。
(3)本发明的自适应层次聚合算法设计轻便，仅依赖一组可学习的参数实现，易于高效地嵌入任何声学模型完成弱监督声音事件检测任务。
附图说明
图1为本发明所述基于自适应层次聚合的弱监督声音事件检测系统框架图；
图2为本发明所述自适应层次聚合算法流程图；
图3(a)为本发明所述弱监督声音事件检测可视化结果的对比图一；
图3(b)为本发明所述弱监督声音事件检测可视化结果的对比图二；
图中：1、原始音频信号，2、信号预处理，3、梅尔频谱特征，4、卷积循环神经网络，5、自适应层次聚合算法模块，6、长短期记忆网络，7、卷积层，8、标准化层，9、ReLU激活层。
具体实施方式
下面结合附图以及具体实施例对本发明作进一步的说明，但本发明的保护范围并不限于此。
如图1所示，本发明基于自适应层次聚合的弱监督声音事件检测系统包括依次相连的声学模型和自适应层次聚合算法模块5，按照音频信号数据流传递过程，依次为信号预处理→声学特征提取→声学模型→层次聚合。信号预处理2的过程是将高维、复杂的原始音频信号1处理成较低维度、便于后续处理的短时、连续的信号帧序列。声学特征提取过程为每帧样本提取符合人耳特性的梅尔频谱特征3，初步过滤冗余信息，提升声学模型建模效率。声学模型可以为任意主流的深度学习模型，声学模型的基准模型为卷积循环神经网络模型，本实施例中选用卷积循环神经网络(CRNN)4，卷积循环神经网络4由6个卷积块和3层长短期记忆网络(LSTM)组成，每个卷积块包含卷积层7、标准化层8、ReLU激活层9。将提取好的梅尔频谱特征3序列送入卷积循环神经网络4中，即可得到帧级别预测概率序列，即声音事件的定位信息，之后通过自适应层次聚合算法模块5计算句级别预测概率，即声音事件的分类信息。
图2自下而上表示帧级别预测概率逐渐聚合得到句级别预测概率的过程，自上而下表示反向传播时梯度传播路径，其中灰度表示梯度的大小。
信号预处理2首先将原始信号按照特定采样率重采样，采样后首先进行预加重处理，弥补高频分量的能量，之后按照指定帧长进行分帧，得到若干连续的较短的帧样本，最后对每帧样本加窗处理，平滑帧信号，防止能量泄露，得到短时连续的信号帧序列，完成信号预处理过程。具体过程如下：从DCASE2017挑战赛提出的大规模弱标注声音事件数据集中选取信号s，原始信号s的采样率为22.5kHz，下采样至16kHz降低复杂度，在卷积循环神经网络4接收之前，信号s需要进行预处理，增加高频分辨率。DCASE17数据集中数据时长均为10秒，即上述信号s共有160000个采样点，此时需要进行分帧处理以降低计算复杂度。在本发明中，帧长设置为1024个采样点(64毫秒)、帧移为664个采样点(41.5毫秒)，即每帧前后保留22.5毫秒的重叠部分以保证帧信号的平滑性。分帧后每条10秒的信号包含240帧样本：s′＝{s′ 1,s′ 2,…,s′ 240}。最后，对每帧样本加窗处理，完成信号的预处理过程。随后，利用短时傅里叶变换将每帧时域信号转换到频率上，利用64个梅尔滤波器对每帧信号进行过滤，得到64维梅尔频谱特征3，即，对于每条信号，卷积循环神经网络4的输入特征维度为240*64。
用F w表示卷积循环神经网络4，给定输入特征X，即可得到帧级别预测概率：
Figure PCTCN2022101361-appb-000010
按照图2所示流程，还需利用自适应层次聚合算法φ hi将
Figure PCTCN2022101361-appb-000011
聚合成句级别预测概率
Figure PCTCN2022101361-appb-000012
从而构造预测误差(公式(6))、训练模型。而卷积循环神经网络4测试阶段则根据已经确定的最优模 型权重和最优聚合策略进行前向计算，即可完成未知数据的声音事件检测。具体过程如下：
1)首先，由于每条信号长度为240帧，因此，自适应层次聚合中可选的子包数量R为240的所有因数集合N +，即N +＝{1,2,3,…,120,240}；利用连续松弛法将可选的子包数量的离散搜索空间转换成可优化的连续搜索空间，能够与卷积循环神经网络4联合优化，实现自动为每类声音事件选择最优子包数量，即自适应地定制特定事件的最优聚合策略；
2)为每类声音事件设置一组低维的、可学习的松弛化参数对应离散搜索空间中所有元素，利用Softmax激活求得搜索空间中所有备选项选择的概率，根据此概率遍历该离散空间，得到每类声音事件的激活期望，实现将离散的搜索空间连续松弛化；具体地：
假设DCASE2017数据集中共有K种声音事件，搜索空间大小为N(即N +中元素个数)，利用一组可学习的参数α∈R K×N将该离散搜索空间松弛化，得到选择不同R的概率集合λ：
Figure PCTCN2022101361-appb-000013
其中，α k为第k维松弛化参数，softmax()为运算符；
3)以第k类声音事件为例，选择N +中某一个元素作为当前确定的子包数量R，将帧级别预测概率
Figure PCTCN2022101361-appb-000014
分割至连续的R个子包中，得到子包集合B＝{b 1,b 2,…,b R}，如图2所示，每条黑色虚线所包含的帧样本属于同一个子包；利用最大值聚合φ max计算每个子包中最大概率值，即最显著的信息，得到子包级预测集合
Figure PCTCN2022101361-appb-000015
Figure PCTCN2022101361-appb-000016
其中，b r为子包集合B中的第r个元素；
4)随后，利用均值聚合φ avg取子包级预测集合的平均概率作为最终句级别预测概率
Figure PCTCN2022101361-appb-000017
Figure PCTCN2022101361-appb-000018
最终，当子包数量为R时，第k类声音事件的激活概率(即句级别预测概率)可以表示为：
Figure PCTCN2022101361-appb-000019
其中，子包数量R决定聚合策略，当子包数量越多时，最大值聚合作用比例越小，均值聚合比例越大，即更多地关注全局信息；当子包数量较少时，最大值聚合作用比例较大，均值聚合比例较少，即更多地关注局部显著信息；自适应层次聚合为每类声音事件自动学习子包数量，实现特定事件定制化的聚合策略；
5)重复过程3)和4)，遍历N +中所有可选的子包数量，即R←N +，得到所有情况下第k类声音事件的句级别预测概率，结合公式(1)所得概率，得到第k类声音事件期望的激活概率：
Figure PCTCN2022101361-appb-000020
其中，
Figure PCTCN2022101361-appb-000021
表示第k类事件选择R个子包数量的概率；
6)利用二进制交叉熵函数(BCELoss)计算所有类别的预测误差L，并完成反向传播，联合优化模型参数和松弛化参数(即训练阶段，训练的模型为现有技术)直至收敛：
Figure PCTCN2022101361-appb-000022
其中，W、α分别为模型参数和松弛化参数，X、Y分别为模型输入的梅尔频谱特征和句级别标签，Y k为第k类声音事件的句级别标签；
7)以上基于连续松弛法联合优化过程完成后，即可得到最优模型权重W *以及最优松弛化参数α *，利用公式(1)即可计算最优松弛化参数下选择不同R的概率λ *；此时，对于第k类声音事件，
Figure PCTCN2022101361-appb-000023
中最大选择概率对应的R即为当前类别最优子包数量
Figure PCTCN2022101361-appb-000024
至此，自适应层次聚合算法完成了为每类声音事件定制一个最优聚合策略
Figure PCTCN2022101361-appb-000025
利用连续松弛法联合优化，仅需引入一组低维松弛化参数，即可高效完成联合优化；相比于模型参数，松弛化参数数量微乎其微，且优化只需关注声音事件的自然属性，如时长、周期等全局特性，而无需关注高维的信号内容，因此易于收敛，从而引导模型参数在最优聚合策略下快速收敛直全局最优。人工选择子包数量并重复训练卷积循环神经网络也能够找到最优聚合策略，但其计算复杂度高达O(N K)，其中N表示搜索空间中可选子包数量大小，K表示声音事件种类数。本发明利用连续松弛法联合优化，仅需引入一组低维松弛化参数，即可将计算复杂度降低至O(N)。
自适应层次聚合是一个由一组独立的、可学习参数控制的独立模块，模块的输入是模型预测的音频定位结果，输出为音频分类结果，自适应层次聚合方法可以方便、有效地嵌入到任何声学模型中实现弱监督声音事件检测；以卷积循环神经网络为基准模型能够同时学习空间和时序上下文特征的多尺度声学特征，为声音事件检测领域的主流模型框架。
以上过程完成了卷积循环神经网络4和自适应层次聚合算法的联合优化，在卷积循环神经网络4测试阶段，给定未知的声音信号，进行预处理和特征提取后，送入训练后的卷积循环神经网络
Figure PCTCN2022101361-appb-000026
中，得到所有待检测目标事件的定位输出(帧级别预测概率)，实现音频定位任务，并根据每类事件特定的最优聚合策略
Figure PCTCN2022101361-appb-000027
得到所有类别的激活概率(句级别预测概率)，实现音频分类任务。
图3(a)、(b)为两条典型音频信号的声音事件定位结果可视化，对比方法为最大值聚合和加权平均聚合。其中，最大值聚合仅能够捕捉显著区域而造成定位边界不完整，尤其在检测时长较长的声音事件时(如“火车声”)，而加权平均聚合总是带来较多的误检测，尤其当检测较短的或多片段的声音事件时(如“尖叫声”和“鸣笛”)。图3(a)、(b)中三种典 型声音事件的定位效果均证实：自适应层次聚合不仅能够捕捉多片段的显著信息从而丢弃冗余信息，还能够在降低误检测率同时保证定位边界的完整，实现最优的声音事件检测性能。
基于与自适应层次聚合的弱监督声音事件检测方法相同的发明构思，本申请还提供了一种电子设备，该电子设备包括一个或多个处理器和一个或多个存储器，存储器中存储了计算机可读代码，其中，计算机可读代码当由一个或多个处理器执行时，进行本发明一种基于自适应层次聚合的弱监督声音事件检测方法的实施。其中，存储器可以包括非易失性存储介质和内存储器；非易失性存储介质可存储操作系统和计算机可读代码。该计算机可读代码包括程序指令，该程序指令被执行时，可使得处理器执行任意一种基于自适应层次聚合的弱监督声音事件检测方法。处理器用于提供计算和控制能力，支撑整个电子设备的运行。存储器为非易失性存储介质中的计算机可读代码的运行提供环境，该计算机可读代码被处理器执行时，可使得处理器执行任意一种基于自适应层次聚合的弱监督声音事件检测方法。
应当理解的是，处理器可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
其中，所述计算机可读存储介质可以是前述实施例所述电子设备的内部存储单元，例如所述计算机设备的硬盘或内存。所述计算机可读存储介质也可以是所述电子设备的外部存储设备，例如所述电子设备上配备的插接式硬盘、智能存储卡(SmartMedia Card，SMC)、安全数字(Secure Digital，SD)卡、闪存卡(Flash Card)等。
所述实施例为本发明的优选的实施方式，但本发明并不限于上述实施方式，在不背离本发明的实质内容的情况下，本领域技术人员能够做出的任何显而易见的改进、替换或变型均属于本发明的保护范围。

Claims (10)

  1. 自适应层次聚合的弱监督声音事件检测方法，其特征在于：
    提取预处理音频信号的声学特征，并输入声学模型，将声学模型预测的帧级别预测概率序列分成若干个连续的子包，利用最大值聚合计算每个子包的显著信息，得到子包级预测集合，利用均值聚合取子包级预测集合的平均概率作为句级别预测概率；
    联合优化声学模型和松弛化参数，直至收敛，得到最优模型权重和最优松弛化参数，根据最优松弛化参数为每类声音事件制定最优聚合策略；
    给定未知的音频信号，进行预处理和特征提取，送入训练后的声学模型，得到所有目标声音事件的帧级别预测概率，实现音频定位任务，并根据每类目标声音事件的最优聚合策略，得到所有目标声音事件类别的句级别预测概率，实现音频分类任务。
  2. 根据权利要求1所述的弱监督声音事件检测方法，其特征在于，所述制定最优聚合策略具体为：利用
    Figure PCTCN2022101361-appb-100001
    计算最优松弛化参数下选择不同R的概率λ *，对于第k类声音事件，
    Figure PCTCN2022101361-appb-100002
    中最大选择概率对应的R即为当前类别最优子包数量
    Figure PCTCN2022101361-appb-100003
    其中：λ为概率集合，R为子包数量，α k为第k维松弛化参数，softmax()为运算符。
  3. 根据权利要求2所述的弱监督声音事件检测方法，其特征在于，所述当子包数量为R时，第k类声音事件的句级别预测概率表示为：
    Figure PCTCN2022101361-appb-100004
    其中：
    Figure PCTCN2022101361-appb-100005
    为第k类声音事件的句级别预测概率，φ hi为自适应层次聚合算法，F w表示声学模型，φ avg表示均值聚合，φ max表示最大值聚合，b r为子包集合B中的第r个元素。
  4. 根据权利要求3所述的弱监督声音事件检测方法，其特征在于，所述第k类声音事件的句级别预测期望概率
    Figure PCTCN2022101361-appb-100006
    为：
    Figure PCTCN2022101361-appb-100007
    其中：
    Figure PCTCN2022101361-appb-100008
    表示第k类事件选择R个子包数量的概率，N +为所有可选的子包数量的集合。
  5. 根据权利要求4所述的弱监督声音事件检测方法，其特征在于，所述联合优化声学模型和松弛化参数采用反向传播进行的：
    Figure PCTCN2022101361-appb-100009
    其中：L所有声音事件类别的平均预测误差，W、α分别为模型参数和松弛化参数，X、Y分别为模型输入的梅尔频谱特征和句级别标签，Y k为第k类声音事件的句级别标签，BCELoss表示二进制交叉熵函数，K为声音事件的类别总数。
  6. 根据权利要求1所述的弱监督声音事件检测方法，其特征在于，所述声学模型为任意 主流的深度学习模型，声学模型的基准模型为卷积循环神经网络模型。
  7. 根据权利要求1所述的弱监督声音事件检测方法，其特征在于，提取的特征为梅尔频谱特征。
  8. 根据权利要求7所述的弱监督声音事件检测方法，其特征在于，所述声学模型训练和验证采用DCASE2017数据集。
  9. 根据权利要求8所述的弱监督声音事件检测方法，其特征在于，所述音频信号下采样至16kHz，帧长和帧移分别设置为1024、664，分帧后每条信号得到240帧样本，梅尔频谱特征为64维。
  10. 一种实现权利要求1-9任一项所述的弱监督声音事件检测方法的系统，其特征在于，包括依次相连的声学模型和自适应层次聚合算法模块，所述声学模型输入预处理和特征提取的音频信号，所述声学模型预测得到帧级别预测概率，所述自适应层次聚合算法模块将帧级别预测概率聚合得到句级别预测概率。
PCT/CN2022/101361 2022-05-16 2022-06-27 自适应层次聚合的弱监督声音事件检测方法及系统 Ceased WO2023221237A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/035,934 US12080319B2 (en) 2022-05-16 2022-06-27 Weakly-supervised sound event detection method and system based on adaptive hierarchical pooling
GB2307040.2A GB2616363B (en) 2022-05-16 2022-06-27 Weakly-supervised sound event detection method and system based on adaptive hierarchical pooling

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210528373.0 2022-05-16
CN202210528373.0A CN114974303B (zh) 2022-05-16 2022-05-16 自适应层次聚合的弱监督声音事件检测方法及系统

Publications (1)

Publication Number Publication Date
WO2023221237A1 true WO2023221237A1 (zh) 2023-11-23

Family

ID=82983548

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/101361 Ceased WO2023221237A1 (zh) 2022-05-16 2022-06-27 自适应层次聚合的弱监督声音事件检测方法及系统

Country Status (2)

Country Link
CN (1) CN114974303B (zh)
WO (1) WO2023221237A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119864049A (zh) * 2024-12-04 2025-04-22 华南农业大学 基于多尺度感知和时域动态增强的鸡群声音事件检测方法
CN120048284A (zh) * 2025-02-25 2025-05-27 江苏大学 类分布与时序上下文协同提示的声音事件检测方法及系统
CN120564108A (zh) * 2025-07-30 2025-08-29 山东大学 基于记忆-边缘引导的弱监督视频显著性目标检测方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117131415B (zh) * 2023-08-11 2025-03-21 广州大学 一种基于噪声事件检测模型的检测方法、装置及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170372725A1 (en) * 2016-06-28 2017-12-28 Pindrop Security, Inc. System and method for cluster-based audio event detection
CN108648748A (zh) * 2018-03-30 2018-10-12 沈阳工业大学 医院噪声环境下的声学事件检测方法
CN110827804A (zh) * 2019-11-14 2020-02-21 福州大学 一种音频帧序列到事件标签序列的声音事件标注方法
CN111933188A (zh) * 2020-09-14 2020-11-13 电子科技大学 一种基于卷积神经网络的声音事件检测方法
CN112036477A (zh) * 2020-08-28 2020-12-04 清华大学 一种高召回率弱标注声音事件检测方法
CN112786029A (zh) * 2020-12-25 2021-05-11 苏州思必驰信息科技有限公司 使用弱监督数据训练vad的方法及装置
CN113707175A (zh) * 2021-08-24 2021-11-26 上海师范大学 基于特征分解分类器与自适应后处理的声学事件检测系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2577570A (en) * 2018-09-28 2020-04-01 Cirrus Logic Int Semiconductor Ltd Sound event detection
CN111933109A (zh) * 2020-07-24 2020-11-13 南京烽火星空通信发展有限公司 一种音频监测方法及系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170372725A1 (en) * 2016-06-28 2017-12-28 Pindrop Security, Inc. System and method for cluster-based audio event detection
CN108648748A (zh) * 2018-03-30 2018-10-12 沈阳工业大学 医院噪声环境下的声学事件检测方法
CN110827804A (zh) * 2019-11-14 2020-02-21 福州大学 一种音频帧序列到事件标签序列的声音事件标注方法
CN112036477A (zh) * 2020-08-28 2020-12-04 清华大学 一种高召回率弱标注声音事件检测方法
CN111933188A (zh) * 2020-09-14 2020-11-13 电子科技大学 一种基于卷积神经网络的声音事件检测方法
CN112786029A (zh) * 2020-12-25 2021-05-11 苏州思必驰信息科技有限公司 使用弱监督数据训练vad的方法及装置
CN113707175A (zh) * 2021-08-24 2021-11-26 上海师范大学 基于特征分解分类器与自适应后处理的声学事件检测系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119864049A (zh) * 2024-12-04 2025-04-22 华南农业大学 基于多尺度感知和时域动态增强的鸡群声音事件检测方法
CN120048284A (zh) * 2025-02-25 2025-05-27 江苏大学 类分布与时序上下文协同提示的声音事件检测方法及系统
CN120564108A (zh) * 2025-07-30 2025-08-29 山东大学 基于记忆-边缘引导的弱监督视频显著性目标检测方法及系统

Also Published As

Publication number Publication date
CN114974303B (zh) 2023-05-12
CN114974303A (zh) 2022-08-30

Similar Documents

Publication Publication Date Title
WO2023221237A1 (zh) 自适应层次聚合的弱监督声音事件检测方法及系统
WO2024045444A1 (zh) 一种视觉问答任务的处理方法、装置、设备和非易失性可读存储介质
US10957341B2 (en) Ultrasonic attack detection employing deep learning
CN113823264B (zh) 语音识别方法、装置、计算机可读存储介质及计算机设备
CN113039555B (zh) 在视频剪辑中进行动作分类的方法、系统及存储介质
CN111294646A (zh) 一种视频处理方法、装置、设备及存储介质
CN112368769A (zh) 端到端流关键词检出
CN107221326A (zh) 基于人工智能的语音唤醒方法、装置和计算机设备
CN111192590A (zh) 语音唤醒方法、装置、设备及存储介质
CN117807495A (zh) 基于多模态数据的模型训练方法、装置、设备及存储介质
CN116596916A (zh) 缺陷检测模型的训练和缺陷检测方法及其装置
CN113160823B (zh) 基于脉冲神经网络的语音唤醒方法、装置及电子设备
CN117063208A (zh) 无监督多模型联合推理系统和方法
CN114898737A (zh) 声学事件检测方法、装置、电子设备和存储介质
US12080319B2 (en) Weakly-supervised sound event detection method and system based on adaptive hierarchical pooling
WO2024183465A1 (zh) 一种模型确定方法和相关装置
CN114882903A (zh) 一种船舶辐射噪声识别方法、装置、设备及可读存储介质
Lim et al. Weakly labeled semi-supervised sound event detection using CRNN with inception module.
CN118199933A (zh) 基于特征选择及transformer-BiGRU的入侵检测方法及系统
CN110490304A (zh) 一种数据处理方法及设备
CN111722696A (zh) 用于低功耗设备的语音数据处理方法和装置
CN115130569B (zh) 音频处理方法、装置及计算机设备、存储介质、程序产品
CN120220013B (zh) 基于双向交互与动态特征增强的时序动作定位方法及装置
CN114299964A (zh) 声线识别模型的训练方法和装置、声线识别方法和装置
US12574820B2 (en) Artificial-intelligence based routing policies for wireless data transmission

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 18035934

Country of ref document: US

ENP Entry into the national phase

Ref document number: 202307040

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20220627

WWE Wipo information: entry into national phase

Ref document number: 2307040.2

Country of ref document: GB

WWP Wipo information: published in national office

Ref document number: 2307040.2

Country of ref document: GB

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22942268

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 2307040.2

Country of ref document: GB

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22942268

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 22942268

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16.06.2025)