WO2019119687A1 - 基于机器学习的防御无声指令控制语音助手的方法 - Google Patents

基于机器学习的防御无声指令控制语音助手的方法 Download PDF

Info

Publication number
WO2019119687A1
WO2019119687A1 PCT/CN2018/083371 CN2018083371W WO2019119687A1 WO 2019119687 A1 WO2019119687 A1 WO 2019119687A1 CN 2018083371 W CN2018083371 W CN 2018083371W WO 2019119687 A1 WO2019119687 A1 WO 2019119687A1
Authority
WO
WIPO (PCT)
Prior art keywords
voice
data
sample
malicious
feature
Prior art date
Application number
PCT/CN2018/083371
Other languages
English (en)
French (fr)
Inventor
徐文渊
冀晓宇
张国明
闫琛
张天晨
张泰民
Original Assignee
浙江大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江大学 filed Critical 浙江大学
Priority to KR1020207007398A priority Critical patent/KR102386155B1/ko
Publication of WO2019119687A1 publication Critical patent/WO2019119687A1/zh
Priority to US16/853,782 priority patent/US11450324B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/26—Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063—Training
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination

Definitions

  • the invention belongs to the field of artificial intelligence voice assistant security, and particularly relates to a method for controlling a voice assistant based on machine learning and defending a silent instruction.
  • voice assistants such as Siri, Google Now, Alexa, Cortana, S Voice, Hi Voice have become more and more popular in people's lives, almost all smart devices (smart phones, tablets) Computers, wearables, and smart audio are equipped with corresponding voice assistants.
  • voice assistants also face a variety of security risks, such as a method called "Dolphin Sound Attack” [Zhang, G., Yan, C., Ji, X., Zhang, T., Zhang, T., &Xu, W. (2017). DolphinAtack: Inaudible Voice Commands.arXiv preprint arXiv: 1708.09537.], using the vulnerability of the smart device microphone circuit, you can silently control the voice assistant to command the smart device to execute the corresponding instructions, for example: Silently turn on the voice assistant, dial the pay phone or monitor the phone, video call, send the paid message, switch the phone to the flight mode, operate the Audi car navigation system, shop, and even unlock the Nexus 7 silently.
  • the process of attack implementation is shown in Figure 1.
  • the attacker first modulates the ordinary audible speech signal into the ultrasonic frequency band by amplitude modulation and then sends it out through the ultrasonic transmitter. At this time, the human ear cannot hear it.
  • the microphone circuit of the smart device can hear and can demodulate the amplitude modulated signal to recover the voice signal before the modulation.
  • the voice assistant recognizes the voice signal and controls the smart device to perform a malicious operation.
  • Visit a malicious website Then make a pass-through download attack and then use the 0day vulnerability to attack the user device.
  • Implant false information The attacker can use the user device to send fake text messages, emails, post fake blog posts, or add fake events to the schedule.
  • the hardware-based solution is to recommend that the manufacturer redesign the microphone circuit so that the high-frequency sound cannot be accepted by the microphone, or the hardware circuit filters out the high-frequency sound.
  • the software-based defense scheme utilizes the difference between audible and inaudible sounds to distinguish between normal and inaudible instructions through machine learning.
  • the prior art successfully distinguishes the normal sound and the inaudible sound command "Hey” by using a Support Vector Machine (SVM).
  • SVM Support Vector Machine
  • the first step use the mobile phone to record 5 audible voice commands ‘Hey’ as a positive sample, and then record the inaudible voice command “Hey” as a negative sample.
  • Step 2 Train the SVM classifier with these positive and negative samples.
  • Step 3 Identify the audible and inaudible voice commands using the trained classifier.
  • the manufacturer is unable to make hardware improvements to the smart devices that have been sold or the recall cost is unacceptable to the manufacturer.
  • the microphone circuit can accept the ultrasonic wave is always a problem.
  • the hardware-based defense scheme is a high-cost, difficult, long-term defense. The solution cannot effectively solve the existing problems.
  • the present invention proposes a method based on machine learning defense "dolphin sound attack”, which can effectively identify whether the received voice instruction is malicious or normal, thereby canceling the malicious instruction.
  • the identification and execution of the software level eliminates the possibility of "dolphin sound attack”.
  • the training phase of the classifier is performed.
  • Step one collecting sample data. Normal speech data and malicious speech data are used as positive and negative samples.
  • Step 2 Preprocessing the voice signal, classifying the collected voice data (divided into two types of normal voice data and malicious voice data) and segmenting the voice data.
  • the segmentation of the speech data includes: finding the beginning and end of each speech signal, and performing data segmentation.
  • Step 3 calculate sample characteristics, normalize and label.
  • To calculate the sample features from the segmented sample data we need to calculate the minimum, average, energy, skewness, kurtosis, spectral mean, spectral variance, spectral standard deviation, irregularity, and average of all data points for each sample. Square root amplitude, spectrum centroid, 11 features. This part of the input is sample data, and the output is the time-frequency domain feature file of each sample data.
  • step four the support vector machine (SVM) classifier is trained.
  • SVM classification algorithm is the core of our detection.
  • An SVM classifier needs to be trained based on positive and negative sample data before using the SVM.
  • the above four steps are the training phase of the SVM classifier.
  • the following is the use of the SVM classifier to detect malicious voice commands.
  • the voice assistant Before the voice assistant receives the voice command to be recognized, it needs to be tested by the SVM classifier. Only when the detection result is a normal command, the voice recognition is performed, otherwise the alarm is issued or discarded. The voice command.
  • step 5 the voice signal received and demodulated by the microphone circuit is preprocessed, as shown in step 2.
  • Step 6 Calculate the features, normalize, see step 3, but does not include labeling operations.
  • step 7 the normalized sample data is input to the SVM classifier for identification.
  • N is the number of sample data points
  • x n is the power consumption value of the nth point
  • ⁇ is the standard deviation
  • a n is the energy at the nth frequency f n after the frequency domain transformation.
  • V s is the spectral variance
  • z(n) is the weighted frequency value
  • y(n) represents the center frequency of the sample point
  • the minimum is the minimum power consumption of all points.
  • the SVM classification algorithm we can judge whether the current sample is a positive sample according to the time domain frequency domain characteristics. This part of the input is a normalized time domain frequency domain feature, and the output is the SVM classification prediction result, that is, whether the current voice is a malicious voice instruction.
  • the final recognition accuracy and reliability largely depend on the selection of feature vectors and the selection of classifiers.
  • the selection of the eigenvectors determines the upper limit of the classifier's classification ability.
  • the training method of the classifier largely determines the rate at which the classification is performed.
  • Figure 1 is a schematic diagram of a voice assistant capable of receiving a malicious voice signal.
  • the upper half of Figure 1 is the spectrum of the signal, and the lower half is the microphone circuit of the smart device.
  • the attacker can modulate the audible low-frequency signal into the ultrasonic frequency band and send it out. At this time, the human ear can't hear it. There is only high-frequency signal in the spectrogram. After receiving the signal, the microphone circuit on the smart device will modulate. The audible low-frequency signal is demodulated. At this time, it can be seen from the spectrogram that there is a low-frequency signal, and this phenomenon is caused by the nonlinear action of the microphone circuit, which is also a loophole in the hardware. The voice assistant recognizes "inaudible voice commands."
  • FIG. 2 is a comparison diagram of a normal speech signal and a malicious speech signal in a time-frequency domain
  • the voice command is “HeySiri”, the upper part is a normal audible voice signal, and the lower part is a voice signal demodulated by the microphone circuit.
  • the demodulated signal can be seen from the frequency domain diagram of the right half.
  • the intensity at the high frequency band is much lower at the high frequency band than the normal speech signal.
  • 3 is a diagram showing the training process of the detector SVM and the detection process of the malicious voice command.
  • FIG. 4 is a flow chart of the improved voice control.
  • the SVM classifier is used as a new module to detect malicious voice signals.
  • Figure 2 is a comparison of the normal speech signal and the malicious speech signal in the time-frequency domain; through Figure 2, it can be found that the ordinary speech signal and the malicious signal (demodulated signal) recorded by the mobile phone are significantly different in the high frequency band. . Therefore, malicious voice commands can be identified by machine learning algorithms. As shown in FIGS. 3 and 4, the present invention performs targeted defense against the "dolphin sound attack” and can detect the silent voice command of the "dolphin sound attack", thereby fundamentally solving the voice assistant, and the smart device is silent voice command. The possibility of control.
  • the following embodiment is a method for controlling a voice assistant based on a machine learning-based defense silent command, and the tested defense device is an iPhone SE.
  • the defense of other different types of smart devices is consistent with this method and will not be further explained.
  • the acquisition of positive and negative samples should take into account different vocals and different voice commands.
  • TIMIT speech library which contains 630 speaker speeches.
  • the languages they speak include the eight dialects of American English, and each person reads 10 speech-rich sentences.
  • Positive sample generation We selected 10 people's voices (including 100 sentences) from the TIMIT speech library as positive samples.
  • the threshold is used to detect, and the obtained sound data is segmented by a length of 20 ms, if the signal strength of four consecutive segments is (x i is the value of the ith data point) is greater than a threshold to determine the starting position of the first segment as the starting position of the entire statement. Similarly, if the signal strength of four consecutive segments is less than a threshold, the start position of the first segment can be determined as the end position of the overall sentence.
  • the voice command can be split according to the start position and the end position.
  • the first features that can be conceived are the average and minimum indicators. However, these indicators cannot fully reflect the difference between normal speech signals and malicious speech signals. We need statistical analysis to select More features that respond effectively to normal, abnormal speech signals. After a series of optimizations, after extracting the features of the speech data, 11 indicators that best reflect the difference between normal and malicious speech are selected from the calculated time-frequency features. We use these features instead of the acquired speech data as input to the next machine learning process.
  • the eigenvectors include: minimum, mean, energy, skewness, kurtosis, spectral mean, spectral variance, spectral standard deviation, irregularity, root mean square amplitude, spectral centroid.
  • each sample feature is set to 1. This part of the input is the time domain frequency domain feature of each sample, and the output is a normalized time domain frequency domain feature.
  • the normalized time-frequency domain features are labeled, the normal voice command feature has a label of 1, and the malicious voice feature label has a value of zero.
  • the classifier After selecting the SVM classifier, the classifier is trained and a detection model is generated for detecting malicious voice commands.
  • the trained SVM detection model can be ported to the voice assistant in the system, where we test on the computer.
  • the normal, malicious voice commands recorded are first subjected to pre-processing and feature extraction operations, and then detected using a trained SVM classifier.
  • the normal speech generation process to be tested We select the voice of 10 people from the TIMIT speech database as the normal speech to be tested.
  • the process of generating malicious voice to be tested using a signal generator (Keysight Technologies.2017.N5172B EXG X ⁇ Series RF Vector Signal Generator, 9 kHz to 6 GHz. http://www.keysight.com/en/pdx ⁇ x201910 ⁇ pn ⁇ N5172B. (2017).) Modulate the voice of the sample to the ultrasonic band through the ultrasonic speaker [Avisoft Bioacoustics.2017.Ultrasonic Dynamic Speaker Vifa.http://www.avisoft.com/usg/vifa.htm. (2017).] Play out (modulation method: amplitude modulation, modulation depth: 100%, carrier frequency: 25kHz), and use the iPhone SE mobile phone to record these inaudible sounds to get demodulated malicious voice signals and treat them as Measure malicious voice.
  • a signal generator Keysight Technologies.2017.N5172B EXG X ⁇ Series RF Vector Signal Generator, 9 kHz to 6 GHz. http://www.keysight

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Telephonic Communication Services (AREA)
  • Telephone Function (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)

Abstract

一种基于机器学习的防御无声指令控制语音助手的方法，步骤如下：1)采集正负样本的数据；2)对正负样本数据进行数据切分；3)样本特征选取与归一化；4)选取分类器进行训练并生成用来对恶意语音指令的检测模型；5)利用得到的检测模型对待检测语音指令进行检测。针对不同型号的智能设备，需要通过该类型的智能设备获取正常的语音命令和恶意的语音命令，并作为正负样本训练一个特定的针对该类型设备的分类器。通过这种定制化的方式，可以很好的解决跨设备无法进行检测和防御的难题。

Description

基于机器学习的防御无声指令控制语音助手的方法 技术领域
本发明属于人工智能语音助手安全领域，具体涉及一种基于机器学习的防御无声指令控制语音助手的方法。
背景技术
作为一种方便有效的人机交互方法，Siri，Google Now，Alexa，Cortana，S Voice，Hi Voice等语音助手在人们的生活中变得越来越流行，几乎所有的智能设备(智能手机，平板电脑，可穿戴设备以及智能音响)都搭载着相应的语音助手。
然而，语音助手也面临着各种各样的安全隐患，例如有一种名为“海豚音攻击”的方法[Zhang,G.,Yan,C.,Ji,X.,Zhang,T.,Zhang,T.,&Xu,W.(2017).DolphinAtack:Inaudible Voice Commands.arXiv preprint arXiv:1708.09537.]，利用智能设备麦克风电路的漏洞，可以无声地控制语音助手从而命令智能设备执行相应的指令，例如：无声的开启语音助手，拨打付费电话或者监听电话，视频通话，发送付费短信，将手机切换到飞行模式，操作奥迪汽车的导航系统，购物，甚至能够无声解锁Nexus 7等。因此，攻击者可以在用户不知道的情况下操纵其智能设备，造成隐私泄露，财产损失等一系列的安全问题。攻击实现的过程如图1，攻击者首先将普通可听见的语音信号通过调幅的方式将其调制到超声波频段然后再通过超声波发送器发送出去，此时人耳是听不到。而智能设备的麦克风电路能够听到并且能够将该调幅信号进行解调，恢复出调制之前的语音信号，此时语音助手识别到该语音信号，并控制智能设备执行恶意的操作。
虽然各种搭载语音助手的智能设备的功能有细微差异，但攻击者能够完成的恶意操作有：
1.访问恶意网站：然后进行路过式下载攻击，然后使用0day漏洞攻击用户设备。
2.监控：攻击者可以拨打语音或者视频电话，从而获取到用户周围的声音 和图像。
3.植入虚假信息：攻击者可以使用用户设备发送虚假短信、邮件、发布假的博文，或者把虚假的事件添加到日程。
4.拒绝服务：攻击者可以打开飞行模式，让设备断网。
5.隐蔽攻击：屏幕显示以及语音反馈都可能暴露攻击，但是黑客可以降低屏幕亮度或者音量，达到隐藏的目的。
“海豚音攻击”是利用智能设备的硬件漏洞而发起的一种新型攻击，目前还没有具体可行的防御方案，现有技术[Zhang,G.,Yan,C.,Ji,X.,Zhang,T.,Zhang,T.,&Xu,W.(2017).DolphinAtack:Inaudible Voice Commands.arXiv preprint arXiv:1708.09537.]提出两种防御方案：基于硬件的防御方案和基于软件的防御方案。
其中基于硬件的解决方案是建议生产厂商重新设计麦克风电路使得高频声音不能被麦克风接受，或者硬件电路识别到高频的声音后将其滤除。
基于软件的防御方案是利用可听声音和不可听声音之间的差异，通过机器学习的方法，区分正常指令和不可听指令。现有技术利用支持向量机(Support Vector Machine,SVM)成功区分出了正常的声音和不可听的声音指令“Hey”。
具体操作如下
第一步：用手机录下5个可听的语音指令‘Hey’作为正样本，然后录下不可听的语音指令“Hey”作为负样本。
第二步：使用这些正负样本训练SVM分类器。
第三步：使用训练好的分类器识别可听与不可听语音指令。
3)分析2)中提及的现有技术存在什么样的不足，即发明所要解决的技术问题。
现有技术提出的基于软硬件的防御方案的不足之处在于：
1.厂家是无法对已经出售的智能设备进行硬件上的改进或者说召回成本是厂家无法承受的。
2.麦克风电路能够接受超声波是一直存在的问题，尽管随着技术，工艺的进步和发展，也未能有效的解决，因此基于硬件的防御方案是一种高成本，难度大，周期长的防御方案，不能有效的解决目前存在的问题。
3.现有技术中基于软件的方案只在一台智能设备，测试了一句语音指令 “Hey”，因此不能确定该方法能否在不同智能设备，不同语音指令，不同人声的防御效果。
发明内容
为了抵御“海豚音攻击”，本发明提出了一种基于机器学习防御“海豚音攻击”的方法，这种方法可以有效的识别接收到的语音指令是恶意的还是正常的指令，进而取消恶意指令的识别和执行，从软件层面杜绝了“海豚音攻击”的可能。
本发明的技术方案具体如下：
首先进行分类器的训练阶段。
步骤一，采集样本数据。正常语音数据和恶意语音数据作为正负样本。
步骤二，语音信号预处理，将采集到的语音数据进行归类(分为两类正常语音数据和恶意语音数据)和语音数据切分。语音数据切分包括：找到每句语音信号的开始和结束部分，进行数据切分。
步骤三，计算样本特征，归一化与贴标签。从切分样本数据中计算出样本特征，我们需要计算每个样本所有数据点的最小值、平均值、能量、偏度、峰度、频谱均值、频谱方差、频谱标准差、不规律性、均方根振幅、频谱质心，11个特征。这一部分输入为样本数据，输出为每个样本数据的时频域特征文件。
为了使我们的分类算法分类准确率更高，样本特征的归一化是需要的。我们需要对这些样本特征做归一化处理，每个样本特征最大值设定为1。这一部分输入为每个样本的时域频域特征，输出为归一化的时域频域特征。
最后，将归一化后的时频域特征贴上标签，正常语音指令的特征为1，恶意语音特征为0.
步骤四，训练支持向量机(SVM)分类器。SVM的分类算法为我们检测的核心。在使用SVM之前需要根据正负样本数据训练一个SVM分类器。
以上四个步骤是SVM分类器的训练阶段。以下是利用SVM分类器检测恶意语音命令，在语音助手接收到待识别的语音命令之前，需要经过SVM分类器的检测，只有检测结果为正常指令时，才进行语音识别，否则发出报警或者丢弃掉该语音命令。
步骤五，将麦克风电路接收并解调后的语音信号进行预处理,见步骤二。
步骤六，计算特征，归一化，见步骤三，但是不包括贴标签的操作。
步骤七，将归一化的样本数据输入到SVM分类器进行识别。
其中各特征定义如下：
平均值：
Figure PCTCN2018083371-appb-000001
能量:
Figure PCTCN2018083371-appb-000002
偏度:
Figure PCTCN2018083371-appb-000003
峰度:
Figure PCTCN2018083371-appb-000004
频谱均值：
Figure PCTCN2018083371-appb-000005
频谱方差：
Figure PCTCN2018083371-appb-000006
频谱标准差：
Figure PCTCN2018083371-appb-000007
不规律性:
Figure PCTCN2018083371-appb-000008
均方根振幅:
Figure PCTCN2018083371-appb-000009
频谱质心：
Figure PCTCN2018083371-appb-000010
其中，N为样本数据点数，x n是第n个点的功耗值，σ是标准差，a n是进行频域变换后第n个频率f n处的能量大小，
Figure PCTCN2018083371-appb-000011
是频谱均值，V s是频谱方差，z(n)是加权频率值，y(n)代表样本点的中心频率；所述的最小值即为所有点的功耗最小值。
利用SVM分类算法我们可以根据时域频域特征判断当前样本是否为正样本。这一部分输入为归一化的时域频域特征，输出为SVM分类预测结果，即当前语音是否为恶意语音指令。
自此，就实现了恶意语音命令的检测和对语音助手的防御。
(1)经过对正常语音信号和恶意语音信号的分析发现,恶意语音信号在高频段是被抑制的。即,从不可听的语音命令还原出来的恶意语音指令在较高频段的强度比较小。不同的手机,不同语音命令,不同说话者在这一点上都是一样的,这是一个普遍的现象。因此,通过机器学习的方式可以进行区分和检测。
(2)对于任何一种机器学习方法,最终的识别精确度与可靠性很大程度上取决于特征向量的选取以及分类器的选取。特征向量的选取决定了分类器分类能力的上限，分类器的训练方法很大程度上决定了进行分类的速率。考虑到语音信号特征提取的重要性，我们首先通过分析语音指令的特点，选取能反映正常语音指令和恶意语音指令之间差异的特征向量，并在此基础上选择了合适的机器学习算 法。
(3)经过测试我们发现,不同智能设备的麦克风电路的非线性作用有很大的差异,对于同一个不可听的语音信号,不同智能设备解调出来的恶意语音信号是有一定的差异的。如果想要建立一个统一的检测模型是不可能或者是低效的而且也没有必要建立一个统一的模型。
针对不同型号的智能设备,需要通过该类型的智能设备获取正常的语音命令和恶意的语音命令,并作为正负样本训练一个特定的针对该类型设备的分类器。通过这种定制化的方式,可以很好的解决跨设备无法进行检测的难题。
附图说明
图1为语音助手能够接受到恶意语音信号的示意图。
图1的上半部分为信号的频谱示意，下半部分位是智能设备的麦克风电路示意。
攻击者可以将可听的低频信号调制到超声波频段发送出来，此时人耳是听不到的，频谱图中只有高频信号，智能设备上的麦克风电路接收到该信号后，会将调制之前可听的低频信号解调出来，此时从频谱图可以看到，有低频的信号产生，而这一现象是由于麦克风电路的非线性作用产生的，也正是这一硬件上的漏洞，使得语音助手能够识别到“不可听的语音指令”。
图2为正常语音信号和恶意语音信号在时频域的对比图；
此语音指令为“HeySiri”，上半部分为正常可听的语音信号，下半部分为麦克风电路解调出来的语音信号，从右半部分的频域图中可以看到，解调后的信号在高频段的强度相比于正常语音信号在高频段的强度低很多。
图3为检测器SVM的训练以及恶意语音指令的检测过程图。
图4为改进后的语音控制流程图,SVM分类器作为新增模块，用来检测恶意语音信号。
具体实施方式
下面对本发明的优选实施方式做出详细的说明。
图2为正常语音信号和恶意语音信号在时频域的对比图；通过图2，可以 发现，手机录到的普通语音信号和恶意信号(解调出来的信号)在高频段是有明显差异的。因此，可以通过机器学习算法识别恶意语音命令。如图3和4所示，本发明针对“海豚音攻击”进行针对性的防御，能够检测出”海豚音攻击”的无声语音指令，从而从根本上解决了语音助手，智能设备被无声语音指令控制的可能性。
以下实施例为基于机器学习的防御无声指令控制语音助手的方法，测试的防御设备为iPhone SE。其它不同类型智能设备的防御与该方法一致，不再做进一步说明。
分类器训练阶段：
正负样本的数据采集。
为了使训练后的分类器能够检测不同人声，语音命令的无声攻击，正负样本的获取上就应该考虑到不同人声和不同语音指令。
因此，我们选用了TIMIT语音库，该数据库含有630个说话人语音，他们说的语言包括美国英语的八大方言，并且每个人阅读10个语音丰富的句子。
正样本的产生方式：我们从TIMIT语音库中选取10个人的语音(共包括100句话)作为正样本。
负样本的产生方式：使用信号发生器(Keysight Technologies.2017.N5172B EXG X‐Series RF Vector Signal Generator,9kHz to 6 GHz.http://www.keysight.com/en/pdx‐x201910‐pn‐N5172B.(2017).)将正样样本的语音调制到超声波频段，通过超声波扬声器[Avisoft Bioacoustics.2017.Ultrasonic Dynamic Speaker Vifa.http://www.avisoft.com/usg/vifa.htm.(2017).]播放出来(调制方式为：调幅，调制深度：100％，载波频率：20‐45kHz)，并用iPhone SE手机录下这些人耳不可听的声音得到解调后的恶意语音信号并作为负样本(共包括10个说话者，100句话)。
1.数据预处理。
采集到以上数据后，首先进行正负样本的划分。然后是语音数据切分。
我们使用了如下语音数据切分算法：
为了检测出每句指令的何时开始和结束，使用阈值来检测，将得到的声音数据以20ms的长度进行切分，如果连续的4个片段的信号强度
Figure PCTCN2018083371-appb-000012
(x i为第i个数据点的值)大于一个阈值，即可确定第一个片段开始位置为整个语句的开始 位置。同理，连续4个片段信号强度小于一个阈值，即可确定第一个片段的开始位置为整体语句的结束位置。根据开始位置和结束位置就可以对语音指令进行切分。
2.样本特征选取，计算与归一化。
考虑到语音信号的特点，首先能想到的特征有平均值，最小值这类指标，但是这些指标不能够完全的反映出正常语音信号和恶意语音信号之间的差异，我们需要经过统计分析，选取更多能够有效反应正常，异常语音信号之间差异的特征。经过一系列优化，我们对语音数据进行特征提取之后，从计算得到的大量时频特征中，选取了最能反映正常和恶意语音之间差异性的11个指标。我们使用这些特征代替采集到的语音数据，将其作为下一步机器学习过程中的输入信息。特征向量包括：最小值、平均值、能量、偏度、峰度、频谱均值、频谱方差、频谱标准差、不规律性、均方根振幅、频谱质心。
为了使我们的分类算法分类准确率更高，需要对样本特征进行归一化处理。每个样本特征最大值设定为1。这一部分输入为每个样本的时域频域特征，输出为归一化的时域频域特征。
最后，将归一化后的时频域特征贴上标签，正常语音指令特征的标签为1，恶意语音特征标签为0.
3.在确定所需提取的特征信息之后，我们进行了机器学习算法的选取。我们使用正常语音和恶意语音数据作为机器学习的训练样本，而对于有限样本学习问题的SVM机器学习算法能够很好的适用于我们这一应用场景。
我们选用SVM机器学习算法有以下三个好处：
(1)由于SVM算法具有非常快的训练速度以及决策速度，使我们能够很快对恶意语音指令做出识别。
(2)由SVM算法本身的特点所决定，该算法对输入数据的维数不敏感，这个优点使我们可以在以后的工作中，寻找新的体现信息差异的特征，在对系统进行改进的同时而不会影响系统的效率。
(3)使用SVM算法还可以使得系统具有自学习功能,随着用于训练的数据增加,可以在分类器训练过程中不断修正各个参数的值，从而在此基础上构造一个较为完善的检测系统。
选取了SVM分类器后进行分类器的训练并生成一个检测模型用来对恶意语 音指令的检测。
恶意语音指令检测过程：
训练好的SVM检测模型可以移植到系统中的语音助手上，在这里，我们在电脑上进行的测试。首先将录制到的正常的，恶意的语音指令进行预处理和特征提取等操作，然后使用训练好的SVM分类器进行检测。
为了验证该方法的可行性，我们首先通过实验的方式测试了该方法对于正常语音命令和恶意语音命令的区分能力。
待测正常语音的产生过程：我们从TIMIT语音数据库中选取10个人的语音作为待测的正常语音。
待测恶意语音的产生过程：使用信号发生器(Keysight Technologies.2017.N5172B EXG X‐Series RF Vector Signal Generator,9 kHz to 6 GHz.http://www.keysight.com/en/pdx‐x201910‐pn‐N5172B.(2017).)将正样样本的语音调制到超声波频段，通过超声波扬声器[Avisoft Bioacoustics.2017.Ultrasonic Dynamic Speaker Vifa.http://www.avisoft.com/usg/vifa.htm.(2017).]播放出来(调制方式为：调幅，调制深度：100％，载波频率：25kHz)，并用iPhone SE手机录下这些人耳不可听的声音得到解调后的恶意语音信号并作为待测恶意语音。
我们测试了10个人，100句正常的语音和100句恶意的语音，最终实验结果显示，无论说话者以及语音指令是怎样的，分类器都能够以98％的准确率区分正常语音指令，以94％的准确率区分恶意语音指令。
通过最终的检测结果可以确定，针对不同的说话者，不同的命令，我们的方法对“海豚音攻击”都是有效的。因此，智能设备的生产厂商只需要升级系统或软件就可以快速，有效地解决目前各个智能设备存在的安全问题。

Claims (3)

  1. 一种基于机器学习的防御无声指令控制语音助手的方法，其特征在于包括如下步骤：
    1)采集正负样本的数据，
    2)对采集到的正负样本数据进行数据切分；
    3)样本特征选取与归一化；
    对语音数据进行特征特取，特征包括，最小值、平均值、能量、偏度、峰度、频谱均值、频谱方差、频谱标准差、不规律性、均方根振幅、频谱质心，将选取的11个特征构成特征向量，特征向量代替采集到的语音数据，将其作为之后过程中的输入信息，
    其中各特征定义如下：
    平均值：
    Figure PCTCN2018083371-appb-100001
    能量:
    Figure PCTCN2018083371-appb-100002
    偏度:
    Figure PCTCN2018083371-appb-100003
    峰度:
    Figure PCTCN2018083371-appb-100004
    频谱均值：
    Figure PCTCN2018083371-appb-100005
    频谱方差：
    Figure PCTCN2018083371-appb-100006
    频谱标准差：
    Figure PCTCN2018083371-appb-100007
    不规律性:
    Figure PCTCN2018083371-appb-100008
    均方根振幅:
    Figure PCTCN2018083371-appb-100009
    频谱质心：
    Figure PCTCN2018083371-appb-100010
    其中，N为样本数据点数，x n是第n个点的功耗值，σ是标准差，a n是进行频域变换后第n个频率f n处的能量大小，
    Figure PCTCN2018083371-appb-100011
    是频谱均值，V s是频谱方差，z(n)是加权频率值，y(n)代表样本点的中心频率；
    对样本特征向量进行归一化处理，每个样本特征最大值设定为1，归一化处理的输入为每个样本的时域频域特征，输出为归一化的时域频域特征；
    最后，将归一化后的时频域特征贴上标签，正常语音指令特征的标签为1，恶意语音特征标签为0；
    4)使用归一化后并贴上标签的时域频域特征作为模型输入，选取机器学习算法进行训练并生成一个检测模型用来对恶意语音指令的检测；
    5)利用步骤4)训练得到的检测模型对待检测语音指令进行检测。
  2. 如权利要求1所述的基于机器学习的防御无声指令控制语音助手的方法，其特征在于所述步骤1)采集的负样本的产生方式为：使用信号发生器将正样本的语音调制到超声波频段，通过超声波扬声器播放出来，所述的调制为调幅，调制深度：100％，载波频率范围20‐45kHz，录下这些声音得到解调后的恶意语音信号并作为负样本数据。
  3. 如权利要求1所述的基于机器学习的防御无声指令控制语音助手的方法，其特征在于数据切分方法为：将得到的声音数据以20ms的长度进行切分，如果连续的4个片段的信号强度
    Figure PCTCN2018083371-appb-100012
    大于一个阈值，其中x i为第i个数据点的值，即可确定第一个片段开始位置为整个语句的开始位置；同理，连续4个片段信号强度小于一个阈值，即可确定第一个片段的开始位置为整体语句的结束位置，根据开始位置和结束位置就可以对语音指令进行切分。
PCT/CN2018/083371 2017-12-19 2018-04-17 基于机器学习的防御无声指令控制语音助手的方法 WO2019119687A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020207007398A KR102386155B1 (ko) 2017-12-19 2018-04-17 머신 러닝 기반의 무음 명령에 의해 음성 어시스턴트가 제어되는 것을 방어하는 방법
US16/853,782 US11450324B2 (en) 2017-12-19 2020-04-21 Method of defending against inaudible attacks on voice assistant based on machine learning

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711374668.2 2017-12-19
CN201711374668.2A CN108172224B (zh) 2017-12-19 2017-12-19 基于机器学习的防御无声指令控制语音助手的方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/853,782 Continuation US11450324B2 (en) 2017-12-19 2020-04-21 Method of defending against inaudible attacks on voice assistant based on machine learning

Publications (1)

Publication Number Publication Date
WO2019119687A1 true WO2019119687A1 (zh) 2019-06-27

Family

ID=62522918

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/083371 WO2019119687A1 (zh) 2017-12-19 2018-04-17 基于机器学习的防御无声指令控制语音助手的方法

Country Status (4)

Country Link
US (1) US11450324B2 (zh)
KR (1) KR102386155B1 (zh)
CN (1) CN108172224B (zh)
WO (1) WO2019119687A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112216304A (zh) * 2020-09-22 2021-01-12 浙江大学 一种基于双麦克风系统检测和定位无声语音指令的方法
CN112839488A (zh) * 2021-01-15 2021-05-25 华南理工大学 一种深度神经网络对抗攻击的检测装置及检测方法
US11264047B2 (en) * 2017-10-20 2022-03-01 Board Of Trustees Of The University Of Illinois Causing a voice enabled device to defend against inaudible signal attacks

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108806702B (zh) * 2018-07-20 2020-07-03 北京航空航天大学 针对超声波语音隐藏攻击的检测方法及装置
CN110875058A (zh) * 2018-08-31 2020-03-10 中国移动通信有限公司研究院 一种语音通信处理方法、终端设备及服务器
EP3848855A4 (en) * 2018-09-19 2021-09-22 Huawei Technologies Co., Ltd. LEARNING METHOD AND DEVICE FOR AN INTENTIONAL DETECTION MODEL AND DEVICE
US11264035B2 (en) 2019-01-05 2022-03-01 Starkey Laboratories, Inc. Audio signal processing for automatic transcription using ear-wearable device
US11264029B2 (en) * 2019-01-05 2022-03-01 Starkey Laboratories, Inc. Local artificial intelligence assistant system with ear-wearable device
US11158315B2 (en) 2019-08-07 2021-10-26 International Business Machines Corporation Secure speech recognition
US11055652B1 (en) * 2019-11-22 2021-07-06 Anvilogic, Inc. System for sharing detection logic through a cloud-based exchange platform
US11399041B1 (en) 2019-11-22 2022-07-26 Anvilogic, Inc. System for determining rules for detecting security threats
US11290483B1 (en) 2020-04-07 2022-03-29 Anvilogic, Inc. Platform for developing high efficacy detection content
CN112235293B (zh) * 2020-10-14 2022-09-09 西北工业大学 一种面向恶意流量检测正负样本均衡生成的过采样方法
CN112489682B (zh) * 2020-11-25 2023-05-23 平安科技（深圳）有限公司 音频处理方法、装置、电子设备和存储介质
CN112581975B (zh) * 2020-12-11 2024-05-17 中国科学技术大学 基于信号混叠和双声道相关性的超声波语音指令防御方法
CN112628695B (zh) * 2020-12-24 2021-07-27 深圳市轻生活科技有限公司 一种语音控制台灯的控制方法和系统
CN113205801B (zh) * 2021-05-08 2024-03-19 国家计算机网络与信息安全管理中心 恶意语音样本的确定方法、装置、计算机设备和存储介质
CN113362836A (zh) * 2021-06-02 2021-09-07 腾讯音乐娱乐科技（深圳）有限公司 训练声码器方法、终端及存储介质
CN113593603A (zh) * 2021-07-27 2021-11-02 浙江大华技术股份有限公司 音频类别的确定方法、装置、存储介质及电子装置
CN114301569B (zh) * 2021-12-30 2024-02-20 浙江大学 一种基于电子设备陶瓷电容发声的麦克风语音注入方法
CN114049884B (zh) * 2022-01-11 2022-05-13 广州小鹏汽车科技有限公司 语音交互方法、车辆、计算机可读存储介质
CN114664311B (zh) * 2022-03-01 2023-05-05 浙江大学 一种记忆网络增强的变分推断无声攻击检测方法
CN114639375B (zh) * 2022-05-09 2022-08-23 杭州海康威视数字技术股份有限公司 基于音频切片调节的智能语音识别安全防御方法及装置
KR102516391B1 (ko) * 2022-09-02 2023-04-03 주식회사 액션파워 음성 구간 길이를 고려하여 오디오에서 음성 구간을 검출하는 방법

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106531159A (zh) * 2016-12-09 2017-03-22 宁波大学 一种基于设备本底噪声频谱特征的手机来源识别方法
CN107393525A (zh) * 2017-07-24 2017-11-24 湖南大学 一种融合特征评估和多层感知器的语音情感识别方法
CN107452371A (zh) * 2017-05-27 2017-12-08 北京字节跳动网络技术有限公司 一种语音分类模型的构建方法及装置
CN107452401A (zh) * 2017-05-27 2017-12-08 北京字节跳动网络技术有限公司 一种广告语音识别方法及装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3337588B2 (ja) * 1995-03-31 2002-10-21 松下電器産業株式会社 音声応答装置
US9544067B2 (en) * 2014-02-27 2017-01-10 Verizon Patent And Licensing Inc. Method and system for transmitting information using ultrasonic messages
EP3164865A1 (en) * 2014-07-04 2017-05-10 Intel Corporation Replay attack detection in automatic speaker verification systems
CN105869630B (zh) * 2016-06-27 2019-08-02 上海交通大学 基于深度学习的说话人语音欺骗攻击检测方法及系统
US10395650B2 (en) * 2017-06-05 2019-08-27 Google Llc Recorded media hotword trigger suppression
WO2019002831A1 (en) * 2017-06-27 2019-01-03 Cirrus Logic International Semiconductor Limited REPRODUCTIVE ATTACK DETECTION
GB2563953A (en) * 2017-06-28 2019-01-02 Cirrus Logic Int Semiconductor Ltd Detection of replay attack
GB201801874D0 (en) * 2017-10-13 2018-03-21 Cirrus Logic Int Semiconductor Ltd Improving robustness of speech processing system against ultrasound and dolphin attacks
US10672416B2 (en) * 2017-10-20 2020-06-02 Board Of Trustees Of The University Of Illinois Causing microphones to detect inaudible sounds and defense against inaudible attacks
US11062703B2 (en) * 2018-08-21 2021-07-13 Intel Corporation Automatic speech recognition with filler model processing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106531159A (zh) * 2016-12-09 2017-03-22 宁波大学 一种基于设备本底噪声频谱特征的手机来源识别方法
CN107452371A (zh) * 2017-05-27 2017-12-08 北京字节跳动网络技术有限公司 一种语音分类模型的构建方法及装置
CN107452401A (zh) * 2017-05-27 2017-12-08 北京字节跳动网络技术有限公司 一种广告语音识别方法及装置
CN107393525A (zh) * 2017-07-24 2017-11-24 湖南大学 一种融合特征评估和多层感知器的语音情感识别方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHANG, GUOMING: "DolphinAttack: Inaudible Voice Commands", CCS'17 PROCEEDINGS OF THE 2017 ACM SIGSAC CONFERENCE ON COMPUTER AND COMMUNICATIONS SECURITY, 3 November 2017 (2017-11-03), pages 103 - 117, XP055621084 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11264047B2 (en) * 2017-10-20 2022-03-01 Board Of Trustees Of The University Of Illinois Causing a voice enabled device to defend against inaudible signal attacks
CN112216304A (zh) * 2020-09-22 2021-01-12 浙江大学 一种基于双麦克风系统检测和定位无声语音指令的方法
CN112839488A (zh) * 2021-01-15 2021-05-25 华南理工大学 一种深度神经网络对抗攻击的检测装置及检测方法

Also Published As

Publication number Publication date
KR20200037399A (ko) 2020-04-08
CN108172224A (zh) 2018-06-15
KR102386155B1 (ko) 2022-04-12
US11450324B2 (en) 2022-09-20
CN108172224B (zh) 2019-08-27
US20200251117A1 (en) 2020-08-06

Similar Documents

Publication Publication Date Title
WO2019119687A1 (zh) 基于机器学习的防御无声指令控制语音助手的方法
Ahmed et al. Void: A fast and light voice liveness detection system
CN109493872B (zh) 语音信息验证方法及装置、电子设备、存储介质
CN103475490B (zh) 一种身份验证方法及装置
US10957341B2 (en) Ultrasonic attack detection employing deep learning
US20150046162A1 (en) Device, system, and method of liveness detection utilizing voice biometrics
US20190042881A1 (en) Acoustic event detection based on modelling of sequence of event subparts
CN108564941A (zh) 语音识别方法、装置、设备及存储介质
CN110853617B (zh) 一种模型训练的方法、语种识别的方法、装置及设备
US11854553B2 (en) Cybersecurity for sensitive-information utterances in interactive voice sessions
US20230401338A1 (en) Method for detecting an audio adversarial attack with respect to a voice input processed by an automatic speech recognition system, corresponding device, computer program product and computer-readable carrier medium
CN110942776B (zh) 一种基于gru的音频防拼接检测方法及系统
US11900927B2 (en) Cybersecurity for sensitive-information utterances in interactive voice sessions using risk profiles
CN113707173B (zh) 基于音频切分的语音分离方法、装置、设备及存储介质
CN116547752A (zh) 虚假音频检测
CN106357411A (zh) 一种身份验证方法和装置
CN109754817A (zh) 信号处理方法及终端设备
WO2022199405A1 (zh) 一种语音控制方法和装置
Li et al. Learning normality is enough: a software-based mitigation against inaudible voice attacks
Khan et al. SpoTNet: A spoofing-aware Transformer Network for Effective Synthetic Speech Detection
Korshunov et al. Joint operation of voice biometrics and presentation attack detection
KR20170010978A (ko) 통화 내용 패턴 분석을 통한 보이스 피싱 방지 방법 및 장치
US11798564B2 (en) Spoofing detection apparatus, spoofing detection method, and computer-readable storage medium
CN109634554B (zh) 用于输出信息的方法和装置
CN109064720B (zh) 位置提示方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18891905

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20207007398

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18891905

Country of ref document: EP

Kind code of ref document: A1