WO2019125988A1 - Secure microphone - Google Patents

Secure microphone Download PDF

Info

Publication number
WO2019125988A1
WO2019125988A1 PCT/US2018/065922 US2018065922W WO2019125988A1 WO 2019125988 A1 WO2019125988 A1 WO 2019125988A1 US 2018065922 W US2018065922 W US 2018065922W WO 2019125988 A1 WO2019125988 A1 WO 2019125988A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio signal
microphone assembly
application
encryption key
processor
Prior art date
Application number
PCT/US2018/065922
Other languages
French (fr)
Inventor
Minho JIN
Inseok HEO
Alexi IVANOV
Francis Tiong
Navjot Singh
Rohit PATURI
Sridhar Krishna NEMALA
Original Assignee
Knowles Electronics, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Knowles Electronics, Llc filed Critical Knowles Electronics, Llc
Publication of WO2019125988A1 publication Critical patent/WO2019125988A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R19/00Electrostatic transducers
    • H04R19/005Electrostatic transducers using semiconductor materials
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/70Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
    • G06F21/82Protecting input, output or interconnection devices
    • G06F21/83Protecting input, output or interconnection devices input devices, e.g. keyboards, mice or controllers thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00Network architectures or network communication protocols for network security
    • H04L63/04Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H04L63/0442Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload wherein the sending and receiving network entities apply asymmetric encryption, i.e. different keys for encryption and decryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R19/00Electrostatic transducers
    • H04R19/04Microphones
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/003Mems transducers or their use

Definitions

  • Microphones are deployed in various types of electronic devices such as cellular phones, mobile devices, headsets, hands free systems, smart televisions, smart speakers (e.g., Google Home, Alexa), portable computers, etc.
  • a microphone converts a sound into a corresponding audio signal electrically representing the sound.
  • electronic devices can be compromised by unauthorized devices through spyware, malware or hacking.
  • unauthorized devices may obtain access to the audio signal from the microphone in a compromised electronic device, and eavesdrop on private conversations.
  • the microphone assembly includes a transducer configured to generate an audio signal in response to a sound.
  • the microphone assembly includes an integrated circuit coupled to the transducer.
  • the integrated circuit is configured to receive the audio signal from the transducer, encrypt the audio signal to generate an encrypted audio signal according to an encryption key, and output the encrypted audio signal through one or more contacts.
  • the microphone assembly includes a housing enclosing the transducer and the integrated circuit and exposing the one or more contacts.
  • the integrated circuit is configured to receive a signal representing the encryption key through the one or more contacts from an external source.
  • the integrated circuit is further configured to locally store the encryption key in response receiving the signal representing the encryption key.
  • the integrated circuit comprises one of an ASIC or a FPGA.
  • the integrated circuit includes one of a CPU, a DSP or a processor core.
  • the integrated circuit is further configured to output the audio signal through the one or more contacts.
  • the integrated circuit is further configured to selectively output either the encrypted audio signal or the audio signal in response to a command from an external source.
  • the device includes a processor configured to generate an encryption key and a decryption key.
  • the device includes a microphone assembly coupled to the processor.
  • the microphone assembly is configured to receive the encryption key from the processor, generate an audio signal in response to a sound, generate an encrypted audio signal by encrypting the audio signal according to the encryption key, and transmit the encrypted audio signal to the processor.
  • the processor is configured to decrypt the encrypted audio signal according to the decryption key to produce decrypted audio data representing the sound.
  • the encryption key and the decryption key comprise an asymmetric key pair.
  • the processor is configured to run software including a device driver for the microphone assembly and an application.
  • the application generates the encryption key and the decryption key.
  • the application communicates with the microphone assembly via the device driver to send the encryption key to the microphone assembly and to receive the encrypted audio signal from the microphone assembly.
  • the application is configured to store the decryption key in a secure storage that is not accessible by any other application or device.
  • the device further includes a network interface, wherein the application is further configured to transmit the decrypted audio data via the network interface.
  • the application comprises one of a voice messaging application, a chat application, a voice assistant application or a voice recorder application.
  • the device comprises one of a cellular phone, a mobile device, a headset, a hands free system, a smart television, a smart speaker or a portable computer.
  • the method includes transmitting an encryption key to a microphone assembly from a processor. In some embodiments, the method includes receiving an encrypted audio signal from the microphone assembly at the processor. In some embodiments, the encrypted audio signal electrical represents a sound captured by the microphone assembly that has been encrypted by the microphone assembly based on the encryption key. In some embodiments, the method includes decrypting, at the processor, the encrypted audio signal to generate decrypted audio data according to a decryption key.
  • the method includes generating the encryption key and the decryption key at the processor.
  • the encryption key and the decryption key comprise an asymmetric key pair.
  • the generating is performed in response to an application being installed in an electronic device comprising the microphone assembly and the processor.
  • the generating is performed at a plurality of points in time to produce updated pairs of encryption keys and decryption keys.
  • Figure 1 is a block diagram of an example electronic device according to embodiments of the present disclosure.
  • Figure 2A is a top plan view of an example microphone assembly according to embodiments of the present disclosure.
  • Figure 2B is a cross-section view of the example microphone assembly of Figure 2A according to embodiments of the present disclosure.
  • Figure 3 is a block diagram of an application of Figure 1 according to embodiments of the present disclosure.
  • Figure 4 is an example interaction diagram illustrating operation of components of the electronic device of Figure 1 according to an embodiment of the present disclosure.
  • a microphone assembly converts a sound into an encrypted audio signal, and outputs the encrypted audio signal through contacts of the microphone assembly.
  • the microphone assembly may be electrically insulated, and only electrically communicate with other electronic devices through the contacts. Because the output of the microphone assembly through the contacts is encrypted, the audio signal can thus be secured.
  • the microphone assembly includes a transducer, an integrated circuit (IC), contacts, and a housing.
  • the housing partially encloses the transducer, the integrated circuit, and the contacts.
  • the transducer may convert a sound into a raw audio signal electrically representing the sound and provide the raw audio signal to the IC.
  • the IC may generate an encrypted audio signal by encrypting the raw audio signal using an encryption key.
  • the housing prevents the raw audio signal from being observed or obtained from outside the microphone assembly.
  • the IC may output the encrypted audio signal to an external device through the contacts. External devices without a proper corresponding decryption key will not be able to decrypt the encrypted audio signal.
  • a system of securing a sound sensed by the microphone assembly may include a processor and the microphone assembly.
  • An application running on the processor may generate an encryption key and a corresponding decryption key.
  • the application may cause the processor to transmit the encryption key to the microphone assembly, and to store the decryption key without transmission.
  • the microphone assembly may perform encryption on the raw audio signal based on the encryption key, and transmit the encrypted audio signal to the processor.
  • the processor may receive the encrypted audio signal, and the application may decrypt the encrypted audio signal according to the decryption key to generate decrypted audio data.
  • the decrypted audio data may thus be accessed by the application without direct access to the raw audio signal. Because the microphone assembly may only output the encrypted audio signal and the application may only have access to the decryption key, other electronic devices or software applications without the decryption key may be precluded from accessing the raw audio signal generated by the microphone assembly.
  • a method of securing a sound sensed by the microphone assembly includes generating an encryption key and a decryption key. The method further includes transmitting the encryption key to the microphone assembly. The method further includes generating, by the microphone assembly, an encrypted audio signal based on the encryption key. The method further includes forwarding the encrypted audio signal to the processor through the contacts of the microphone assembly. An application executing on the processor may decrypt the encrypted audio signal based on the decryption key.
  • the electronic device 100 may be a cellular phone, mobile device, headset, hearing aid device, smart televisions, smart speakers, etc.
  • the electronic device 100 includes a microphone assembly 110 and a processor 130.
  • the microphone assembly 110 senses a sound, and transmits an audio signal representing the sensed sound in an encrypted form to the processor 130 through the driver 120.
  • These components may be embodied as hardware components that are electrically coupled to each other through conductive wires or traces.
  • the electronic device 100 includes more, fewer, or different components than shown in Figure 1.
  • audio may be captured at a mobile device and then transmitted to the cloud for further processing, perhaps including some of the processing performed by processor 130 as described below.
  • processor 130 there can be several microphone assemblies 110 which can communicate with one or more processors 130.
  • the microphone assembly 110 is an electrical component that senses a sound and outputs an audio signal in an encrypted format.
  • the microphone assembly 110 converts the sensed sound into a raw audio signal.
  • the raw audio signal is an electrical signal representing the sensed sound.
  • the microphone assembly 110 receives an encryption key signal 122 electrically representing an encryption key from the processor 130, and stores a copy of the encryption key within the microphone assembly 110 using the encryption key signal 122.
  • the microphone assembly 110 thereafter encrypts the audio signal according to the encryption key and a key-based encryption algorithm and transmits the encrypted audio signal 118 to the processor 130, while protecting the raw audio signal from unauthorized devices.
  • microphone assembly 110 is configured with the capability to perform encryption according to aspects of the present embodiments, it is not necessary that it always does so.
  • microphone assembly 110 is selectively controlled to transmit either the encrypted audio signal 118 or an unencrypted audio signal according to a command provided by the processor 130, for example.
  • a user may configure the processor 130 to enable or disable encryption of the audio signal through a user interface.
  • the microphone assembly 110 is electrically insulated except through designated contacts of the microphone assembly 110. Accordingly, external devices including the processor 130 and other devices (not shown) may not access the raw audio signal. Instead, the external devices may access only the encrypted audio signal 118. However, unauthorized devices without the decryption key may not decrypt the encrypted audio signal 118. Accordingly, the sound sensed by the microphone assembly 110 can be protected from unauthorized devices.
  • the processor 130 is a CPU or processor core that executes instructions to perform various functions or applications disclosed herein.
  • the instructions may be stored by a non- transitory computer readable medium.
  • the instructions executed by the processor 130 constitute software applications such as an operating system (OS) 180, a driver 120, and an application 160. These software applications operate together to cause the processor 130 to generate and transmit the encryption key signal 122, and to receive the encrypted audio signal 118.
  • OS operating system
  • the processor 130 executes more, fewer, or different software applications than shown in FIG. 1.
  • one or more functionalities of these software applications may be performed by an application specific integrated circuit or a field programmable gate array.
  • the OS 180 is a software platform that manages hardware or software resources of the device 100. Specifically, the OS 180 allows the application 160 to operate on the processor 130 and to communicate with driver 120. Examples of the OS 180 operating on a mobile hand-held device include iOS, ANDROID, WINDOWS MOBILE, WINDOWS, LINUX, etc.
  • the application 160 is a software program that operates to utilize sound sensed by the microphone assembly 110 (e.g., chat or other messaging applications, voice command applications, audio/video recording applications, audio/video conferencing applications, etc.).
  • the application 160 may present a user interface to a user of the electronic device 100 for allowing the user to utilize the application 160.
  • the application 160 generates an encryption key 132 and a corresponding decryption key for a key-based
  • the application 160 provides the encryption key 132 represented in a digital representation to the driver 120 for transmission to microphone assembly 110, and stores the corresponding decryption key in its own secure storage without transmission or the ability of other applications to access it.
  • the driver 120 is a device driver software application that provides an interface between the microphone assembly 110 and the application 160.
  • the driver 120 receives the encryption key 132 represented in a digital representation from application 160, and causes the processor 130 to generate an encryption key signal 122 electrically representing the encryption key 132.
  • the driver 120 causes the processor 130 to transmit the encryption key signal 122 to the microphone assembly 110.
  • the driver 120 receives encrypted audio signal 118 received by the processor 130 from microphone assembly 110, and forwards encrypted audio data 128 corresponding to the encrypted audio signal 118 to the application 160.
  • the application 160 receives the encrypted audio data 128 from the driver 120, and decrypts the encrypted audio data 128 according to the decryption key using the key-based decryption algorithm.
  • the application 160 may further utilize the decrypted audio data, such as forwarding the decrypted audio data to another application or causing the decrypted audio data to be communicated to a network in a voice messaging or voice assistant application example.
  • an unauthorized application executing on processor 130 may also gain access to the encrypted audio data, and perhaps also the encryption key, it cannot decrypt the data without the decryption key, thereby securing the sound captured by the microphone assembly 110.
  • any application including an unauthorized application running in processor 130 may gain access to driver 120, including its inputs and outputs.
  • any application including an unauthorized application running in processor 130 e.g., malware
  • driver 120 including its inputs and outputs.
  • audio data from microphone assembly 110 is not encrypted according to the present embodiments, this allows the unauthorized application to store, record or forward audio captured by microphone assembly 110, which can include a user’s speech for example.
  • an unauthorized application may still be able to access the encrypted audio data 128 and even the encryption key 132 from driver 120.
  • the application 160 since the application 160 according to embodiments securely maintains the decryption key corresponding to encryption key 132, the unauthorized application will not be able to decrypt the data 128. Additionally or alternatively, when microphone assembly 110 is included in a separate device as processor 130, such as when microphone assembly 110 is included in a Bluetooth headset, an unauthorized entity may be able to intercept encrypted audio signal 118. However, such an entity would not be able to access the decryption key maintained by application 160, and so would not be able to decrypt the intercepted encrypted audio signal 118.
  • application 160 can be an agent application that performs decryption of audio captured from microphone assembly 110, and such other applications can register with application 160 to obtain the decrypted audio.
  • application 160 can run in the background. Additionally or alternatively, some or all of the functionality of application 160 can be included in driver 120.
  • the microphone assembly 110 includes a transducer 210, an integrated circuit 220 (also referred to as an“IC 220” herein), contacts 228, a substrate 230, and a cover 240. Together, these components operate to sense a sound, and output an encrypted audio signal 118 indicative of the sensed sound in an encrypted form through the contacts 228.
  • IC 220 also referred to as an“IC 220” herein
  • the cover 240 and the substrate 230 constitute a housing 250 that insulates the electrical signals generated by transducer 210 from external devices except the contacts 228.
  • the IC 220 may be the only interface that electrically communicates with external devices through the contacts 228.
  • the microphone assembly 110 includes more, fewer, or different components than shown in Figures 2 A and 2B.
  • the transducer 210 is an electrical component that senses sound, and generates an electrical audio signal indicative of the sensed sound.
  • the transducer 210 may be a microelectromechanical systems (MEMS) transducer.
  • the transducer 210 may include a conductive back plate and a conductive diaphragm positioned in a distance relationship with the conductive back plate.
  • the diaphragm is configured to move in relation to the back plate in response to incident acoustic wave.
  • the movement of the diaphragm in relation to the back plate causes a capacitance associated with the transducer 210 to vary.
  • the change in the capacitance of the transducer 210 in response to the acoustic wave can be measured and converted into a corresponding raw audio signal.
  • the raw audio signal is an electrical signal that represents the sensed sound.
  • the IC 220 is an electrical circuit electrically coupled between the transducer 210 and the contacts 228.
  • IC 220 may include one or more processors (e.g. CPU, DSP, processor core, etc.) executing instructions (e.g. firmware stored in memory in or accessible to IC 220) to perform the functionalities described herein and below.
  • processors e.g. CPU, DSP, processor core, etc.
  • instructions e.g. firmware stored in memory in or accessible to IC 220
  • IC 220 is implemented by an application specific integrated circuit or a field programmable gate array configured to perform similar functionalities. The implementation choice can be driven by various factors.
  • implementing the IC 220 with a DSP allows flexibility in terms of signal processing (e.g., encryption, filtering, amplification, etc.) performed on the raw audio signal, whereas implementing the IC 220 with an application specific integrated circuit allows improved performance in terms of power and speed for a smaller area.
  • signal processing e.g., encryption, filtering, amplification, etc.
  • application specific integrated circuit allows improved performance in terms of power and speed for a smaller area.
  • the IC 220 receives the raw audio signal from the transducer 210 through a connection 225.
  • the connection 225 may be a bonding wire or a conductive trace directly coupling between the IC 220 and the transducer 210.
  • the IC 220 generates an encrypted audio signal 118 by encrypting the raw audio signal, and outputs the encrypted audio signal 118 through the contacts 228.
  • the IC 220 may receive an encryption key signal 122 indicating the encryption key from the processor 130 through one of the contacts 228, and store the encryption key indicated by the encryption key signal 122 in a register or memory (not shown) in or accessible to IC 220.
  • the IC 220 may thereafter encrypt the raw audio signal according to the stored encryption key using a key-based encryption algorithm executed by IC 220, as will be described in more detail below.
  • the IC 220 may perform other signal processing such as filtering or amplification on the raw audio signal before the encryption.
  • the IC 220 may extract features from the raw audio signal such as MFCC or pitch, and further include these extracted features in the encrypted audio signal 118, as well as encoded symbols (e.g., recognition results, acoustic event detection results, etc.).
  • IC 220 is configured to only forward feature vectors extracted from the raw audio signal in the encrypted audio signal 118, in which case encryption may or may not be necessary because the extraction of feature vectors from an audio signal is generally not reversible.
  • the housing 250 is a component that partially encloses the transducer 210 and the IC 220.
  • the housing 250 may include the substrate 230 and the cover 240.
  • the substrate 230 and the cover 240 electrically insulate the transducer 210 from external devices, and only allow an electrical communication between the IC 220 and external devices through the contacts 228. Because the housing 250 secures the raw audio signal output from the transducer 210 and because the only electrical communication available from the IC 220 is the encrypted audio signal 118 through the contacts 228, the sound sensed by the transducer 210 is secured from unauthorized devices that do not have access to a proper decryption key.
  • the substrate 230 is a mechanical structure, on which the transducer 210 and the IC 220 can be disposed.
  • the substrate 230 may be a printed circuit board (PCB) or a flexible circuit board (FCB).
  • the substrate 230 may include an inner surface 242 and an outer surface 246.
  • the transducer 210 and the IC 220 may be placed on the inner surface 242, where the contacts 228 may be placed on the outer surface 246.
  • the substrate 230 includes, between the inner surface 242 and the outer surface 246, an opening 232 and a via connection 248, at which the transducer 210 and the IC 220 can be located, respectively.
  • the diaphragm of the transducer 210 may be located at the opening 232 of the inner surface 242, such that an acoustic wave may enter the opening 232 of the outer surface 246 and travel towards the diaphragm.
  • electrical pads of the IC 220 may contact the via connection 248 comprising conductive materials, such that an electric signal (e.g., the encrypted audio signal 118) from the IC 220 can be provided to external devices (e.g., processor 130) through the contacts 228 or an electrical signal (e.g., encryption key signal 122) from external devices (e.g., processor 130) can be provided to the IC 220 through the contacts 228.
  • the cover 240 is located on the inner surface 242 of the substrate 230 to shield the transducer 210, the IC 220, and the connection 225 within the cover 240.
  • an output from the transducer 210 or the connection 225 between the transducer 210 and the IC 220 may be isolated, such that the transducer 210 may only output the raw audio signal to the IC 220. Accordingly, the raw audio signal produced by the transducer 210 is secured from unauthorized devices.
  • application 160 can be a voice assistant application executing on an electronic device such as a mobile phone.
  • the application is implemented by software modules including an audio interface 340, a user interface 350 and a network interface 360.
  • application 160 comprises software modules including a key generator 310, a decryption key store 320, and a decryptor 330.
  • the application 160 includes more, fewer, or different modules than shown in Figure 3.
  • audio interface 340 includes functionality for receiving an encrypted audio signal from microphone assembly 110 via driver 120 and for communicating encryption keys to microphone assembly 110 via driver 120.
  • User interface 350 includes functionality for interfacing with a user of the electronic device (e.g., a mobile phone) such as providing a display of controls or outputting voice prompts and responses for controlling the operation of the voice assistant application.
  • Network interface 360 operates to interact with networking functionality of the electronic device so as to communicate decrypted audio data from the user to a contact via a messaging server, for example. Audio interface 340, user interface 350 and network interface 360 can be implemented in various ways known to those skilled in the art, and so further details thereof will be omitted here for sake of clarity of the present embodiments.
  • the key generator 310 generates an encryption key and a corresponding decryption key for a key-based encryption algorithm.
  • the key generator 310 may generate the encryption key and a decryption key pair during an initial setup of application 160 on the electronic device. For example, after downloading the application 160 from a network (e.g. an online application store or database), an application setup process may execute to cause the key generator 310 to generate the encryption key and decryption key pair.
  • the key generator 310 may provide the generated encryption key to the driver 120 via the audio interface 340, and store the generated decryption key at the decryption key store 320.
  • the key generator 310 may periodically generate new key pairs that replace the encryption key used by microphone assembly 110 and the decryption key used by decryptor 330. Such periodic changes can be used to boost the security as a single or fixed encryption key might be attacked. Triggers for key pair changes may include timer expiration, start and end of audio signal transmission, initiation of other applications using audio, user request for encryption, etc.
  • the length of the key generated by the key generator 310 may always be the same, or it may vary depending on the configuration and context, such as allowable latency, sampling rate, and security level. For example, if the application requires near-realtime response, then the length of key should be minimized at the expense of security level. In non realtime applications, the key length may vary to its maximum value so that boost the security level.
  • Decryption key store 320 may be implemented depending on the type of device or operating system. Some illustrative examples include Java Secret Key class and Android key store system. Importantly, however, it should be noted that in these and other embodiments, the decryption key is securely stored such that only application 160 can access it. Moreover, it should be noted that although decryption key store 320 as illustrated in Figure 3 is shown as being included in application 160, this is not necessary in all embodiments. For example, decryption key store 320 may be implemented by a non-volatile memory of the electronic device that is accessible by application 160.
  • Key generator 310 generates a pair of keys in accordance with the particular key- based encryption algorithm to be used.
  • the keys are asymmetric, wherein the encryption key is a public key and the decryption key is a private key.
  • the encryption algorithm is the publicly available Pretty Good Privacy (PGP) application, Rivest-Shamir-Adleman (RSA), Elliptic Curve Cryptography (ECC), etc.
  • PGP Pretty Good Privacy
  • RSA Rivest-Shamir-Adleman
  • ECC Elliptic Curve Cryptography
  • the encryption key and decryption key are symmetric private keys.
  • the decryptor 330 receives the encrypted audio data 128 from the driver 120, and decrypts the encrypted audio data 128.
  • the decryptor 330 retrieves the decryption key from the decryption key store 320, and decrypts the encrypted audio data 128 according to the decryption key and the key -based encryption algorithm.
  • the decryptor 330 is the only component that can retrieve the decryption key. Hence, only the decryptor 330 may decrypt the encrypted audio data 128.
  • the encrypted data forms a message block.
  • the audio data stream becomes blocks of message frames concatenated.
  • the stream of data received has to be identified as proper blocks of message frames. The beginning and ending location within a stream that composes a frame has to be identified.
  • a vendor of microphone assembly 110 having the encryption capabilities of the present embodiments can provide a software development kit (SDK) that allows a developer of application 160 to incorporate the key- based security functionality of the present embodiments, in addition to allowing application 160 to communicate with microphone assembly 110 via driver 120.
  • SDK software development kit
  • the application 160 created using the SDK is automatically authorized to access the content sensed by the microphone assembly 110 through the driver 120.
  • the vendor of microphone assembly 110 can update driver 120 to include a registry of authorized applications that have been developed using the vendor’s SDK and may only forward encrypted audio to such authorized applications.
  • FIG 4 illustrated is an example interaction diagram 400 illustrating an operation of components of the electronic device 100 of Figure 1 according to an embodiment of the present disclosure.
  • Figure 4 illustrates an interaction of the microphone assembly 110, the driver 120, and the application 160.
  • the interaction among the microphone assembly 110, the driver 120, and the application 160 includes more, fewer, or different steps than shown in Figure 4.
  • the application 160 performs an initial setup 405, for example when the application 160 is first downloaded or installed in the electronic device 100. As part of or in addition to the initial setup, the application 160 may generate 410 a pair of encryption key and decryption key. The application 160 provides 415 the generated encryption key to the driver 120. The application 160 may store the generated decryption key at the decryption key store 320 without transmitting or otherwise allowing the decryption key to be accessed by any other entity except application 160.
  • the driver 120 receives the encryption key from the application 160, and generates an encryption key signal 122 electrically representing the encryption key.
  • the driver 120 causes the processor 130 to transmit 420 the encryption key signal 122 to the microphone assembly 110.
  • the microphone assembly 110 receives the encryption key signal 122 from the processor 130 and locally stores a digital representation of the encryption key using the encryption key signal.
  • the microphone assembly 110 generates 430 an audio signal.
  • a transducer 210 in the microphone assembly 110 may sense a sound or an acoustic wave incident on the microphone assembly 110 and generate a raw audio signal electrically representing the sensed sound.
  • the microphone assembly 110 encrypts 440 the raw audio signal according to the encryption key indicated by the encryption key signal 122 from the processor 130 and the key-based encryption algorithm.
  • the microphone assembly 110 may perform filtering, amplification, or other signal processing on the audio signal prior to the encryption.
  • the microphone assembly 110 transmits 445 the encrypted audio signal to the processor 130.
  • the driver 120 receives encrypted audio data electrically representing the encrypted audio signal received by the processor 130.
  • the driver 120 provides 450 the encrypted audio data to the application 160.
  • the driver 120 may optionally determine whether the application 160 is an authorized application before forwarding the encrypted audio data.
  • the application 160 receives the encrypted audio data from the driver 120, and decrypts 460 the audio data. For example, upon receiving the encrypted audio data, the application 160 retrieves the decryption key, and decrypts the audio data according to the decryption key and the key-based encryption algorithm. The application 160 may further use the decrypted audio data, for example communicating the decrypted audio data to a remote device.
  • any two components so associated can also be viewed as being “operably connected,” or “operably coupled,” to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being “operably couplable,” to each other to achieve the desired functionality.
  • operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.
  • phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.” Further, unless otherwise noted, the use of the words“approximate,”“about,” “around,”“substantially,” etc., mean plus or minus ten percent.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Telephone Function (AREA)

Abstract

The present disclosure describes devices and methods for improving security of an audio signal generated by a microphone by encrypting the audio signal. In one aspect, a microphone assembly converts a sound into an encrypted audio signal, and outputs the encrypted audio signal through contacts of the microphone assembly. The microphone assembly may be electrically insulated, and only electrically communicate with other electronic devices through the contacts. Because the output of the microphone assembly through the contacts is encrypted, the audio signal can thus be secured.

Description

SECURE MICROPHONE
CROSS-REFERENCE TO RELATED APPLICATION
[0001] The present application claims priority to U.S. Provisional Patent Application No. 62/608,550, filed December 20, 2017, the contents of which are incorporated herein by reference in their entirety.
BACKGROUND
[0002] Microphones are deployed in various types of electronic devices such as cellular phones, mobile devices, headsets, hands free systems, smart televisions, smart speakers (e.g., Google Home, Alexa), portable computers, etc. A microphone converts a sound into a corresponding audio signal electrically representing the sound. Often, electronic devices can be compromised by unauthorized devices through spyware, malware or hacking. Moreover, unauthorized devices may obtain access to the audio signal from the microphone in a compromised electronic device, and eavesdrop on private conversations.
SUMMARY
[0003] Various embodiments disclosed herein are related to a microphone assembly. In some embodiments, the microphone assembly includes a transducer configured to generate an audio signal in response to a sound. In some embodiments, the microphone assembly includes an integrated circuit coupled to the transducer. In some embodiments, the integrated circuit is configured to receive the audio signal from the transducer, encrypt the audio signal to generate an encrypted audio signal according to an encryption key, and output the encrypted audio signal through one or more contacts. In some embodiments, the microphone assembly includes a housing enclosing the transducer and the integrated circuit and exposing the one or more contacts. [0004] In some embodiments, the integrated circuit is configured to receive a signal representing the encryption key through the one or more contacts from an external source.
[0005] In some embodiments, the integrated circuit is further configured to locally store the encryption key in response receiving the signal representing the encryption key.
[0006] In some embodiments, the integrated circuit comprises one of an ASIC or a FPGA.
[0007] In some embodiments, the integrated circuit includes one of a CPU, a DSP or a processor core.
[0008] In some embodiments, the integrated circuit is further configured to output the audio signal through the one or more contacts.
[0009] In some embodiments, the integrated circuit is further configured to selectively output either the encrypted audio signal or the audio signal in response to a command from an external source.
[0010] Various embodiments disclosed herein are related to a device. In some
embodiments, the device includes a processor configured to generate an encryption key and a decryption key. In some embodiments, the device includes a microphone assembly coupled to the processor. In some embodiments, the microphone assembly is configured to receive the encryption key from the processor, generate an audio signal in response to a sound, generate an encrypted audio signal by encrypting the audio signal according to the encryption key, and transmit the encrypted audio signal to the processor. In some embodiments, the processor is configured to decrypt the encrypted audio signal according to the decryption key to produce decrypted audio data representing the sound.
[0011] In some embodiments, the encryption key and the decryption key comprise an asymmetric key pair.
[0012] In some embodiments, the processor is configured to run software including a device driver for the microphone assembly and an application. In some embodiments, the application generates the encryption key and the decryption key. [0013] In some embodiments, the application communicates with the microphone assembly via the device driver to send the encryption key to the microphone assembly and to receive the encrypted audio signal from the microphone assembly.
[0014] In some embodiments, the application is configured to store the decryption key in a secure storage that is not accessible by any other application or device.
[0015] In some embodiments, the device further includes a network interface, wherein the application is further configured to transmit the decrypted audio data via the network interface.
[0016] In some embodiments, the application comprises one of a voice messaging application, a chat application, a voice assistant application or a voice recorder application.
[0017] In some embodiments, the device comprises one of a cellular phone, a mobile device, a headset, a hands free system, a smart television, a smart speaker or a portable computer.
[0018] Various embodiments disclosed herein are related to a method. In some
embodiments, the method includes transmitting an encryption key to a microphone assembly from a processor. In some embodiments, the method includes receiving an encrypted audio signal from the microphone assembly at the processor. In some embodiments, the encrypted audio signal electrical represents a sound captured by the microphone assembly that has been encrypted by the microphone assembly based on the encryption key. In some embodiments, the method includes decrypting, at the processor, the encrypted audio signal to generate decrypted audio data according to a decryption key.
[0019] In some embodiments, the method includes generating the encryption key and the decryption key at the processor.
[0020] In some embodiments, the encryption key and the decryption key comprise an asymmetric key pair. [0021] In some embodiments, the generating is performed in response to an application being installed in an electronic device comprising the microphone assembly and the processor.
[0022] In some embodiments, the generating is performed at a plurality of points in time to produce updated pairs of encryption keys and decryption keys.
BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The foregoing and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several embodiments in accordance with the disclosure and are, therefore, not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings.
[0024] Figure 1 is a block diagram of an example electronic device according to embodiments of the present disclosure.
[0025] Figure 2A is a top plan view of an example microphone assembly according to embodiments of the present disclosure.
[0026] Figure 2B is a cross-section view of the example microphone assembly of Figure 2A according to embodiments of the present disclosure.
[0027] Figure 3 is a block diagram of an application of Figure 1 according to embodiments of the present disclosure.
[0028] Figure 4 is an example interaction diagram illustrating operation of components of the electronic device of Figure 1 according to an embodiment of the present disclosure.
[0029] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.
DETAILED DESCRIPTION
[0030] The present disclosure describes devices and methods for improving security of an audio signal generated by a microphone by encrypting the audio signal. In one aspect, a microphone assembly converts a sound into an encrypted audio signal, and outputs the encrypted audio signal through contacts of the microphone assembly. The microphone assembly may be electrically insulated, and only electrically communicate with other electronic devices through the contacts. Because the output of the microphone assembly through the contacts is encrypted, the audio signal can thus be secured.
[0031] In one or more embodiments, the microphone assembly includes a transducer, an integrated circuit (IC), contacts, and a housing. The housing partially encloses the transducer, the integrated circuit, and the contacts. The transducer may convert a sound into a raw audio signal electrically representing the sound and provide the raw audio signal to the IC. The IC may generate an encrypted audio signal by encrypting the raw audio signal using an encryption key. The housing prevents the raw audio signal from being observed or obtained from outside the microphone assembly. The IC may output the encrypted audio signal to an external device through the contacts. External devices without a proper corresponding decryption key will not be able to decrypt the encrypted audio signal.
[0032] In one or more embodiments, a system of securing a sound sensed by the microphone assembly is disclosed. The system may include a processor and the microphone assembly. An application running on the processor may generate an encryption key and a corresponding decryption key. The application may cause the processor to transmit the encryption key to the microphone assembly, and to store the decryption key without transmission. The microphone assembly may perform encryption on the raw audio signal based on the encryption key, and transmit the encrypted audio signal to the processor. The processor may receive the encrypted audio signal, and the application may decrypt the encrypted audio signal according to the decryption key to generate decrypted audio data. The decrypted audio data may thus be accessed by the application without direct access to the raw audio signal. Because the microphone assembly may only output the encrypted audio signal and the application may only have access to the decryption key, other electronic devices or software applications without the decryption key may be precluded from accessing the raw audio signal generated by the microphone assembly.
[0033] It should be noted that, although the present embodiments will be described herein with reference to a key -based encryption algorithm example, this is non-limiting, and other embodiments may include the use of other encryption and decryption algorithms, which may or may not be key-based.
[0034] In one or more embodiments, a method of securing a sound sensed by the microphone assembly is disclosed. The method includes generating an encryption key and a decryption key. The method further includes transmitting the encryption key to the microphone assembly. The method further includes generating, by the microphone assembly, an encrypted audio signal based on the encryption key. The method further includes forwarding the encrypted audio signal to the processor through the contacts of the microphone assembly. An application executing on the processor may decrypt the encrypted audio signal based on the decryption key.
EXAMPLE APPARATUS
[0035] Referring to Figure 1, illustrated is a block diagram of an example electronic device 100 according to embodiments of the present disclosure. The electronic device 100 may be a cellular phone, mobile device, headset, hearing aid device, smart televisions, smart speakers, etc. The electronic device 100 includes a microphone assembly 110 and a processor 130. The microphone assembly 110 senses a sound, and transmits an audio signal representing the sensed sound in an encrypted form to the processor 130 through the driver 120. These components may be embodied as hardware components that are electrically coupled to each other through conductive wires or traces. In some embodiments, the electronic device 100 includes more, fewer, or different components than shown in Figure 1. It should be noted, moreover, that the present disclosure is not limited to embodiments having all components included in a single device, and includes embodiments where some components are housed in separate devices and communicate with each other via wired and/or wireless means. In a voice assistant application embodiment, for example, audio may be captured at a mobile device and then transmitted to the cloud for further processing, perhaps including some of the processing performed by processor 130 as described below. In these and other embodiments, there can be several microphone assemblies 110 which can communicate with one or more processors 130.
[0036] The microphone assembly 110 is an electrical component that senses a sound and outputs an audio signal in an encrypted format. In one aspect, the microphone assembly 110 converts the sensed sound into a raw audio signal. The raw audio signal is an electrical signal representing the sensed sound. In one aspect, the microphone assembly 110 receives an encryption key signal 122 electrically representing an encryption key from the processor 130, and stores a copy of the encryption key within the microphone assembly 110 using the encryption key signal 122. The microphone assembly 110 thereafter encrypts the audio signal according to the encryption key and a key-based encryption algorithm and transmits the encrypted audio signal 118 to the processor 130, while protecting the raw audio signal from unauthorized devices. It should be noted that although microphone assembly 110 is configured with the capability to perform encryption according to aspects of the present embodiments, it is not necessary that it always does so. In some embodiments, microphone assembly 110 is selectively controlled to transmit either the encrypted audio signal 118 or an unencrypted audio signal according to a command provided by the processor 130, for example. In these and other embodiments, a user may configure the processor 130 to enable or disable encryption of the audio signal through a user interface.
[0037] In one aspect, the microphone assembly 110 is electrically insulated except through designated contacts of the microphone assembly 110. Accordingly, external devices including the processor 130 and other devices (not shown) may not access the raw audio signal. Instead, the external devices may access only the encrypted audio signal 118. However, unauthorized devices without the decryption key may not decrypt the encrypted audio signal 118. Accordingly, the sound sensed by the microphone assembly 110 can be protected from unauthorized devices.
[0038] The processor 130 is a CPU or processor core that executes instructions to perform various functions or applications disclosed herein. The instructions may be stored by a non- transitory computer readable medium. In one aspect, the instructions executed by the processor 130 constitute software applications such as an operating system (OS) 180, a driver 120, and an application 160. These software applications operate together to cause the processor 130 to generate and transmit the encryption key signal 122, and to receive the encrypted audio signal 118. In other embodiments, the processor 130 executes more, fewer, or different software applications than shown in FIG. 1. In some embodiments, one or more functionalities of these software applications may be performed by an application specific integrated circuit or a field programmable gate array.
[0039] The OS 180 is a software platform that manages hardware or software resources of the device 100. Specifically, the OS 180 allows the application 160 to operate on the processor 130 and to communicate with driver 120. Examples of the OS 180 operating on a mobile hand-held device include iOS, ANDROID, WINDOWS MOBILE, WINDOWS, LINUX, etc.
[0040] The application 160 is a software program that operates to utilize sound sensed by the microphone assembly 110 (e.g., chat or other messaging applications, voice command applications, audio/video recording applications, audio/video conferencing applications, etc.). In one example, the application 160 may present a user interface to a user of the electronic device 100 for allowing the user to utilize the application 160. The application 160 generates an encryption key 132 and a corresponding decryption key for a key-based
encryption/decryption algorithm. The application 160 provides the encryption key 132 represented in a digital representation to the driver 120 for transmission to microphone assembly 110, and stores the corresponding decryption key in its own secure storage without transmission or the ability of other applications to access it. [0041] The driver 120 is a device driver software application that provides an interface between the microphone assembly 110 and the application 160. The driver 120 receives the encryption key 132 represented in a digital representation from application 160, and causes the processor 130 to generate an encryption key signal 122 electrically representing the encryption key 132. The driver 120 causes the processor 130 to transmit the encryption key signal 122 to the microphone assembly 110. Moreover, the driver 120 receives encrypted audio signal 118 received by the processor 130 from microphone assembly 110, and forwards encrypted audio data 128 corresponding to the encrypted audio signal 118 to the application 160.
[0042] The application 160 receives the encrypted audio data 128 from the driver 120, and decrypts the encrypted audio data 128 according to the decryption key using the key-based decryption algorithm. The application 160 may further utilize the decrypted audio data, such as forwarding the decrypted audio data to another application or causing the decrypted audio data to be communicated to a network in a voice messaging or voice assistant application example. Although an unauthorized application executing on processor 130 may also gain access to the encrypted audio data, and perhaps also the encryption key, it cannot decrypt the data without the decryption key, thereby securing the sound captured by the microphone assembly 110.
[0043] More particularly, the present applicant recognizes various potential vulnerabilities to hacking that are present in connection with devices having configurations similar to device 100, but are solved by the present embodiments. For example, any application including an unauthorized application running in processor 130 (e.g., malware) may gain access to driver 120, including its inputs and outputs. As such, if audio data from microphone assembly 110 is not encrypted according to the present embodiments, this allows the unauthorized application to store, record or forward audio captured by microphone assembly 110, which can include a user’s speech for example. Even if the audio data from microphone assembly 110 is encrypted, an unauthorized application may still be able to access the encrypted audio data 128 and even the encryption key 132 from driver 120. However, because the application 160 according to embodiments securely maintains the decryption key corresponding to encryption key 132, the unauthorized application will not be able to decrypt the data 128. Additionally or alternatively, when microphone assembly 110 is included in a separate device as processor 130, such as when microphone assembly 110 is included in a Bluetooth headset, an unauthorized entity may be able to intercept encrypted audio signal 118. However, such an entity would not be able to access the decryption key maintained by application 160, and so would not be able to decrypt the intercepted encrypted audio signal 118.
[0044] It should be noted that various alternatives to including decryption functionality in an application 160 (e.g. chat or other messaging applications, voice command or assistant applications, audio/video recording applications, audio/video conferencing applications, etc.) are possible. For example, application 160 can be an agent application that performs decryption of audio captured from microphone assembly 110, and such other applications can register with application 160 to obtain the decrypted audio. In these and other embodiments, the application 160 can run in the background. Additionally or alternatively, some or all of the functionality of application 160 can be included in driver 120.
[0045] Referring to Figure 2A, illustrated is a top plan view of an example microphone assembly 110 according to embodiments of the present disclosure. Referring to Figure 2B, illustrated is a cross-section view of the example microphone assembly 110 of Figure 2A according to embodiments of the present disclosure. In one embodiment, the microphone assembly 110 includes a transducer 210, an integrated circuit 220 (also referred to as an“IC 220” herein), contacts 228, a substrate 230, and a cover 240. Together, these components operate to sense a sound, and output an encrypted audio signal 118 indicative of the sensed sound in an encrypted form through the contacts 228. In one aspect, the cover 240 and the substrate 230 constitute a housing 250 that insulates the electrical signals generated by transducer 210 from external devices except the contacts 228. In some embodiments, the IC 220 may be the only interface that electrically communicates with external devices through the contacts 228. In other embodiments, the microphone assembly 110 includes more, fewer, or different components than shown in Figures 2 A and 2B.
[0046] The transducer 210 is an electrical component that senses sound, and generates an electrical audio signal indicative of the sensed sound. In one implementation, the transducer 210 may be a microelectromechanical systems (MEMS) transducer. The transducer 210 may include a conductive back plate and a conductive diaphragm positioned in a distance relationship with the conductive back plate. The diaphragm is configured to move in relation to the back plate in response to incident acoustic wave. The movement of the diaphragm in relation to the back plate causes a capacitance associated with the transducer 210 to vary. The change in the capacitance of the transducer 210 in response to the acoustic wave can be measured and converted into a corresponding raw audio signal. The raw audio signal is an electrical signal that represents the sensed sound.
[0047] The IC 220 is an electrical circuit electrically coupled between the transducer 210 and the contacts 228. In some embodiments, IC 220 may include one or more processors (e.g. CPU, DSP, processor core, etc.) executing instructions (e.g. firmware stored in memory in or accessible to IC 220) to perform the functionalities described herein and below. In other embodiments, IC 220 is implemented by an application specific integrated circuit or a field programmable gate array configured to perform similar functionalities. The implementation choice can be driven by various factors. For example, implementing the IC 220 with a DSP allows flexibility in terms of signal processing (e.g., encryption, filtering, amplification, etc.) performed on the raw audio signal, whereas implementing the IC 220 with an application specific integrated circuit allows improved performance in terms of power and speed for a smaller area.
[0048] In one aspect, the IC 220 receives the raw audio signal from the transducer 210 through a connection 225. The connection 225 may be a bonding wire or a conductive trace directly coupling between the IC 220 and the transducer 210. The IC 220 generates an encrypted audio signal 118 by encrypting the raw audio signal, and outputs the encrypted audio signal 118 through the contacts 228. The IC 220 may receive an encryption key signal 122 indicating the encryption key from the processor 130 through one of the contacts 228, and store the encryption key indicated by the encryption key signal 122 in a register or memory (not shown) in or accessible to IC 220. The IC 220 may thereafter encrypt the raw audio signal according to the stored encryption key using a key-based encryption algorithm executed by IC 220, as will be described in more detail below. The IC 220 may perform other signal processing such as filtering or amplification on the raw audio signal before the encryption. In addition, the IC 220 may extract features from the raw audio signal such as MFCC or pitch, and further include these extracted features in the encrypted audio signal 118, as well as encoded symbols (e.g., recognition results, acoustic event detection results, etc.). In alternative embodiments, IC 220 is configured to only forward feature vectors extracted from the raw audio signal in the encrypted audio signal 118, in which case encryption may or may not be necessary because the extraction of feature vectors from an audio signal is generally not reversible.
[0049] The housing 250 is a component that partially encloses the transducer 210 and the IC 220. The housing 250 may include the substrate 230 and the cover 240. The substrate 230 and the cover 240 electrically insulate the transducer 210 from external devices, and only allow an electrical communication between the IC 220 and external devices through the contacts 228. Because the housing 250 secures the raw audio signal output from the transducer 210 and because the only electrical communication available from the IC 220 is the encrypted audio signal 118 through the contacts 228, the sound sensed by the transducer 210 is secured from unauthorized devices that do not have access to a proper decryption key.
[0050] The substrate 230 is a mechanical structure, on which the transducer 210 and the IC 220 can be disposed. The substrate 230 may be a printed circuit board (PCB) or a flexible circuit board (FCB). The substrate 230 may include an inner surface 242 and an outer surface 246. The transducer 210 and the IC 220 may be placed on the inner surface 242, where the contacts 228 may be placed on the outer surface 246. In one aspect, the substrate 230 includes, between the inner surface 242 and the outer surface 246, an opening 232 and a via connection 248, at which the transducer 210 and the IC 220 can be located, respectively. For example, the diaphragm of the transducer 210 may be located at the opening 232 of the inner surface 242, such that an acoustic wave may enter the opening 232 of the outer surface 246 and travel towards the diaphragm. For another example, electrical pads of the IC 220 may contact the via connection 248 comprising conductive materials, such that an electric signal (e.g., the encrypted audio signal 118) from the IC 220 can be provided to external devices (e.g., processor 130) through the contacts 228 or an electrical signal (e.g., encryption key signal 122) from external devices (e.g., processor 130) can be provided to the IC 220 through the contacts 228.
[0051] The cover 240 is located on the inner surface 242 of the substrate 230 to shield the transducer 210, the IC 220, and the connection 225 within the cover 240. In this configuration, an output from the transducer 210 or the connection 225 between the transducer 210 and the IC 220 may be isolated, such that the transducer 210 may only output the raw audio signal to the IC 220. Accordingly, the raw audio signal produced by the transducer 210 is secured from unauthorized devices.
[0052] Referring to Figure 3, illustrated is a block diagram of an example application 160 of Figure 1 according to embodiments of the present disclosure. In this example, application 160 can be a voice assistant application executing on an electronic device such as a mobile phone. In one embodiment such as that illustrated in Figure 3, the application is implemented by software modules including an audio interface 340, a user interface 350 and a network interface 360. In addition, to implement the security aspects of the present embodiments, application 160 comprises software modules including a key generator 310, a decryption key store 320, and a decryptor 330. In some embodiments, the application 160 includes more, fewer, or different modules than shown in Figure 3.
[0053] Generally, audio interface 340 includes functionality for receiving an encrypted audio signal from microphone assembly 110 via driver 120 and for communicating encryption keys to microphone assembly 110 via driver 120. User interface 350 includes functionality for interfacing with a user of the electronic device (e.g., a mobile phone) such as providing a display of controls or outputting voice prompts and responses for controlling the operation of the voice assistant application. Network interface 360 operates to interact with networking functionality of the electronic device so as to communicate decrypted audio data from the user to a contact via a messaging server, for example. Audio interface 340, user interface 350 and network interface 360 can be implemented in various ways known to those skilled in the art, and so further details thereof will be omitted here for sake of clarity of the present embodiments.
[0054] The key generator 310 generates an encryption key and a corresponding decryption key for a key-based encryption algorithm. The key generator 310 may generate the encryption key and a decryption key pair during an initial setup of application 160 on the electronic device. For example, after downloading the application 160 from a network (e.g. an online application store or database), an application setup process may execute to cause the key generator 310 to generate the encryption key and decryption key pair. The key generator 310 may provide the generated encryption key to the driver 120 via the audio interface 340, and store the generated decryption key at the decryption key store 320. In these and other embodiments, for enhanced security, the key generator 310 may periodically generate new key pairs that replace the encryption key used by microphone assembly 110 and the decryption key used by decryptor 330. Such periodic changes can be used to boost the security as a single or fixed encryption key might be attacked. Triggers for key pair changes may include timer expiration, start and end of audio signal transmission, initiation of other applications using audio, user request for encryption, etc.
[0055] The length of the key generated by the key generator 310 may always be the same, or it may vary depending on the configuration and context, such as allowable latency, sampling rate, and security level. For example, if the application requires near-realtime response, then the length of key should be minimized at the expense of security level. In non realtime applications, the key length may vary to its maximum value so that boost the security level.
[0056] Decryption key store 320 may be implemented depending on the type of device or operating system. Some illustrative examples include Java Secret Key class and Android key store system. Importantly, however, it should be noted that in these and other embodiments, the decryption key is securely stored such that only application 160 can access it. Moreover, it should be noted that although decryption key store 320 as illustrated in Figure 3 is shown as being included in application 160, this is not necessary in all embodiments. For example, decryption key store 320 may be implemented by a non-volatile memory of the electronic device that is accessible by application 160.
[0057] Key generator 310 generates a pair of keys in accordance with the particular key- based encryption algorithm to be used. In one example, the keys are asymmetric, wherein the encryption key is a public key and the decryption key is a private key. In this and other examples, the encryption algorithm is the publicly available Pretty Good Privacy (PGP) application, Rivest-Shamir-Adleman (RSA), Elliptic Curve Cryptography (ECC), etc. In another example, the encryption key and decryption key are symmetric private keys. [0058] The decryptor 330 receives the encrypted audio data 128 from the driver 120, and decrypts the encrypted audio data 128. More particularly, upon receiving the encrypted audio data 128, the decryptor 330 retrieves the decryption key from the decryption key store 320, and decrypts the encrypted audio data 128 according to the decryption key and the key -based encryption algorithm. In one aspect, the decryptor 330 is the only component that can retrieve the decryption key. Hence, only the decryptor 330 may decrypt the encrypted audio data 128. By mechanically and electrically insulating the connection 225 between the transducer 210 and the IC 220 and by limiting the access to the decryption key to the decryptor 330, the security of the audio sensed by the transducer 210 can be improved.
[0059] The encrypted data forms a message block. After encryption the audio data stream becomes blocks of message frames concatenated. At the decryptor the stream of data received has to be identified as proper blocks of message frames. The beginning and ending location within a stream that composes a frame has to be identified.
[0060] There are many methods in achieving this frame synchronization. One possible way would be for the encryptor to insert special signature data pattern as a message frame periodically or in a pseudo random fashion. At the decryptor the frame boundary can be recovered by continuously checking whether the received pattern matches the signature pattern.
[0061] Those skilled in the art will understand various ways to adapt an application such as a voice messaging application with the security functionality of the present embodiments after being taught by the present examples. For example, a vendor of microphone assembly 110 having the encryption capabilities of the present embodiments can provide a software development kit (SDK) that allows a developer of application 160 to incorporate the key- based security functionality of the present embodiments, in addition to allowing application 160 to communicate with microphone assembly 110 via driver 120. In one aspect, the application 160 created using the SDK is automatically authorized to access the content sensed by the microphone assembly 110 through the driver 120. For example, the vendor of microphone assembly 110 can update driver 120 to include a registry of authorized applications that have been developed using the vendor’s SDK and may only forward encrypted audio to such authorized applications. [0062] Referring to Figure 4, illustrated is an example interaction diagram 400 illustrating an operation of components of the electronic device 100 of Figure 1 according to an embodiment of the present disclosure. In particular, Figure 4 illustrates an interaction of the microphone assembly 110, the driver 120, and the application 160. In other embodiments, the interaction among the microphone assembly 110, the driver 120, and the application 160 includes more, fewer, or different steps than shown in Figure 4.
[0063] The application 160 performs an initial setup 405, for example when the application 160 is first downloaded or installed in the electronic device 100. As part of or in addition to the initial setup, the application 160 may generate 410 a pair of encryption key and decryption key. The application 160 provides 415 the generated encryption key to the driver 120. The application 160 may store the generated decryption key at the decryption key store 320 without transmitting or otherwise allowing the decryption key to be accessed by any other entity except application 160.
[0064] The driver 120 receives the encryption key from the application 160, and generates an encryption key signal 122 electrically representing the encryption key. The driver 120 causes the processor 130 to transmit 420 the encryption key signal 122 to the microphone assembly 110.
[0065] The microphone assembly 110 receives the encryption key signal 122 from the processor 130 and locally stores a digital representation of the encryption key using the encryption key signal. The microphone assembly 110 generates 430 an audio signal. For example, a transducer 210 in the microphone assembly 110 may sense a sound or an acoustic wave incident on the microphone assembly 110 and generate a raw audio signal electrically representing the sensed sound. Moreover, the microphone assembly 110 encrypts 440 the raw audio signal according to the encryption key indicated by the encryption key signal 122 from the processor 130 and the key-based encryption algorithm. The microphone assembly 110 may perform filtering, amplification, or other signal processing on the audio signal prior to the encryption. The microphone assembly 110 transmits 445 the encrypted audio signal to the processor 130. [0066] The driver 120 receives encrypted audio data electrically representing the encrypted audio signal received by the processor 130. The driver 120 provides 450 the encrypted audio data to the application 160. The driver 120 may optionally determine whether the application 160 is an authorized application before forwarding the encrypted audio data.
[0067] The application 160 receives the encrypted audio data from the driver 120, and decrypts 460 the audio data. For example, upon receiving the encrypted audio data, the application 160 retrieves the decryption key, and decrypts the audio data according to the decryption key and the key-based encryption algorithm. The application 160 may further use the decrypted audio data, for example communicating the decrypted audio data to a remote device.
[0068] The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are illustrative, and that in fact many other architectures can be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being "operably connected," or "operably coupled," to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being "operably couplable," to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.
[0069] With respect to the use of plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity. [0070] It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but is not limited to," etc.).
[0071] It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., "a" and/or "an" should typically be interpreted to mean "at least one" or "one or more"); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, typically means at least two recitations, or two or more recitations).
[0072] Furthermore, in those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to "at least one of A, B, or C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B." Further, unless otherwise noted, the use of the words“approximate,”“about,” “around,”“substantially,” etc., mean plus or minus ten percent.
[0073] The foregoing description of illustrative embodiments has been presented for purposes of illustration and of description. It is not intended to be exhaustive or limiting with respect to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired from practice of the disclosed embodiments. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents.

Claims

WHAT IS CLAIMED IS:
1. A microphone assembly comprising:
a transducer configured to generate an audio signal in response to a sound;
an integrated circuit coupled to the transducer, the integrated circuit configured to: receive the audio signal from the transducer,
encrypt the audio signal to generate an encrypted audio signal according to an encryption key, and
output the encrypted audio signal through one or more contacts; and a housing enclosing the transducer and the integrated circuit and exposing the one or more contacts.
2. The microphone assembly of claim 1, wherein the integrated circuit is configured to receive a signal representing the encryption key through the one or more contacts from an external source.
3. The microphone assembly of claim 2, wherein the integrated circuit is further configured to locally store the encryption key in response receiving the signal representing the encryption key.
4. The microphone assembly of claim 1, wherein the integrated circuit comprises one of an ASIC or a FPGA.
5. The microphone assembly of claim 1, wherein the integrated circuit includes one of a CPU, a DSP or a processor core.
6. The microphone assembly of claim 1, wherein the integrated circuit is further configured to output the audio signal through the one or more contacts.
7. The microphone assembly of claim 6, wherein the integrated circuit is further configured to selectively output either the encrypted audio signal or the audio signal in response to a command from an external source.
8. A device comprising:
a processor configured to generate an encryption key and a decryption key; and a microphone assembly coupled to the processor, the microphone assembly configured to:
receive the encryption key from the processor,
generate an audio signal in response to a sound,
generate an encrypted audio signal by encrypting the audio signal according to the encryption key, and
transmit the encrypted audio signal to the processor,
wherein the processor is configured to decrypt the encrypted audio signal according to the decryption key to produce decrypted audio data representing the sound.
9. The device of claim 8, wherein the encryption key and the decryption key comprise an asymmetric key pair.
10. The device of claim 8, wherein the processor is configured to run software including a device driver for the microphone assembly and an application, and wherein the application generates the encryption key and the decryption key.
11. The device of claim 10, wherein the application communicates with the microphone assembly via the device driver to send the encryption key to the microphone assembly and to receive the encrypted audio signal from the microphone assembly.
12. The device of claim 10, wherein the application is configured to store the decryption key in a secure storage that is not accessible by any other application or device.
13. The device of claim 10, further comprising a network interface, wherein the application is further configured to transmit the decrypted audio data via the network interface.
14. The device of claim 10, wherein the application comprises one of a voice messaging application, a chat application, a voice assistant application or a voice recorder application.
15. The device of claim 8, wherein the device comprises one of a cellular phone, a mobile device, a headset, a hands free system, a smart television, a smart speaker or a portable computer.
16. A method comprising:
transmitting an encryption key to a microphone assembly from a processor;
receiving an encrypted audio signal from the microphone assembly at the processor, the encrypted audio signal electrical representing a sound captured by the microphone assembly that has been encrypted by the microphone assembly based on the encryption key; and
decrypting, at the processor, the encrypted audio signal to generate decrypted audio data according to a decryption key.
17. The method of claim 16, further comprising generating the encryption key and the decryption key at the processor.
18. The method of claim 17, wherein the encryption key and the decryption key comprise an asymmetric key pair.
19. The method of claim 17, wherein the generating is performed in response to an application being installed in an electronic device comprising the microphone assembly and the processor.
20. The method of claim 17, wherein the generating is performed at a plurality of points in time to produce updated pairs of encryption keys and decryption keys.
PCT/US2018/065922 2017-12-20 2018-12-17 Secure microphone WO2019125988A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762608550P 2017-12-20 2017-12-20
US62/608,550 2017-12-20

Publications (1)

Publication Number Publication Date
WO2019125988A1 true WO2019125988A1 (en) 2019-06-27

Family

ID=65003552

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/065922 WO2019125988A1 (en) 2017-12-20 2018-12-17 Secure microphone

Country Status (1)

Country Link
WO (1) WO2019125988A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112818376A (en) * 2021-03-08 2021-05-18 郑州铁路职业技术学院 Encryption method for anti-theft recording information transmission

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030131230A1 (en) * 2002-01-09 2003-07-10 International Business Machines Corporation Secured cellular telephone communications system, method, and computer program product
WO2016077228A1 (en) * 2014-11-11 2016-05-19 Invensense, Inc. Secure audio sensor
US20160350068A1 (en) * 2015-06-01 2016-12-01 Nagravision S.A. Methods and systems for conveying encrypted data to a communication device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030131230A1 (en) * 2002-01-09 2003-07-10 International Business Machines Corporation Secured cellular telephone communications system, method, and computer program product
WO2016077228A1 (en) * 2014-11-11 2016-05-19 Invensense, Inc. Secure audio sensor
US20160350068A1 (en) * 2015-06-01 2016-12-01 Nagravision S.A. Methods and systems for conveying encrypted data to a communication device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112818376A (en) * 2021-03-08 2021-05-18 郑州铁路职业技术学院 Encryption method for anti-theft recording information transmission
CN112818376B (en) * 2021-03-08 2024-03-26 郑州铁路职业技术学院 Encryption method for anti-theft recording information transmission

Similar Documents

Publication Publication Date Title
JP6938702B2 (en) Hearing devices with communication protection and related methods
US10681082B2 (en) Hearing device with communication protection and related method
JP6807153B2 (en) Devices and related methods for secure hearing device communication
EP3032845B1 (en) Hearing device configured to authenticate a mode request and related method
US11546699B2 (en) Hearing device with service mode and related method
CN103973696B (en) A kind of data processing method of voice call
JP2018129793A (en) Communication method of hearing system and relevant device
CN112182624A (en) Encryption method, encryption device, storage medium and electronic equipment
WO2019125988A1 (en) Secure microphone
CN103974243B (en) A kind of data handling system of voice call
CN103986711B (en) A kind of data processing method of voice call
CN103974242B (en) A kind of data processing method of voice call
WO2014137343A1 (en) System and method for secure voip communication
CN103986712B (en) A kind of data processing method of voice call
EP4429274A1 (en) Data encryption for a hearing device
CN115694804A (en) Method, device and equipment for realizing safety communication between equipment
KR20240003681A (en) Electronic device for providing encryption service and method of operating the same
CN104066081B (en) A kind of data handling system of voice call
CN104066080B (en) A kind of data processing method of voice call
CN116709307A (en) Call encryption method and device
SI24698A (en) An encryption coding modules
CN103987036A (en) Data processing system for voice communication
CN103986579A (en) Data processing system for voice communication
CN103997732A (en) Data processing system in voice communication

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18830652

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18830652

Country of ref document: EP

Kind code of ref document: A1