US10659876B2 - Distributed microphone array and sound source positioning system applicable thereto - Google Patents

Distributed microphone array and sound source positioning system applicable thereto Download PDF

Info

Publication number
US10659876B2
US10659876B2 US16/086,166 US201616086166A US10659876B2 US 10659876 B2 US10659876 B2 US 10659876B2 US 201616086166 A US201616086166 A US 201616086166A US 10659876 B2 US10659876 B2 US 10659876B2
Authority
US
United States
Prior art keywords
microphone array
signal processing
sound source
processing unit
microphone
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/086,166
Other versions
US20190116418A1 (en
Inventor
Swee Ann Teo
Rui Zhan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Espressif Systems Shanghai Co Ltd
Original Assignee
Espressif Systems Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Espressif Systems Shanghai Co Ltd filed Critical Espressif Systems Shanghai Co Ltd
Assigned to ESPRESSIF SYSTEMS (SHANGHAI) PTE LTD reassignment ESPRESSIF SYSTEMS (SHANGHAI) PTE LTD ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TEO, SWEE ANN, ZHAN, Rui
Assigned to ESPRESSIF SYSTEMS (SHANGHAI) CO., LTD. reassignment ESPRESSIF SYSTEMS (SHANGHAI) CO., LTD. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: ESPRESSIF SYSTEMS (SHANGHAI) PTE., LTD.
Publication of US20190116418A1 publication Critical patent/US20190116418A1/en
Assigned to ESPRESSIF SYSTEMS (SHANGHAI) PTE LTD reassignment ESPRESSIF SYSTEMS (SHANGHAI) PTE LTD CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECT THE ADDRESS PREVIOUSLY RECORDED AT REEL: 047108 FRAME: 0547. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Assignors: TEO, SWEE ANN, ZHAN, Rui
Assigned to ESPRESSIF SYSTEMS (SHANGHAI) CO., LTD. reassignment ESPRESSIF SYSTEMS (SHANGHAI) CO., LTD. CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY ADDRESS PREVIOUSLY RECORDED ON REEL 049316 FRAME 0885. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Assignors: ESPRESSIF SYSTEMS (SHANGHAI) PTE., LTD.
Application granted granted Critical
Publication of US10659876B2 publication Critical patent/US10659876B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/027—Spatial or constructional arrangements of microphones, e.g. in dummy heads
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
    • H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
    • H04R2201/401—2D or 3D arrays of transducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
    • H04R2201/405—Non-uniform arrays of transducers or a plurality of uniform arrays with different transducer spacing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2420/00—Details of connection covered by H04R, not provided for in its groups
    • H04R2420/07—Applications of wireless loudspeakers or wireless microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field

Definitions

  • the present disclosure relates to a microphone array technology and, in particular, to a distributed microphone array and sound source positioning system having the same.
  • Microphone array technology has important and wide applications in the field of audio signal processing. Within a spatial range in which audio signal acquisition is required, a microphone array is constituted by multiple microphones deployed in different positions.
  • a sound pick-up range of the microphone array is much extended.
  • the pick-up range of the microphone array is extended from less than 0.5 meters of the traditional pick-up by a single microphone to several meters, or even dozens of meters.
  • the mainstream microphone products in the current market include small-sized wired microphone arrays, small-sized wireless microphone arrays (mainly using Wi-Fi access) and suspended microphone arrays.
  • most small-sized microphone arrays are desktop microphone arrays, wherein an audio mixer 101 is integrated with multiple microphones 102 and Wi-Fi device, and has a physical dimension generally less than 0.3 meter.
  • the distances between the microphones 102 are small, and space diversity gains of the audio signal processing are limited; and the mixer 101 output may be connected to a terminal device such as an audio device 103 in a wired or wireless mode.
  • Proper increase in the distances between microphones may provide higher space diversity gains and a better directional sound pick-up effect due to the long wavelength of audio signals.
  • a suspended microphone array is a microphone array product providing long distances between microphones, wherein microphones 202 suspended on the top 201 of a space (for example, a large meeting place) acquire specific audio signals in the space, the audio signals are inputted into a mixer 203 by specific cables to make signal processing, and then the output signals of the mixer 203 are transmitted into a speaker 205 through a signal line 204 , and finally the speaker 205 outputs the acquired audio signals.
  • the connections between the microphones and the mixer of the mainstream products listed above are all wired.
  • the small-sized integrated mode strongly limits the physical dimension of the microphone array, and thus limits the sound pick-up range and space diversity gains of the microphone array; also, the cable connection mode of the suspended microphone array with long distances between microphones increases the difficulty of microphone deployment, and the limited number of input ports of the mixer also limit the number of the deployed microphones, thereby restraining the physical coverage area of the microphone array in practical application.
  • the present disclosure provides a distributed microphone array and a sound source positioning system having the same, which provide a large span of sound pick-up range, higher space diversity gains and a more accurate three-dimensional spatial sound source positioning and achieve directional acquisition of multiple sound sources.
  • the present disclosure provides a distributed microphone array.
  • the array includes a plurality of microphone array nodes which are deployed in a distributed manner.
  • Each of the plurality of microphone array nodes is provided with one or more microphone audio acquisition modules.
  • the microphone array node is configured to communicatively connect to an external signal processing unit via a wireless network, and upload, an unprocessed audio signal or an audio signal which is simply processed on the microphone array node, to the signal processing unit.
  • the wireless network adopts 802.11 Wi-Fi communication, Bluetooth communication, Zigbee communication or any wireless network transmission mode.
  • Each of the plurality of microphone array nodes further includes a wireless device or an Internet of Things device with any application function.
  • Each of the plurality of microphone array nodes further includes:
  • wireless transmission and control module where an input end of the wireless transmission and control module is communicatively connected to the one or more microphone audio acquisition modules, and the wireless transmission and control module is communicatively connected to the signal processing unit via the wireless network;
  • a function module which is communicatively connected to the wireless transmission and control module and is any application function device.
  • the one or more microphone audio acquisition module and the function module are communicatively connected to the wireless transmission and control module through a data interface; and the data interface adopts UART, SPI, I2S, USB, SDIO, I2C or any standard interface.
  • the present disclosure also provides a sound source positioning system.
  • the sound source positioning system includes:
  • the signal processing unit which is configured to communicatively connect to the distributed microphone array via a wireless network, receive an unprocessed audio signal uploaded by the distributed microphone array, and perform audio signal processing of audio mixing, noise suppression, echo cancellation or three-dimensional spatial sound source positioning.
  • the positioning system includes one or more signal processing units connected to the wireless network.
  • the signal processing unit is an independent device in the wireless network or integrated with an access point (AP) in a Wi-Fi wireless network.
  • AP access point
  • the positioning system further includes an optical monitoring device.
  • the distributed microphone array and sound source positioning system having the same has the advantages in that all the wireless microphone array nodes deployed in the wireless network form a large microphone array in the present disclosure.
  • microphone signal acquisition and transmission are based on a wireless mode, and the distributed deployment of microphones is no longer limited by physical dimensions or cable connections, so that a considerable number of microphones can be deployed and the distances between the microphones as well as the coverage area of the microphone array can be greatly increased.
  • the deployment of the wireless microphone array nodes is simple and flexible, thereby achieving deployment of a large quantity of microphones with long distances.
  • the spatial positions of the wireless microphone array nodes are diversified, so that the wireless microphone array has a large span of sound pick-up range and abundant acquired signal samples to achieve better echo and noise suppression in the signal processing unit, thereby providing higher space diversity gains and more accurate three-dimensional spatial sound source positioning and achieving directional acquisition of multiple sound sources.
  • the microphone audio acquisition module according to the present disclosure can be integrated with an existing network device conveniently.
  • the mode of adding the microphone audio acquisition module to an existing network device in an acquisition space is often used in order to make the microphone deployment more convenient and flexible.
  • a lower cost can be achieved without cables, and the spatial positions for audio signal acquisition are not limited to the same surface and hence are diversified.
  • the signal processing unit according to the present disclosure can be a powerful processing unit deployed in the wireless network according to practical application to provide a stronger signal processing capability, and can cooperate with other monitoring device such as cameras, to achieve the application for complex dynamic audio and video positioning and monitoring acquisition of multiple sound sources.
  • FIG. 1 is a structural view of a small-sized microphone array in the prior art.
  • FIG. 2 is a structural view of a suspended microphone array in the prior art.
  • FIG. 3 is a structural view of wireless microphone array nodes according to the present disclosure.
  • FIG. 4 is a system diagram of a sound source positioning system based on a distributed microphone array according to embodiment one of the present disclosure.
  • FIG. 5 is a system diagram of a sound source positioning system based on a distributed microphone array according to embodiment two of the present disclosure.
  • the present disclosure discloses a large-scale distributed microphone array adopting wireless access.
  • the array includes a plurality of microphone array nodes which are distributed in a space and a wireless network.
  • Each of the plurality of microphone array nodes is provided with one or more microphone audio acquisition modules.
  • the microphone array node is communicatively connected to an external signal processing unit via the wireless network, and uploads, an unprocessed audio signal or an audio signal which is simply processed on the microphone array node, to the signal processing unit to perform audio mixing and other audio signal processing.
  • the microphone array node acquires audio signals by microphone audio acquisition modules, and wirelessly transmits the acquired audio signals to the signal processing unit. The microphone array node does not further process the acquired audio signals.
  • Each microphone array node is connected to one or more signal processing units of the wireless network through a wireless transmission module.
  • the signal processing unit may be an independent device in the wireless network or be integrated with other devices in the wireless network, for example, integrated with an access point (AP) in a Wi-Fi wireless network.
  • AP access point
  • the wireless network where the microphone array nodes are located may can adopt 802.11 Wi-Fi communication (for example, Wi-Fi communication or Wi-Fi Internet of Things), Bluetooth wireless communication, Zigbee wireless communication or any wireless network transmission mode.
  • 802.11 Wi-Fi communication standard is one of the most widely used wireless access technology standards.
  • the Wi-Fi Internet of Things access mode is one of the most widely used, low-cost and best extensible Internet of Things access modes.
  • a Wi-Fi network is used as an example in the present disclosure.
  • the microphone array node further includes a wireless device or an Internet of Things device with any application function.
  • the wireless microphone array node may include any wireless device or Internet of Things device supporting other application functions along with a microphone audio acquisition module, for example, a Wi-Fi light group device along with the microphone module, or a Wi-Fi household Internet of Things device along with the microphone module.
  • the wireless microphone array node may also provide only the function of audio signal acquisition.
  • FIG. 3 is a structural view of microphone array nodes according to an embodiment.
  • the microphone array node includes a wireless transmission and a control module 301 , two microphone audio acquisition modules 302 and a function module 303 .
  • the two microphone audio acquisition modules 302 are used for audio signal digital acquisition.
  • the function module 303 is a device providing any application function, for example, household appliance controlling, lighting controlling and other data acquisition and processing.
  • the wireless transmission and control module 301 is communicatively connected to the microphone audio acquisition modules 302 and the function module 303 via a control signal and a data interface, and is communicatively connected to an external signal processing unit via a wireless network.
  • Each wireless microphone array node is connected to one or more signal processing units in the wireless network through the wireless transmission and control module 301 .
  • the microphone audio acquisition modules 302 and the function module 303 are communicatively connected to the wireless transmission and control module 301 through a data interface.
  • the data interface may adopt UART, SPI, I2S, USB, SDIO, I2C, any standard interface, or any non-standard private interface.
  • the present disclosure further discloses a sound source positioning system to which a distributed microphone array is applicable.
  • the positioning system includes: the above-mentioned distributed microphone array and one or more signal processing units.
  • the distributed microphone array is communicatively connected to the signal processing units via a wireless network.
  • the distributed microphone array is used for directionally acquiring a single sound source or multiple sound sources.
  • the signal processing unit is communicatively connected to the distributed microphone array via the wireless network, receives an unprocessed audio signal uploaded by the distributed microphone array or an audio signal which is simply processed on the microphone array node, and performs audio signal processing of audio mixing, noise suppression, echo cancellation or three-dimensional spatial sound source positioning.
  • the signal processing unit is an independent device in the wireless network, or is integrated in functionality with other device in the wireless network, for example, integrated with an access point (AP) in a Wi-Fi wireless network.
  • AP access point
  • the sound source positioning system can be further provided with an optical monitoring device or other monitoring devices, such as cameras, to achieve the application of complex dynamic audio and video positioning and monitoring acquisition of multiple sound sources.
  • FIG. 4 illustrates a first embodiment of a sound source positioning system based on a distributed microphone array.
  • the distributed microphone array is a simple wireless microphone array based on Wi-Fi.
  • the Wi-Fi wireless microphone array includes four wireless microphone array nodes: a first node 401 , a second node 402 , a third node 403 and a forth node 404 .
  • the first node 401 and the second node 402 each are formed of a Wi-Fi lighting lamp group suspended on the top of the space, which is integrated with a microphone audio acquisition module.
  • the third node 403 is a desktop-type Wi-Fi microphone node in which two audio acquisition modules are provided.
  • the fourth node 404 is a Wi-Fi projector integrated with a microphone audio acquisition module.
  • An integrated Wi-Fi access point (AP) and signal processing unit 405 is communicatively connected to the first node 401 , the second node 402 , the third node 403 and the fourth node 404 via a wireless network and is configured for carrying out audio mixing of the audio signals uploaded by the first node 401 , the second node 402 , the third node 403 and the forth node 404 .
  • the audio signals acquired by the Wi-Fi microphone array nodes are transmitted via Wi-Fi wireless links to the integrated Wi-Fi access point and signal processing unit 405 .
  • the audio signals After performing processing such as echo and noise suppression and signal enhancement by the signal processing unit, the audio signals are transmitted to an audio device to broadcast in real time, or are transmitted to a database for further data processing and storage. Such the transmission can be made via Wi-Fi wireless links or wired links according to the practical application.
  • FIG. 5 illustrates a second embodiment of a sound source positioning system based on a distributed microphone array.
  • the system is a large real-time automatic audio and video monitoring system based on a Wi-Fi wireless microphone array.
  • the system includes ten nodes, three integrated Wi-Fi access point and signal processing units wirelessly communicating with the ten nodes, a central signal processing unit and monitoring system communicatively connected to the three Wi-Fi access point and signal processing units, and four cameras communicatively connected to the central signal processing unit and monitoring system.
  • the system shown by this embodiment may be deployed in a large public area (for example, a shopping center, an open space of a supermarket, a station hall of a railway station, etc.).
  • a large public area for example, a shopping center, an open space of a supermarket, a station hall of a railway station, etc.
  • the wireless Wi-Fi microphone array in this embodiment includes ten nodes.
  • Each of the ten nodes is formed of a suspended Wi-Fi lighting lamp group integrated with a microphone audio acquisition module.
  • the ten nodes are divided into three node groups (that is, a first node group 501 , a second node group 502 and a third node group 503 ).
  • the first node group 501 , the second node group 502 and the third node group 503 correspond to three Wi-Fi access points integrated with signal processing units (that is, a first Wi-Fi access point and signal processing unit 504 , a second Wi-Fi access point and signal processing unit 505 and a third Wi-Fi access point and signal processing unit 506 ), respectively.
  • the first node group 501 , the second node group 502 and the third node group 503 overlap with each other, so that two Wi-Fi microphone array nodes are located in two node groups, respectively, in order to increase inputted signal sources, facilitate data processing by the signal processing units, and provide more stable audio acquisition coverage at the borders of the microphone array node groups.
  • the first Wi-Fi access point and signal processing unit 504 , the second Wi-Fi access point and signal processing unit 505 and the third Wi-Fi access point and signal processing unit 506 perform only simple signal processing, such as voice noise suppression, directional signal acquisition, and simple sound source positioning, etc.
  • a more advanced signal processing and monitoring unit is located at the central signal processing unit and monitoring system 507 .
  • Output signals of the three Wi-Fi access point and signal processing units are inputted into the central signal processing unit and monitoring system 507 .
  • This system can perform recognition and extraction for multiple sound sources, accurate three-dimensional positioning, speech recognition, sensitive information extraction, monitoring for a sound source video signal flow, and tracking of the specific sound source in a wide range, for example.
  • the monitoring for the sound source video signal flow means that the monitoring system module performs further video monitoring of one or more specific sound sources according to the audio information processed by the central signal processing unit, and call the cameras 508 near the locations of the sound sources to lock and amplify the sound sources and track and monitor the sound sources in a wide range.
  • the central signal processing unit and monitoring system 507 can conveniently perform dynamic monitoring of multiple sound sources, that is, Based on acquisition of the recognized information by the microphone array and camera group, audio and video information of sound sources in the area are dynamically acquired to generate dynamic monitoring information flows, which includes:
  • the content of the above-mentioned dynamic monitoring information flows is dynamically acquired and updated according to a specific algorithm of the processing unit so as to comprehensively monitor the area in real time.

Landscapes

  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Abstract

Disclosed is a distributed microphone array. The array comprises: a plurality of distributed microphone array nodes, wherein each microphone array node is provided with one or a plurality of microphone audio acquisition modules; and the microphone array node is connected to an external signal processing unit by means of wireless network communication, and uploads an unprocessed audio signal or an audio signal which has undergone simple signal processing on the array node to the signal processing unit. In the present invention, all the wireless microphone array nodes distributed in a wireless network form a large microphone array. In a wireless microphone array, microphone signal acquisition and transmission are based on a wireless type, and the distribution of microphones is no longer limited by a physical dimension or a cabling connection, so that the distribution of a significant amount of microphones can be achieved, and the distance between the microphones and the coverage area of the microphone array can be greatly increased.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
This application is the United States national phase under 35 U.S.C. § 371 of PCT International Patent Application No. PCT/CN2016/113578, filed on Dec. 30, 2016, which claims benefit of priority of Chinese Application No. 201610535135.7, filed on Jul. 8, 2016.
TECHNICAL FIELD
The present disclosure relates to a microphone array technology and, in particular, to a distributed microphone array and sound source positioning system having the same.
BACKGROUND
Microphone array technology has important and wide applications in the field of audio signal processing. Within a spatial range in which audio signal acquisition is required, a microphone array is constituted by multiple microphones deployed in different positions.
Compared with traditional audio signal acquisition by a single microphone, a sound pick-up range of the microphone array is much extended. Depending on the size of the microphone array, the pick-up range of the microphone array is extended from less than 0.5 meters of the traditional pick-up by a single microphone to several meters, or even dozens of meters. By use of signals acquired by multiple microphones, specific audio signals can be effectively enhanced in a noise environment so that a good noise suppression effect can be achieved; and a directional sound pick-up operation can be implemented and voices from audio signal sources in specific directions can be acquired; and sound sources can be positioned conveniently.
The mainstream microphone products in the current market include small-sized wired microphone arrays, small-sized wireless microphone arrays (mainly using Wi-Fi access) and suspended microphone arrays.
As shown in FIG. 1, most small-sized microphone arrays are desktop microphone arrays, wherein an audio mixer 101 is integrated with multiple microphones 102 and Wi-Fi device, and has a physical dimension generally less than 0.3 meter. The distances between the microphones 102 are small, and space diversity gains of the audio signal processing are limited; and the mixer 101 output may be connected to a terminal device such as an audio device 103 in a wired or wireless mode.
Proper increase in the distances between microphones may provide higher space diversity gains and a better directional sound pick-up effect due to the long wavelength of audio signals.
As shown in FIG. 2, a suspended microphone array is a microphone array product providing long distances between microphones, wherein microphones 202 suspended on the top 201 of a space (for example, a large meeting place) acquire specific audio signals in the space, the audio signals are inputted into a mixer 203 by specific cables to make signal processing, and then the output signals of the mixer 203 are transmitted into a speaker 205 through a signal line 204, and finally the speaker 205 outputs the acquired audio signals.
The connections between the microphones and the mixer of the mainstream products listed above are all wired. The small-sized integrated mode strongly limits the physical dimension of the microphone array, and thus limits the sound pick-up range and space diversity gains of the microphone array; also, the cable connection mode of the suspended microphone array with long distances between microphones increases the difficulty of microphone deployment, and the limited number of input ports of the mixer also limit the number of the deployed microphones, thereby restraining the physical coverage area of the microphone array in practical application.
SUMMARY
The present disclosure provides a distributed microphone array and a sound source positioning system having the same, which provide a large span of sound pick-up range, higher space diversity gains and a more accurate three-dimensional spatial sound source positioning and achieve directional acquisition of multiple sound sources.
To achieve this, the present disclosure provides a distributed microphone array. The array includes a plurality of microphone array nodes which are deployed in a distributed manner. Each of the plurality of microphone array nodes is provided with one or more microphone audio acquisition modules. The microphone array node is configured to communicatively connect to an external signal processing unit via a wireless network, and upload, an unprocessed audio signal or an audio signal which is simply processed on the microphone array node, to the signal processing unit.
The wireless network adopts 802.11 Wi-Fi communication, Bluetooth communication, Zigbee communication or any wireless network transmission mode.
Each of the plurality of microphone array nodes further includes a wireless device or an Internet of Things device with any application function.
Each of the plurality of microphone array nodes further includes:
a wireless transmission and control module, where an input end of the wireless transmission and control module is communicatively connected to the one or more microphone audio acquisition modules, and the wireless transmission and control module is communicatively connected to the signal processing unit via the wireless network;
a function module, which is communicatively connected to the wireless transmission and control module and is any application function device.
The one or more microphone audio acquisition module and the function module are communicatively connected to the wireless transmission and control module through a data interface; and the data interface adopts UART, SPI, I2S, USB, SDIO, I2C or any standard interface.
The present disclosure also provides a sound source positioning system. The sound source positioning system includes:
the above-mentioned distributed microphone array, which directionally acquires a single sound source or multiple sound sources; and
the signal processing unit, which is configured to communicatively connect to the distributed microphone array via a wireless network, receive an unprocessed audio signal uploaded by the distributed microphone array, and perform audio signal processing of audio mixing, noise suppression, echo cancellation or three-dimensional spatial sound source positioning.
The positioning system includes one or more signal processing units connected to the wireless network.
The signal processing unit is an independent device in the wireless network or integrated with an access point (AP) in a Wi-Fi wireless network.
The positioning system further includes an optical monitoring device.
Compared with the microphone array technology in the prior art, the distributed microphone array and sound source positioning system having the same according to the present disclosure has the advantages in that all the wireless microphone array nodes deployed in the wireless network form a large microphone array in the present disclosure. In the wireless microphone array, microphone signal acquisition and transmission are based on a wireless mode, and the distributed deployment of microphones is no longer limited by physical dimensions or cable connections, so that a considerable number of microphones can be deployed and the distances between the microphones as well as the coverage area of the microphone array can be greatly increased.
The deployment of the wireless microphone array nodes is simple and flexible, thereby achieving deployment of a large quantity of microphones with long distances. Also, the spatial positions of the wireless microphone array nodes are diversified, so that the wireless microphone array has a large span of sound pick-up range and abundant acquired signal samples to achieve better echo and noise suppression in the signal processing unit, thereby providing higher space diversity gains and more accurate three-dimensional spatial sound source positioning and achieving directional acquisition of multiple sound sources.
The microphone audio acquisition module according to the present disclosure can be integrated with an existing network device conveniently. Thus, in practical application, the mode of adding the microphone audio acquisition module to an existing network device in an acquisition space is often used in order to make the microphone deployment more convenient and flexible. In this mode, a lower cost can be achieved without cables, and the spatial positions for audio signal acquisition are not limited to the same surface and hence are diversified.
The signal processing unit according to the present disclosure can be a powerful processing unit deployed in the wireless network according to practical application to provide a stronger signal processing capability, and can cooperate with other monitoring device such as cameras, to achieve the application for complex dynamic audio and video positioning and monitoring acquisition of multiple sound sources.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 is a structural view of a small-sized microphone array in the prior art.
FIG. 2 is a structural view of a suspended microphone array in the prior art.
FIG. 3 is a structural view of wireless microphone array nodes according to the present disclosure.
FIG. 4 is a system diagram of a sound source positioning system based on a distributed microphone array according to embodiment one of the present disclosure.
FIG. 5 is a system diagram of a sound source positioning system based on a distributed microphone array according to embodiment two of the present disclosure.
 DETAILED DESCRIPTION
The embodiments of the present disclosure will be further described in detail with reference to the accompanying drawings.
The present disclosure discloses a large-scale distributed microphone array adopting wireless access. The array includes a plurality of microphone array nodes which are distributed in a space and a wireless network. Each of the plurality of microphone array nodes is provided with one or more microphone audio acquisition modules. The microphone array node is communicatively connected to an external signal processing unit via the wireless network, and uploads, an unprocessed audio signal or an audio signal which is simply processed on the microphone array node, to the signal processing unit to perform audio mixing and other audio signal processing.
The microphone array node acquires audio signals by microphone audio acquisition modules, and wirelessly transmits the acquired audio signals to the signal processing unit. The microphone array node does not further process the acquired audio signals.
Each microphone array node is connected to one or more signal processing units of the wireless network through a wireless transmission module. The signal processing unit may be an independent device in the wireless network or be integrated with other devices in the wireless network, for example, integrated with an access point (AP) in a Wi-Fi wireless network.
Further, the wireless network where the microphone array nodes are located may can adopt 802.11 Wi-Fi communication (for example, Wi-Fi communication or Wi-Fi Internet of Things), Bluetooth wireless communication, Zigbee wireless communication or any wireless network transmission mode. At present, the 802.11 Wi-Fi communication standard is one of the most widely used wireless access technology standards. In the field of Internet of Things, the Wi-Fi Internet of Things access mode is one of the most widely used, low-cost and best extensible Internet of Things access modes. A Wi-Fi network is used as an example in the present disclosure.
Further, the microphone array node further includes a wireless device or an Internet of Things device with any application function. The wireless microphone array node may include any wireless device or Internet of Things device supporting other application functions along with a microphone audio acquisition module, for example, a Wi-Fi light group device along with the microphone module, or a Wi-Fi household Internet of Things device along with the microphone module. The wireless microphone array node may also provide only the function of audio signal acquisition.
Reference is made to FIG. 3, which is a structural view of microphone array nodes according to an embodiment. The microphone array node includes a wireless transmission and a control module 301, two microphone audio acquisition modules 302 and a function module 303.
The two microphone audio acquisition modules 302 are used for audio signal digital acquisition.
The function module 303 is a device providing any application function, for example, household appliance controlling, lighting controlling and other data acquisition and processing.
The wireless transmission and control module 301 is communicatively connected to the microphone audio acquisition modules 302 and the function module 303 via a control signal and a data interface, and is communicatively connected to an external signal processing unit via a wireless network.
Each wireless microphone array node is connected to one or more signal processing units in the wireless network through the wireless transmission and control module 301.
The microphone audio acquisition modules 302 and the function module 303 are communicatively connected to the wireless transmission and control module 301 through a data interface. The data interface may adopt UART, SPI, I2S, USB, SDIO, I2C, any standard interface, or any non-standard private interface.
The present disclosure further discloses a sound source positioning system to which a distributed microphone array is applicable. The positioning system includes: the above-mentioned distributed microphone array and one or more signal processing units. The distributed microphone array is communicatively connected to the signal processing units via a wireless network.
The distributed microphone array is used for directionally acquiring a single sound source or multiple sound sources.
The signal processing unit is communicatively connected to the distributed microphone array via the wireless network, receives an unprocessed audio signal uploaded by the distributed microphone array or an audio signal which is simply processed on the microphone array node, and performs audio signal processing of audio mixing, noise suppression, echo cancellation or three-dimensional spatial sound source positioning.
The signal processing unit is an independent device in the wireless network, or is integrated in functionality with other device in the wireless network, for example, integrated with an access point (AP) in a Wi-Fi wireless network.
Further, according to practical application, the sound source positioning system can be further provided with an optical monitoring device or other monitoring devices, such as cameras, to achieve the application of complex dynamic audio and video positioning and monitoring acquisition of multiple sound sources.
Reference is made to FIG. 4, which illustrates a first embodiment of a sound source positioning system based on a distributed microphone array. The distributed microphone array is a simple wireless microphone array based on Wi-Fi.
The Wi-Fi wireless microphone array includes four wireless microphone array nodes: a first node 401, a second node 402, a third node 403 and a forth node 404.
The first node 401 and the second node 402 each are formed of a Wi-Fi lighting lamp group suspended on the top of the space, which is integrated with a microphone audio acquisition module. The third node 403 is a desktop-type Wi-Fi microphone node in which two audio acquisition modules are provided. The fourth node 404 is a Wi-Fi projector integrated with a microphone audio acquisition module.
An integrated Wi-Fi access point (AP) and signal processing unit 405 is communicatively connected to the first node 401, the second node 402, the third node 403 and the fourth node 404 via a wireless network and is configured for carrying out audio mixing of the audio signals uploaded by the first node 401, the second node 402, the third node 403 and the forth node 404.
The audio signals acquired by the Wi-Fi microphone array nodes are transmitted via Wi-Fi wireless links to the integrated Wi-Fi access point and signal processing unit 405. After performing processing such as echo and noise suppression and signal enhancement by the signal processing unit, the audio signals are transmitted to an audio device to broadcast in real time, or are transmitted to a database for further data processing and storage. Such the transmission can be made via Wi-Fi wireless links or wired links according to the practical application.
Reference is made to FIG. 5, which illustrates a second embodiment of a sound source positioning system based on a distributed microphone array. Specifically, the system is a large real-time automatic audio and video monitoring system based on a Wi-Fi wireless microphone array. The system includes ten nodes, three integrated Wi-Fi access point and signal processing units wirelessly communicating with the ten nodes, a central signal processing unit and monitoring system communicatively connected to the three Wi-Fi access point and signal processing units, and four cameras communicatively connected to the central signal processing unit and monitoring system.
The system shown by this embodiment may be deployed in a large public area (for example, a shopping center, an open space of a supermarket, a station hall of a railway station, etc.).
The wireless Wi-Fi microphone array in this embodiment includes ten nodes. Each of the ten nodes is formed of a suspended Wi-Fi lighting lamp group integrated with a microphone audio acquisition module. The ten nodes are divided into three node groups (that is, a first node group 501, a second node group 502 and a third node group 503). The first node group 501, the second node group 502 and the third node group 503 correspond to three Wi-Fi access points integrated with signal processing units (that is, a first Wi-Fi access point and signal processing unit 504, a second Wi-Fi access point and signal processing unit 505 and a third Wi-Fi access point and signal processing unit 506), respectively. The first node group 501, the second node group 502 and the third node group 503 overlap with each other, so that two Wi-Fi microphone array nodes are located in two node groups, respectively, in order to increase inputted signal sources, facilitate data processing by the signal processing units, and provide more stable audio acquisition coverage at the borders of the microphone array node groups.
The first Wi-Fi access point and signal processing unit 504, the second Wi-Fi access point and signal processing unit 505 and the third Wi-Fi access point and signal processing unit 506 perform only simple signal processing, such as voice noise suppression, directional signal acquisition, and simple sound source positioning, etc.
A more advanced signal processing and monitoring unit is located at the central signal processing unit and monitoring system 507. Output signals of the three Wi-Fi access point and signal processing units are inputted into the central signal processing unit and monitoring system 507. This system can perform recognition and extraction for multiple sound sources, accurate three-dimensional positioning, speech recognition, sensitive information extraction, monitoring for a sound source video signal flow, and tracking of the specific sound source in a wide range, for example.
The monitoring for the sound source video signal flow means that the monitoring system module performs further video monitoring of one or more specific sound sources according to the audio information processed by the central signal processing unit, and call the cameras 508 near the locations of the sound sources to lock and amplify the sound sources and track and monitor the sound sources in a wide range.
As mentioned above, in a large area monitored by the system, the central signal processing unit and monitoring system 507 can conveniently perform dynamic monitoring of multiple sound sources, that is, Based on acquisition of the recognized information by the microphone array and camera group, audio and video information of sound sources in the area are dynamically acquired to generate dynamic monitoring information flows, which includes:
{audio flow 1, location 1, video flow 1;
audio flow 2, location 2, video flow 2;
. . .
audio flow n, location n, video flow n}.
The content of the above-mentioned dynamic monitoring information flows is dynamically acquired and updated according to a specific algorithm of the processing unit so as to comprehensively monitor the area in real time.
While the content of the present disclosure has been described in detail with reference to the above-mentioned preferred embodiments, the above description is not to be construed as limiting the present disclosure. Various modifications and substitutions are all apparent after those skilled in the art read the above content. Therefore, the scope of protection of the present disclosure is defined by the appended claims.

Claims (6)

What is claimed is:
1. A sound source positioning system, comprising:
a distributed microphone array which directionally acquires a single sound source or multiple sound sources, wherein the distributed microphone array comprises: a plurality of microphone array nodes which are deployed in a distributed manner, and a signal processing unit external to the plurality of microphone array nodes, wherein each of the plurality of microphone array nodes is provided with one or more microphone audio acquisition modules, wherein the each of the microphone array nodes is configured to communicatively connect to the signal processing unit via a wireless network, and directly upload to the signal processing unit an unprocessed audio signal;
wherein the signal processing unit is configured to communicatively connect to the distributed microphone array via the wireless network, receive the unprocessed audio signal uploaded by the distributed microphone array, and perform audio signal processing of audio mixing, noise suppression, echo cancellation or three-dimensional spatial sound source positioning; the signal processing unit is integrated with an access point of the wireless network; and
a central signal processing unit and monitoring system which is configured to receive output signals of the signal processing unit, and perform recognition and extraction for multiple sound sources, accurate three-dimensional positioning, speech recognition, sensitive information extraction, monitoring for a sound source video signal flow, and tracking of a specific sound source in a wide range.
2. The sound source positioning system according to claim 1, further comprising an optical monitoring device.
3. The sound source positioning system according to claim 1, wherein
the wireless network adopts one of the following wireless network transmission modes: 802.11 Wi-Fi communication, Bluetooth communication, and Zigbee Communication.
4. The sound source positioning system according to claim 1, wherein
the each of the plurality of microphone array, nodes further comprises a wireless device or an Internet of Things device with at least one application function.
5. The sound source positioning system according to claim 1, wherein the each of the plurality of microphone array nodes further comprises:
a wireless transmission and control module, wherein an input end of the wireless transmission and control module is communicatively connected to the one or more microphone audio acquisition modules, and the wireless transmission and control module is communicatively connected to the signal processing unit via the wireless network; and
a function module, which is communicatively connected to the wireless transmission and control module and is a device with at least one application function.
6. The sound source positioning system according to claim 5, wherein
the one or more microphone audio acquisition modules and the function module are communicatively connected to the wireless transmission and control module through a data interface; and the data interface comprises one of the following standard interfaces: UART, SPI, I2S, USB, SDIO and I2C.
US16/086,166 2016-07-08 2016-12-30 Distributed microphone array and sound source positioning system applicable thereto Active US10659876B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201610535135 2016-07-08
CN201610535135.7 2016-07-08
CN201610535135.7A CN106028227B (en) 2016-07-08 2016-07-08 Distributed microphone array and its applicable sonic location system
PCT/CN2016/113578 WO2018006568A1 (en) 2016-07-08 2016-12-30 Distributed microphone array and sound source positioning system applicable thereto

Publications (2)

Publication Number Publication Date
US20190116418A1 US20190116418A1 (en) 2019-04-18
US10659876B2 true US10659876B2 (en) 2020-05-19

Family

ID=57109785

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/086,166 Active US10659876B2 (en) 2016-07-08 2016-12-30 Distributed microphone array and sound source positioning system applicable thereto

Country Status (3)

Country Link
US (1) US10659876B2 (en)
CN (1) CN106028227B (en)
WO (1) WO2018006568A1 (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106028227B (en) 2016-07-08 2019-05-24 乐鑫信息科技（上海）股份有限公司 Distributed microphone array and its applicable sonic location system
CN106199611B (en) * 2016-08-19 2020-04-07 乐鑫信息科技（上海）股份有限公司 Sonar-integrated Internet of things equipment and space environment detection system applicable to same
CN106448677A (en) * 2016-10-26 2017-02-22 江苏金米智能科技有限责任公司 A voice positioning control system used in a building and based on a wireless sensing network
CN107102296B (en) * 2017-04-27 2020-04-14 大连理工大学 Sound source positioning system based on distributed microphone array
CN107527626A (en) * 2017-08-30 2017-12-29 北京嘉楠捷思信息技术有限公司 Audio identification system
CN107767868A (en) * 2017-10-23 2018-03-06 深圳北鱼信息科技有限公司 Microphone array and speech control system
CN108376547A (en) * 2018-05-08 2018-08-07 贵州大学 A kind of array speech collecting system and acquisition method based on Arduino
JP7120077B2 (en) * 2019-02-27 2022-08-17 トヨタ自動車株式会社 driving support system
CN110095755B (en) * 2019-04-01 2021-03-12 云知声智能科技股份有限公司 Sound source positioning method
CN112929606A (en) * 2021-01-29 2021-06-08 世邦通信股份有限公司 Audio and video acquisition method and device and storage medium
CN115175049B (en) * 2022-09-07 2022-12-09 杭州兆华电子股份有限公司 Master-slave mode microphone array system
WO2024129713A1 (en) * 2022-12-13 2024-06-20 Stryker Corporation Active noise cancellation in a medical environment

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1426047A (en) 2001-12-12 2003-06-25 雅马哈株式会社 Mixer device and music device capable of communicating with mixer device
CN103841360A (en) 2013-12-11 2014-06-04 三亚中兴软件有限责任公司 Distributed video conference achieving method and system, video conference terminal and audio and video integrated device
KR101532712B1 (en) 2014-02-06 2015-07-01 김철호 Auditor suitability Sound System for Mass Public Facilities
US20150286459A1 (en) * 2012-12-21 2015-10-08 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Filter and method for informed spatial filtering using multiple instantaneous direction-of-arrival estimates
CN105206281A (en) 2015-09-14 2015-12-30 胡旻波 Voice enhancement device based on distributed microphone array network
CN105376531A (en) 2015-10-30 2016-03-02 苏州扬佛自动化设备有限公司 Intelligent security and protection system based on IOT (Internet of Things)
CN106028227A (en) 2016-07-08 2016-10-12 乐鑫信息科技（上海）有限公司 Distributed microphone array and sound source positioning system employing same
US20170358313A1 (en) * 2016-06-09 2017-12-14 Sonos, Inc. Dynamic Player Selection for Audio Signal Processing
US20190019295A1 (en) * 2015-08-14 2019-01-17 Nokia Technologies Oy Monitoring

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100493183C (en) * 2005-12-28 2009-05-27 陈沨 Remote wireless realtime multi-media-messaging service security system
CN102082989A (en) * 2010-11-17 2011-06-01 深圳市驭天健科技有限公司 ZigBee protocol-based all-digital wireless sound reinforcement device for teaching
CN102137316B (en) * 2010-12-29 2014-06-18 北京恒智兴业数码科技有限公司 Digital high-definition multichannel microphone system
KR101355910B1 (en) * 2013-01-16 2014-01-29 주식회사 큐앤웍스 Wireless microphone system using smart phone
CN205961412U (en) * 2016-07-08 2017-02-15 乐鑫信息科技（上海）有限公司 Distributing type microphone array and sound localization system that is suitable for thereof

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1426047A (en) 2001-12-12 2003-06-25 雅马哈株式会社 Mixer device and music device capable of communicating with mixer device
US20150286459A1 (en) * 2012-12-21 2015-10-08 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Filter and method for informed spatial filtering using multiple instantaneous direction-of-arrival estimates
CN103841360A (en) 2013-12-11 2014-06-04 三亚中兴软件有限责任公司 Distributed video conference achieving method and system, video conference terminal and audio and video integrated device
KR101532712B1 (en) 2014-02-06 2015-07-01 김철호 Auditor suitability Sound System for Mass Public Facilities
US20190019295A1 (en) * 2015-08-14 2019-01-17 Nokia Technologies Oy Monitoring
CN105206281A (en) 2015-09-14 2015-12-30 胡旻波 Voice enhancement device based on distributed microphone array network
CN105376531A (en) 2015-10-30 2016-03-02 苏州扬佛自动化设备有限公司 Intelligent security and protection system based on IOT (Internet of Things)
US20170358313A1 (en) * 2016-06-09 2017-12-14 Sonos, Inc. Dynamic Player Selection for Audio Signal Processing
CN106028227A (en) 2016-07-08 2016-10-12 乐鑫信息科技（上海）有限公司 Distributed microphone array and sound source positioning system employing same

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
International Search Report for PCT/CN2016/113578 dated Mar. 31, 2017.
Zhang Hui-Xin, "Sound Source Localization Distributed Microphone Array", Zhang, Hui-xin, Yan, An-bin, Wang, Xin, Jiang Wei, "Sound Source Localization Distributed Microphone Array", China Academic Journal Electronic Publishing House, vol. 41, No. 2, Feb. 2016 (pp. 77-81).

Also Published As

Publication number Publication date
CN106028227A (en) 2016-10-12
WO2018006568A1 (en) 2018-01-11
US20190116418A1 (en) 2019-04-18
CN106028227B (en) 2019-05-24

Similar Documents

Publication Publication Date Title
US10659876B2 (en) Distributed microphone array and sound source positioning system applicable thereto
JP6799573B2 (en) Terminal bracket and Farfield voice dialogue system
US8451315B2 (en) System and method for distributed meeting capture
EP3148223A2 (en) A method of relating a physical location of a loudspeaker of a loudspeaker system to a loudspeaker identifier
US9326064B2 (en) Microphone array configuration and method for operating the same
CN106548783B (en) Voice enhancement method and device, intelligent sound box and intelligent television
US20170070822A1 (en) Method for determining or verifying spatial relations in a loudspeaker system
JP6465913B2 (en) Method and system for self-adaptive transmission of communication signals
CN205249484U (en) Microphone linear array reinforcing directive property adapter
CN107889001B (en) Expandable microphone array and establishing method thereof
CN113273222A (en) Framework for processing sensor data in smart home system
US20190089839A1 (en) Sound Signal Processing Device and Sound Signal Processing Method
CN108184201A (en) audio testing method, audio test system
CN104468146B (en) The broadcast intercom system of IP based network
US20140093099A1 (en) Wireless Microphone Structure
US20220284893A1 (en) Wireless lighting control systems for intelligent luminaires
CN109286790B (en) Directional monitoring system based on sound source positioning and monitoring method thereof
CN205961412U (en) Distributing type microphone array and sound localization system that is suitable for thereof
US9432566B2 (en) Camera system capable of expanding functions and an amount of video stream
WO2022093464A1 (en) Technologies for providing media control device quick response (qr) code functionality
KR102003675B1 (en) System and method for detecting trouble of line-array speaker
KR20170113911A (en) Monitoring system and method
JP7019177B2 (en) Lighting equipment
EP3364410A1 (en) System and method for voice recognition
KR102430456B1 (en) Mobile communication service system using internet networks using utp cables

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

AS Assignment

Owner name: ESPRESSIF SYSTEMS (SHANGHAI) PTE LTD, CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TEO, SWEE ANN;ZHAN, RUI;REEL/FRAME:047108/0547

Effective date: 20180904

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

AS Assignment

Owner name: ESPRESSIF SYSTEMS (SHANGHAI) CO., LTD., CHINA

Free format text: CHANGE OF NAME;ASSIGNOR:ESPRESSIF SYSTEMS (SHANGHAI) PTE., LTD.;REEL/FRAME:049316/0885

Effective date: 20181126

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

AS Assignment

Owner name: ESPRESSIF SYSTEMS (SHANGHAI) PTE LTD, CHINA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECT THE ADDRESS PREVIOUSLY RECORDED AT REEL: 047108 FRAME: 0547. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNORS:TEO, SWEE ANN;ZHAN, RUI;REEL/FRAME:049741/0374

Effective date: 20180904

AS Assignment

Owner name: ESPRESSIF SYSTEMS (SHANGHAI) CO., LTD., CHINA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY ADDRESS PREVIOUSLY RECORDED ON REEL 049316 FRAME 0885. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME;ASSIGNOR:ESPRESSIF SYSTEMS (SHANGHAI) PTE., LTD.;REEL/FRAME:049763/0310

Effective date: 20181126

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY