CN111629041B - Remote bird sound signal transmission system - Google Patents
Remote bird sound signal transmission system Download PDFInfo
- Publication number
- CN111629041B CN111629041B CN202010429163.7A CN202010429163A CN111629041B CN 111629041 B CN111629041 B CN 111629041B CN 202010429163 A CN202010429163 A CN 202010429163A CN 111629041 B CN111629041 B CN 111629041B
- Authority
- CN
- China
- Prior art keywords
- bird sound
- bird
- sound
- acquisition terminal
- birdsound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention discloses a remote bird sound signal transmission system, which comprises: the system comprises a plurality of bird sound acquisition terminals, gateway equipment and a cloud server; each bird sound acquisition terminal is used for acquiring sound signals of the monitoring area, generating bird sound audio files according to the sound signals and then sending the bird sound audio files to the gateway equipment through the built-in wireless communication module when receiving access requests sent by the gateway equipment; the gateway equipment is used for sending an access request to each bird sound acquisition terminal, then collecting bird sound audio files of each terminal, establishing communication connection with the cloud server through the wireless network module, and uploading the bird sound audio files to the cloud server; and the cloud server is used for storing and forwarding the bird sound audio file after receiving the bird sound audio file. By implementing the embodiment of the invention, the timeliness of bird sound collection can be improved under the condition of not influencing the normal activity of birds.
Description
Technical Field
The invention relates to the field of biological monitoring, in particular to a remote bird sound signal transmission system.
Background
Birds are one of important members in nature, the bird sound is one biological characteristic of the birds, can be used for identifying the varieties of the birds, and has important significance in biodiversity monitoring and ecological environment protection. The existing bird sound collection method generally comprises the steps of placing a recording device in a related field area for recording and storing according to a set program, and carrying out manual processing and analysis on data after a worker regularly retrieves a memory card. The method can continuously record the sound data of the target area for two months or three months, does not need manual guard, and has poor real-time performance. In order to enhance timeliness, the bird sounds can be recorded by workers in the field, but the recorded area range is limited, the recorded bird sound data is less, and the activities of birds can be influenced by people in the field. Or the birds to be detected are captured back from the field, and the bird sound is collected in the laboratory environment, although the bird sound quality is excellent, the normal activities of the birds are seriously influenced.
Disclosure of Invention
The embodiment of the invention provides a remote bird sound signal transmission system which can improve the timeliness of bird sound collection under the condition that the normal activity of birds is not influenced.
An embodiment of the present invention provides a remote bird sound signal transmission system, including:
the system comprises a plurality of bird sound acquisition terminals, gateway equipment and a cloud server;
each bird sound acquisition terminal is used for acquiring sound signals of a monitoring area, generating bird sound audio files according to the sound signals and then sending the bird sound audio files to the gateway equipment through a built-in wireless communication module when receiving an access request sent by the gateway equipment;
the gateway device is used for sending an access request to each bird sound acquisition terminal, then receiving a bird sound audio file of each bird sound acquisition terminal, establishing communication connection with the cloud server through a built-in wireless network module, and then uploading the bird sound audio file to the cloud server;
and the cloud server is used for storing and forwarding the bird sound audio file after receiving the bird sound audio file.
Further, each of the bird sound collection terminals is further configured to send bird sound file information corresponding to the bird sound audio file and device state information of the respective bird sound collection terminal to the gateway device;
the gateway equipment is further used for sending the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to the cloud server.
Further, the cloud server comprises a storage server, a message server and an application server;
the storage server is used for receiving the bird sound audio file sent by the gateway equipment after the communication connection is established with the gateway equipment, and then storing the bird sound audio file;
the message server is used for receiving the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment sent by the gateway equipment after establishing communication connection with the gateway equipment, then storing the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment, and forwarding the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to the application server;
the application server is used for extracting the corresponding bird sound audio files from the storage server according to the bird sound file information, then sending the bird sound audio files and the bird sound file information to a bird sound monitoring platform, and sending the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to an equipment management platform.
Further, each of the bird sound collection terminals includes: the device comprises a microphone, a preamplifier, an audio codec, a main control chip, a memory and a wireless communication module;
each bird sound acquisition terminal is used for acquiring sound signals of a monitoring area and generating a bird sound audio file according to the sound signals, and specifically comprises the following steps:
the microphone collects the sound of the monitoring area and transmits the sound to the preamplifier;
the preamplifier gains the sound signal and then transmits the sound signal after the gain to the audio codec;
the audio codec converts the gained sound signals into digital signals to generate audio data, and then transmits the audio data to the main control chip;
the main control chip receives the audio data in real time by utilizing a ping-pong buffer technology, obtains a segment containing the bird sound in the audio data through a bird sound extraction algorithm, generates the bird sound audio file, and stores the bird sound audio file in the memory.
Further, the wireless communication module is a LoRa module, and each bird sound collection terminal and the gateway device are networked in a star topology structure.
Further, each bird sound collection terminal sends the bird sound audio file, the bird sound file information and the equipment state information of each bird sound collection terminal to the gateway equipment in the following manner:
the bird sound acquisition terminal splits the bird sound audio file to obtain a plurality of first bird sound audio data blocks which are sequentially arranged;
the method comprises the steps that the birdsound acquisition terminal receives an access request which is sent by the gateway device and carries a reset data packet, feeds a connection data packet which carries the birdsound file information and the equipment state information of the birdsound acquisition terminal back to the gateway device, so that the gateway device extracts the birdsound file information and the equipment state information of the birdsound acquisition terminal when receiving the connection data packet, and sends a response data packet to the birdsound acquisition terminal;
after receiving the response data packet, the birdsound acquisition terminal sends a plurality of content data packets carrying the first birdsound audio data block to the gateway device one by one until all the first birdsound audio data blocks are sent; when the gateway equipment receives a content data packet, judging whether a first bird sound audio data block in the content data packet is a last bird sound data block, if not, continuously sending a response data packet to the bird sound acquisition terminal so that the bird sound acquisition terminal continuously sends the content data packet after receiving the response data packet; and if so, sending a disconnection data packet to the bird sound acquisition terminal so that the bird sound acquisition terminal finishes sending the content data packet.
Furthermore, each of the bird sound collection terminals is further configured to delete the bird sound audio file stored in the memory after all the first bird sound audio data blocks are sent.
Further, the gateway device sends the bird sound audio file to the storage server by:
the gateway equipment splits the bird sound audio file to obtain a plurality of second bird sound audio data blocks which are sequentially arranged;
the gateway equipment sends a first communication instruction to the storage server through the wireless network module, so that the storage server establishes HTTP communication with the gateway equipment after receiving the first communication instruction;
and the gateway equipment sends a plurality of data transmission instructions carrying second bird sound audio data blocks to the storage server one by one through the wireless network module until all the second bird sound audio data blocks are completely transmitted.
Further, the gateway device sends the bird sound file information, the device state information of each bird sound collecting terminal, and the state information of the gateway device to the message server in the following manner:
the gateway equipment sends a second communication instruction to the message server through the wireless network module, so that the message server establishes MQTT communication with the gateway equipment after receiving the second communication instruction;
the gateway equipment combines the bird sound file information in a JSON format to generate a first character string, and then sends the first character string to the message server;
and the gateway equipment combines the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment in a JSON format to generate a second character string, and then sends the second character string to the message server.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a remote birdsound signal transmission system which comprises a plurality of birdsound acquisition terminals, gateway equipment and a cloud server; after each bird sound acquisition terminal acquires bird sound data, the bird sound data are sent to the gateway equipment by using a wireless communication technology, and then the corresponding bird sound data are uploaded to the cloud server by the gateway equipment; as long as a user has a requirement for acquiring the bird sound data, the bird sound data can be quickly acquired through the remote bird sound signal transmission system; compared with the prior action of regularly retrieving the memory card; on the one hand, timeliness of the bird sound acquisition is improved, and on the other hand, convenience of bird sound acquisition is improved.
Drawings
Fig. 1 is a system architecture diagram of a remote bird sound signal transmission system according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a bird sound collection terminal according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a gateway device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the remote bird sound signal transmission system includes:
the system comprises a plurality of bird sound acquisition terminals, gateway equipment and a cloud server;
each bird sound acquisition terminal is used for acquiring sound signals of a monitoring area, generating a bird sound audio file according to the sound signals and then sending the bird sound audio file to the gateway equipment through a built-in wireless communication module when receiving an access request sent by the gateway equipment;
the gateway device is used for sending an access request to each birdsong acquisition terminal, then receiving a birdsong audio file of each birdsong acquisition terminal, establishing communication connection with the cloud server through a built-in wireless network module, and then uploading the birdsong audio file to the cloud server;
and the cloud server is used for storing and forwarding the bird sound audio file after receiving the bird sound audio file.
The following describes the above devices and their functions one by one:
as shown in fig. 2, each bird sound collection terminal includes a microphone, a preamplifier, an audio codec, a main control chip, a memory, and an LoRa module;
preferably, each bird sound acquisition terminal is in wireless networking connection with the gateway equipment through a LoRa module and in a star topology structure;
each bird sound acquisition terminal is deployed in an acquisition monitoring area, sound acquisition is carried out through a microphone in each bird sound acquisition terminal, and the monitoring range of the microphone reaches 50m 2 Obtaining a sound signal within a frequency response range of 20Hz-20 kHz; then transmitting the sound signal to the preamplifier, performing gain on the sound signal by the amplifier, and then transmitting the sound signal after the gain to the audio codec; the audio coding and decoding chip converts the sound signal into a digital signal to generate audio data, the sampling rate of 32kHz and the quantization precision of 16 bits. Then transmitting the audio data to a main control chip, and receiving the audio data by the main control chip in real time by utilizing a ping-pong buffer technology; in the present invention, the length of the sound signal per frame is 10ms. The main control chip obtains a fragment V of the bird sound contained in the audio data through a bird sound extraction algorithm; performing MP3 compression on the fragments V containing the bird sounds at a code rate of 128kbps to obtain a file F1 (namely, a bird sound audio file) and storing the file into a memory (which can also be a memory card); thereby realizing the recording of the bird sound; when receiving an access request sent by the gateway equipment, the bird sound acquisition terminal acquires the bird sound audio file, the bird sound file information and the bird sound acquisition terminal in the following waySending the standby state information to the gateway device:
the bird sound extraction algorithm adopts an automatic segmentation algorithm based on prior probability to extract bird sound signals in the audio data, the audio data are converted from a time domain to a frequency domain through Fourier transform to be analyzed, noise variance is obtained through a noise spectrum estimation algorithm, pure spectrum estimation is obtained through a minimum mean square error estimation algorithm, and a prior signal-to-noise ratio is obtained through a guide decision method, so that a prior probability value of a sound frame is obtained; and judging whether the current frame has a sound signal or not through a preset threshold value according to the prior probability value of the sound frame, and collecting continuous multi-frame sound frame signals to obtain a segment V containing the bird sound.
The specific scheme of the birdsound extraction algorithm is as follows:
input signal y of the lambda frame in (lambda) carrying out sub-band separation by an orthogonal mirror filter to obtain a low-frequency sub-band y l (lambda) and the high-frequency subband y h (lambda). Since the main energy of most bird singing is concentrated below 8kHz, the high-frequency sub-band has no excessive information for distinguishing bird sounds from noises, and the subsequent steps only carry out on the low-frequency sub-band y l (lambda) analysis. Low frequency subband y l (lambda) sampling frequency F S =16000, number of sampling points N l ＝160。
For low frequency subband signal y l (lambda, N) to obtain the current analysis frame signal y (lambda, N), see formula (1), where N is F The total length after the frame stack is usually an integer power of 2, N F 256; n is sampling point number, N =0,1, \8230, N F -1。
Adopting a mixed flat-top Hanning window function w (N) of formula (2), windowing the signal y (lambda, N), and executing N F Performing point Fourier transform, and obtaining the magnitude spectrum Y (lambda, k) of the signal with noise by taking the modulus value, see formula (3), wherein k is the frequency point number, k =0,1, \ 8230;, N F -1。
Due to the symmetry of the fourier transform, the subsequent steps only analyze the first N frequency points of the spectrum, where N = N F The default value of/2 +1, namely N, is 129.
Noisy signal power spectrum Y 2 (λ, k) the intra smoothing by equation (4) yields S' (λ, k), where W is the normalized Hanning window function and the window length is (2 XN) W +1)，1≤N W ≤5，N W Is 1, i.e., W = [0.25,0.5,0.25 = [0.25,0.25 ]]。
And performing interframe smoothing on the S' (lambda, k) by using a formula (5) to obtain S (lambda, k), wherein alpha is S Is an interframe smoothing factor, 0.5 < alpha S ＜1，α S Is 0.8.
S(λ,k)＝α S ×S(λ-1,k)+(1-α S )×S′(λ,k)k＝0,1,…,N-1 (5)
Noise spectrum estimation algorithm based on minimum value search, wherein the minimum value S of the smooth power spectral density S (lambda, k) is updated after R continuous estimates of the smooth power spectral density S (lambda, k) are obtained min (lambda, k) is shown in formula (6), wherein min {. Cndot } is the minimum operator, R is the number of search frames, and the value of the minimum operator is related to the frame number lambda, which is shown in formula (7).
S min (λ,k)＝min{S(λ′,k)|(λ′＝(λ-R+1),…,λ)} (6)
Obtaining noise power spectrum estimation D according to (8) type smooth updating 2 (λ, k) in which [ α ] m ·S min (λ,k)]As noiseDecision threshold, 2 < alpha m ＜8，α m Is 5. Alpha is alpha D In order to be a weight factor, the weight factor,
noise power spectrum D as the number of frames increases 2 The change in (λ, k) gradually stabilizes.
And combining the historical prior signal-to-noise ratio, and obtaining the prior signal-to-noise ratio estimation xi (lambda, k) by a guide decision method of a formula (9).
Where γ (λ, k) is the a posteriori signal-to-noise ratio estimate, i.e.
X 2 (lambda-1, k) is the bird sound power spectrum estimation of the previous frame, the amplitude spectrum estimation is obtained by the equation (11),
representing the historical a priori signal-to-noise ratio. Alpha is alpha ξ (λ, k) is a weight adjustment factor, and is obtained by equation (10). When the instantaneous signal-to-noise ratio is larger, the weight alpha of the current prior signal-to-noise ratio maximum likelihood estimation value is increased ξ (λ, k) makes the estimate of ξ (λ, k) more accurate.
Wherein alpha is h Is a constant, 0 < alpha h ＜0.5，α h Is 0.1.
The bird's acoustic amplitude spectrum estimate X (λ, k) is updated by equation (11) based on the minimum mean square error estimation criterion.
Wherein u is the power exponent of the amplitude spectrum estimation, u is more than 0 and less than or equal to 2, and the default value of u is 0.5.Γ (·) is a gamma function, and is calculated as formula (12), where Γ (1.25) ≈ 0.9064 when u = 0.5. Phi (-) is a confluent hyper-geometric function, the calculation formula is shown in the formula (13), the calculation complexity of the function is high, and the calculation can be simplified by using an approximate function in the practical engineering application.
And performing interframe smoothing on the prior signal-to-noise ratio xi (lambda, k) to obtain a smooth prior signal-to-noise ratio zeta (lambda, k), see equation (14). Wherein alpha is ζ For inter-frame smoothing factor, 0 < alpha ζ ＜1，α ζ Is 0.7.
ζ(λ,k)＝α ζ ×ζ(λ-1,k)+(1-α ζ )×ξ(λ,k) (14)
And smoothing the prior signal-to-noise ratio zeta (lambda, k) to represent the ratio of the bird sound power to the noise power of the frequency point k, solving the mean value of zeta (lambda, k) and substituting into a formula (15) to obtain the prior probability p (lambda) of the sound frame.
When the probability value P (lambda) is larger than the set threshold value P t If yes, the frame is judged to be a sound frame, otherwise, the frame is judged to be a noise frame. Wherein the threshold value P t Needs to be adjusted to the best effect through experiments, and the P is more than or equal to 0.2 and less than or equal to P t ≤0.8，P t Is 0.5. Input signal y corresponding to voiced frames for successive r frames in (lambda) the set yields a voiced segment signal, denoted as V = { y = in (λ-r+1),y in (λ-r+2),…,y in (λ)}。
The bird sound acquisition terminal splits the bird sound audio file to obtain a plurality of first bird sound audio data blocks which are sequentially arranged;
the bird sound acquisition terminal receives an access request which is sent by the gateway equipment and carries a reset data packet, feeds back a connection data packet which carries the bird sound file information and the bird sound acquisition terminal equipment state information to the gateway equipment, so that the gateway equipment extracts the bird sound file information and the equipment state information of the bird sound acquisition terminal when receiving the connection data packet, and sends a response data packet to the bird sound acquisition terminal;
after receiving the response data packet, the birdsound acquisition terminal sends a plurality of content data packets carrying the first birdsound audio data block to the gateway device one by one until all the first birdsound audio data blocks are sent; when the gateway equipment receives a content data packet, judging whether a first bird sound audio data block in the content data packet is a last bird sound data block, if not, continuously sending a response data packet to the bird sound acquisition terminal so that the bird sound acquisition terminal continuously sends the content data packet after receiving the response data packet; and if so, sending a disconnection data packet to the bird sound acquisition terminal so that the bird sound acquisition terminal finishes sending the content data packet.
Preferably, each of the bird sound collection terminals is further configured to delete the bird sound audio file stored in the memory after all the first bird sound audio data blocks are sent.
Specifically, the method comprises the following steps: since the internal buffer size of the LoRa module is 512 bytes, and the data size of the file F1 (bird audio file) is large, it is necessary to perform packet transmission. In order to ensure stable transmission between the gateway and the terminal, a communication data packet format as shown in table 1 is defined, data packet verification is required for each reception, and if the verification is wrong, the communication process is automatically ended. In addition, the process of mutual response of the two parties is limited in time, and the communication process is automatically ended if the response is overtime. Gateway equipment also embeds has the loRa module, and the both sides pass through the loRa module and accomplish data transmission.
Table 1 wireless communication packet format definition
Table 1 shows the format definition of the wireless communication data packet when data transmission is performed between each bird sound collection terminal and the gateway; the following describes the wireless data transmission procedure between the gateway and each bird sound collection terminal in detail with reference to table 1:
1. the birdsound acquisition terminal reads a file F1 (namely, a birdsound audio file) from the memory and loads the file to the data buffer a, and the file is split by 480 bytes to obtain K data blocks which are marked as a (1), a (2), (8230) and a (K).
2. The gateway device selects a bird sound collection terminal through the address to send the R packet (namely, the reset data packet). The LoRa address of the gateway is 0x0000, the LoRa addresses of the plurality of terminals are not repeated, and the value range is 0x0001-0xFFFE.
3. And after receiving the R packet, the bird sound acquisition terminal reads the date and time in the R packet and updates locally, integrates the current bird sound audio file information and the terminal equipment state information into a C packet (namely the connection data packet) and sends the C packet to the gateway equipment.
4. And when the gateway equipment receives the packet C, the gateway equipment indicates that the link is successful, obtains the bird sound file information and the terminal equipment state information, sends a packet A (namely the response data packet) to the bird sound acquisition terminal if the file to be transmitted is to be transmitted, starts to enter a file transmission state, and sends N packets (namely the disconnection data packet) to end the communication process if no file to be transmitted is to be transmitted.
5. When receiving the A packet, the birdsound acquisition terminal sends a file data block (namely the first birdsound audio data block) to the gateway, wherein the file data block is sent by an S packet for a (1), a (2) and a (8230), and a (K-1), and the file data block is sent by an E packet for the last data block a (K). It should be noted that both the S packet and the E packet are the content data packets, where the E packet represents the end of the file.
6. And when the gateway equipment receives the S packet or the E packet, reading the data content in the S packet or the E packet and storing the data content in a data buffer b. If the S packet is received, the received audio data block is not the last bird sound data block, the A packet is continuously sent to the bird sound acquisition terminal, and file transmission is continuously carried out; the bird sound acquisition terminal continues to execute the step 5 after receiving the packet A; if the E packet is received, the last bird sound audio data block is considered to be received, an N packet is sent to the bird sound acquisition terminal to finish the communication process, the content of the data buffer area b is stored in the memory card, a file F2 is obtained, and the content of the file F1 is consistent with that of the file F2.
7. And after the bird sound acquisition terminal receives the N packets, deleting the file F1 in the memory, and waiting for the next access of the gateway. (prevention of memory shortage due to accumulation of conventional bird's sound audio files in memory)
8. The gateway device accesses the next bird sound collecting device and returns to the step 1 to continue the execution.
It should be noted that the main control chip of each bird sound collection terminal communicates with the respective LoRa module through a serial port; in the transmission flow of the gateway equipment and each bird sound acquisition terminal, the bird sound acquisition terminal waits for an access request of the gateway equipment at any time, and besides wireless data transmission, tasks such as sound acquisition, bird sound extraction, data compression, file storage and the like need to be executed; in order to reduce the CPU occupancy rate of a main control chip of the bird sound acquisition terminal, the sending and receiving of the serial port are respectively mapped to a direct memory access channel of the main control chip, the transmission flow is designed into a finite state machine, the state transfer is realized by matching with a timer, and the robustness of the program is improved by combining with a watchdog timer. The gateway equipment also adopts the method to solve the problem of real-time wireless communication, and effectively improves the stability of wireless data transmission.
The current bird sound audio file information in the above-mentioned C packet (i.e., the connection packet) includes: the method comprises the following steps of acquiring the name of a bird sound audio file, the size of the bird sound audio file, the date and time corresponding to the collected bird sound and the name of a collecting terminal device corresponding to the collected bird sound;
the terminal device status information in the C packet (i.e., the connection packet) includes: the device name of the bird sound acquisition terminal, the battery power of the bird sound acquisition terminal, the recording device state of the bird sound acquisition terminal and the used storage space of the storage of the bird sound acquisition terminal.
As shown in fig. 3, the gateway device includes an LoRa module, a main control chip, a wireless network module, and a memory (which may also be a memory card); the LoRa module is used for carrying out data transmission with the bird sound acquisition terminal, and the wireless network module can be any one of 2G/3G/4G wireless network module for carry out data transmission with the cloud server.
In the invention, the cloud server comprises a storage server, a message server and an application server. The storage server is used for storing the bird sound file data for a long time and is equivalent to a bird sound database; the message server is used for recording the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment and pushing the message to the application server; and the application server forwards the data to the bird sound monitoring platform and the equipment management platform, so as to provide a data basis for systems such as visual monitoring, species identification and the like.
It should be noted that in a preferred embodiment, the bird sound file information in the message server includes: the method comprises the following steps of accessing a network storage path of a bird sound audio file, the size of the bird sound audio file, the date and time corresponding to the collected bird sound and the name of a collecting terminal device corresponding to the collected bird sound; the details are shown in Table 2.
The device state information in the message server includes: acquiring equipment state information of each bird sound acquisition terminal and state information of gateway equipment; the device state information of each bird sound acquisition terminal comprises a device name of the bird sound acquisition terminal, the battery power of the bird sound acquisition terminal, the recording device state of the bird sound acquisition terminal and the used storage space of the memory of the bird sound acquisition terminal, and is specifically shown in table 3; the state information of the gateway device includes a name of the gateway device, a battery level of the gateway device, a used storage space of a memory of the gateway device, and a number of connections between the gateway device and the bird sound collection terminal, which are described in table 4.
TABLE 2 bird song file information content
TABLE 3 Equipment status information content of each bird sound acquisition terminal
Table 4 status information content of gateway device
In a preferred embodiment, the storage server is configured to receive the bird sound audio file sent by the gateway device after establishing a communication connection with the gateway device, and then store the bird sound audio file;
the message server is used for receiving the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment sent by the gateway equipment after establishing communication connection with the gateway equipment, then storing the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment, and forwarding the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to the application server;
the application server is used for extracting the corresponding bird sound audio file from the storage server according to the bird sound file information and then sending the bird sound audio file and the bird sound file information to the bird sound monitoring platform; and the device management platform is also used for sending the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to the equipment management platform.
The gateway device sends the bird sound audio file to the storage server in the following manner:
the gateway equipment splits the bird sound audio file to obtain a plurality of second bird sound audio data blocks which are sequentially arranged;
the gateway equipment sends a first communication instruction to the storage server through the wireless network module, so that the storage server establishes HTTP communication with the gateway equipment after receiving the first communication instruction;
and the gateway equipment sends a plurality of data transmission instructions carrying second bird sound audio data blocks to the storage server one by one through the wireless network module until all the second bird sound audio data blocks are completely transmitted.
The gateway equipment sends the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to the message server in the following modes:
the gateway equipment sends a second communication instruction to the message server through the wireless network module, so that the message server establishes MQTT communication with the gateway equipment after receiving the second communication instruction;
the gateway equipment combines the bird sound file information in a JSON format to generate a first character string, and then sends the first character string to the message server;
and the gateway equipment combines the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment in a JSON format to generate a second character string, and then sends the second character string to the message server.
The following describes in detail the data transmission between the gateway device and each server:
in the invention, the communication between the gateway and the cloud only relates to the storage server and the message server, the communication protocols adopted by the gateway and the message server are HTTP (hyper text transfer protocol) and MQTT (message queue telemetry transport), and the communication instructions related to the network module are shown in tables 5 and 6 respectively.
TABLE 5 HTTP communication Instructions
TABLE 6 MQTT communication Instructions
The following describes data communication between the gateway and the cloud server with reference to tables 5 and 6;
the method specifically comprises the following steps:
1. since the internal buffer size of the wireless network module of the gateway device is 1460 bytes, and the data size of the file F2 is large, the packet transmission is also performed. Therefore, the gateway device reads the file F2 and loads the file to the data buffer c, and splits the file with 1152 bytes to obtain M data blocks (i.e., the second birdsong data block) which are denoted as c (1), c (2), \ 8230;, and c (M).
2. The gateway device establishes HTTP communication with the storage server by the T1 instruction (i.e., the above-described first communication instruction).
3. The gateway device constructs a write operation request header according to table 7, fetches data block c (M) as the transmission content (where M =1,2, \ 8230;, M), and transmits to the storage server through the T2 instruction; table 7 is specifically as follows:
TABLE 7 request packet content for write operations
4. And the gateway equipment waits for the storage server to send the feedback information, reads the feedback information through the T3 instruction, and if the 200 state code is returned, the writing operation is successful. And similarly, repeating the step 3, and sequentially sending the second bird sound audio data blocks c (m) until the file is finished.
5. And after the file is uploaded, the gateway equipment deletes the file F2 in the memory card. At this point, there is a corresponding file F3 on the storage server, and the content is consistent with files F1 and F2. And obtaining a network storage access path of the bird sound audio file by combining the name of the bird sound audio file, wherein the format of the network storage access path of the bird sound audio file is "< server domain name >/< storage folder >/< file name >".
6. The gateway device closes the HTTP communication through the T4 instruction and disconnects the link with the storage server.
7. The gateway equipment configures the MQTT environment through a T5 instruction (namely the second communication instruction), starts the MQTT function through a T6 instruction, and establishes MQTT communication with the message server through a T7 instruction.
8. And combining the bird sound file information according to the table 2 in a JSON format to obtain a character string J1 (namely the first character string), and sending the character string J1 to a message server through a T8 instruction. At this time, the message server will automatically forward the content of the character string J1 to the application server, and after the application server analyzes the message packet, the application server obtains the bird sound file F3 from the storage server and then forwards the bird sound file to the bird sound monitoring platform.
9. In order to facilitate management of the gateway and the terminals, the gateway is further configured to upload, to the message server, device state information (table 3) of each bird sound collection terminal and state information (table 4) of the gateway device; combining the device state information of each bird sound acquisition terminal and the state information of the gateway device, sorting according to the contents of the table 3 and the table 4, combining in a JSON format to obtain a character string J2 (namely the second character string), and sending the character string J2 to a message server through a T8 instruction. At this time, the message server will automatically forward the content of the character string J2 to the application server, and the application server will forward the content to the device management platform after analyzing the content.
10. The link is disconnected from the message server by the T9 instruction.
By implementing the embodiment of the invention, the following beneficial effects are achieved:
1. the LoRa technology is used as a medium for wireless data transmission between the gateway and each bird sound acquisition terminal, one-to-many networking communication is realized by combining a star topology structure, the transmission distance is longer, and the power consumption is lower.
2. The cloud server has massive access points and storage capacity, and is maintained and updated by technicians for a long time, the stability and the safety of the cloud server are more reliable than those of a self-built server, and the message server can automatically inform a third-party application server, so that the cloud server is favorable for expansion of applications such as bird monitoring and bird sound recognition.
3. The 2G/3G/4G wireless network module is adopted to realize the communication between the gateway and the cloud server, and the communication system is compatible with various communication systems, so that the access of the Internet is more convenient and flexible.
4. Each fragment containing the bird sound has the date and time and the record of equipment, and the research on the space-time distribution rule of the birds is facilitated.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (6)
1. A remote birdsong signal transmission system, comprising: the system comprises a plurality of bird sound acquisition terminals, gateway equipment and a cloud server;
each bird sound acquisition terminal is used for acquiring sound signals of a monitoring area, generating a bird sound audio file according to the sound signals and then sending the bird sound audio file to the gateway equipment through a built-in wireless communication module when receiving an access request sent by the gateway equipment; the audio file management system is also used for sending the bird sound file information corresponding to the bird sound audio file and the equipment state information of each bird sound acquisition terminal to the gateway equipment;
the gateway device is used for sending an access request to each birdsong acquisition terminal, then receiving a birdsong audio file of each birdsong acquisition terminal, establishing communication connection with the cloud server through a built-in wireless network module, and then uploading the birdsong audio file to the cloud server; the system is also used for sending the bird sound file information, the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to the cloud server; the bird sound file information comprises a network storage access path of the bird sound audio file, the size of the bird sound audio file, the date and time corresponding to the collected bird sound and the name of the collecting terminal device corresponding to the collected bird sound; the equipment state information comprises the equipment name of the bird sound acquisition terminal, the battery power of the bird sound acquisition terminal, the recording equipment state of the bird sound acquisition terminal and the used storage space of a memory of the bird sound acquisition terminal; the state information of the gateway equipment comprises the name of the gateway equipment, the battery capacity of the gateway equipment, the used storage space of a memory of the gateway equipment and the number of the gateway equipment connected with the bird sound acquisition terminal;
the cloud server is used for storing and forwarding the bird sound audio file after receiving the bird sound audio file;
wherein, each bird sound acquisition terminal includes: the device comprises a microphone, a preamplifier, an audio codec, a main control chip, a memory and a wireless communication module;
each bird sound acquisition terminal is used for acquiring sound signals of a monitoring area and generating a bird sound audio file according to the sound signals, and specifically comprises the following steps:
the microphone collects the sound of the monitoring area and transmits the sound to the preamplifier;
the preamplifier gains the sound signal and then transmits the sound signal after the gain to the audio codec;
the audio codec converts the gained sound signals into digital signals to generate audio data, and then transmits the audio data to the main control chip;
the main control chip receives the audio data in real time by utilizing a ping-pong buffer technology, obtains a segment containing a bird sound in the audio data by a bird sound extraction algorithm, generates a bird sound audio file, and stores the bird sound audio file in the memory; the bird sound extraction algorithm specifically comprises the following steps:
the bird sound extraction algorithm adopts an automatic segmentation algorithm based on prior probability to extract bird sound signals in the audio data, the audio data are converted from a time domain to a frequency domain through Fourier transform to be analyzed, a noise variance is obtained through a noise spectrum estimation algorithm, a pure spectrum estimation is obtained through a minimum mean square error estimation algorithm, and a prior signal-to-noise ratio is obtained through a guide decision method so as to obtain a prior probability value of a sound frame; judging whether a sound signal exists in the current frame or not through a preset threshold value according to the prior probability value of the sound frame, and collecting continuous multi-frame sound frame signals to obtain a fragment containing the bird sound;
the wireless communication module is a LoRa module, and each bird sound acquisition terminal and the gateway equipment are networked in a star topology structure; the main control chip is communicated with the LoRa module through a serial port, the sending and receiving of the serial port are mapped to the direct memory access channel of the main control chip respectively, the transmission process is set to be a finite state machine, and state transfer is achieved through matching of a timer.
2. The remote birdsound signal transmission system of claim 1, wherein the cloud server includes a storage server, a message server, and an application server;
the storage server is used for receiving the bird sound audio file sent by the gateway equipment after establishing communication connection with the gateway equipment and then storing the bird sound audio file;
the message server is configured to receive, after establishing communication connection with the gateway device, the bird sound file information, the device state information of each bird sound collection terminal, and the state information of the gateway device sent by the gateway device, then store the bird sound file information, the device state information of each bird sound collection terminal, and the state information of the gateway device, and forward the bird sound file information, the device state information of each bird sound collection terminal, and the state information of the gateway device to the application server;
the application server is used for extracting the corresponding bird sound audio file from the storage server according to the bird sound file information and then sending the bird sound audio file and the bird sound file information to the bird sound monitoring platform; and sending the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment to an equipment management platform.
3. The remote birdsound signal transmission system according to claim 1, wherein each of the birdsound collection terminals transmits the birdsound audio file, the birdsound file information, and the device state information of the respective birdsound collection terminal to the gateway device by:
the bird sound acquisition terminal splits the bird sound audio file to obtain a plurality of first bird sound audio data blocks which are sequentially arranged;
the method comprises the steps that the birdsound acquisition terminal receives an access request which is sent by the gateway device and carries a reset data packet, feeds back a connection data packet which carries the birdsound file information and the equipment state information of the birdsound acquisition terminal to the gateway device, so that the gateway device extracts the birdsound file information and the equipment state information of the birdsound acquisition terminal when receiving the connection data packet, and sends a response data packet to the birdsound acquisition terminal;
after receiving the response data packet, the birdsound acquisition terminal sends a plurality of content data packets carrying the first birdsound audio data block to the gateway device one by one until all the first birdsound audio data blocks are sent; when the gateway equipment receives a content data packet, judging whether a first bird sound audio data block in the content data packet is a last bird sound data block, if not, continuously sending a response data packet to the bird sound acquisition terminal so that the bird sound acquisition terminal continuously sends the content data packet after receiving the response data packet; and if so, sending a disconnection data packet to the bird sound acquisition terminal so that the bird sound acquisition terminal finishes sending the content data packet.
4. The remote birdsound signal transmission system according to claim 3, wherein each of the birdsound acquisition terminals is further configured to delete the birdsound audio file stored in the memory after all the first birdsound audio data blocks are transmitted.
5. The remote birdsound signal transmission system of claim 2, wherein the gateway device transmits the birdsound audio file to the storage server by:
the gateway equipment splits the bird sound audio file to obtain a plurality of second bird sound audio data blocks which are sequentially arranged;
the gateway equipment sends a first communication instruction to the storage server through the wireless network module, so that the storage server establishes HTTP communication with the gateway equipment after receiving the first communication instruction;
and the gateway equipment sends a plurality of data transmission instructions carrying second bird sound audio data blocks to the storage server one by one through the wireless network module until all the second bird sound audio data blocks are completely transmitted.
6. The remote birdsound signal transmission system according to claim 2, wherein the gateway device transmits the birdsound file information, the device state information of each birdsound acquisition terminal, and the state information of the gateway device to the message server by:
the gateway equipment sends a second communication instruction to the message server through the wireless network module, so that the message server establishes MQTT communication with the gateway equipment after receiving the second communication instruction;
the gateway equipment combines the file information in a JSON format to generate a first character string, and then sends the first character string to the message server;
and the gateway equipment combines the equipment state information of each bird sound acquisition terminal and the state information of the gateway equipment in a JSON format to generate a second character string, and then sends the second character string to the message server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010429163.7A CN111629041B (en) | 2020-05-20 | 2020-05-20 | Remote bird sound signal transmission system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010429163.7A CN111629041B (en) | 2020-05-20 | 2020-05-20 | Remote bird sound signal transmission system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111629041A CN111629041A (en) | 2020-09-04 |
| CN111629041B true CN111629041B (en) | 2023-02-17 |
Family
ID=72260031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010429163.7A Active CN111629041B (en) | 2020-05-20 | 2020-05-20 | Remote bird sound signal transmission system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111629041B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113225400A (en) * | 2021-05-08 | 2021-08-06 | 南京林业大学 | Bird population density monitoring system and method based on singing of singing birds |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103905559A (en) * | 2014-04-14 | 2014-07-02 | 重庆工商职业学院 | Valuable and rare bird population distribution detection system based on birdcall voiceprint characteristics |
| CN108922548A (en) * | 2018-08-20 | 2018-11-30 | 深圳园林股份有限公司 | A kind of bird based on deep learning, frog intelligent monitoring method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4206876B2 (en) * | 2003-09-10 | 2009-01-14 | ヤマハ株式会社 | Communication device and program for transmitting state of remote place |
-
2020
- 2020-05-20 CN CN202010429163.7A patent/CN111629041B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103905559A (en) * | 2014-04-14 | 2014-07-02 | 重庆工商职业学院 | Valuable and rare bird population distribution detection system based on birdcall voiceprint characteristics |
| CN108922548A (en) * | 2018-08-20 | 2018-11-30 | 深圳园林股份有限公司 | A kind of bird based on deep learning, frog intelligent monitoring method |
Non-Patent Citations (1)
| Title |
|---|
| "面向鸟声传感网数据中心的边缘虚拟网桥技术研究";吴烨;《中国优秀硕士学位论文全文数据库 (信息科技辑)》;20170215;正文第2章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111629041A (en) | 2020-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019227579A1 (en) | Conference information recording method and apparatus, computer device, and storage medium | |
| CN110049270A (en) | Multi-person conference speech transcription method, apparatus, system, equipment and storage medium | |
| AU2016260156B2 (en) | Method and device for improving audio processing performance | |
| CN102592591B (en) | Dual-band speech encoding | |
| CN102907077B (en) | For the system and method for the intelligent audio record of mobile device | |
| CN102111314A (en) | Smart home voice control system and method based on Bluetooth transmission | |
| WO2016011875A1 (en) | Method, system, and related device for data transmission | |
| CN111629041B (en) | Remote bird sound signal transmission system | |
| WO2015103836A1 (en) | Voice control method and device | |
| US9774743B2 (en) | Silence signatures of audio signals | |
| US7050974B1 (en) | Environment adaptation for speech recognition in a speech communication system | |
| CN105744208A (en) | Video conference control system and control method | |
| KR20170140188A (en) | Method and apparatus for audio content recognition | |
| US10276191B2 (en) | Speech section detection device, voice processing system, speech section detection method, and computer program product | |
| CN112672086B (en) | Audio and video equipment data acquisition, analysis, early warning system | |
| CN109510891A (en) | Voice control recording device and method | |
| CN101478616A (en) | Instant voice communication method | |
| CN116319467A (en) | Depth synthesis audio detection method and system based on bidirectional flow of IDC machine room | |
| CN111341351A (en) | Voice activity detection method and device based on self-attention mechanism and storage medium | |
| WO2019000877A1 (en) | Audio data processing method and device | |
| Sakhri et al. | An energy-efficient wmsn-based system for endangered birds monitoring | |
| CN112382306B (en) | Method and device for separating speaker audio | |
| CN116472705A (en) | Conference content display method, conference system and conference equipment | |
| CN110730394A (en) | Intelligent optical network unit configuration method and device based on voice interaction | |
| CN113223508A (en) | Management method of dual-mode TWS Bluetooth headset |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |