CN113763974A - Packet loss compensation method and device, electronic equipment and storage medium - Google Patents
Packet loss compensation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113763974A CN113763974A CN202111014837.8A CN202111014837A CN113763974A CN 113763974 A CN113763974 A CN 113763974A CN 202111014837 A CN202111014837 A CN 202111014837A CN 113763974 A CN113763974 A CN 113763974A
- Authority
- CN
- China
- Prior art keywords
- packet
- compensation
- packet loss
- determining
- pitch period
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000009499 grossing Methods 0.000 claims abstract description 14
- 238000005070 sampling Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 6
- 230000003247 decreasing effect Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 11
- 230000003287 optical effect Effects 0.000 description 6
- 210000001260 vocal cord Anatomy 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
- H04L43/0829—Packet loss
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/10—Flow control between communication endpoints
- H04W28/14—Flow control between communication endpoints using intermediate storage
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Environmental & Geological Engineering (AREA)
- Quality & Reliability (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a packet loss compensation method, a device, an electronic device and a storage medium, wherein the method comprises the following steps: if a packet loss event is monitored, acquiring a historical voice packet from a cache; determining a pitch period according to the historical voice packet; delaying the target data in the historical voice packet by the pitch period, and determining a first compensation packet of the current packet loss; and smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into the cache. According to the technical scheme, packet loss compensation is completed by repeatedly crossing packet loss data through the fundamental tone period, and meanwhile, the compensation packet is smoothed through an overlap-add method, so that the correlation and the continuity of the compensation packet are guaranteed, a good packet loss hiding effect is achieved, and a new thought is provided for packet loss compensation of wireless voice equipment.
Description
Technical Field
The embodiment of the invention relates to the technical field of communication, in particular to a packet loss compensation method and device, electronic equipment and a storage medium.
Background
With the rapid development of wireless networks, more and more wireless devices are applied to people's daily life, such as wireless bluetooth headsets, audio and video, sound equipment, and the like. However, due to the uneven network environment, packet loss occurs in the transmitted audio data due to network jitter, network failure, network abnormality and the like, so that interruption exists in the audio data, and the user experience is reduced.
Currently, in order to reduce the packet loss rate of audio data, a compensation packet is inserted to compensate for the lost data packet. The compensation packet is typically simple, such as a silence packet, a noise packet, or a repetition of previous packets. However, such compensation has a discontinuous feeling due to packet loss, and is only suitable for a very small packet loss interval and an extremely low packet loss rate. When the packet loss interval is large or the packet loss rate is high, the compensated audio data is obviously different from the audio of the original audio data, so that the sound effect of the compensated audio data is poor, and improvement is needed urgently.
Disclosure of Invention
The invention provides a packet loss compensation method, a device, an electronic device and a storage medium, which can improve the correlation and the continuity of a compensation packet and have a better packet loss hiding effect.
In a first aspect, an embodiment of the present invention provides a packet loss compensation method, where the method includes:
if a packet loss event is monitored, acquiring a historical voice packet from a cache;
determining a pitch period according to the historical voice packet;
delaying the target data in the historical voice packet by the pitch period, and determining a first compensation packet of the current packet loss;
and smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into the cache.
In a second aspect, an embodiment of the present invention further provides a packet loss compensation apparatus, where the apparatus includes:
the historical packet acquisition module is used for acquiring a historical voice packet from a cache if a packet loss event is monitored;
the period determining module is used for determining a pitch period according to the historical voice packet;
a first determining module, configured to delay target data in the historical voice packet by the pitch period, and determine a first compensation packet that is currently lost;
and the second determining module is used for smoothing the first compensation packet to obtain a second compensation packet with the current packet loss and storing the second compensation packet into the cache.
In a third aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the packet loss compensation method according to any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the packet loss compensation method according to any embodiment of the present invention.
According to the packet loss compensation method, the packet loss compensation device, the packet loss compensation equipment and the packet loss compensation storage medium provided by the embodiment of the invention, if a packet loss event is monitored, a historical voice packet is obtained from a cache; determining a pitch period according to the historical voice packet; delaying the target data in the historical voice packet by the pitch period, and determining a first compensation packet of the current packet loss; and smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into the cache. According to the technical scheme, packet loss compensation is completed by repeatedly crossing packet loss data through the fundamental tone period, and meanwhile, the compensation packet is smoothed through an overlap-add method, so that the correlation and the continuity of the compensation packet are guaranteed, a good packet loss hiding effect is achieved, and a new thought is provided for packet loss compensation of wireless voice equipment.
Drawings
Fig. 1A is a flowchart of a packet loss compensation method according to an embodiment of the present invention;
fig. 1B is a block diagram of a continuous packet loss compensation scheme according to an embodiment of the present invention;
fig. 2A is a flowchart of a packet loss compensation method according to a second embodiment of the present invention;
FIG. 2B is a schematic diagram of pitch period detection provided by the second embodiment of the present invention;
FIG. 2C is a diagram illustrating an overlap-add smoothing scheme according to a second embodiment of the present invention;
fig. 3 is a block diagram of a packet loss compensation apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1A is a flowchart of a packet loss compensation method according to an embodiment of the present invention, which is applicable to a case of packet loss when audio data is transmitted, and is particularly applicable to a case that the audio data transmitted by a wireless voice device has packet loss due to network jitter, network failure, network abnormality, and the like, so that the audio data has interruption, which causes a decrease in user experience. The method can be executed by the packet loss compensation device provided by the embodiment of the present invention, and the device can be implemented in a software and/or hardware manner and can be integrated on an electronic device.
Specifically, as shown in fig. 1A, the packet loss compensation method provided in the embodiment of the present invention may include the following steps:
and S110, if a packet loss event is monitored, acquiring a historical voice packet from a cache.
The packet loss refers to the fact that data of one or more data packets cannot reach a destination through a network. The network environments of the wireless voice devices are uneven, and when audio data are transmitted, packet loss occurs due to network jitter, network faults, network abnormalities and the like, so that the wireless voice devices cannot reach the wireless voice device terminal through wireless network connection. The wireless voice device comprises a wireless Bluetooth headset, a sound box and the like, and the wireless mode comprises 2.4G, WIFI, Bluetooth and other wireless connection modes.
The cache stores historical voice packets, so-called historical voice packets, which can be understood as voice data packets transmitted in a period of time before packet loss. When a packet loss event is monitored, a historical voice packet needs to be acquired from the cache, so that a compensation packet can be determined according to the historical voice packet conveniently in the follow-up process.
And S120, determining a pitch period according to the historical voice packet.
Sound signals can be classified into unvoiced sounds and voiced sounds according to the way in which the vocal cords of a person vibrate. The voiced sound needs the periodic vibration of the vocal cords, and has obvious periodicity, the frequency of the vocal cords vibration is called the fundamental frequency, and the corresponding period is the fundamental period. The pitch frequency range of the human voice is 65HZ-400Hz, and the corresponding pitch period range is 15ms-2.5 ms.
Optionally, the pitch period may be calculated by using an average amplitude difference method, and the calculation complexity of calculating the pitch period by using the average amplitude difference method is lower than that of calculating the pitch period by using a common autocorrelation method and a frequency domain cepstrum method.
S130, delaying the target data in the historical voice packet by a pitch period, and determining a first compensation packet of the current packet loss.
The audio of the pronunciation of a person during speaking is a continuous gradual process in frequency domain and amplitude, and the audio data has a lot of repeated data in time, which is called as a time redundancy characteristic. The method for compensating the lost voice data packet in the embodiment of the invention utilizes the characteristic to perform prediction compensation on the lost data packet, and based on the gradual change characteristic of sound, the pitch periods of adjacent periods are approximately considered to be the same or the change trend is stable.
Therefore, the first compensation packet of the current lost packet, namely the compensation data closest to the lost frame, can be determined according to the target data and the pitch period in the historical voice packet. Specifically, the voice data of a pitch period before packet loss in the historical voice packet may be used as target data, and the target data may be delayed based on the pitch period to obtain the first compensation packet.
S140, smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into a cache.
If the first compensation packet is directly used for packet loss compensation, obviously, the voice continuity at the compensation position is poor. In order to enhance the continuity of the voice before and after packet loss compensation, the voice data of the compensation packet may be smoothed. Illustratively, the first 1/2 packet data of the first compensation packet and the last 1/2 packet data of the historical speech packet in the previous pitch period can be overlapped and added through windowing to obtain the first 1/2 packet data of the second compensation packet; the first 1/4 packet data of the first compensation packet may also be overlapped and added with the last 1/4 packet data of the historical speech packet in the previous pitch period by windowing to obtain the first 1/4 packet data of the second compensation packet. In this embodiment, the size of the overlap-add packet is not limited, and may be adjusted according to actual circumstances.
According to the technical scheme of the embodiment, if a packet loss event is monitored, a historical voice packet is obtained from a cache; determining a pitch period according to the historical voice packet; delaying target data in a historical voice packet by a pitch period, and determining a first compensation packet of a current packet loss; and smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into a cache. Packet loss compensation is completed by repeatedly crossing packet loss data through fundamental tone periods, and meanwhile, the compensation packet is smoothed through an overlap-add method, so that the correlation and the continuity of the compensation packet are ensured, a good packet loss hiding effect is achieved, and a new thought is provided for packet loss compensation of wireless voice equipment.
On the basis of the technical scheme, if the packet loss event is continuously monitored, the gain of each compensation packet which continuously loses packets is set to be decreased progressively frame by frame. Because the compensated voice data is non-real voice, the influence on the real voice can be reduced by reducing the energy of the compensated voice data, and larger distortion is avoided, so that when a continuous packet loss event is monitored, the second compensated data after smoothing is attenuated, and a better voice effect can be obtained. Fig. 1B is a block diagram of a continuous packet loss compensation scheme. Illustratively, the compensation packet gain for consecutive packet losses is decremented by 80% from frame to avoid introducing greater signal distortion. The formula is expressed as:
Gain_Scale＝(80％)m-1
wherein, Gain _ Scale represents the Gain coefficient of the compensation frame, and m represents the number of continuous packet loss.
Example two
Fig. 2A is a flowchart of a packet loss compensation method according to a second embodiment of the present invention, which is further optimized based on the above embodiment, and the pitch period is determined "according to a historical voice packet" and is optimized "according to the historical voice packet, an average amplitude difference is calculated, and a minimum average amplitude difference is determined; determining the number of sampling points according to the minimum average amplitude difference; and determining the pitch period according to the number of the sampling points. ".
Specifically, as shown in fig. 2A, the method includes:
and S210, if a packet loss event is monitored, acquiring a historical voice packet from a cache.
The network environments of the wireless voice devices are uneven, and when audio data are transmitted, packet loss occurs due to network jitter, network faults, network abnormalities and the like, so that the wireless voice devices cannot reach the wireless voice device terminal through wireless network connection. The wireless voice device comprises a wireless Bluetooth headset, a sound box and the like, and the wireless mode comprises 2.4G, WIFI, Bluetooth and other wireless connection modes.
The cache stores historical voice packets, so-called historical voice packets, which can be understood as voice data packets transmitted in a period of time before packet loss. When a packet loss event is monitored, a historical voice packet needs to be acquired from the cache, so that a compensation packet can be determined according to the historical voice packet conveniently in the follow-up process. The length of the buffer in this embodiment is 45ms, which is the length of 6 bluetooth packets (7.5 ms).
S220, calculating the average amplitude difference according to the historical voice packets, and determining the minimum average amplitude difference.
Sound signals can be classified into unvoiced sounds and voiced sounds according to the way in which the vocal cords of a person vibrate. The voiced sound needs the periodic vibration of the vocal cords, and has obvious periodicity, the frequency of the vocal cords vibration is called the fundamental frequency, and the corresponding period is the fundamental period.
In this embodiment, the pitch period is calculated by using an average amplitude difference method, and the calculation process is as follows:
wherein R isn(k) Is the average amplitude difference, xnAnd representing the voice data stored in the cache, and k is the number of sampling points.
And S230, determining the number of sampling points according to the minimum average amplitude difference.
So that the average amplitude difference Rn(k) The minimum k value is the number of sampling points corresponding to the detected pitch period.
And S240, determining the pitch period according to the number of the sampling points.
E.g. kminAt a sampling rate of 8kHz, 20, a sampling time of 2.5ms, kmax120 at an 8kHz sampling rate and 15 ms. From the number of sample points, the pitch period can be determined. Fig. 2B is a schematic diagram of pitch period detection.
And S250, delaying the target data in the historical voice packet by a pitch period, and determining a first compensation packet of the current packet loss.
The audio of the pronunciation of a person during speaking is a continuous gradual process in frequency domain and amplitude, and the audio data has a lot of repeated data in time, which is called as a time redundancy characteristic. The method for compensating the lost voice data packet in the embodiment of the invention utilizes the characteristic to perform prediction compensation on the lost data packet, and based on the gradual change characteristic of sound, the pitch periods of adjacent periods are approximately considered to be the same or the change trend is stable.
Therefore, the first compensation packet of the current lost packet, namely the compensation data closest to the lost frame, can be determined according to the target data and the pitch period in the historical voice packet. Specifically, the voice data of a pitch period before packet loss in the historical voice packet may be used as target data, and the target data may be delayed based on the pitch period to obtain the first compensation packet.
S260, smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into a cache.
If the first compensation packet is directly used for packet loss compensation, obviously, the voice continuity at the compensation position is poor. In order to enhance the continuity of the voice before and after packet loss compensation, the voice data of the compensation packet may be smoothed. Fig. 2C is a schematic diagram of an overlap-add smoothing scheme. The first 1/2 packet data of the first compensation packet may be added by windowing overlapping with the last 1/2 packet data of the historical speech packet at its previous pitch period to obtain the first 1/2 packet data of the second compensation packet, which is represented as:
y1(n)＝y(n)*h(n)+x(n)*h(k-n)，n＝0，1，2，…，k
where k is the number of sampling points of 1/2 packets 30(8kHz sampling rate, 60 points per packet), y (n) is the first 1/2 packet of the first compensation packet, and x (n) is the last 1/2 packet of the previous pitch period.
According to the technical scheme, a pitch period determining mode is provided, the pitch period is determined through a minimum amplitude difference method, complexity of determining the pitch period can be reduced, packet loss compensation is completed through repeated spanning of packet loss data of the pitch period, meanwhile, a compensation packet is smoothed through an overlap-add method, correlation and continuity of the compensation packet are guaranteed, and meanwhile compensation efficiency is improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a packet loss compensation apparatus according to a third embodiment of the present invention, where the apparatus is adapted to execute the packet loss compensation method according to the third embodiment of the present invention, and can ensure correlation and continuity of compensation packets. As shown in fig. 3, the apparatus includes a history packet obtaining module 310, a period determining module 320, a first determining module 330, and a second determining module 340.
The history packet obtaining module 310 is configured to obtain a history voice packet from a cache if a packet loss event is monitored;
a period determining module 320, configured to determine a pitch period according to the historical voice packet;
a first determining module 330, configured to delay target data in a historical voice packet by a pitch period, and determine a first compensation packet of a current packet loss;
the second determining module 340 is configured to smooth the first compensation packet to obtain a second compensation packet with a current packet loss, and store the second compensation packet in the buffer.
According to the technical scheme of the embodiment, if a packet loss event is monitored, a historical voice packet is obtained from a cache; determining a pitch period according to the historical voice packet; delaying target data in a historical voice packet by a pitch period, and determining a first compensation packet of a current packet loss; and smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into a cache. Packet loss compensation is completed by repeatedly crossing packet loss data through fundamental tone periods, and meanwhile, the compensation packet is smoothed through an overlap-add method, so that the correlation and the continuity of the compensation packet are ensured, a good packet loss hiding effect is achieved, and a new thought is provided for packet loss compensation of wireless voice equipment.
Preferably, the second determining module 340 is further specifically configured to overlap and add the first 1/2 packet data of the first compensation packet and the last 1/2 packet data of the previous pitch period in the historical speech packet by windowing, so as to obtain the first 1/2 packet data of the second compensation packet.
Preferably, the apparatus further comprises: and the gain setting module is used for setting the gain of each compensation packet which continuously loses packets to be decreased frame by frame if packet loss events are continuously monitored.
Preferably, the period determining module 320 specifically includes: the device comprises an amplitude difference calculating unit, a sampling point calculating unit and a period determining unit. The amplitude difference calculating unit is used for calculating average amplitude difference according to the historical voice packets and determining the minimum average amplitude difference; the sampling point calculating unit is used for determining the number of sampling points according to the minimum average amplitude difference; and the period determining unit is used for determining the pitch period according to the number of the sampling points.
The packet loss compensation device provided by the embodiment of the invention can execute the packet loss compensation method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 4 is only an example and should not bring any limitation to the function and the scope of use of the embodiment of the present invention.
As shown in FIG. 4, electronic device 12 is embodied in the form of a general purpose computing device. The components of electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, implementing the packet loss compensation method provided by the embodiment of the present invention.
EXAMPLE five
Embodiment 5 of the present invention further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the method for compensating for packet loss according to any embodiment of the present invention is implemented.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (8)
1. A packet loss compensation method, the method comprising:
if a packet loss event is monitored, acquiring a historical voice packet from a cache;
determining a pitch period according to the historical voice packet;
delaying the target data in the historical voice packet by the pitch period, and determining a first compensation packet of the current packet loss;
smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, and storing the second compensation packet into the cache;
smoothing the first compensation packet to obtain a second compensation packet of the current packet loss, including:
and overlapping and adding the front 1/2 packet data of the first compensation packet and the rear 1/2 packet data of the previous pitch period in the historical voice packet through windowing to obtain the front 1/2 packet data of the second compensation packet.
2. The method of claim 1, further comprising:
and if the packet loss event is continuously monitored, setting the gain of each compensation packet which continuously loses packets to be decreased frame by frame.
3. The method according to claim 1, wherein determining a pitch period from the historical speech packets comprises:
calculating an average amplitude difference according to the historical voice packet, and determining a minimum average amplitude difference;
determining the number of sampling points according to the minimum average amplitude difference;
and determining the pitch period according to the number of the sampling points.
4. A packet loss compensation apparatus, comprising:
the historical packet acquisition module is used for acquiring a historical voice packet from a cache if a packet loss event is monitored;
the period determining module is used for determining a pitch period according to the historical voice packet;
a first determining module, configured to delay target data in the historical voice packet by the pitch period, and determine a first compensation packet that is currently lost;
the second determining module is used for smoothing the first compensation packet to obtain a second compensation packet with the current packet loss and storing the second compensation packet into the cache;
the second determining module is further specifically configured to overlap and add, by windowing, the first 1/2 packet data of the first compensation packet and the last 1/2 packet data of the previous pitch period in the historical speech packet, so as to obtain the first 1/2 packet data of the second compensation packet.
5. The apparatus of claim 4, further comprising:
and the gain setting module is used for setting the gain of each compensation packet which continuously loses packets to be decreased frame by frame if packet loss events are continuously monitored.
6. The apparatus of claim 4, wherein the period determination module comprises:
the amplitude difference calculation unit is used for calculating average amplitude difference according to the historical voice packets and determining the minimum average amplitude difference;
the sampling point calculating unit is used for determining the number of sampling points according to the minimum average amplitude difference;
and the period determining unit is used for determining the pitch period according to the number of the sampling points.
7. An electronic device, characterized in that the device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the packet loss compensation method according to any one of claims 1 to 3.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the packet loss compensation method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111014837.8A CN113763974B (en) | 2021-08-31 | 2021-08-31 | Packet loss compensation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111014837.8A CN113763974B (en) | 2021-08-31 | 2021-08-31 | Packet loss compensation method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113763974A true CN113763974A (en) | 2021-12-07 |
| CN113763974B CN113763974B (en) | 2023-05-16 |
Family
ID=78792189
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111014837.8A Active CN113763974B (en) | 2021-08-31 | 2021-08-31 | Packet loss compensation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113763974B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114615164A (en) * | 2022-03-09 | 2022-06-10 | 网易（杭州）网络有限公司 | Packet loss rate detection method and device, storage medium and electronic equipment |
| CN117676185A (en) * | 2023-12-05 | 2024-03-08 | 无锡中感微电子股份有限公司 | Packet loss compensation method and device for audio data and related equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012070340A1 (en) * | 2010-11-26 | 2012-05-31 | 株式会社エヌ・ティ・ティ・ドコモ | Concealment signal generating device, concealment signal generation method and concealment signal generation program |
| CN106788876A (en) * | 2015-11-19 | 2017-05-31 | 电信科学技术研究院 | A kind of method and system of voice Discarded Packets compensation |
| CN106898356A (en) * | 2017-03-14 | 2017-06-27 | 建荣半导体（深圳）有限公司 | A kind of bag-losing hide method suitable for Bluetooth voice call, device and blue tooth voice process chip |
-
2021
- 2021-08-31 CN CN202111014837.8A patent/CN113763974B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012070340A1 (en) * | 2010-11-26 | 2012-05-31 | 株式会社エヌ・ティ・ティ・ドコモ | Concealment signal generating device, concealment signal generation method and concealment signal generation program |
| CN106788876A (en) * | 2015-11-19 | 2017-05-31 | 电信科学技术研究院 | A kind of method and system of voice Discarded Packets compensation |
| CN106898356A (en) * | 2017-03-14 | 2017-06-27 | 建荣半导体（深圳）有限公司 | A kind of bag-losing hide method suitable for Bluetooth voice call, device and blue tooth voice process chip |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114615164A (en) * | 2022-03-09 | 2022-06-10 | 网易（杭州）网络有限公司 | Packet loss rate detection method and device, storage medium and electronic equipment |
| CN114615164B (en) * | 2022-03-09 | 2023-12-29 | 网易（杭州）网络有限公司 | Packet loss rate detection method and device, storage medium and electronic equipment |
| CN117676185A (en) * | 2023-12-05 | 2024-03-08 | 无锡中感微电子股份有限公司 | Packet loss compensation method and device for audio data and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113763974B (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108615535B (en) | Voice enhancement method and device, intelligent voice equipment and computer equipment | |
| CN109767792B (en) | Voice endpoint detection method, device, terminal and storage medium | |
| JP6636937B2 (en) | Transient suppression depending on the situation | |
| CN113763974B (en) | Packet loss compensation method and device, electronic equipment and storage medium | |
| US20120239394A1 (en) | Erroneous detection determination device, erroneous detection determination method, and storage medium storing erroneous detection determination program | |
| JP6290429B2 (en) | Speech processing system | |
| WO2012158156A1 (en) | Noise supression method and apparatus using multiple feature modeling for speech/noise likelihood | |
| EP2589047A2 (en) | Speech audio processing | |
| US20140148933A1 (en) | Sound Feature Priority Alignment | |
| CN112489665B (en) | Voice processing method and device and electronic equipment | |
| CN110070884B (en) | Audio starting point detection method and device | |
| US9484044B1 (en) | Voice enhancement and/or speech features extraction on noisy audio signals using successively refined transforms | |
| US10529331B2 (en) | Suppressing key phrase detection in generated audio using self-trigger detector | |
| US20150348572A1 (en) | Detecting a user's voice activity using dynamic probabilistic models of speech features | |
| JP2021089438A (en) | Selective adaptation and utilization of noise reduction technique in invocation phrase detection | |
| JPWO2019031268A1 (en) | Information processing device and information processing method | |
| CN114596870A (en) | Real-time audio processing method and device, computer storage medium and electronic equipment | |
| CN111883135A (en) | Voice transcription method and device and electronic equipment | |
| JP6182895B2 (en) | Processing apparatus, processing method, program, and processing system | |
| CN111261143B (en) | Voice wakeup method and device and computer readable storage medium | |
| JP6098149B2 (en) | Audio processing apparatus, audio processing method, and audio processing program | |
| CN110070885B (en) | Audio starting point detection method and device | |
| US5812967A (en) | Recursive pitch predictor employing an adaptively determined search window | |
| US9307334B2 (en) | Method for calculating audio latency in real-time audio processing system | |
| CN113160846B (en) | Noise suppression method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |