CN105810201B - Voice activity detection method and its system - Google Patents
Voice activity detection method and its system Download PDFInfo
- Publication number
- CN105810201B CN105810201B CN201410853931.6A CN201410853931A CN105810201B CN 105810201 B CN105810201 B CN 105810201B CN 201410853931 A CN201410853931 A CN 201410853931A CN 105810201 B CN105810201 B CN 105810201B
- Authority
- CN
- China
- Prior art keywords
- signal
- noise
- present frame
- noise ratio
- spectrum density
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Noise Elimination (AREA)
- Time-Division Multiplex Systems (AREA)
Abstract
The present invention provides a kind of voice activity detection method and its systems.Wherein, this method comprises: calculating the spectrum density of audio signal present frame;Estimate the desired value of noise spectrum density;The desired value of spectrum density and the noise spectrum density based on the present frame, calculates the signal-to-noise ratio of present frame;With signal-to-noise ratio and pre-determined threshold based on the present frame, voice activity detection result is generated.Therefore, voice activity detection result is related with the probability statistical distribution of noise, to overcome influence of the noise to testing result.Meanwhile pre-determined threshold is dynamic threshold and related with the variation of noise, may make the noisy environment of the adaptive present frame of testing result.
Description
Technical field
The present invention relates to speech recognition technology more particularly to a kind of voice activity detection methods and its system.
Background technique
Voice activity detection (Voice Activity detection, VAD) is also referred to as speech detection, in speech processes
In for detecting the presence or absence of voice, so that voice segments and the non-speech segment in signal be separated.VAD can be used for echo and disappear
It removes, noise suppression, language person identification and speech recognition etc..
Traditional vad algorithm often selects the features such as short-time energy, spectrum energy, the zero-crossing rate of audio signal to be judged.
Therefore, in the environment of pure voice environment and high s/n ratio, better performances.And the back unstable in low signal-to-noise ratio or noise
Under scape environment, testing result can be influenced by feature of noise amount, so that performance be caused to decline.
With the continuous development of speech recognition technology, the requirement to voice activity detection is also higher and higher.Therefore, it is necessary to one
Kind of VAD detection method, can in the presence of a harsh environment, such as noise is unstable or the environment of low signal-to-noise ratio in, still keep good
Good detection performance.
Summary of the invention
Problems solved by the invention make the performance of voice activity detection in the environment of noise is unstable or low signal-to-noise ratio not
It can decline.
To solve the above problems, the present invention provides a kind of voice activity detection methods, comprising: it is current to calculate audio signal
The spectrum density of frame;Calculate the desired value of noise spectrum density;The phase of spectrum density and the noise spectrum density based on the present frame
Prestige value calculates the signal-to-noise ratio of present frame;And signal-to-noise ratio and pre-determined threshold based on the present frame, generate voice activity detection
As a result.
Optionally, the desired value of noise spectrum density is what the statistical distribution based on noise calculated.
Optionally, the calculating of the signal-to-noise ratio is based on formula:
Wherein, SNR indicates signal-to-noise ratio.
Optionally, the pre-determined threshold is dynamic threshold and changes with the variation of signal-to-noise ratio.
Optionally, the calculating of the dynamic threshold is based on formula:
Wherein, γ indicates that dynamic threshold, D indicate the variance of signal-to-noise ratio, PFAIndicate the probability of false-alarm.
Optionally, when the signal-to-noise ratio is greater than the pre-determined threshold, the voice activity detection of generation is the result is that the sound
The present frame of frequency signal is voice segments;When the signal-to-noise ratio is less than the pre-determined threshold, the voice activity detection result of generation
The present frame for being the audio signal is non-speech segment.
The present invention also provides a kind of voice activity detection systems, comprising: receiving unit, for receiving audio signal;Place
Unit is managed, for calculating the signal-to-noise ratio of present frame, wherein the signal-to-noise ratio of the present frame is based on the audio signal present frame
Spectrum density and noise spectrum density desired value calculate;And judging unit, being configured to can be based on the letter of the present frame
Make an uproar than and pre-determined threshold, generate voice activity detection result.
Optionally, the processing unit includes: that spectrum of first computing unit for calculating the audio signal present frame is close
Degree;Second computing unit is used to calculate the desired value of noise spectrum density;And third computing unit, for calculating the present frame
Signal-to-noise ratio.
Optionally, the desired value of the noise spectrum density is what the statistical distribution based on noise calculated.
Optionally, the calculating of the signal-to-noise ratio is based on formula:
Wherein, SNR indicates signal-to-noise ratio.
Optionally, the pre-determined threshold is dynamic threshold and changes with the variation of signal-to-noise ratio.
Optionally, the processing unit further comprises: the 4th computing unit, for calculating the dynamic threshold.
Optionally, the calculating of the dynamic threshold is based on formula:
Wherein, γ indicates that dynamic threshold, D indicate the variance of signal-to-noise ratio, PFAIndicate the probability of false-alarm.
Optionally, when the signal-to-noise ratio is greater than the pre-determined threshold, the voice activity detection of generation is the result is that the sound
The present frame of frequency signal is voice segments;When the signal-to-noise ratio is less than the pre-determined threshold, the voice activity detection result of generation
Present frame for the audio signal is non-speech segment.
Compared with prior art, technical solution of the present invention has the advantage that
Firstly, VAD judging result of the invention is the statistical distribution based on noise, rather than the statistics for passing through voice signal
What distribution generated.Specifically, voice activity detection method provided by the invention needs to count the probability distribution of noise, and it is based on
The desired value of this estimation noise spectrum density, and then generate judging result.Since in real life, noise is to belong to steadily in the long term
Signal, therefore, as long as the probability distribution statistical of noise is appropriate, VAD judging result either working as in voice signal to be measured
When previous frame is in steady noise environment, when being in the environment of unstable noise, can have preferable detection performance.
Further, when carrying out VAD judgement, using dynamic threshold, and the variation of the dynamic threshold and noise has
It closes, so that dynamic threshold can change with the variation of noise signal, with adaptive current background environment well.
Detailed description of the invention
Fig. 1 is the flow diagram of the voice activity detection method of one embodiment of the invention;
Fig. 2 is the structural schematic diagram of the voice activity detection system of one embodiment of the invention;And
Fig. 3 is the structural schematic diagram of the processing unit in the voice activity detection system of one embodiment of the invention.
Specific embodiment
To make the above purposes, features and advantages of the invention more obvious and understandable, with reference to the accompanying drawing to the present invention
Specific embodiment be described in detail.
Referring to Fig.1, the voice activity detection method 100 of one embodiment of the invention is illustrated.This approach includes the following steps.
S101 calculates the spectrum density of audio signal present frame.
In the method 100, voice activity detection carries out frame by frame.Specifically, audio signal is divided into multiple frames, so
Each frame of the audio signal is detected respectively afterwards, and then determines the voice segments and non-speech segment of audio signal.Wherein, often
The length range of one frame can be set as 10ms to 30ms.Therefore, the present frame of audio signal as currently needs to carry out voice
The frame of activity detection.
In some embodiments, the spectrum density of the present frame is frequency spectral density (the Power Spectral of present frame
Density, PSD), thus the power that the audio signal for calculating present frame has in cell frequency, using as present frame
The measurement of characteristic quantity.
In some embodiments, the calculating of the spectrum density of audio signal present frame can use Pasteur's Power estimation algorithm
(Bartlett Algorithm).In some embodiments, the calculating of the spectrum density of present frame can also use period nomography.
The present invention to the algorithm of present frame spectrum density with no restriction.
S103 calculates the desired value of noise spectrum density.
The desired value of noise spectrum density is that the statistical distribution based on noise signal calculates, wherein the statistical of noise
Cloth is carried out under the pure noisy environment of not voice signal.
In some embodiments, the estimation of noise spectrum density can use the algorithm with smaller variance, such as above-mentioned
Pasteur's algorithm, to improve the detection performance of VAD.This is because method 100 carries out VAD judgement in the present frame to audio signal
When, i.e., when judging present frame for voice segments or non-speech segment, it is related to the variance of noise, this in the following step can be detailed
It is thin to illustrate.
It is noted that voice activity detection method 100 provided by the invention is built upon on some assumed conditions
's.Specifically, assuming initially that voice signal and noise signal are independent from each other, incidence relation is not present from each other;Its
Secondary hypothesis noise signal is steady in a long-term, and voice signal is short-term stability.In addition, these hypothesis are to meet practical situation
, it is also that there is real utility value based on these method and systems assumed therefore.
S105, the desired value of spectrum density and noise spectrum density based on present frame calculate signal-to-noise ratio (SNR).
The calculating of signal-to-noise ratio can be based on formula:
Wherein, the desired value of noise spectrum density is that statistical distribution signal-based calculates, therefore, the letter of present frame
It makes an uproar and calculates acquisition than being also based on the statistical distribution of noise.
S107, the variance based on SNR calculate dynamic threshold γ.
The calculating of dynamic threshold γ can be based on formula:
Wherein, D indicates the variance of SNR, PFAIndicate false-alarm (False Alarm) probability.False-alarm probability refers to that noise is missed
It is judged to the probability of voice, that is, in the case where no voice signal, Signal to Noise Ratio (SNR) is judged as the probability greater than γ.
Therefore, dynamic threshold is related with the variance of SNR, i.e., related with the variation of SNR, meanwhile, when noise is unstable,
The variance of SNR can change, so, the value of dynamic threshold can change with the variation of noise, therefore, provided by the invention
Voice activity detection method 100 is capable of the variation of adaptive noise, thus in the environment of noise is unstable or signal-to-noise ratio, detection
Performance will not decline.
In addition, dynamic threshold also with false-alarm probability PFAIt is related, in practical applications, it can be come by controlling false-alarm probability
Improve the performance of voice activity detection method 100.
S109 is based on SNR and γ, generates the VAD judging result of present frame.
Wherein, when SNR is greater than γ, the VAD judging result of generation is that present frame is voice segments, i.e., audio signal is current
Frame is voice segments;When SNR is less than γ, the VAD judging result of generation is that present frame is non-speech segment, i.e., audio signal is current
Frame is non-speech segment.
In some embodiments, the generation of VAD judging result may be based on the SNR of present frame and fixed door limit value generates
's.Specifically, then determining present frame for voice segments when SNR is greater than the fixed door limit value;When SNR is less than the fixed threshold,
Then determine present frame for non-speech segment.
Therefore, VAD method 100 provided by the invention is the statistical distribution based on noise, rather than voice-based statistics
What distribution carried out.Meanwhile variation (the noise of the generation of VAD judging result and real-time signal-to-noise ratio (SNR of present frame) and noise
The variance of ratio) it is related.So as to overcome noise unstable or low signal-to-noise ratio influence caused by VAD judging result.
In addition, the generation of VAD judging result only needs to consider the SNR of present frame, without considering priori and posteriority
SNR.Therefore, speech detection method 100 provided by the invention is simpler, so as to improve the efficiency of detection.
Referring to Fig. 2, the voice activity detection system 200 of one embodiment of the invention is illustrated.The system includes: receiving unit
201 for receiving audio signal;Processing unit 203 is used to calculate the signal-to-noise ratio of the audio signal present frame, wherein the noise
Desired value than being spectrum density and noise spectrum density based on present frame calculates acquisition;And judging unit 205 is for generating
VAD judging result, wherein the judging result is that the signal-to-noise ratio calculated based on processing unit 203 and pre-determined threshold are generated.
Referring to Fig. 3, processing unit 203 includes the spectrum density that the first computing unit 2031 is used to calculate present frame, the second meter
Calculate the noise that desired value and third computing unit 2035 of the unit 2033 for calculating noise spectrum density are used to calculate present frame
Than.
The signal-to-noise ratio (SNR) that third computing unit 2035 calculates is based on formula:
Wherein, the desired value of noise spectrum density is that statistical distribution signal-based calculates, therefore, the letter of present frame
It makes an uproar and calculates acquisition than being also based on the statistical distribution of noise.In addition, the statistical distribution of noise is in no voice signal activity
Pure noise in the case of carry out.
In some embodiments, the calculating of audio signal present frame spectrum density and noise spectrum density can be composed using Pasteur
Algorithm for estimating (Bartlett Algorithm), period nomography etc..The present invention to when this with no restriction.But it is worth noting
, the estimation of noise spectrum density is preferably with the algorithm with smaller variance, such as above-mentioned Pasteur's algorithm, to improve
The detection performance of VAD method, this is because the judging unit of the system 200 is that the variance based on noise generates VAD judging result
's.
The spectrum density can be frequency spectral density (Power Spectral Density, PSD), to calculate current
The power having in the audio signal cell frequency of frame, as the measurement to present frame.
In some embodiments, the pre-determined threshold is dynamic threshold, the dynamic threshold calculated by processing unit 203 and
Come.Specifically, processing unit 203 still further comprises the 4th computing unit 2037, and for calculating dynamic threshold, the dynamic gate
The calculating of limit γ can be based on formula:
Wherein, D indicates the variance of SNR, PFAIndicate false-alarm (False Alarm) probability.False-alarm probability refers to that noise is missed
It is judged to the probability of voice, that is, in the case where no voice signal, Signal to Noise Ratio (SNR) is judged as the probability greater than γ.
Therefore, dynamic threshold is related with the variance of SNR, i.e., related with the variation of SNR, meanwhile, when noise is unstable,
The variance of SNR can change, so, the value of dynamic threshold can change with the variation of noise, therefore, the judgement knot of VAD
Fruit is capable of the variation of adaptive noise.
In addition, dynamic threshold also with false-alarm probability PFAIt is related, in practical applications, can by reduce false-alarm probability,
Improve the accuracy of VAD judgement.
Judging unit 205 is when carrying out VAD judgement, if the Signal to Noise Ratio (SNR) of present frame is greater than dynamic threshold γ, currently
Frame is judged as voice segments;If the Signal to Noise Ratio (SNR) of present frame is less than dynamic threshold γ, present frame is judged as non-voice
Section.
In system 200 provided by the invention, the generation of VAD judging result and real-time signal-to-noise ratio (SNR of present frame) with
And the variation (variance of signal-to-noise ratio) of noise is related.To overcome, noise is unstable or low signal-to-noise ratio makes VAD judging result
At influence.In addition, when carrying out VAD judgement to each frame, it is only necessary to the case where considering current SNR, without considering priori
The case where with posteriority SNR, judgment method is simpler.
System 200 can be by carrying out VAD judgement to audio signal, with the voice segments of the determining audio signal and non-language frame by frame
Segment.
System 200 can further include execution unit 207 be configured to can: VAD based on judging unit 205 judgement
As a result different operations is executed to the different frame of audio signal (voice segments and non-speech segment), for example, identification, decoding etc..
Although present disclosure is as above, present invention is not limited to this.Anyone skilled in the art are not departing from this
It in the spirit and scope of invention, can make various changes or modifications, therefore protection scope of the present invention should be with claim institute
Subject to the range of restriction.
Claims (9)
1. a kind of voice activity detection method characterized by comprising
Calculate the spectrum density of audio signal present frame;
Calculate the desired value of noise spectrum density;
The desired value of spectrum density and the noise spectrum density based on the present frame, calculates the signal-to-noise ratio of present frame;With
Signal-to-noise ratio and pre-determined threshold based on the present frame generate voice activity detection result;
Wherein, the pre-determined threshold is dynamic threshold;The calculating of the dynamic threshold is based on formula:
Wherein, γ indicates that dynamic threshold, D indicate the variance of signal-to-noise ratio, PFAIndicate the probability of false-alarm.
2. the method according to claim 1, wherein wherein the desired value of noise spectrum density is the system based on noise
Score what cloth calculated.
3. the method according to claim 1, wherein wherein the calculating of the signal-to-noise ratio is based on formula:
Wherein, SNR indicates signal-to-noise ratio.
4. the method according to claim 1, wherein being generated when the signal-to-noise ratio is greater than the pre-determined threshold
Voice activity detection the result is that the audio signal present frame be voice segments;When the signal-to-noise ratio is less than the pre-determined threshold
When, the voice activity detection of generation is the result is that the present frame of the audio signal is non-speech segment.
5. a kind of voice activity detection system characterized by comprising
Receiving unit, for receiving audio signal;
Processing unit, for calculating the signal-to-noise ratio of present frame, wherein the signal-to-noise ratio of the present frame is based on the audio signal
What the spectrum density of present frame and the desired value of noise spectrum density calculated;And
Judging unit, be configured to can signal-to-noise ratio and pre-determined threshold based on the present frame, generate voice activity detection result;
Wherein, the pre-determined threshold is dynamic threshold;
The processing unit includes: the 4th computing unit, for being calculated using the following equation the dynamic threshold:
Wherein, γ indicates that dynamic threshold, D indicate the variance of signal-to-noise ratio, PFAIndicate the probability of false-alarm.
6. system according to claim 5, which is characterized in that the processing unit includes: the first computing unit based on
Calculate the spectrum density of the audio signal present frame;Second computing unit is used to calculate the desired value of noise spectrum density;And third
Computing unit, for calculating the signal-to-noise ratio of the present frame.
7. system according to claim 5, which is characterized in that the desired value of the noise spectrum density is the system based on noise
Score what cloth calculated.
8. system according to claim 5, which is characterized in that wherein the calculating of the signal-to-noise ratio is based on formula:
Wherein, SNR indicates signal-to-noise ratio.
9. system according to claim 5, which is characterized in that when the signal-to-noise ratio is greater than the pre-determined threshold, generate
Voice activity detection the result is that the audio signal present frame be voice segments;When the signal-to-noise ratio is less than the pre-determined threshold
When, the voice activity detection result of generation is that the present frame of the audio signal is non-speech segment.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410853931.6A CN105810201B (en) | 2014-12-31 | 2014-12-31 | Voice activity detection method and its system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201410853931.6A CN105810201B (en) | 2014-12-31 | 2014-12-31 | Voice activity detection method and its system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105810201A CN105810201A (en) | 2016-07-27 |
CN105810201B true CN105810201B (en) | 2019-07-02 |
Family
ID=56464829
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201410853931.6A Active CN105810201B (en) | 2014-12-31 | 2014-12-31 | Voice activity detection method and its system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105810201B (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106356070B (en) * | 2016-08-29 | 2019-10-29 | 广州市百果园网络科技有限公司 | A kind of acoustic signal processing method and device |
CN106384597B (en) * | 2016-08-31 | 2020-01-21 | 广州市网星信息技术有限公司 | Audio data processing method and device |
CN106910507B (en) * | 2017-01-23 | 2020-04-24 | 中国科学院声学研究所 | Detection and identification method and system |
CN107393553B (en) * | 2017-07-14 | 2020-12-22 | 深圳永顺智信息科技有限公司 | Auditory feature extraction method for voice activity detection |
CN107910016B (en) * | 2017-12-19 | 2021-07-27 | 河海大学 | Noise tolerance judgment method for noisy speech |
WO2019183747A1 (en) * | 2018-03-26 | 2019-10-03 | 深圳市汇顶科技股份有限公司 | Voice detection method and apparatus |
CN112053702B (en) * | 2020-09-30 | 2024-03-19 | 北京大米科技有限公司 | Voice processing method and device and electronic equipment |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1354455A (en) * | 2000-11-18 | 2002-06-19 | 深圳市中兴通讯股份有限公司 | Sound activation detection method for identifying speech and music from noise environment |
CN1783211A (en) * | 2004-11-25 | 2006-06-07 | Lg电子株式会社 | Speech detection method |
CN101010722A (en) * | 2004-08-30 | 2007-08-01 | 诺基亚公司 | Detection of voice activity in an audio signal |
CN101080765A (en) * | 2005-05-09 | 2007-11-28 | 株式会社东芝 | Voice activity detection apparatus and method |
CN101197130A (en) * | 2006-12-07 | 2008-06-11 | 华为技术有限公司 | Sound activity detecting method and detector thereof |
CN101599269A (en) * | 2009-07-02 | 2009-12-09 | 中国农业大学 | Sound end detecting method and device |
CN102800322A (en) * | 2011-05-27 | 2012-11-28 | 中国科学院声学研究所 | Method for estimating noise power spectrum and voice activity |
CN103903634A (en) * | 2012-12-25 | 2014-07-02 | 中兴通讯股份有限公司 | Voice activation detection (VAD), and method and apparatus for the VAD |
-
2014
- 2014-12-31 CN CN201410853931.6A patent/CN105810201B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1354455A (en) * | 2000-11-18 | 2002-06-19 | 深圳市中兴通讯股份有限公司 | Sound activation detection method for identifying speech and music from noise environment |
CN101010722A (en) * | 2004-08-30 | 2007-08-01 | 诺基亚公司 | Detection of voice activity in an audio signal |
CN1783211A (en) * | 2004-11-25 | 2006-06-07 | Lg电子株式会社 | Speech detection method |
CN101080765A (en) * | 2005-05-09 | 2007-11-28 | 株式会社东芝 | Voice activity detection apparatus and method |
CN101197130A (en) * | 2006-12-07 | 2008-06-11 | 华为技术有限公司 | Sound activity detecting method and detector thereof |
CN101599269A (en) * | 2009-07-02 | 2009-12-09 | 中国农业大学 | Sound end detecting method and device |
CN102800322A (en) * | 2011-05-27 | 2012-11-28 | 中国科学院声学研究所 | Method for estimating noise power spectrum and voice activity |
CN103903634A (en) * | 2012-12-25 | 2014-07-02 | 中兴通讯股份有限公司 | Voice activation detection (VAD), and method and apparatus for the VAD |
Also Published As
Publication number | Publication date |
---|---|
CN105810201A (en) | 2016-07-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105810201B (en) | Voice activity detection method and its system | |
Aneeja et al. | Single frequency filtering approach for discriminating speech and nonspeech | |
Moattar et al. | A simple but efficient real-time voice activity detection algorithm | |
Gerkmann et al. | Noise power estimation based on the probability of speech presence | |
US20140358552A1 (en) | Low-power voice gate for device wake-up | |
Zaw et al. | The combination of spectral entropy, zero crossing rate, short time energy and linear prediction error for voice activity detection | |
CN106098076B (en) | One kind estimating time-frequency domain adaptive voice detection method based on dynamic noise | |
CN103366739A (en) | Self-adaptive endpoint detection method and self-adaptive endpoint detection system for isolate word speech recognition | |
CN105810214B (en) | Voice-activation detecting method and device | |
KR20180067920A (en) | System and method for end-point detection of speech based in harmonic component | |
Li et al. | Non-stationary noise power spectral density estimation based on regional statistics | |
Verteletskaya et al. | Voice activity detection for speech enhancement applications | |
Hirszhorn et al. | Transient interference suppression in speech signals based on the OM-LSA algorithm | |
WO2016028254A1 (en) | Methods and apparatus for speech segmentation using multiple metadata | |
CN108847218A (en) | A kind of adaptive threshold adjusting sound end detecting method, equipment and readable storage medium storing program for executing | |
CN106486133B (en) | One kind is uttered long and high-pitched sounds scene recognition method and equipment | |
Kim et al. | Voice activity detection based on conditional MAP criterion incorporating the spectral gradient | |
Momeni et al. | Single-channel speech presence probability estimation using inter-frame and inter-band correlations | |
Varela et al. | Combining pulse-based features for rejecting far-field speech in a HMM-based voice activity detector | |
CN112102818B (en) | Signal-to-noise ratio calculation method combining voice activity detection and sliding window noise estimation | |
CN116830191A (en) | Automatic speech recognition parameters based on hotword attribute deployment | |
WO2021197566A1 (en) | Noise supression for speech enhancement | |
Zhang et al. | Robust voice activity detection feature design based on spectral kurtosis | |
CN106409312B (en) | Audio classifier | |
Zhang et al. | An endpoint detection algorithm based on MFCC and spectral entropy using BP NN |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |