CN1529528A - Multi sampling rate array signal noise-removing method - Google Patents

Multi sampling rate array signal noise-removing method Download PDF

Info

Publication number
CN1529528A
CN1529528A CNA031434355A CN03143435A CN1529528A CN 1529528 A CN1529528 A CN 1529528A CN A031434355 A CNA031434355 A CN A031434355A CN 03143435 A CN03143435 A CN 03143435A CN 1529528 A CN1529528 A CN 1529528A
Authority
CN
China
Prior art keywords
signal
filter
noise
master reference
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CNA031434355A
Other languages
Chinese (zh)
Other versions
CN1224287C (en
Inventor
曾庆宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN 03143435 priority Critical patent/CN1224287C/en
Publication of CN1529528A publication Critical patent/CN1529528A/en
Application granted granted Critical
Publication of CN1224287C publication Critical patent/CN1224287C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Landscapes

  • Soundproofing, Sound Blocking, And Sound Damping (AREA)
  • Filters That Use Time-Delay Elements (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

The system comprises a remote image receiving service center and a mobile communication handset with a pick-up head. Method for receiving remote on site images includes following steps: (1) user register the mobile communication handset with a pick-up head to the remote image receiving service center, and unique id code is stored to database in computer; (2) when on site images is needed to transfer to remote receiving service center, user dials dedicated line telephone of the said center through registered mobile phone; (3) with information dialed from the registered phone being received, the center discriminates unique id code of incoming call and puts the on site phone through; (4) pick-up head starts to take on site images, and the hand set sends on site images to the remote image receiving service center; (5) the said center receives and processes the remote on site images, and sorts on site images to relevant fold according to unique id code of handset of incoming call.

Description

Multi-sampling rate array signal Denoising Method
(1) technical field:
The present invention relates to signal processing and identification, particularly the array signal Denoising Method.
(2) background technology:
Signal noise silencing all has important use in fields such as communication and signal identifications and is worth.In numerous signal noise silencing methods, adaptive noise cancellation method (Adaptive Noise Cancellation abbreviates ANC as) can adapt to multiple noise circumstance, and is little to signal impairment, can be described as a kind of most important signal noise silencing method.In recent years, researchers find: the actual de-noising performance based on the ANC noise-canceling system of two transducers is quite limited, especially true in the broadband noise occasion, improve de-noising effect, often obtain to form array, and obtained many achievements in research in array signal de-noising field by increasing number of sensors.
The performance of array signal de-noising generally improves along with the increase of transducer number, and when using the ANC method owing to should not contain useful signal in each reference sensor, thereby make that the distance between master reference and the reference sensor usually need be very big, this just makes that the overall dimension of sensor array is bigger.Yet, many practical matter requirement sensor array overall dimensions must be very little, and such as in mobile phone and hands-free phone, sensor array (be microphone array this moment) should be able to embed wherein, this just requires the transducer number in the array can not be too much, and distance each other must be very little.But, transducer number meeting restricting signal denoising effect very little, very short distance then causes cross-talk (Cross-talk) phenomenon of signal and noise very serious, causes common ANC method to lose efficacy.How reach the signal noise silencing effect of a plurality of transducers by transducer number seldom, and how from crosstalk signal, to offset noise by the little and stable algorithm of amount of calculation and extract useful signal, to be miniature array carry out two of signal noise silencing by the ANC method has much challenging problem.
Up to now, at the occasion of two-way voice signal and noise cross-talk only, several schemes have been seen, as: " the international acoustics that is published in 1985, voice and signal processing proceeding " the 3rd volume 1253-1256 page or leaf, people such as Jin Se write, name is called " a kind of new sef-adapting filter structure is carried out noise cancellation in the occasion of crosstalking some experiments and notional result " (R.L.Zinser, G.Mirchandani, J.B.Evans.Some Experimental and Theoretical Results Using a New Adaptive FilterStructure for Noise Cancellation in the Presence of Cross-talk.Proc.ICASSP, Tampa, Vol.3,1253-1256,1985) literary composition, be published in " IEEE Circuits and Systems journal " the 39th volume the 10th phase 681-694 page or leaf in 1992, rice explains that people such as reaching the Buddhist nun writes, name is called " a kind of new adaptive noise cancellation scheme in the occasion of crosstalking " (G.Mirchandani, R.L.Zinser, J.B.Evans.A New Adaptive Noise Cancellation Scheme in the Presence of Crosstalk.IEEETransaction on Circuits and System, Vol.39, No.10,681-694,1992), be published in " IEEE signal processing system collected works " the 605-614 page or leaf in October, 1999, " asymmetric anti-crosstalk the Adaptive Noise Canceller " (S.M.Kuo that writes by Guo Dengren, W.M.Peng.AsymmetricCrosstalk-Resistant Adaptive Noise Canceler.Proc.IEEE Workshop on SignalProcessing Systems, October, 605-614,1999), but the noise cancellation method of two paths of signals only, its actual voice de-noising effect is limited, moreover, it is found that said method often causes the algorithm instability easily, even usually disperse.A very stable method is write " the anti-adaptive noise cancellation of crosstalking " by people such as Ma Hawan, this article is published in nineteen ninety " biomedical engineering annual " the 18th volume 57-67 page or leaf (G Madhavan, H.D.Bruin.Crosstalk Resistant Adaptive NoiseCancellation.Annals of Biomedical Engineering, Vol.18,57-67,1990), this method has used three grades of filtering systems to handle, amount of calculation is bigger, and at medical signals.As at more than the anti-crosstalk signal Denoising Method of the array of two-way, do not see the achievement in research report as yet.
Blind source partition method also is used to signal noise silencing in recent years, obtained some theoretical results, but amount of calculation is big, is unfavorable for real-time implementation, and needs the transducer number more than the noise source number, and therefore blind source separation signal Denoising Method and many practical applications still have certain distance.
In addition, for the voice signal noise-canceling system that does not use the ANC method, have the earliest and the quite ripe subtractive method of spectrums based on single microphone, this method can make voice noise-canceling system volume minimum, and operand is also little, can real-time implementation.But fatal " music noise " problem can occur, cause the voice signal intelligibility after its de-noising to descend, even it is to be difficult to accept, and, only being applicable to the stationary noise environment, the scope of application is less.
(3) summary of the invention:
The present invention will disclose the multi-sampling rate array signal Denoising Method that a kind of only use transducer number seldom can reach the signal noise silencing effect of a plurality of transducers, make and realize bigger de-noising amount by less transducer; The invention also discloses the anti-adaptive noise cancellation method of crosstalking of a kind of two-stage array, the signal cross-talk problem of the miniature array that the transducer of solution close proximity is formed, this method can be offset noise by the little and stable algorithm of amount of calculation and extract signal from crosstalk signal.
The present invention adopts array signal multi-sampling rate method to carry out signal sampling, carrying out signal noise silencing then handles, can be to adopt any in the existing noise cancellation method when carrying out denoising Processing, as the ANC method, array ANC method, active noise silencing method etc., and for the transducer close proximity, when having the situation that noise and signal crosstalk mutually in the array (miniature array is especially true), should adopt the anti-crosstalk noise method of offset of array, particularly adopt the anti-adaptive noise cancellation method of crosstalking of two-stage array among the present invention, it is the same with the method for above-mentioned Ma Hawan, it is a kind of highly stable method, and lack one-level than three grades of filtering systems of Ma Hawan, directly send noise cancellation signal, not only reduced operand, also usually increase than the de-noising effect of three grades of filtering systems from partial filter.
1〉array signal multi-sampling rate method is as follows:
(1) in the sensor array of forming by two or more transducers, select a transducer or one group of transducer as master reference, other conduct is with reference to transducer.
Such as: to the individual transducer M of the N+1 in the array (N 〉=1) 0, M 1..., M N, select M 0Be master reference, M 1..., M NBe reference sensor.
When selecting one group of transducer as master reference, before the signal noise silencing that carries out subsequently, need the signal that each master reference sampling is obtained is carried out forming actual one road master reference signal after the computing, operation method can be such as methods such as linearity summations.
(2) signal to obtaining from master reference, use lower sampling rate to carry out digitlization, the general common sampling rate of signal of using is carried out digitlization, (as: voice are generally 8KHz), and, then use higher sampling rate to carry out digitlization to the signal that each reference sensor obtains, be preferably integral multiple to calculate with convenient in the speed of master reference sampling rate, (as: sampling rate that is several times as much as the normal speech processing, for example 24KHz used usually in voice).
Such as: to master reference M 0The signal x that picks up 0(t) get x with sampling rate f digitlization 0(k), x 0(k)=x 0(t) | T=k/f, k=0,1,2 ... (also discrete signal being used identical mark with corresponding digital here); And to reference sensor M iThe signal x that picks up i(t) then get x with sampling rate f '=pf digitlization i(k '), p>1 is a positive integer, i=1 ..., N, x i(k ')=x i(t) | T=k '/f ', k '=0,1,2 ...
Certainly, also can use the sampling rate identical even higher, but before carrying out signal noise silencing even afterwards, abandon some sampled points, form the lower in fact sampling rate of actual master reference with reference sensor to master reference signals sampling speed.
(3) understand for convenient, the digital signal that each reference sensor can be obtained is considered as having with master reference behind the down-sampling uniformly-spaced the ordered series of numbers digital signal of identical sampling rate, thereby obtain the ordered series of numbers reference signal, so each reference sensor is just as being several reference sensors.
Such as: to each by reference sensor M iThe x that obtains i(k ') all makes x i ( j ) ( k ) = { x i ( j ) , x i ( p + j ) , · · · , x i ( pk + j ) , · · · } , J=0,1 ..., p-1i=1 ..., N then gets Np reference signal x that has identical sampling rate with master reference i (j)(k).
2〉the anti-adaptive noise cancellation method of crosstalking of two-stage array, concrete grammar is:
(1) with the filter A of the digital signal input first order system of all reference sensors in initial pure noise stage, adjust the coefficient of filter A, make the digital signal of filter A output and the digital signal error power minimum of master reference.
Such as: the coefficient w that adjusts following formula median filter A makes error e 1(k) quadratic sum minimum
e 1(k)=x 0(k)-w x(k)
Wherein
w=(w 1,w 2,…,w N)
w i = ( w i ( 0 ) , w i ( 1 ) , · · · , w i ( p - 1 ) )
w i ( j ) = ( w i ( - L ) ( j ) , · · · , w i ( - 1 ) ( j ) , w i 0 ( j ) , w i 1 ( j ) , · · · , w iL ( j ) )
x(k)=[ x 1(k), x 2(k),…, x N(k)] T
x ‾ i ( k ) = [ x ‾ i ( 0 ) ( k ) , x ‾ i ( 1 ) ( k ) , · · · , x ‾ i ( p - 1 ) ( k ) ] T
x ‾ i ( j ) ( k ) = [ x i ( j ) ( k - L ) , · · · , x i ( j ) ( k - 1 ) ,
x i ( j ) ( k ) , x i ( j ) ( k + 1 ) , · · · , x i ( j ) ( k + L ) ] T
And following the example of of L seen next step.
(2) the filter A of first order system preferably uses finite impulse response filter, can adopt the causal filter form, also can adopt the non-causal filter form may arrive the adverse effect that arrives reference sensor behind the master reference earlier to overcome noise, promptly should and postpone input A (can contain time of delay) with each reference signal just with negative, postpone to count L should greater than L 0 = d f c , Wherein d is the ultimate range between master reference and the reference sensor, and f is the sampling rate of master reference, and c is a signal velocity, generally gets L=qceil (L 0), ceil (L here 0) for being not less than L 0Smallest positive integral, under existing device condition, q is taken as 10~50 positive integer usually, so filter A exponent number is Np (2L+1), N is the reference sensor number..
(3) has stage of signal subsequently at pure noise, keep the coefficient of filter A constant, consequential signal after its output signal and master reference digital signal are subtracted each other, input as the median filter B of second level system, and the coefficient of adjustment filter B, make the digital signal of filter B output and the digital signal error power minimum of master reference.
Such as: the coefficient w that adjusts following formula median filter B makes error e 2(k) quadratic sum minimum
e 2(k)=x 0(k)-w′ e 1(k)
Wherein
w′=(w′ -L′,…,w′ -1,w′ 0,w′ 1,…,w′ L′)
e 1(k)=[e 1(k-L′),…,e 1(k-1),e 1(k),e 1(k+1),…,e 1(k+L′)] T
And following the example of of L ' seen next step.
Delay sampling when (4) L ' basis in the exponent number of filter B the is determined filter A L that counts determines the common desirable 2L~4L of L ' under existing device condition.
(5) output of filter B be the de-noising of need extracting digital signal.
(6) during the noise or signal environment change, repeat above-mentioned (1)~(4), the coefficient of readjusting filter A and B is to adapt to new environment.
(7) this method supposition noise is uncorrelated with signal.
Said method can be used for voice communication and speech recognition systems such as mobile phone, phone, computer peripheral, also can be used for systems such as underwater signal reception, array antenna communication, biomedicine signals extraction and active noise silencing.
Because the present invention adopts array signal multi-sampling rate method to carry out signal sampling, reference sensor is used the sampling rate that is several times as much as the common sampling rate of signal, so only use the minority transducer, even only two transducers can be obtained the array de-noising effect that a plurality of transducers are formed, and adopt the anti-adaptive noise cancellation method of crosstalking of two-stage array to carry out signal noise silencing, can be with simple algorithm, cheap cost, solve the signal cross-talk problem between the transducer of close proximity preferably.And the present invention is not only the cross-interference issue that solves two paths of signals, also solved the cross-interference issue of multiple signals.Test shows: the present invention uses many speed sampling method (MSR) and the anti-cross-talk adaptive noise cancellation of two-stage array (ACRANC) method, the voice de-noising effect is significantly improved, and allow microphone close proximity each other, system based on them can be microminiaturized, also be easy to real-time implementation, and be applicable to multiple noise circumstance, so especially be used for the voice de-noising of hands-free phone, mobile phone etc.
(4) description of drawings:
Fig. 1 is the schematic diagram of the anti-adaptive noise cancellation method of crosstalking of two-stage array;
Fig. 2 is microphone 1 picks up in one embodiment of the present invention the voice and the mixed signal figure of noise;
Fig. 3 is microphone 2 picks up in one embodiment of the present invention the voice and the mixed signal figure of noise;
Fig. 4 carries out voice signal figure after the de-noising with the ACRANC method in one embodiment of the present invention;
Fig. 5 carries out voice signal figure after the de-noising with MSR and ACRANC method in one embodiment of the present invention.
(5) embodiment
Fig. 1 is the schematic diagram of the anti-adaptive noise cancellation method of crosstalking of two-stage array, wherein x 0Be the digital signal that master reference picks up, x 1, x 2..., x NBe the digital signal that reference sensor picks up, A, B are respectively the first order and second level filter, y 1, y 2Be respectively the output of filter A, B, and e 1=x 0-y 1, e 2=x 0-y 2
Be the validity of checking this method, provide a non-limiting examples of the present invention and concrete result of the test thereof below.In this test, the mini microphone battle array only is made up of at a distance of the microphone of the little button size of 2cm two centers, the sampling rate of main microphone signal is 8KHz, reference microphone signals sampling speed is 24KHz, voice are real voice " this section voice are used for test ", noise when noise is the broadcast receiver off resonance (being broadband noise), pure noise more than 1.5 seconds before beginning, voice is arranged, test is 54 square metres common indoor carrying out, and indoor placement has numerous equipment such as computer, tables and chairs.
Fig. 2, Fig. 3 have provided two microphone M in the array respectively 0, M 1The noisy speech signal x that collects 0, x 1, only drawn among the figure from 0.5 second to the 3.125th second the noise and the mixed signal figure of voice, by audition, the voice in two mixed signals are all by the noise severe contamination, and the hearer does not hear the said content of speaker.
The voice noise cancellation signal of Fig. 4 for only using the anti-cross-talk adaptive noise cancellation of two-stage array (ACRANC) method to obtain, the voice noise cancellation signal of Fig. 5 for using multi-sampling rate (MSR) method and ACRANC method to obtain simultaneously.All only drawn from 0.5 second to 3.125 seconds relevant signal graph among Fig. 4 and Fig. 5, and from 0 to 0.5 second be the initial convergence process of algorithm when optimizing filter coefficient.
Mixed signal x among Fig. 2 0, be 2.970dB in the signal to noise ratio of voice that the voice stage is arranged and noise; And the x among Fig. 3 1Corresponding signal to noise ratio is 3.879dB; Among Fig. 4 after the de-noising the corresponding signal to noise ratio of voice be increased to 5.657dB, its leaching process median filter A and B all adopt cause and effect type finite impulse response filter, exponent number is appointed respectively and is taken as 98 and 48, if the exponent number of filter A is reduced to 32, the signal to noise ratio of voice then is reduced to 5.248dB after the gained de-noising; The corresponding signal to noise ratio of the voice that extract has but reached 15.323dB among Fig. 5, in its de-noising process, filter A and B also all adopt cause and effect type finite impulse response filter, exponent number appoints equally respectively and is taken as 98 and 48 that (if look the reference signal of input filter A is signal behind three road down-samplings, then the exponent number of this three road signals input A is 32), therefore its amount of calculation is identical with the amount of calculation of de-noising voice signal in obtaining Fig. 4, but its voice de-noising effect is greatly improved.By audition, noise reduces to some extent among Fig. 4, can understand said content; And noise is much smaller especially among Fig. 5, and said content is clear to be understood well.

Claims (7)

1, multi-sampling rate array signal Denoising Method comprises signal sampling method and signal noise silencing facture, it is characterized in that: described signal sampling method is an array signal multi-sampling rate method, and it is to sample with following method:
(1) in the sensor array of being made up of two or more transducers, select a transducer or one group of transducer as master reference, other conduct is with reference to transducer;
(2) signal to obtaining from master reference or master reference group uses lower sampling rate to carry out digitlization, and to the signal that each reference sensor obtains, then uses higher sampling rate to carry out digitlization.
2, array signal Denoising Method according to claim 1 is characterized in that: use the common sampling rate of this signal to carry out digitlization to the master reference signal.
3, array signal Denoising Method according to claim 1 is characterized in that: reference sensor signals sampling speed is the integral multiple of master reference signals sampling speed.
4, array signal Denoising Method according to claim 2 is characterized in that: reference sensor signals sampling speed is the integral multiple of master reference signals sampling speed.
5, according to any one described array signal Denoising Method in the claim 1~4, it is characterized in that: described signal noise silencing facture is the anti-adaptive noise cancellation method of crosstalking of two-stage array, is specially:
(1) as if the master reference that has more than, with synthetic one road master reference digital signal of digital signal of each road master reference sampling gained;
(2) use the two-stage filtering system totally two filters carry out signal noise silencing, and the supposition noise is uncorrelated with signal;
(3) with the first order filter A of the digital signal input system of the reference sensor in initial pure noise stage, adjust the coefficient of filter A, make the digital signal and the master reference digital signal error power minimum of filter A output;
(4) with each reference signal and delay input A thereof, postponing counts determines according to the sampling rate of the ultimate range between master reference and the reference sensor, master reference and the propagation velocity of signal;
(5) has stage of signal subsequently at pure noise, keep the coefficient of filter A constant, consequential signal after its output is subtracted each other with the master reference digital signal, input as the median filter B of second level system, and the coefficient of adjustment filter B, make the digital signal of filter B output and the digital signal error power minimum of master reference;
(6) exponent number of filter B is counted according to the delay sampling of determining filter A and is determined;
(7) output of filter B be the de-noising of need extracting digital signal;
When (8) noise or signal environment of living in changed, the coefficient of readjusting filter A and B was to adapt to new environment.
6, according to any one described array signal Denoising Method in the claim 1~4, it is characterized in that: filter A uses finite impulse response filter, adopt the causal filter form, perhaps adopt the non-causal filter form may arrive the adverse effect that arrives reference sensor behind the master reference earlier to overcome noise.
7, array signal Denoising Method according to claim 5, it is characterized in that: filter A uses finite impulse response filter, adopt the causal filter form, perhaps adopt the non-causal filter form may arrive the adverse effect that arrives reference sensor behind the master reference earlier to overcome noise.
CN 03143435 2003-09-28 2003-09-28 Multi sampling rate array signal noise-removing method Expired - Fee Related CN1224287C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 03143435 CN1224287C (en) 2003-09-28 2003-09-28 Multi sampling rate array signal noise-removing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 03143435 CN1224287C (en) 2003-09-28 2003-09-28 Multi sampling rate array signal noise-removing method

Publications (2)

Publication Number Publication Date
CN1529528A true CN1529528A (en) 2004-09-15
CN1224287C CN1224287C (en) 2005-10-19

Family

ID=34286583

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 03143435 Expired - Fee Related CN1224287C (en) 2003-09-28 2003-09-28 Multi sampling rate array signal noise-removing method

Country Status (1)

Country Link
CN (1) CN1224287C (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104754465A (en) * 2013-12-31 2015-07-01 展讯通信(上海)有限公司 Self-adaptive signal enhancing method and system
CN109243482A (en) * 2018-10-30 2019-01-18 深圳市昂思科技有限公司 Improve the miniature array voice de-noising method of ACRANC and Wave beam forming

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104754465A (en) * 2013-12-31 2015-07-01 展讯通信(上海)有限公司 Self-adaptive signal enhancing method and system
CN104754465B (en) * 2013-12-31 2018-06-05 展讯通信(上海)有限公司 A kind of adaptive signal enhancement method and system
CN109243482A (en) * 2018-10-30 2019-01-18 深圳市昂思科技有限公司 Improve the miniature array voice de-noising method of ACRANC and Wave beam forming
CN109243482B (en) * 2018-10-30 2022-03-18 深圳市昂思科技有限公司 Micro-array voice noise reduction method for improving ACROC and beam forming

Also Published As

Publication number Publication date
CN1224287C (en) 2005-10-19

Similar Documents

Publication Publication Date Title
CN1122970C (en) Signal noise reduction by time-domain spectral subtraction using fixed filters
CN100337270C (en) Device and method for eliminating voice communication terminal background noise
CN101079267A (en) Collecting sound device with directionality, collecting sound method with directionality and memory product
CN1166160C (en) Arrangement for suppressing interfering component of input signal
CN1282156C (en) Audio signal bandwidth extension
CN1197422C (en) Sound close detection for mobile terminal and other equipment
CN102969002B (en) Microphone array speech enhancement device capable of suppressing mobile noise
CN101911724A (en) Speech enhancement using multiple microphones on multiple devices
US6754623B2 (en) Methods and apparatus for ambient noise removal in speech recognition
JP2012522255A (en) Audio signal classification method and apparatus
CN1719516A (en) Adaptive filter device and adaptive filtering method
CN1476180A (en) Sound echo inhibitor for hand free voice communication
CN1292985A (en) Improved signal localization arrangement
CN1711799A (en) Array device and portable terminal
CN1565144A (en) Directional audio signal processing using an oversampled filterbank
CN1515108A (en) Single ended line probing in DSL system
CN101061743A (en) Method and apparatus for audio signal enhancement
CN101030383A (en) Sound source separating device, method, and program
CN103913778B (en) Nuclear magnetic resonance signal real-time noise offsetting device for multiple near-end reference coils
CN1668058A (en) Recursive least square difference based subband echo canceller
CN1121764C (en) Transmission device and system
CN1367978A (en) Adaptability to multiple hands-free communication accessories
CN1224287C (en) Multi sampling rate array signal noise-removing method
CN1369173A (en) Symmetry based subband acoustic echo cancellation
CN1768555A (en) Method and apparatus for reducing an interference noise signal fraction in a microphone signal

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C19 Lapse of patent right due to non-payment of the annual fee
CF01 Termination of patent right due to non-payment of annual fee