CN1443006A - Mixed sound system of intelligent controlled video frequency conference and method of controlling conference course - Google Patents
Mixed sound system of intelligent controlled video frequency conference and method of controlling conference course Download PDFInfo
- Publication number
- CN1443006A CN1443006A CN 03102814 CN03102814A CN1443006A CN 1443006 A CN1443006 A CN 1443006A CN 03102814 CN03102814 CN 03102814 CN 03102814 A CN03102814 A CN 03102814A CN 1443006 A CN1443006 A CN 1443006A
- Authority
- CN
- China
- Prior art keywords
- data
- voice data
- spokesman
- chairman
- people
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
An audio mixing system for intelligent control of video meeting is divided into the client end including three terminals for chairman of the meeting, speaker assigned by the chairman and a couple of visitors of the meeting, the server end including network interface, central processor, starting flash storage, program flash storage, random storage, buffer, data bus and address bus, and the client end and the server end are connected to be a hard ware system integrated with functions of voice collecting, processing and transmitting. The controlling method for the meeting is to judge whether it is the chair man or the speaker assigned by the chairman or the visitor of the meeting according to the speaker status information "brought" by the voide data and then to decide whether the audio data of this path should be sent out or not as per the actual situation at the moment.
Description
Technical field
The present invention relates to a kind of in video conference the mixer system of Based Intelligent Control meeting process and the method for control meeting process.Be applicable to holding video conference in network.
Background technology
At present, the server end of video conference has generally all used mixer, is used to mix different participants' sound, and mixed voice data is sent to each client.In video conference, there is the different role of some identity, " chairman " arranged, " spokesman " of chairman's appointment, and " auditor "., need file an application to the chairman by operation interface if chip in as the auditor, just obtain right to speak after chairman agrees, system begins to transmit this people's that chips in voice data; Equally, after speech finishes, propose to withdraw from application by operation interface to the chairman, withdraw from speech after chairman agrees, system stops to send this circuit-switched data.Because the auditor adds each time or withdraws from speech, all need once " challenge-response " process of experience, auditor in this process/chip in people and chairman need carry out extra operation, have so not only increased the load of network, can not concentrate on meeting itself.
Summary of the invention
The technical problem to be solved in the present invention is: a kind of mixer system of Based Intelligent Control video conference and the method for control meeting process are provided, this mixer system has solved the troublesome operation that adds and withdraw from speech in the video conference process, " challenge-response " process of making becomes and there is no need, reduced the load of network, made more convenient to operate.
The technical solution adopted in the present invention is: the mixer system of Based Intelligent Control video conference, this mixer system are divided into customer end A and server end B, wherein:
I) customer end A comprises spokesman b and three kinds of terminals of several auditor c of chairman a, chairman's appointment;
Ii) server end B comprises network interface, central processing unit, startup flash memory, program flash memory, random asccess memory, buffer and data/address bus and address bus;
Iii) customer end A and server end B connect into a sound collection, processing, transmission hardware system;
Iv) the voice data of customer end A at first enters modulus converter A/D by microphone, and A/D becomes the data flow of PCM form with audio signal digitizing, is sent to server end B by network interface, and its characteristics are:
Start from starting flash memory when v) server end is switched on, the acoustic processing program that will be solidificated in then in the program flash memory is written into the memory field, central processing unit is from the memory field call instruction, these instructions are upper threshold according to three threshold values setting, threshold value lower limit and sound dwell time logic determines go out whether this circuit-switched data is participated in audio mixing, do not process for non-participating PCM stream, carry out the audio mixing computing for the data flow of participating in audio mixing, the audio mixing algorithm promptly is that each circuit-switched data is carried out linear superposition, specific algorithm is provided by the program that is solidificated in the program flash memory, is written into the memory field for the central processing unit scheduling when operation;
Vi) the data behind the audio mixing still keep the PCM form, and these PCM streams are sent to relevant terminal by network interface, change the PCM circulation into analog signal at terminal D/A transducer, output to audio-frequence player device.
At server end B, what compare with three threshold values of described setting is " zero-crossing rate " of voice data, and promptly signal wave passes the number of times of transverse axis (zero level) in the unit interval, and mixer after obtaining sampled data at every turn, zero-crossing rate to data is analyzed, and two kinds of situations are arranged:
I) if exceed certain numerical value, promptly upper threshold assert that then data are " sound ", only are identified as sound data and just participate in audio mixing;
Ii) set the sound dwell time, the zero-crossing rate of this section in the time added up, if numerical value less than a certain specific value, promptly the threshold value lower limit then can predicate " noiseless ", as long as be identified as noiselessly, just should withdraw from speech immediately.
The method of Based Intelligent Control video conference process of the present invention comprises the step I of customer end A realization and the Step II that server end B realizes, wherein:
Step I shows as 1), the customer end A program judges spokesman's condition information I of voice data " incidentally ", if the spokesman b of chairman a or chairman's appointment directly sends voice data to server B, if not, judges whether it is people (auditor) c that chips in;
2), client-side program is obtained spokesman's condition information I incessantly, and parse maximum two spokesman ID, contrast the ID of self, can draw two simple facts, i.e. " whether self chip in the people " and " current whether can chipping in ", if have one and self to equate among two ID that parse then be the people c that chips in, continue to send voice data, if do not wait then self be not the people c that chips in to server;
3), judge whether and can chip in according to spokesman's condition information I again, if two ID that parse are all non-0, the people that chips in is described, and the quota has been filled, do not send data, if having only non-0 or two an of ID all is 0, then current state can be chipped in, and beginning sends voice data to server.
Step II shows as 1), server end B detects the voice data that customer end A sends in network after, obtain the ID in this circuit-switched data, if the voice data that the spokesman b terminal of chairman a or chairman's appointment is sent, directly ginseng is mixed, otherwise assert it is people (auditor) c that chips in;
2) server calculates the total zero-crossing rate A in zero-crossing rate R and the time T earlier, program judges whether the current people of chipping in according to the ID that parses then, if, investigate according to value A and whether to become noiselessly,, carry out information setting if become noiselessly, from spokesman's condition information I, reject this road ID, and stop (transmission) and mix this road voice data, if do not become noiselessly, continue (transmission) and mix this road voice data;
3) if not the current people that chips in, judge whether to become according to value R sound, if become sound, carry out information setting, from spokesman's condition information I, add this road ID, and beginning (transmission) mixes this road voice data, if do not become soundly, abandon this packet.
The invention has the beneficial effects as follows: because the present invention is the operation of having simplified auditor/chip in people and chairman by " sound/no sound detection ", " challenge-response " process is become be there is no need, reduced the load of network, made the participant can concentrate on meeting itself.
Description of drawings
Fig. 1 is a hardware block diagram of the present invention.
Fig. 2 is the workflow diagram of customer end A.
Fig. 3 is the workflow diagram of server end B.
Embodiment
Mixer system of the present invention is divided into customer end A and server end B, and the spokesman b of chairman a, chairman's appointment is arranged client terminal and several auditors/people c chips in.
Server end B is forming (consulting Fig. 1) by network interface 1 (100BASE-T), central processing unit 2 (MPC860), random asccess memory 3, startup flash memory 4, program flash memory 5, buffer 6, data/ address bus 7,9 and address bus 8,10 aspect the hardware realization.
The voice data of customer end A at first enters modulus converter A/D by microphone, and A/D becomes the data flow of PCM (pulse code modulation) form with audio signal digitizing, is sent to server end B by network interface 1.
Start from starting flash memory 4 during server end B energising, the acoustic processing program that will be solidificated in then in the program flash memory 5 is written into the memory field, central processing unit 2 is from the memory field call instruction, these instructions are upper threshold according to three threshold values setting, threshold value lower limit and sound dwell time logic determines go out whether this circuit-switched data is participated in audio mixing, do not process for non-participating PCM stream, carry out the audio mixing computing for the data flow of participating in audio mixing, the audio mixing algorithm promptly is that each circuit-switched data is carried out linear superposition, specific algorithm is provided by the program that is solidificated in the program flash memory 5, is written into the memory field for central processing unit 2 scheduling when operation; Data behind the audio mixing still keep the PCM form, and these PCM streams are sent to relevant terminal by network interface 1, change the PCM circulation into analog signal at terminal D/A transducer, output to audio-frequence player device.
In the acoustic processing program of server end B, what compare with these threshold values that preset (upper threshold, threshold value lower limit and sound dwell time) is " zero-crossing rate " of voice data, be that signal wave passes the number of times of transverse axis (zero level) in the unit interval, mixer after obtaining sampled data at every turn, zero-crossing rate to data is analyzed, if exceed certain numerical value, i.e. upper threshold, assert that then data are " sound ", only are identified as sound data and just participate in audio mixing; Set the sound dwell time, the zero-crossing rate of this section in the time added up, if numerical value less than a certain specific value, promptly the threshold value lower limit then can predicate " noiseless ", as long as be identified as noiselessly, just should withdraw from speech immediately.
The acoustic processing program of server end is also set " the current people's of chipping in situation " information, and it is bundled in the voice data of uninterrupted transmission, transmits to client.Client is equipped with " information analysis program ", the voice data that client is received by parsing, will be wherein the information I of " the current people's of chipping in situation " parse, determine directly whether this locality is necessary to send voice data to server.
By the analysis of front as can be known, spokesman's condition information I here is " piggy backed " client by voice data, so play a part tie, and information setting is to be driven by the result of sound detection to change initiation into, what need here to obtain is two great shifts, sound to noiseless transformation and noiseless to sound transformation.By these two transformations, add and withdraw from the operation of meeting automatically, and " examining " process replaces the chairman to finish automatically according to information I by customer end A, realize the automatic ordered control of meeting process.For example, for each terminal, independently non-0 a numerical value ID is all arranged, length is 1 byte, number N is 2 if the maximum that system allows is chipped in, we just are defined as 2 byte longs to information I so, its content is exactly respectively two people's that chip in ID, (is necessary to illustrate why be two, because 4 people of general maximum permission talk simultaneously, remove a, b is so the number of chipping in mostly is 2 most), not hard to imagine, if have only the chip in people or the people that do not chip in, so Dui Ying position just is 0.
Shown in Figure 2 is the workflow diagram of customer end A, the steps include:
1), the customer end A program judges spokesman's condition information I of voice data " incidentally ", if the spokesman of chairman or chairman's appointment directly sends voice data to server B, if not, judges whether it is the people that chips in;
2), client-side program is obtained spokesman's condition information I incessantly, and parse maximum two spokesman ID, contrast the ID of self, can draw two simple facts, i.e. " whether self chip in the people " and " current whether can chipping in ", if have one and self to equate among two ID that parse then be the people that chips in, continue to send voice data, if do not wait then self be not the people that chips in to server;
3), judge whether and can chip in according to spokesman's condition information I again, if two ID that parse are all non-0, the people that chips in is described, and the quota has been filled, do not send data, if having only non-0 or two an of ID all is 0, then current state can be chipped in, and beginning is to server sounding sound data.
Shown in Figure 3 is the workflow diagram of server end B, the steps include:
1), server end B detects the voice data that customer end A sends in network after, obtain the ID in this circuit-switched data, if the voice data that spokesman's terminal of chairman or chairman's appointment is sent, directly ginseng is mixed, otherwise assert it is that the people that chips in is the auditor;
2) server calculates the total zero-crossing rate A in zero-crossing rate R and the time T earlier, program judges whether the current people of chipping in according to the ID that parses then, if, investigate according to value A and whether to become noiselessly,, carry out information setting if become noiselessly, from spokesman's condition information I, reject this road ID, and stop (transmission) and mix this road voice data, if do not become noiselessly, continue (transmission) and mix this road voice data;
3) if not the current people that chips in, judge whether to become according to value R sound, if become sound, carry out information setting, from spokesman's condition information I, add this road ID, and beginning (transmission) mixes this road voice data, if do not become soundly, abandon this packet.
Claims (3)
1, a kind of mixer system of Based Intelligent Control video conference, this mixer are divided into customer end A and server end B, wherein:
I) customer end A comprises spokesman b and three kinds of terminals of several auditor c of chairman a, chairman's appointment;
Ii) server end B comprises network interface (1), central processing unit (2), starts flash memory (4), program flash memory (5), random asccess memory (3), buffer (6) and data/address bus (7,9) and address bus (8,10);
Iii) customer end A and server end B connect into a sound collection, processing, transmission hardware system;
Iv) the voice data of customer end A at first enters modulus converter A/D by microphone, and A/D becomes the data flow of PCM form with audio signal digitizing, is sent to server end B by network interface (1), it is characterized in that:
Start from starting flash memory (4) when v) server end is switched on, the acoustic processing program that will be solidificated in then in the program flash memory (5) is written into the memory field, central processing unit (2) is from the memory field call instruction, these instructions are upper threshold according to three threshold values setting, threshold value lower limit and sound dwell time logic determines go out whether this circuit-switched data is participated in audio mixing, do not process for non-participating PCM stream, carry out the audio mixing computing for the data flow of participating in audio mixing, the audio mixing algorithm promptly is that each circuit-switched data is carried out linear superposition, specific algorithm is provided by the program that is solidificated in the program flash memory (5), is written into the memory field for central processing unit (2) scheduling when operation;
Vi) the data behind the audio mixing still keep the PCM form, and these PCM streams are sent to relevant terminal by network interface (1), change the PCM circulation into analog signal at terminal D/A transducer, output to audio-frequence player device.
2, the mixer system of Based Intelligent Control video conference according to claim 1, it is characterized in that: at server end B, what compare with three threshold values of described setting is " zero-crossing rate " of voice data, be that signal wave passes the number of times of transverse axis (zero level) in the unit interval, mixer after obtaining sampled data at every turn, zero-crossing rate to data is analyzed, and two kinds of situations are arranged:
I) if exceed certain numerical value, promptly upper threshold assert that then data are " sound ", only are identified as sound data and just participate in audio mixing;
Ii) set the sound dwell time, the zero-crossing rate of this section in the time added up, if numerical value less than a certain specific value, promptly the threshold value lower limit then can predicate " noiseless ", as long as be identified as noiselessly, just should withdraw from speech immediately.
3, a kind of method of Based Intelligent Control video conference process, this method comprise the step I of customer end A realization and the Step II that server end B realizes, wherein:
Step I shows as 1), the customer end A program judges spokesman's condition information I of voice data " incidentally ", if the spokesman b of chairman a or chairman's appointment directly sends voice data to server B, if not, judges whether it is people (auditor) c that chips in;
2), client-side program is obtained spokesman's condition information I incessantly, and parse maximum two spokesman ID, contrast the ID of self, can draw two simple facts, i.e. " whether self chip in the people " and " current whether can chipping in ", if have one and self to equate among two ID that parse then be the people c that chips in, continue to send voice data, if do not wait then self be not the people c that chips in to server;
3), judge whether and can chip in according to spokesman's condition information I again, if two ID that parse are all non-0, the people that chips in is described, and the quota has been filled, do not send data, if having only non-0 or two an of ID all is 0, then current state can be chipped in, and beginning sends voice data to server.
Step II shows as 1), server end B detects the voice data that customer end A sends in network after, obtain the ID in this circuit-switched data, if the voice data that the spokesman b terminal of chairman a or chairman's appointment is sent, directly ginseng is mixed, otherwise assert it is people (auditor) c that chips in;
2) server calculates the total zero-crossing rate A in zero-crossing rate R and the time T earlier, program judges whether the current people of chipping in according to the ID that parses then, if, investigate according to value A and whether to become noiselessly,, carry out information setting if become noiselessly, from spokesman's condition information I, reject this road ID, and stop (transmission) and mix this road voice data, if do not become noiselessly, continue (transmission) and mix this road voice data;
3) if not the current people that chips in, judge whether to become according to value R sound, if become sound, carry out information setting, from spokesman's condition information I, add this road ID, and beginning (transmission) mixes this road voice data, if do not become soundly, abandon this packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 03102814 CN1206860C (en) | 2003-01-16 | 2003-01-16 | Mixed sound system of intelligent controlled video frequency conference and method of controlling conference course |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 03102814 CN1206860C (en) | 2003-01-16 | 2003-01-16 | Mixed sound system of intelligent controlled video frequency conference and method of controlling conference course |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1443006A true CN1443006A (en) | 2003-09-17 |
| CN1206860C CN1206860C (en) | 2005-06-15 |
Family
ID=27796563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 03102814 Expired - Fee Related CN1206860C (en) | 2003-01-16 | 2003-01-16 | Mixed sound system of intelligent controlled video frequency conference and method of controlling conference course |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1206860C (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100399744C (en) * | 2005-04-30 | 2008-07-02 | 腾讯科技（深圳）有限公司 | Method for realizing group chatting |
| CN100401765C (en) * | 2005-03-24 | 2008-07-09 | 华为技术有限公司 | Video conference controlling method |
| CN101374344B (en) * | 2008-10-20 | 2011-10-26 | 杭州优能通信系统有限公司 | Wireless emergency communication synthesis scheduling system |
| CN101373442B (en) * | 2008-09-02 | 2011-11-09 | 广东威创视讯科技股份有限公司 | Method for transmitting user operation case |
| CN1997057B (en) * | 2005-11-26 | 2012-01-18 | 沃福森微电子股份有限公司 | Audio device |
| CN101502089B (en) * | 2006-07-28 | 2013-07-03 | 西门子企业通讯有限责任两合公司 | Method for carrying out an audio conference, audio conference device, and method for switching between encoders |
| CN106162043A (en) * | 2015-04-14 | 2016-11-23 | 杭州施强网络科技有限公司 | Multimedia file demenstration method in a kind of video conferencing system |
| CN106534762A (en) * | 2016-11-16 | 2017-03-22 | 深圳市捷视飞通科技股份有限公司 | Low-time-delay distributed audio processing method and system |
| CN107040746A (en) * | 2017-03-31 | 2017-08-11 | 北京奇艺世纪科技有限公司 | Multi-video chat method and device based on Voice command |
| CN109510905A (en) * | 2018-12-06 | 2019-03-22 | 中通天鸿（北京）通信科技股份有限公司 | The sound mixing method and system of multi-path voice |
| CN109817237A (en) * | 2019-03-06 | 2019-05-28 | 小雅智能平台（深圳）有限公司 | A kind of audio automatic processing method, terminal and computer readable storage medium |
| CN109859753A (en) * | 2019-02-26 | 2019-06-07 | 北京华夏电通科技有限公司 | Voice-activated method and device applied to digital court |
| CN109976700A (en) * | 2019-01-25 | 2019-07-05 | 广州富港万嘉智能科技有限公司 | A kind of method, electronic equipment and the storage medium of the transfer of recording permission |
-
2003
- 2003-01-16 CN CN 03102814 patent/CN1206860C/en not_active Expired - Fee Related
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100401765C (en) * | 2005-03-24 | 2008-07-09 | 华为技术有限公司 | Video conference controlling method |
| CN100399744C (en) * | 2005-04-30 | 2008-07-02 | 腾讯科技（深圳）有限公司 | Method for realizing group chatting |
| CN1997057B (en) * | 2005-11-26 | 2012-01-18 | 沃福森微电子股份有限公司 | Audio device |
| CN101502089B (en) * | 2006-07-28 | 2013-07-03 | 西门子企业通讯有限责任两合公司 | Method for carrying out an audio conference, audio conference device, and method for switching between encoders |
| CN101373442B (en) * | 2008-09-02 | 2011-11-09 | 广东威创视讯科技股份有限公司 | Method for transmitting user operation case |
| CN101374344B (en) * | 2008-10-20 | 2011-10-26 | 杭州优能通信系统有限公司 | Wireless emergency communication synthesis scheduling system |
| CN106162043A (en) * | 2015-04-14 | 2016-11-23 | 杭州施强网络科技有限公司 | Multimedia file demenstration method in a kind of video conferencing system |
| CN106534762A (en) * | 2016-11-16 | 2017-03-22 | 深圳市捷视飞通科技股份有限公司 | Low-time-delay distributed audio processing method and system |
| CN107040746A (en) * | 2017-03-31 | 2017-08-11 | 北京奇艺世纪科技有限公司 | Multi-video chat method and device based on Voice command |
| CN107040746B (en) * | 2017-03-31 | 2019-11-15 | 北京奇艺世纪科技有限公司 | Multi-video chat method and device based on voice control |
| CN109510905A (en) * | 2018-12-06 | 2019-03-22 | 中通天鸿（北京）通信科技股份有限公司 | The sound mixing method and system of multi-path voice |
| CN109510905B (en) * | 2018-12-06 | 2020-10-30 | 中通天鸿（北京）通信科技股份有限公司 | Multi-channel voice mixing method and system |
| CN109976700A (en) * | 2019-01-25 | 2019-07-05 | 广州富港万嘉智能科技有限公司 | A kind of method, electronic equipment and the storage medium of the transfer of recording permission |
| CN109859753A (en) * | 2019-02-26 | 2019-06-07 | 北京华夏电通科技有限公司 | Voice-activated method and device applied to digital court |
| CN109817237A (en) * | 2019-03-06 | 2019-05-28 | 小雅智能平台（深圳）有限公司 | A kind of audio automatic processing method, terminal and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1206860C (en) | 2005-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1206860C (en) | Mixed sound system of intelligent controlled video frequency conference and method of controlling conference course | |
| US20060026002A1 (en) | Method and system for handling audio signals of conference | |
| RU2293368C2 (en) | Method (versions) and system (versions) for controlling conferencs and control unit for multimedia/speech system | |
| US10244120B2 (en) | Method for carrying out an audio conference, audio conference device, and method for switching between encoders | |
| EP2367343B1 (en) | Audio mixing | |
| DE602004001637T2 (en) | Method and device for improving disturbing signals in an audio / video conference | |
| US6807563B1 (en) | Automatic teleconferencing control system | |
| US8290134B2 (en) | Managing conference calls via a talk queue | |
| EP1391103B1 (en) | Efficient buffer allocation for current and predicted active speakers in voice conferencing systems | |
| US20030149724A1 (en) | Multi-point video conferencing scheme | |
| US7580375B1 (en) | Scalable moderated audio conferencing for multicast and unicast endpoints and gateways | |
| WO2000072563A1 (en) | Automatic teleconferencing control system | |
| CN1672394A (en) | Conference server dynamically determining information streams to be received by a conference bridge | |
| CN100344140C (en) | Video telephone conference system and its audio/video processing method | |
| CN106130747A (en) | A kind of method of Conference control, system and mobile terminal | |
| WO2001045326A2 (en) | Method and device for controlling a telecommunication conference | |
| CN100484175C (en) | Method and system of implementing report of current speaker during conference | |
| US20100061536A1 (en) | Method for carrying out a voice conference and voice conference system | |
| CN1610401A (en) | Terminal entry and exit method in multi-point meeting | |
| Smith et al. | Speaker selection for tandem-free operation VoIP conference bridges | |
| Baskaran et al. | Audio mixer with automatic gain controller for software based multipoint control unit | |
| CN1277401C (en) | Mixing method of telephone meeting | |
| CN1543181A (en) | A distributed mix processing method | |
| CN103095939B (en) | Conference voice control method and system | |
| CN1455547A (en) | Method of realizing mass business in communication system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20080319 Pledge (preservation): Preservation |
|
| PD01 | Discharge of preservation of patent |
Date of cancellation: 20080919 Pledge (preservation): Preservation registration |
|
| PD01 | Discharge of preservation of patent |
Date of cancellation: 20080919 Pledge (preservation): Preservation registration |
|
| C19 | Lapse of patent right due to non-payment of the annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |