CN109822587A - It is a kind of for the voice hospital guide robot head neck device of factories and miness hospital and control - Google Patents
It is a kind of for the voice hospital guide robot head neck device of factories and miness hospital and control Download PDFInfo
- Publication number
- CN109822587A CN109822587A CN201910163672.7A CN201910163672A CN109822587A CN 109822587 A CN109822587 A CN 109822587A CN 201910163672 A CN201910163672 A CN 201910163672A CN 109822587 A CN109822587 A CN 109822587A
- Authority
- CN
- China
- Prior art keywords
- voice
- module
- hospital
- mouth
- neck
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000009471 action Effects 0.000 claims abstract description 52
- 230000033001 locomotion Effects 0.000 claims abstract description 40
- 230000007246 mechanism Effects 0.000 claims abstract description 33
- 238000010276 construction Methods 0.000 claims abstract description 8
- 239000002184 metal Substances 0.000 claims description 55
- 229910052751 metal Inorganic materials 0.000 claims description 55
- 238000000034 method Methods 0.000 claims description 21
- 230000005540 biological transmission Effects 0.000 claims description 13
- 230000008878 coupling Effects 0.000 claims description 10
- 238000010168 coupling process Methods 0.000 claims description 10
- 238000005859 coupling reaction Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 7
- 101000965313 Legionella pneumophila subsp. pneumophila (strain Philadelphia 1 / ATCC 33152 / DSM 7513) Aconitate hydratase A Proteins 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 6
- 238000001228 spectrum Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000009432 framing Methods 0.000 claims description 4
- 102220605052 Histone H4-like protein type G_S61A_mutation Human genes 0.000 claims description 3
- 101000642268 Homo sapiens Speckle-type POZ protein Proteins 0.000 claims description 3
- 235000015429 Mirabilis expansa Nutrition 0.000 claims description 3
- 244000294411 Mirabilis expansa Species 0.000 claims description 3
- 102220479869 Protein FAM180A_S62A_mutation Human genes 0.000 claims description 3
- 102220579099 Protein TSSC4_S62D_mutation Human genes 0.000 claims description 3
- 102100036422 Speckle-type POZ protein Human genes 0.000 claims description 3
- 102220347773 c.185C>G Human genes 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 229940079593 drug Drugs 0.000 claims description 3
- 239000003814 drug Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 235000013536 miso Nutrition 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 230000005855 radiation Effects 0.000 claims description 3
- 230000000241 respiratory effect Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000003595 spectral effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims 1
- 239000010931 gold Substances 0.000 claims 1
- 229910052737 gold Inorganic materials 0.000 claims 1
- NJPPVKZQTLUDBO-UHFFFAOYSA-N novaluron Chemical compound C1=C(Cl)C(OC(F)(F)C(OC(F)(F)F)F)=CC=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F NJPPVKZQTLUDBO-UHFFFAOYSA-N 0.000 claims 1
- 230000003993 interaction Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004377 microelectronic Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Abstract
The present invention relates to a kind of for the voice hospital guide robot head neck device of factories and miness hospital and control；Factories and miness hospital is solved due to lacking professional hospital guide personnel, divides the problem of examining low efficiency；The neck part device includes head device, neck device and control system, and the head device is set to and neck device top；The head device includes head construction, mouth action mechanism, voice module；The neck device includes neck support and cervical motion mechanism；The control system is unstable for hospital's ambient noise to the control of voice module, the larger situation of intensity fluctuation, traditional short-time energy is substituted as characteristic quantity using low frequency energy, improve the accuracy of speech recognition under complicated noise, main hospital department road voice guidance work is completed according to speech recognition content, and while voice dialogue, robot realizes anthropomorphic mouth action, and the movement of neck pitching revolution, increase the interaction capabilities of hospital guide robot.
Description
Technical field
The invention belongs to intellect service robot field more particularly to a kind of voice hospital guide robots for factories and miness hospital
Neck part device and control.
Background technique
With the development of China's intelligence manufacture industry and the continuous improvement of scientific and technological level, intellect service robot is daily
Application in life is also more and more extensive, and intellect service robot has been able to be applied to tour guide, medical treatment, welcome etc. each at present
Field, they can become the following mankind most right-hand man and most intimate partner surely.
The present invention is applied to voice hospital guide's service field in factories and miness hospital, and factories and miness hospital belongs to affiliated hospital, enterprise, scale
Not as good as Grade III Class A hospital and medical worker's amount of flow it is big, the problem of lacking professional hospital guide personnel, cause point to examine low efficiency, this hair
It is bright provide it is a kind of for the voice hospital guide robot head neck device of factories and miness hospital and control, but due under factories and miness hospital scene
Ambient noise it is unstable and intensity has compared with macrorelief so that general speech recognition algorithm is under the lower environment of signal-to-noise ratio,
Vad algorithm based on short-time energy almost can not work normally.
Summary of the invention
The present invention solves the above problem, provide a kind of voice hospital guide robot head neck device for factories and miness hospital and
Control, it can share the medical high operating pressure by stages of factories and miness patient in hospital, improve dividing for hospital and examine efficiency, and in hospital
The voice messaging of high accuracy identification clients, makes rapidly voice according to the voice messaging recognized in complicated noise circumstance
Respond, specifically include voice show the way, department inquiry, expert's brief introduction, while speech response clients, the head device and
Neck device completes the movement of anthropomorphic mouth action and neck pitching, revolution, improve hospital guide robot technology sense and
Intelligence degree.
To solve the above-mentioned problems, the first object of the present invention is to provide a kind of voice hospital guide's machine for factories and miness hospital
Device department of human head and neck device, second is designed to provide a kind of control of voice hospital guide's robot head neck device for factories and miness hospital
Method processed.
First technical solution adopted by the present invention is:
A kind of voice hospital guide's robot head neck device for factories and miness hospital, including head device, neck device and control system
System, the head device are set to and neck device top；
The head device includes head construction, mouth action mechanism, voice module；
The neck device includes neck support, cervical motion mechanism；
The control system based on main control chip, connect with the voice module voice messaging of clients for identification and
The interrogation information of clients is responded, the main control chip is connect with mouth action mechanism and cervical motion mechanism controls, is completed
The movement of anthropomorphic mouth action and neck pitching, revolution.
Further, the head construction includes face's support plate, the first support plate of mouth, the second support plate of mouth；Institute
It states face's support plate to be arranged vertically with the first support plate of mouth and the second support plate of mouth, and is fixedly connected.
Further, the mouth action mechanism include mouth action control module, stepper motor, motor fixed rack,
Metal shaft coupling, mouth transmission bracket and chin component, the stepper motor pass through motor fixed rack and mouth second
Fagging is fixedly connected, and the input terminal of the output shaft and metal shaft coupling of the stepper motor is affixed, the metal shaft coupling it is another
One end is connect with the input terminal of mouth transmission bracket, the end of the mouth transmission bracket and chin member.
Further, PB0, PB1, PB2, PB3 pin of the main control chip of the control system are dynamic with the mouth respectively
Make IN1, IN2, IN3, IN4 pin connection of control module, the positive and negative anodes of the mouth action control module and the power supply of 5V connect
It connecing, output end OUT1, OUT2 of the mouth action control module are connect with the positive and negative anodes input terminal of the stepper motor respectively,
For controlling hospital guide robot mouth action.
Further, the voice module include master control borad, microphone, speech recognition module, speech utterance module, loudspeaker with
And horn bracket, the microphone are fixed in face's support plate, and are connect with the monophonic entrance of speech recognition module, the loudspeaker
Be vertically arranged and be fixedly connected with the first support plate of mouth and the second support plate of mouth by horn bracket, the loudspeaker are just
Cathode is connect with the output positive and negative charge of speech utterance module, the master control borad respectively with speech recognition module, speech utterance mould
The control connection of block, mouth action control module, cervical motion mechanism.
Further, the main control chip of the control system is arranged on master control borad, PA4, PA5 of the main control chip,
PA6, PA7 pin are connect with MISO, MOSI, SCK, NSS pin of speech recognition module respectively to be communicated by SPI protocol, transmits language
Sound identification information, RST, WR, IRQ pin of the speech recognition module respectively with PB12, PB13, PB14 pin of main control chip
Connection；Input terminal of MICP, MICN pin of the speech recognition module as the positive and negative anodes of the microphone, wherein MICP is positive
Pole input terminal, MICN are negative input；SPOP, SPON of the speech recognition module respectively with the IN+ of speech utterance module,
IN- connection, OUT+, OUT- of the speech utterance module are connected respectively at the positive and negative anodes of the loudspeaker for exporting robot
Response voice.
Further, the neck support includes neck metallic support, neck metab, incidence connector.
Further, the cervical motion mechanism include uniaxial metal steering engine, the first metal steering wheel, twin shaft metal steering engine,
Second metal steering wheel；The output shaft of the single shaft metal steering engine is connect with the first metal steering wheel, the first metal steering wheel
Output end be fixed in the card slot of neck metallic support, neck metallic support lower end and twin shaft metal steering engine are affixed, institute
It states the output shaft of twin shaft metal steering engine vertically downward, and is connect with the second metal steering wheel, the second metal steering wheel is fixed
In the card slot of neck metab.
Further, the PB7 pin of the main control chip is connect with the OUT pin of uniaxial metal steering engine；The master control core
The PB8 pin of piece is connect with the OUT pin of twin shaft metal steering engine respectively.
Second technical solution adopted by the present invention is:
Based on a kind of control method that voice hospital guide's robot head neck device for factories and miness hospital is realized, including it is following
Step:
Step S1, the voice messaging of the described microphone acquisition clients converts acoustic waves into audio digital signals and is passed to speech recognition mould
Inside block；
Step S2, the described speech recognition module pre-processes audio digital signals, and present invention employs improve pretreatment
Algorithm improves the accuracy of speech recognition under complicated noise；
Step S3, the described speech recognition module will carry out acoustic feature extraction by pretreated voice signal；
Step S4, show that acoustic feature certain section of voice messaging in identification network belongs to the probability of some delimiter symbol；
Step S5, the decoding that language model and pronunciation dictionary will be passed through by the voice messaging of acoustic model, from candidate text
The character string sequence of maximum probability is found out in sequence, the text results of speech recognition are transferred to master control by final speech recognition module
Chip；
Step S6, corresponding response language is completed in the corresponding instruction of text information that main control chip is recognized according to speech recognition module
Sound and the mouth action and cervical motion for cooperating voice.
Further, improved Preprocessing Algorithm specifically includes in the step S2:
Step S201, audio digital signals are by transmission functionHigh-pass digital filter to radio-frequency head
Divide and aggravated, the influence of removal lip radiation increases the high frequency resolution of voice；
Step S202, framing is carried out to audio digital signals according to the short-term stationarity of voice signal；
Step S203, windowing process is carried out to voice signal, emphasizes the speech waveform near sampling n, weakens the wave of rest part
Shape, the long 25ms of window, it is 10ms that window, which moves, and every frame has 410 sampled points, substitutes general short-time energy as special using low frequency energy
Sign amount, allows the voice signal to be measured to be by the voice signal that a FIR low pass filter obtains, pass through formulaJudge low frequency energy, wherein,Voice signal is detected for band,For FIR low pass filter coefficient,It is the order of filter,It is filtered voice signal, passes through training estimation back
The low frequency energy of scape noise presets the threshold values of low frequency energy, for noise circumstance under hospital's scene is unstable has burst point
The characteristics of spot noise, to the cusp noise for using intermediate value Fuzzy Processing to filter out burst after voice signal adding window, in dimension
SequenceIn, if window size is m, theThe output of a Fuzzy Processing is,It is to be arranged from big to small in window
The value of median afterwards:,, export and obtain the voice of removal cusp
Signal.
Further, acoustic feature described in step S3, which extracts, uses MFCC speech feature extraction technology, specifically includes:
Step S301, Fast Fourier Transform (FFT) (FFT) is carried out to the signal after adding window,,Frequency spectrum is obtained, whereinFor the voice signal of input, N indicates the points of Fourier transformation；
Step S302, by actual frequencySpatial scaling isFrequencyScale, conversion formula are；
Step S303, the spectral coefficient that transformation obtains is filtered with sequence triangular filter, frequency spectrum is smoothed, and
The effect of harmonic carcellation highlights the formant of original voice, reduces operand.
Step S304, the logarithmic energy of each filter group is calculated, formula is,；
Step S305, by energy value obtained in the previous stepThrough discrete cosine transform, obtain MFCC coefficient:,, whereinIt is MFCC coefficient order,It is triangle
The number of filter.
Further, identification network described in step S4 is taken using gauss hybrid models-Hidden Markov Model GMM-HMM
It builds, has the characteristics that training speed is fast, model is small, is easy to transplant, the conditional probability that step S3 obtains MFCC characteristic parameter is defeated
Enter to obtain the probability of the corresponding phoneme of this section of speech frame or syllable into S4.
Further, it is to control voice module by main control chip to complete that voice is responded in step S6, and specific steps are such as
Under:
Step S61A, as described in step S5, the voice messaging recognized is converted into text information and passes through SPI communication by voice module
Agreement is to main control chip, if recognizing: " how to get to is emergency department ", " how to get to is Respiratory Medicine ", " doctor's Cao Xueqin brief introduction " etc. are pre-
If text information, main control chip transmits corresponding ASCII character to the voice module, and waiting buffer area to be sent is null event:
Number is written in while (SPI_I2S_GetFlagStatus (FLASH_SPIx, SPI_I2S_FLAG_TXE)==RESET)
According to register, the data to be written write-in is sent buffer area: SPI_I2S_SendData (FLASH_SPIx, byte)；
Step S61B, after the described voice module receives main control chip corresponding instruction, initialization is by specified sequence setting deposit
Device, starts play position and resets nMP3Pos=0, and a FIF0 register (word every time is written in the MP3 data inside serial Flash
Section) nMp3Pos++, modifies BA, 17 registers, open interrupt allows EX0=1.
Further, cooperate the mouth action of voice and cervical motion Method Of Accomplishment as follows in step S6:
Step S62A, main control chip to the voice module transmit corresponding ASCII character while control mouth action mechanism and
Cervical motion mechanism action；
Step S62B, it initializes: opening peripheral clock, configuration initialisation structures body, calls out structural body initialization function；
Step S62C, pulsewidth is configured by duty ratio, is configured by TIM_SetCompare1 () function and wants motor rotation
Angle；
Step S62D, several angle values are set according to the natural law of mankind's mouth action and cervical motion, makes the hospital guide
The naturally anthropomorphic degree of robot motion is high.
The present invention has the advantages that compared with the existing technology
1, the present invention provides a kind of for the voice hospital guide robot head neck device of factories and miness hospital and control, using a kind of improvement
Audio recognition method, be directed to hospital's voice environment environment complicated and changeable, be different from the general VAD based on short-time energy
Algorithm provides a kind of improved voice activation algorithm based on low frequency energy, significantly reduces the interference of high frequency section energy,
Still the voice messaging of clients more can be accurately detected in the lower situation of signal-to-noise ratio；
2, hospital guide's robot head neck device for mechanical of the present invention is simple for structure generous, occupies little space powerful practical,
The problem of present invention is applied to hospital diagnosis-guiding field, solves factories and miness hospital due to lacking professional hospital guide personnel, point examines low efficiency,
It not only solves the above problem to a certain extent and also improves the overall image of hospital, the demeanor of hospital guide person represents hospital
Image, change people and come into the viewpoint of the robot " pleasnt to the eye but of no use " of life to first；
3, the basic guiding doctor's hospital guide's problem of voice answer-back may be implemented in hospital guide robot of the present invention, including chemically examines process, pays
Expense process, is handled the problems such as into-hospital procedures at department position, and while voice answering, anthropomorphic natural mouth is may be implemented in robot
Portion's movement and cervical motion make clients obtain affectional pleasure, and medical treatment process is convenient, is conducive to the rehabilitation of disease.
Detailed description of the invention
Fig. 1 is apparatus of the present invention main view；
Fig. 2 is apparatus of the present invention rear structure figure；
Fig. 3 is the partial view of mouth action mechanism of the present invention；
Fig. 4 is neck structure drawing of device of the present invention；
Fig. 5 is control system main control chip circuit diagram of the present invention；
Fig. 6 is voice module circuit diagram of the present invention；
Fig. 7 is mouth action control module of the present invention, stepper motor, twin shaft metal steering engine, uniaxial metal steering engine circuit diagram；
Fig. 8 is Whole Work Flow figure of the present invention；
Fig. 9 is speech recognition flow chart of the present invention；
Figure 10 is the improved Preprocessing Algorithm schematic diagram of the present invention.
In figure: head device 1, neck device 2, control system 3, head construction 1-1, mouth action mechanism 1-2, voice mould
Block 1-3, face support plate 1-1A, the first support plate of mouth 1-1B, the second support plate of mouth 1-1C, mouth action control module 1-
2A, stepper motor 1-2B, motor fixed rack 1-2C, metal shaft coupling 1-2D, mouth transmission bracket 1-2E, chin component 1-
2F, master control borad 1-3A, microphone 1-3B, speech recognition module 1-3C, speech utterance module 1-3D, loudspeaker 1-3E, horn bracket 1-
3F, neck support 2-1, cervical motion mechanism 2-2, neck metallic support 2-1A, neck metab 2-1B, incidence connector
2-1C, uniaxial metal steering engine 2-2A, the first metal steering wheel 2-2B, twin shaft metal steering engine 2-2C, the second metal steering wheel 2-2D.
Specific embodiment
Below with reference to attached drawing, the present invention is described in detail.
Specific embodiment one
A kind of voice hospital guide's robot head neck device for factories and miness hospital, as shown in Figure 1 and Figure 2, including head device 1, neck
Part device 2 and control system 3, the head device 1 are set to and 2 top of neck device；
The head device 1 includes head construction 1-1, mouth action mechanism 1-2, voice module 1-3；
The neck device 2 includes neck support 2-1, cervical motion mechanism 2-2；
The control system 3 connect the voice messaging of clients for identification with the voice module 1-3 based on main control chip
And respond the interrogation information of clients, the main control chip and mouth action mechanism 1-2 and cervical motion mechanism 2-2 control
The movement of anthropomorphic mouth action and neck pitching, revolution is completed in connection.
Specific embodiment two
As shown in Figure 1, Figure 2, Figure 3 shows, on the basis of specific embodiment one, the head construction 1-1 includes face's support plate
1-1A, the first support plate of mouth 1-1B, the second support plate of mouth 1-1C；Face's support plate 1-1A and the first support plate of mouth
1-1B and the second support plate of mouth 1-1C are arranged vertically, and are fixedly connected.
The mouth action mechanism 1-2 includes mouth action control module 1-2A, stepper motor 1-2B, motor fixed rack
1-2C, metal shaft coupling 1-2D, mouth transmission bracket 1-2E and chin component 1-2F, the stepper motor 1-2B pass through motor
Fixed bracket 1-2C is fixedly connected with the second support plate of mouth 1-1C, the output shaft and metal shaft coupling of the stepper motor 1-2B
The input terminal of 1-2D is affixed, and the other end of the metal shaft coupling 1-2D is connect with the input terminal of mouth transmission bracket 1-2E, institute
End and the chin component 1-2F for stating mouth transmission bracket 1-2E are affixed.
The voice module 1-3 includes master control borad 1-3A, microphone 1-3B, speech recognition module 1-3C, speech utterance module
1-3D, loudspeaker 1-3E and horn bracket 1-3F, the microphone 1-3B is fixed on face support plate 1-1A, and and speech recognition
The monophonic entrance of module 1-3C connects, and the loudspeaker 1-3E passes through horn bracket 1-3F and mouth the first support plate 1-1B and mouth
Portion the second support plate 1-1C is vertically arranged and is fixedly connected, and the positive and negative anodes of the loudspeaker 1-3E are defeated with speech utterance module 1-3D's
Outlet positive and negative anodes connection, the master control borad 1-3A respectively with speech recognition module 1-3C, speech utterance module 1-3D, mouth action
The control connection of control module 1-2A, cervical motion mechanism 2-2.
Microphone 1-3B is the prior art in present embodiment, and manufacturer is Guangdong Jia Xin microelectronics monopolized store, model
52DB。
Speech recognition module 1-3C is the prior art in present embodiment, and manufacturer is the good letter microelectronics franchise in Guangdong
Shop, model LD3320.
Speech utterance module 1-3D is the prior art in present embodiment, and manufacturer is that Shenzhen ties up the limited public affairs of core science and technology
Department, model PAM8406.
Loudspeaker 1-3E is the prior art in present embodiment, and manufacturer is Shenzhen Wei Xin Science and Technology Ltd., model
For 4 Ω of speaker 5W.
As shown in Figure 2, Figure 4 shows, the neck support 2-1 includes neck metallic support 2-1A, neck metab 2-1B, head
Neck connector 2-1C；The cervical motion mechanism 2-2 includes uniaxial metal steering engine 2-2A, the first metal steering wheel 2-2B, twin shaft
Metal steering engine 2-2C, the second metal steering wheel 2-2D；The output shaft and the first metal steering wheel of the single shaft metal steering engine 2-2A
The output end of 2-2B connection, the first metal steering wheel 2-2B is fixed in the card slot of neck metallic support 2-1A, the neck
The lower end metallic support 2-1A and twin shaft metal steering engine 2-2C are affixed, the output shaft of the twin shaft metal steering engine 2-2C vertically downward,
And connect with the second metal steering wheel 2-2D, the second metal steering wheel 2-2D is fixed on the card slot of neck metab 2-1B
In.
As shown in Fig. 5, Fig. 6, Fig. 7, PB0, PB1, PB2, PB3 pin of the main control chip of the control system 3 respectively with
IN1, IN2, IN3, IN4 pin of the mouth action control module 1-2A connects, the mouth action control module 1-2A's
The power supply of positive and negative anodes and 5V connection, output end OUT1, OUT2 of the mouth action control module 1-2A respectively with the stepping
The positive and negative anodes input terminal of motor 1-2B connects, for controlling hospital guide robot mouth action；The master control of the control system 3
Chip layout on master control borad 1-3A, PA4, PA5, PA6, PA7 pin of the main control chip respectively with speech recognition module 1-
MISO, MOSI, SCK, NSS pin connection of 3C is communicated by SPI protocol, transmits voice recognition information, the speech recognition module
RST, WR, IRQ pin of 1-3C is connect with PB12, PB13, PB14 pin of main control chip respectively；The speech recognition module 1-
Input terminal of MICP, MICN pin of 3C as the positive and negative anodes of the microphone 1-3B, wherein MICP is electrode input end, and MICN is
Negative input；SPOP, SPON of the speech recognition module 1-3C connects with IN+, IN- of speech utterance module 1-3D respectively
It connects, OUT+, OUT- of the speech utterance module 1-3D is connected respectively at the positive and negative anodes of the loudspeaker 1-3E for exporting machine
The response voice of people；The PB7 pin of the main control chip is connect with the OUT pin of uniaxial metal steering engine 2-2A；The master control core
The PB8 pin of piece is connect with the OUT pin of twin shaft metal steering engine 2-2C respectively.
The main control chip model STM32F407.
The course of work:
The microphone 1-3B acquires clients voice messaging, and sound wave is converted to audio digital signals and is passed to language by the microphone 1-3B
Sound identification module 1-3C, by audio digital signals by pre-processing, the high frequency for increasing voice including preemphasis first is differentiated
Secondly rate carries out adding window framing to audio digital signals, using improved Preprocessing Algorithm, specifically passed using low frequency energy substitution
The short-time energy of system carries out end-point detection as characteristic quantity, to audio digital signals, uses MFCC speech feature extraction skill later
Art and the method for GMM-HMM model training identification network carry out the matching of acoustic model, using language model and pronunciation
Audio digital signals are converted into the text information of section by dictionary, realize speech identifying function, speech recognition module 1-3C will be literary
This information passes to main control chip, and main control chip decides whether to continue to execute according to preset recognized list, if being matched to
Literal order in recognized list, then the main control chip controls the corresponding instruction of this voice and controls the speech utterance module 1-
3D exports response voice and exports sound by loudspeaker 1-3E, and mouth action control module and cervical motion mechanism are complete at the same time
Pitching motion is turned round at mouth opening and closing movement and neck, completing complete human-computer dialogue process, (interactive voice, incidence movement are handed over
Mutually).
Specific embodiment three
As shown in Fig. 8, Fig. 9, Figure 10, a kind of controlling party that voice hospital guide's robot head neck device for factories and miness hospital is realized
Method, comprising the following steps:
Step S1, the voice messaging of the described microphone 1-3B acquisition clients converts acoustic waves into audio digital signals and is passed to voice knowledge
Inside other module 1-3C；
Step S2, the described speech recognition module 1-3C pre-processes audio digital signals, pre- present invention employs improving
Processing Algorithm improves the accuracy of speech recognition under complicated noise；
Step S3, the described speech recognition module 1-3C will carry out acoustic feature extraction by pretreated voice signal；
Step S4, show that acoustic feature certain section of voice messaging in identification network belongs to the probability of some delimiter symbol；
Step S5, the decoding that language model and pronunciation dictionary will be passed through by the voice messaging of acoustic model, from candidate text
The character string sequence of maximum probability is found out in sequence, the text results of speech recognition are transferred to by final speech recognition module 1-3C
Main control chip；
Step S6, corresponding return is completed in the corresponding instruction of text information that main control chip is recognized according to speech recognition module 1-3C
It answers voice and cooperates the mouth action and cervical motion of voice.
Further, improved Preprocessing Algorithm specifically includes in the step S2:
Step S201, audio digital signals are by transmission functionHigh-pass digital filter to radio-frequency head
Divide and aggravated, the influence of removal lip radiation increases the high frequency resolution of voice；
Step S202, framing is carried out to audio digital signals according to the short-term stationarity of voice signal；
Step S203, windowing process is carried out to voice signal, emphasizes the speech waveform near sampling n, weakens the wave of rest part
Shape, the long 25ms of window, it is 10ms that window, which moves, and every frame has 410 sampled points, substitutes general short-time energy as special using low frequency energy
Sign amount, allows the voice signal to be measured to be by the voice signal that a FIR low pass filter obtains, pass through formulaJudge low frequency energy, wherein,For band detection voice letter
Number,For FIR low pass filter coefficient,It is the order of filter,It is filtered voice signal, is estimated by training
The low frequency energy of ambient noise is calculated to preset the threshold values of low frequency energy, for noise circumstance under hospital's scene it is unstable have it is prominent
The characteristics of hair tip spot noise, to the cusp noise for using intermediate value Fuzzy Processing to filter out burst after voice signal adding window, one
Dimension sequenceIn, if window size is m, theThe output of a intermediate value Fuzzy Processing is,It is in window from big
The value of median after to minispread:,, export and obtain removal cusp
Voice signal.
Further, acoustic feature described in step S3, which extracts, uses MFCC speech feature extraction technology, specifically includes:
Step S301, Fast Fourier Transform (FFT) (FFT) is carried out to the signal after adding window,,Frequency spectrum is obtained, whereinFor the voice signal of input, N indicates the points of Fourier transformation；
Step S302, by actual frequencySpatial scaling isFrequencyScale, conversion formula are；
Step S303, the spectral coefficient that transformation obtains is filtered with sequence triangular filter, frequency spectrum is smoothed, and
The effect of harmonic carcellation highlights the formant of original voice, reduces operand.
Step S304, the logarithmic energy of each filter group is calculated, formula is,；
Step S305, by energy value obtained in the previous stepThrough discrete cosine transform, obtain MFCC coefficient:,, whereinIt is MFCC coefficient order,It is triangle filter
The number of wave device.
Further, identification network described in step S4 is taken using gauss hybrid models-Hidden Markov Model GMM-HMM
It builds, has the characteristics that training speed is fast, model is small, is easy to transplant, the conditional probability that step S3 obtains MFCC characteristic parameter is defeated
Enter to obtain the probability of the corresponding phoneme of this section of speech frame or syllable into S4.
Further, it is to control voice module by main control chip to complete that voice is responded in step S6, and specific steps are such as
Under:
Step S61A, as described in step S5, the voice messaging recognized is converted into text information and passes through SPI communication by voice module
Agreement is to main control chip, if being matched to: " how to get to is emergency department ", " how to get to is Respiratory Medicine ", " doctor's Cao Xueqin brief introduction " etc. are pre-
If text information, then main control chip transmits corresponding ASCII character to the voice module, waits buffer area to be sent for empty thing
Part: while (SPI_I2S_GetFlagStatus (FLASH_SPIx, SPI_I2S_FLAG_TXE)==RESET), write-in
The data to be written write-in is sent buffer area: SPI_I2S_SendData (FLASH_SPIx, byte) by data register；
Step S61B, after the described voice module receives main control chip corresponding instruction, initialization is by specified sequence setting deposit
Device, starts play position and resets nMP3Pos=0, and a FIF0 register (word every time is written in the MP3 data inside serial Flash
Section) nMp3Pos++, modifies BA, 17 registers, open interrupt allows EX0=1.
Further, cooperate the mouth action of voice and cervical motion Method Of Accomplishment as follows in step S6:
Step S62A, main control chip to the voice module transmit corresponding ASCII character while control mouth action mechanism and
Cervical motion mechanism action；
Step S62B, it initializes: opening peripheral clock, configuration initialisation structures body, calls out structural body initialization function；
Step S62C, pulsewidth is configured by duty ratio, is configured by TIM_SetCompare1 () function and wants motor rotation
Angle；
Step S62D, several angle values are set according to the natural law of mankind's mouth action and cervical motion, makes the hospital guide
The naturally anthropomorphic degree of robot motion is high.
Claims (7)
1. a kind of voice hospital guide's robot head neck device for factories and miness hospital, which is characterized in that including head device (1),
Neck device (2) and control system (3), the head device (1) are set to and neck device (2) top；
The head device (1) includes head construction (1-1), mouth action mechanism (1-2), voice module (1-3)；
The neck device (2) includes neck support (2-1), cervical motion mechanism (2-2)；
The control system (3) connect the voice of clients for identification with the voice module (1-3) based on main control chip
Information and the interrogation information for responding clients, the main control chip and mouth action mechanism (1-2) and cervical motion mechanism
(2-2) control connection, completes the movement of anthropomorphic mouth action and neck pitching, revolution.
2. a kind of voice hospital guide's robot head neck device for factories and miness hospital according to claim 1, which is characterized in that
The head construction (1-1) includes face's support plate (1-1A), the first support plate of mouth (1-1B), the second support plate of mouth (1-
1C)；Face's support plate (1-1A) is arranged vertically with the first support plate of mouth (1-1B) and the second support plate of mouth (1-1C),
And it is fixedly connected.
3. a kind of voice hospital guide's robot head neck device for factories and miness hospital according to claim 1, which is characterized in that
The mouth action mechanism (1-2) includes mouth action control module (1-2A), stepper motor (1-2B), motor fixed rack
(1-2C), metal shaft coupling (1-2D), mouth transmission bracket (1-2E) and chin component (1-2F), the stepper motor (1-
2B) it is fixedly connected by motor fixed rack (1-2C) with the second support plate of mouth (1-1C), the stepper motor (1-2B)
The input terminal of output shaft and metal shaft coupling (1-2D) is affixed, the other end and mouth the transmission branch of the metal shaft coupling (1-2D)
The input terminal of frame (1-2E) connects, and the end of the mouth transmission bracket (1-2E) and chin component (1-2F) are affixed.
4. a kind of voice hospital guide's robot head neck device for factories and miness hospital according to claim 1, which is characterized in that
The voice module (1-3) includes master control borad (1-3A), microphone (1-3B), speech recognition module (1-3C), speech utterance module
(1-3D), loudspeaker (1-3E) and horn bracket (1-3F), the microphone (1-3B) are fixed on face's support plate (1-1A), and
It is connect with the monophonic entrance of speech recognition module (1-3C), the loudspeaker (1-3E) pass through horn bracket (1-3F) and mouth the
One support plate (1-1B) is vertically arranged and is fixedly connected with the second support plate of mouth (1-1C), the positive and negative anodes of the loudspeaker (1-3E)
Connect with the output positive and negative charge of speech utterance module (1-3D), the master control borad (1-3A) respectively with speech recognition module (1-
3C), the control connection of speech utterance module (1-3D), mouth action control module (1-2A), cervical motion mechanism (2-2).
5. a kind of voice hospital guide's robot head neck device for factories and miness hospital according to claim 1, which is characterized in that
The neck support (2-1) includes neck metallic support (2-1A), neck metab (2-1B), incidence connector (2-
1C)；The cervical motion mechanism (2-2) includes uniaxial metal steering engine (2-2A), the first metal steering wheel (2-2B), twin shaft metal rudder
Machine (2-2C), the second metal steering wheel (2-2D)；The output shaft and the first metal steering wheel of the single shaft metal steering engine (2-2A)
(2-2B) connection, the output end of the first metal steering wheel (2-2B) are fixed in the card slot of neck metallic support (2-1A), institute
It states the neck metallic support lower end (2-1A) and twin shaft metal steering engine (2-2C) is affixed, the output of the twin shaft metal steering engine (2-2C)
Axis vertically downward, and is connect with the second metal steering wheel (2-2D), and the second metal steering wheel (2-2D) is fixed on neck gold
In the card slot for belonging to pedestal (2-1B).
6. a kind of voice hospital guide's robot head neck device for factories and miness hospital according to claim 1, which is characterized in that
PB0, PB1, PB2, PB3 pin of the main control chip of the control system (3) respectively with the mouth action control module (1-
IN1, IN2, IN3, IN4 pin connection 2A), the positive and negative anodes of the mouth action control module (1-2A) and the power supply of 5V connect
It connects, output end OUT1, OUT2 of the mouth action control module (1-2A) are positive and negative with the stepper motor (1-2B) respectively
Pole input terminal connection, for controlling hospital guide robot mouth action；The main control chip of the control system (3) is arranged in master
Control plate (1-3A) on, PA4, PA5, PA6, PA7 pin of the main control chip respectively with the MISO of speech recognition module (1-3C),
The connection of MOSI, SCK, NSS pin is communicated by SPI protocol, transmits voice recognition information, the speech recognition module (1-3C)
RST, WR, IRQ pin are connect with PB12, PB13, PB14 pin of main control chip respectively；The speech recognition module (1-3C)
Input terminal of MICP, MICN pin as the positive and negative anodes of the microphone (1-3B), wherein MICP is electrode input end, and MICN is negative
Pole input terminal；SPOP, SPON of the speech recognition module (1-3C) connect with IN+, IN- of speech utterance module (1-3D) respectively
It connects, OUT+, OUT- of the speech utterance module (1-3D) are connected respectively at the positive and negative anodes of the loudspeaker (1-3E) for exporting
The response voice of robot；The PB7 pin of the main control chip is connect with the OUT pin of uniaxial metal steering engine (2-2A)；It is described
The PB8 pin of main control chip is connect with the OUT pin of twin shaft metal steering engine (2-2C) respectively.
7. a kind of based on a kind of any voice hospital guide's robot head neck device for factories and miness hospital of claim 1 to 6
The control method of realization, which comprises the following steps:
Step S1, the voice messaging of described microphone (1-3B) the acquisition clients converts acoustic waves into audio digital signals and is passed to voice
Identification module (1-3C) is internal；
Step S2, the described speech recognition module (1-3C) pre-processes audio digital signals, and present invention employs improve
Preprocessing Algorithm improves the accuracy of speech recognition under complicated noise；
Step S3, the described speech recognition module (1-3C) will carry out acoustic feature extraction by pretreated voice signal；
Step S4, show that acoustic feature certain section of voice messaging in identification network belongs to the probability of some delimiter symbol；
Step S5, the decoding that language model and pronunciation dictionary will be passed through by the voice messaging of acoustic model, from candidate text
The character string sequence of maximum probability is found out in sequence, final speech recognition module (1-3C) transmits the text results of speech recognition
To main control chip；
Step S6, the corresponding instruction of text information that main control chip is recognized according to speech recognition module (1-3C) is completed corresponding
It responds voice and cooperates the mouth action and cervical motion of voice；
Further, improved Preprocessing Algorithm specifically includes in the step S2:
Step S201, audio digital signals are by transmission functionHigh-pass digital filter to high frequency section
It is aggravated, the influence of removal lip radiation increases the high frequency resolution of voice；
Step S202, framing is carried out to audio digital signals according to the short-term stationarity of voice signal；
Step S203, windowing process is carried out to voice signal, emphasizes the speech waveform near sampling n, weakens the wave of rest part
Shape, the long 25ms of window, it is 10ms that window, which moves, and every frame has 410 sampled points, substitutes general short-time energy as special using low frequency energy
Sign amount, allows the voice signal to be measured to be by the voice signal that a FIR low pass filter obtains, judged by formula low
Frequency energy, wherein,Voice signal is detected for band,For FIR low pass filter coefficient, It is the order of filter,It is filtered voice signal, background is estimated by training
The low frequency energy of noise presets the threshold values of low frequency energy, has burst cusp for noise circumstance under hospital's scene is unstable
The characteristics of noise, to the cusp noise for using intermediate value Fuzzy Processing to filter out burst after voice signal adding window, in dimension sequence
ColumnIn, if window size is m, theThe output of a intermediate value Fuzzy Processing is,Be in window from big to small
The value of median after arrangement:,, export and obtain removal cusp
Voice signal；
Further, acoustic feature described in step S3, which extracts, uses MFCC speech feature extraction technology, specifically includes:
Step S301, Fast Fourier Transform (FFT) (FFT) is carried out to the signal after adding window,,Frequency spectrum is obtained, whereinFor the voice signal of input, N indicates the points of Fourier transformation；
Step S302, by actual frequencySpatial scaling isFrequencyScale, conversion formula are；
Step S303, the spectral coefficient that transformation obtains is filtered with sequence triangular filter, frequency spectrum is smoothed, and
The effect of harmonic carcellation highlights the formant of original voice, reduces operand；
Step S304, the logarithmic energy of each filter group is calculated, formula is,；
Step S305, by energy value obtained in the previous stepThrough discrete cosine transform, obtain MFCC coefficient:,, whereinIt is MFCC coefficient order,It is triangle filtering
The number of device；
Further, identification network described in step S4 is built using gauss hybrid models-Hidden Markov Model GMM-HMM, is had
Have that training speed is fast, model is small, is easy to the characteristics of transplanting, the conditional probability that MFCC characteristic parameter obtains is input to S4 by step S3
In obtain the probability of the corresponding phoneme of this section of speech frame or syllable；
Further, it is to control voice module by main control chip to complete that voice is responded in step S6, the specific steps are as follows:
Step S61A, as described in step S5, the voice messaging recognized is converted into text information and passes through SPI communication by voice module
Agreement is to main control chip, if being matched to: " how to get to is emergency department ", " how to get to is Respiratory Medicine ", " doctor's Cao Xueqin brief introduction " etc. are pre-
If text information, then main control chip transmits corresponding ASCII character to the voice module, waits buffer area to be sent for empty thing
Part: while (SPI_I2S_GetFlagStatus (FLASH_SPIx, SPI_I2S_FLAG_TXE)==RESET), write-in
The data to be written write-in is sent buffer area: SPI_I2S_SendData (FLASH_SPIx, byte) by data register；
Step S61B, after the described voice module receives main control chip corresponding instruction, initialization is by specified sequence setting deposit
Device, starts play position and resets nMP3Pos=0, and a FIF0 register (word every time is written in the MP3 data inside serial Flash
Section) nMp3Pos++, modifies BA, 17 registers, open interrupt allows EX0=1；
Further, cooperate the mouth action of voice and cervical motion Method Of Accomplishment as follows in step S6:
Step S62A, main control chip to the voice module transmit corresponding ASCII character while control mouth action mechanism and
Cervical motion mechanism action；
Step S62B, it initializes: opening peripheral clock, configuration initialisation structures body, calls out structural body initialization function；
Step S62C, pulsewidth is configured by duty ratio, the angle for wanting motor rotation is configured by TIM_SetCompare1 () function
Degree；
Step S62D, several angle values are set according to the natural law of mankind's mouth action and cervical motion, makes the hospital guide
The naturally anthropomorphic degree of robot motion is high.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910163672.7A CN109822587B (en) | 2019-03-05 | 2019-03-05 | Control method for head and neck device of voice diagnosis guide robot for factory and mine hospitals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910163672.7A CN109822587B (en) | 2019-03-05 | 2019-03-05 | Control method for head and neck device of voice diagnosis guide robot for factory and mine hospitals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109822587A true CN109822587A (en) | 2019-05-31 |
| CN109822587B CN109822587B (en) | 2022-05-31 |
Family
ID=66865298
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910163672.7A Expired - Fee Related CN109822587B (en) | 2019-03-05 | 2019-03-05 | Control method for head and neck device of voice diagnosis guide robot for factory and mine hospitals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109822587B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110931034A (en) * | 2019-11-27 | 2020-03-27 | 深圳市悦尔声学有限公司 | Pickup noise reduction method for built-in earphone of microphone |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS5837765A (en) * | 1981-08-28 | 1983-03-05 | Toshiba Corp | Diagnostic management system |
| JP2008216735A (en) * | 2007-03-06 | 2008-09-18 | Fujitsu Ltd | Reception robot and method of adapting to conversation for reception robot |
| CN106965193A (en) * | 2017-03-31 | 2017-07-21 | 旗瀚科技有限公司 | A kind of intelligent robot diagnosis guiding system |
| JP2018001403A (en) * | 2016-07-07 | 2018-01-11 | 深▲せん▼狗尾草智能科技有限公司Ｓｈｅｎｚｈｅｎ Ｇｏｗｉｌｄ Ｒｏｂｏｔｉｃｓ Ｃｏ．，Ｌｔｄ． | Method, system and robot body for synchronizing voice and virtual operation |
| CN107901046A (en) * | 2017-11-03 | 2018-04-13 | 深圳市易特科信息技术有限公司 | A guide and examine auxiliary robot for hospital |
| CN108942973A (en) * | 2018-09-29 | 2018-12-07 | 哈尔滨理工大学 | Science and technology center's guest-greeting machine department of human head and neck device with temperature and humidity casting function |
-
2019
- 2019-03-05 CN CN201910163672.7A patent/CN109822587B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS5837765A (en) * | 1981-08-28 | 1983-03-05 | Toshiba Corp | Diagnostic management system |
| JP2008216735A (en) * | 2007-03-06 | 2008-09-18 | Fujitsu Ltd | Reception robot and method of adapting to conversation for reception robot |
| JP2018001403A (en) * | 2016-07-07 | 2018-01-11 | 深▲せん▼狗尾草智能科技有限公司Ｓｈｅｎｚｈｅｎ Ｇｏｗｉｌｄ Ｒｏｂｏｔｉｃｓ Ｃｏ．，Ｌｔｄ． | Method, system and robot body for synchronizing voice and virtual operation |
| CN106965193A (en) * | 2017-03-31 | 2017-07-21 | 旗瀚科技有限公司 | A kind of intelligent robot diagnosis guiding system |
| CN107901046A (en) * | 2017-11-03 | 2018-04-13 | 深圳市易特科信息技术有限公司 | A guide and examine auxiliary robot for hospital |
| CN108942973A (en) * | 2018-09-29 | 2018-12-07 | 哈尔滨理工大学 | Science and technology center's guest-greeting machine department of human head and neck device with temperature and humidity casting function |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110931034A (en) * | 2019-11-27 | 2020-03-27 | 深圳市悦尔声学有限公司 | Pickup noise reduction method for built-in earphone of microphone |
| CN110931034B (en) * | 2019-11-27 | 2022-05-24 | 深圳市悦尔声学有限公司 | Pickup noise reduction method for built-in earphone of microphone |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109822587B (en) | 2022-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105575388B (en) | Emotional speech processing | |
| CN103996155A (en) | Intelligent interaction and psychological comfort robot service system | |
| US20150325240A1 (en) | Method and system for speech input | |
| WO2019023877A1 (en) | Specific sound recognition method and device, and storage medium | |
| JPS58130393A (en) | Voice recognition equipment | |
| CN108847234B (en) | Lip language synthesis method and device, electronic equipment and storage medium | |
| CN102543073A (en) | Shanghai dialect phonetic recognition information processing method | |
| CN109036395A (en) | Personalized speaker control method, system, intelligent sound box and storage medium | |
| CN109036437A (en) | Accents recognition method, apparatus, computer installation and computer readable storage medium | |
| Wand et al. | Deep Neural Network Frontend for Continuous EMG-Based Speech Recognition. | |
| CN112837401B (en) | Information processing method, device, computer equipment and storage medium | |
| CN109872713A (en) | A kind of voice awakening method and device | |
| CN111554281B (en) | Vehicle-mounted man-machine interaction method for automatically identifying languages, vehicle-mounted terminal and storage medium | |
| CN108831463A (en) | Lip reading synthetic method, device, electronic equipment and storage medium | |
| CN110349565B (en) | Auxiliary pronunciation learning method and system for hearing-impaired people | |
| JP2001166789A (en) | Method and device for voice recognition of chinese using phoneme similarity vector at beginning or end | |
| CN106683676A (en) | Voice recognition system for robot control | |
| US20230298616A1 (en) | System and Method For Identifying Sentiment (Emotions) In A Speech Audio Input with Haptic Output | |
| CN109215634A (en) | A kind of method and its system of more word voice control on-off systems | |
| CN109822587A (en) | It is a kind of for the voice hospital guide robot head neck device of factories and miness hospital and control | |
| Kanabur et al. | An extensive review of feature extraction techniques, challenges and trends in automatic speech recognition | |
| CN109300478A (en) | A kind of auxiliary Interface of person hard of hearing | |
| Zhang et al. | A multi-channel/multi-speaker articulatory database in Mandarin for speech visualization | |
| CN111833878A (en) | Chinese voice interaction non-inductive control system and method based on raspberry Pi edge calculation | |
| Wang et al. | Across-speaker articulatory normalization for speaker-independent silent speech recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220531 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |