CN109998874A - A kind of intelligent blind-guiding device - Google Patents
A kind of intelligent blind-guiding device Download PDFInfo
- Publication number
- CN109998874A CN109998874A CN201910331588.1A CN201910331588A CN109998874A CN 109998874 A CN109998874 A CN 109998874A CN 201910331588 A CN201910331588 A CN 201910331588A CN 109998874 A CN109998874 A CN 109998874A
- Authority
- CN
- China
- Prior art keywords
- main control
- control chip
- sensor
- chip
- connect
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 5
- 230000002093 peripheral effect Effects 0.000 claims abstract description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 210000001367 artery Anatomy 0.000 claims 1
- 210000003462 vein Anatomy 0.000 claims 1
- 230000001771 impaired effect Effects 0.000 abstract description 5
- 230000004888 barrier function Effects 0.000 abstract description 4
- 230000003993 interaction Effects 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000000034 method Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 206010047571 Visual impairment Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000001568 sexual effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
Abstract
The invention discloses a kind of intelligent blind-guiding devices, including voice recognition chip, camera, sensor, main control chip, host computer, control device and chipspeech, the sensor connects main control chip by pulse width modulation module (PWM), analog-digital converter (A/D), universal asynchronous receiving-transmitting transmitter (UART) and Serial Peripheral Interface (SPI) (SPI)；Main control chip is connect by bluetooth unit with host computer, map platform and artificial intelligence platform in host computer feed back mobile route to main control chip, main control chip is connected with control device and chipspeech, control device drives blind-guide device, chipspeech guides travel information by loudspeaker to visually impaired people, solves the problems, such as that visually impaired people goes on a journey, realizes the human-computer interaction with visually impaired people, achieve the effect that avoid barrier, navigator fix, substantially improves the life of visually impaired people.
Description
Technical field
The present invention relates to blind person's guide fields, more particularly to a kind of intelligent blind-guiding device.
Background technique
According to statistics, always there are 16,910,000 visually impaired peoples in China, and due to dysopia, their trip is very inconvenient；By logical
Well-trained seeing-eye dog help guidance is crossed, seeing-eye dog can provide convenience really, but seeing-eye dog rare numbers at present, be far less than
Demand, and a qualified seeing-eye dog needs that a large amount of manpower and material resources is spent to carry out screening training, seeing-eye dog sexual valence for synthesis
It is higher than not.
Along with the rapid development of science and technology, this field of artificial intelligence has obtained universal answer in social various industries
With nowadays scientific and technical development has had developed a variety of blind person's walking aids, and helpful blind person avoids collision and the side of discrimination
To new Electronic Blind Navigation, but Environment Obstacles information finally can only be supplied to blind person with certain visual manner by new Electronic Blind Navigation.
Summary of the invention
The object of the present invention is to provide a kind of intelligent blind-guiding device, solves the problems, such as that blind person goes on a journey, can more be handed over blind person
Mutually, the guidance for realizing multi-function domestic scene, substantially improves blindman.
To solve the above-mentioned problems, the present invention provides a kind of intelligent blind-guiding device, which is characterized in that includes: microphone,
Voice recognition chip, camera, sensor, main control chip, host computer, control device, chipspeech, loudspeaker, the wheat
Gram wind is connect by analog-to-digital conversion device with voice recognition chip；The camera is connected by asynchronous receiving-transmitting transmitter and main control chip
It connects；The sensor includes inductance sensor, UV sensor, rain sensor, ultrasonic sensor and warm and humid sensing
Device, wherein inductance sensor, UV sensor, rain sensor are connect by analog-digital converter with main control chip, ultrasonic wave
Sensor is connect by pulse width modulation module with main control chip, and warm and humid sensor is connect by monobus with main control chip, described
Bluetooth unit is equipped in main control chip, main control chip passes through asynchronous receiving-transmitting transmitter for the bluetooth of bluetooth unit and master system
Unit connection；The main control chip is connect by pulse width modulation module with control device, control device include driving motor with
Steering-engine, main control chip are connect by asynchronous receiving-transmitting transmitter with chipspeech；Chipspeech is put by power
Big device is connect with loudspeaker.
Preferably, the voice recognition chip is LD3320 voice recognition chip, is used for speech recognition, and this is integrated with language
Sound recognition processor and some external circuits, thus can be directly integrated in existing product can be thus achieved speech recognition,
Acoustic control, good in interactive function.
Preferably, the main control chip is equipped with clock chip, and clock chip is connected by Serial Peripheral Interface (SPI) and main control chip
It connects, clock chip is DS1302 clock chip, carries out timing for the time.
Preferably, the camera is OpenMV camera, and openMV carries MicroPython resolver, by embedding
Enter and programmed in formula using Python, when calling directly Find_biobs function, obtains a list, the letter comprising all color lumps
Breath, searches for each color lump using Python, so that it may all information is matched, to realize identification traffic lights and realize avoidance function
Energy.
Preferably, main control chip is KEA128 main control chip, at navigation routine processing, avoidance processing, human-computer interaction
Reason the tasks such as communicates with host computer and is possessed the embedded flash memory of 128KB based on Cortex-M0+ kernel, had outstanding
EMC/ESD compatibility, can adapt to hot environment, radiated emissions are lower, provide expansible, reliable and stable high-performance solution party
Case, to accomplish at low cost, stability is good.
Preferably, the host computer connects Iflytek artificial intelligence platform and Gao De by SDK Software Development Kit
Map open platform, Iflytek artificial intelligence platform is for realizing speech recognition, speech synthesis, the task of semantic understanding；It is high
Positioning, route planning, navigation of the moral map open platform for offer, realize active function of leading the way.
Preferably, the main control chip is equipped with I/O interface and connect respectively with warning light and buzzer, receives for sensor
To sounding an alarm when to obstacle.
Preferably, the chipspeech is SYN6288 chipspeech, is used for speech synthesis, synthetic effect is more
Add nature.
Detailed description of the invention
Fig. 1 is the entire block diagram of intelligent blind-guiding device in the embodiment of the present invention.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.Implement below
Example is not intended to limit the scope of the invention for illustrating the present invention.
In the description of the present invention, it is to be understood that, term " on ", "lower", "left", "right", "top", "bottom" etc. indicate
Orientation or positional relationship be based on the orientation or positional relationship shown in the drawings, be merely for convenience of description the present invention and simplification retouch
It states, rather than the device or element of indication or suggestion meaning must have a particular orientation, be constructed and operated in a specific orientation,
Therefore it is not considered as limiting the invention.
As shown in connection with fig. 1, a kind of intelligent blind-guiding device for schematically showing the embodiment of the present invention includes: Mike
Wind 1, voice recognition chip 2, camera 3, sensor 4, main control chip 5, host computer 6, control device 7, chipspeech 8,
Loudspeaker 9, the microphone 1 are connect by analog-to-digital conversion device with voice recognition chip 2；The voice recognition chip 2 is for real-time
Monitoring data are handled when receiving order data then being sent back to main control chip 5, and it is concurrent that main control chip 5 handles voice recognition commands
It returns speech synthesis and blind person carries out human-computer interaction；3 module of camera is used to detect red green equal lamps and detection barrier, camera shooting
First 3 are connect by asynchronous receiving-transmitting transmitter with main control chip 5；The sensor 4 includes inductance sensor 401, ultraviolet light biography
Sensor 402, rain sensor 403, ultrasonic sensor 404, warm and humid sensor 405, inductance sensor 401 are default for incuding
Electromagnetism copper wire feedback on road is used for main control chip 5, UV sensor 402 and warm and humid sensor 405 to environment
Detection converts measurable electric signal feedback to main control chip 5 for ultraviolet light and temperature and moisture signals, rain sensor 403 and super
Sonic sensor 404 is for being converted into electric signal feedback to main control chip 5 when detecting ponding or obstacle；Wherein, the biography
In sensor 4, inductance sensor 401, UV sensor 402, rain sensor 403 pass through analog-digital converter and main control chip 5
Connection, ultrasonic sensor 404 connects by pulse width modulation module with main control chip 5, warm and humid sensor 405 pass through monobus and
Main control chip 5 connects；The main control chip 5 is logical for navigation routine processing, avoidance processing, man-machine dialogue system and host computer 6
The tasks such as letter, are equipped with bluetooth unit 10 in main control chip 5, main control chip 5 by asynchronous receiving-transmitting transmitter by bluetooth unit 10 with
The bluetooth unit 11 of 6 system of host computer connects；The host computer 6 interacts position with main control chip 5 for connecting network platform realization
Confidence breath and navigation information；The control device 7 includes driving motor 701 and steering-engine 702, the shifting for blind-guide device
Dynamic, control device 7 includes driving motor 701 and steering-engine 702, is connect by pulse width modulation module with main control chip 5；Institute
It states chipspeech 8 and is converted into speech signal output for receiving the signal of main control chip 5, chipspeech 8 passes through different
Step receiving-transmitting transmitter is connect with main control chip 5；Loudspeaker 9 are connect by power amplifier with chipspeech 8, for voice
The information for conveying main control chip 5 to export.
Blind person says the position for needing to reach by microphone 1, and microphone 1 is transferred signals to by analog-to-digital conversion device
LD3320 voice recognition chip 2, LD3320 speech chip real-time monitoring signal, LD3320 speech chip turn the voice received
Change text data into and send back to the processing of KEA128 main control chip 5, KEA128 main control chip 5 by predefined grammer with receive
Data are matched, and the bluetooth unit 11 of host computer 6 is then connected to by bluetooth unit 10, and 6 system of host computer is soft by SDK
Part development kit connects Amap open platform, and after the success of Amap path planning, specific route result is dealt into
On the machine 6 of position, path is dealt into main control chip 5 by bluetooth 11 and handled by host computer 6, the processing of main control chip 5
NaviInfo.getlconType () function obtains control signal, drives to driving motor 701 and steering-engine 702, together
When, main control chip 5 sends the signal to SYN6288 chipspeech 8 by asynchronous receiving-transmitting transmitter, and SYN6288 voice closes
At chip 8 by power amplifier transfer to loudspeaker 9, voice feedback is carried out to user；
During blind-guide device guides blind person to destination, the OpenCV camera 3 in equipment can detect red green section
Wherein the color of lamp still walks on to determine whether to stand to wait, while transmitting a signal to main control chip 5, and control voice closes
Voice reminder is carried out at chip 8；
OpenCV camera 3 and ultrasonic sensor 404 detect barrier, meeting feedback signal to master control in traveling process
Chip 5, main control chip 5 controls driving motor 701 and steering-engine 702 is realized and detoured；
In traveling process ultrasonic sensor 404 and rain sensor 403 encounter detect barrier or ponding when,
Electric signal can be fed back to control chipspeech 8 to main control chip 5 and carry out voice reminder, and trigger warning light 13 and buzzer
14 remind pedestrian.
Main control chip 5 is additionally provided with clock chip 12, and clock chip 12 is connect by Serial Peripheral Interface (SPI) with main control chip 5,
Clock chip 12 is DS1302 clock chip, can provide real-time temporal information, clocking capability to blind person.
DTH11 Temperature Humidity Sensor, UV sensor 402 when blind person's trip can detect temperature and humidity around, ultraviolet
The information such as line intensity are conveyed by chipspeech 8.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, without departing from the technical principles of the invention, several improvement and replacement can also be made, these are improved and replacement
Also it should be regarded as protection scope of the present invention.
Claims (8)
1. a kind of intelligent blind-guiding device, which is characterized in that include: microphone (1), voice recognition chip (2), camera (3),
Sensor (4), main control chip (5), host computer (6), control device (7), chipspeech (8), loudspeaker (9), the Mike
Wind (1) is connect by analog-to-digital conversion device with voice recognition chip (2)；The camera (3) passes through asynchronous receiving-transmitting transmitter and master
Control chip (5) connection；The sensor (4) includes inductance sensor (401), UV sensor (402), rain sensor
(403), ultrasonic sensor (404) and warm and humid sensor (405), wherein inductance sensor (401), UV sensor
(402), rain sensor (403) is connect by analog-digital converter with main control chip (5), and ultrasonic sensor (404) passes through arteries and veins
Wide modulation module is connect with main control chip (5), and warm and humid sensor (405) is connect by monobus with main control chip (5)；The master
Control chip (5) in be equipped with bluetooth unit (10), main control chip (5) by asynchronous receiving-transmitting transmitter by bluetooth unit (10) with it is upper
The bluetooth unit (11) of machine (6) system connects；The control device (7) includes driving motor (701) and steering-engine
(702), main control chip (5) is connect by pulse width modulation module with control device (7)；Main control chip (5) is passed by asynchronous receiving-transmitting
Defeated device is connect with chipspeech (8)；Chipspeech (8) is connect by power amplifier with loudspeaker (9).
2. intelligent blind-guiding device according to claim 1, which is characterized in that the voice recognition chip (2) is LD3320
Voice recognition chip.
3. intelligent blind-guiding device according to claim 1, which is characterized in that the main control chip (5) is equipped with clock chip
(12), clock chip (12) is connect by Serial Peripheral Interface (SPI) with main control chip (5), and clock chip (12) is DS1302 clock core
Piece.
4. intelligent blind-guiding device according to claim 1, which is characterized in that the camera (3) is OpenCV camera.
5. intelligent blind-guiding device according to claim 1, which is characterized in that the main control chip (5) is KEA128 master control
Chip.
6. intelligent blind-guiding device according to claim 1, which is characterized in that the host computer (6) passes through SDK software development
Kit connects Iflytek artificial intelligence platform and Amap open platform.
7. intelligent blind-guiding device according to claim 1, which is characterized in that the main control chip (5) is equipped with I/O interface point
It is not connect with warning light (13) and buzzer (14).
8. intelligent blind-guiding device according to claim 1, which is characterized in that the chipspeech (8) is SYN6288
Chipspeech.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910331588.1A CN109998874A (en) | 2019-04-24 | 2019-04-24 | A kind of intelligent blind-guiding device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910331588.1A CN109998874A (en) | 2019-04-24 | 2019-04-24 | A kind of intelligent blind-guiding device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109998874A true CN109998874A (en) | 2019-07-12 |
Family
ID=67174118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910331588.1A Pending CN109998874A (en) | 2019-04-24 | 2019-04-24 | A kind of intelligent blind-guiding device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109998874A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110680686A (en) * | 2019-11-06 | 2020-01-14 | 青岛港湾职业技术学院 | Intelligent voice blind guiding system based on AI open platform and use method |
| CN111281760A (en) * | 2020-03-17 | 2020-06-16 | 罗芳芳 | Auxiliary walking training device for nursing old people |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103126862A (en) * | 2013-02-04 | 2013-06-05 | 江苏科技大学 | Outdoor blind guiding robot based on global position system (GPS), general packet radio service (GPRS) and radio frequency identification devices (RFID) and navigational positioning method |
| CN204170056U (en) * | 2014-10-10 | 2015-02-25 | 北京农业职业学院 | A kind of embedded intelligence apparatus for guiding blind |
| CN205339481U (en) * | 2016-01-05 | 2016-06-29 | 燕山大学 | Blind appearance is led to wearable intelligence |
| CN106214436A (en) * | 2016-07-22 | 2016-12-14 | 上海师范大学 | A kind of intelligent blind guiding system based on mobile phone terminal and blind-guiding method thereof |
| CN206102931U (en) * | 2016-07-22 | 2017-04-19 | 西北师范大学 | Supplementary glasses of blind person's intelligence pronunciation |
| CN107374921A (en) * | 2017-07-20 | 2017-11-24 | 安顺学院 | A kind of blind man navigation assistant devices based on internet |
| CN107822840A (en) * | 2017-12-04 | 2018-03-23 | 安徽理工大学 | A kind of intelligent blind-guiding crutch based on dsp chip |
| US20180164849A1 (en) * | 2016-12-09 | 2018-06-14 | BeeInventor Limited | Integrated intelligent head-mounted device |
| CN109481248A (en) * | 2018-12-26 | 2019-03-19 | 浙江师范大学 | A kind of smart guide glasses |
| CN210186001U (en) * | 2019-04-24 | 2020-03-27 | 广东海洋大学 | Intelligent blind guiding device |
-
2019
- 2019-04-24 CN CN201910331588.1A patent/CN109998874A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103126862A (en) * | 2013-02-04 | 2013-06-05 | 江苏科技大学 | Outdoor blind guiding robot based on global position system (GPS), general packet radio service (GPRS) and radio frequency identification devices (RFID) and navigational positioning method |
| CN204170056U (en) * | 2014-10-10 | 2015-02-25 | 北京农业职业学院 | A kind of embedded intelligence apparatus for guiding blind |
| CN205339481U (en) * | 2016-01-05 | 2016-06-29 | 燕山大学 | Blind appearance is led to wearable intelligence |
| CN106214436A (en) * | 2016-07-22 | 2016-12-14 | 上海师范大学 | A kind of intelligent blind guiding system based on mobile phone terminal and blind-guiding method thereof |
| CN206102931U (en) * | 2016-07-22 | 2017-04-19 | 西北师范大学 | Supplementary glasses of blind person's intelligence pronunciation |
| US20180164849A1 (en) * | 2016-12-09 | 2018-06-14 | BeeInventor Limited | Integrated intelligent head-mounted device |
| CN107374921A (en) * | 2017-07-20 | 2017-11-24 | 安顺学院 | A kind of blind man navigation assistant devices based on internet |
| CN107822840A (en) * | 2017-12-04 | 2018-03-23 | 安徽理工大学 | A kind of intelligent blind-guiding crutch based on dsp chip |
| CN109481248A (en) * | 2018-12-26 | 2019-03-19 | 浙江师范大学 | A kind of smart guide glasses |
| CN210186001U (en) * | 2019-04-24 | 2020-03-27 | 广东海洋大学 | Intelligent blind guiding device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110680686A (en) * | 2019-11-06 | 2020-01-14 | 青岛港湾职业技术学院 | Intelligent voice blind guiding system based on AI open platform and use method |
| CN111281760A (en) * | 2020-03-17 | 2020-06-16 | 罗芳芳 | Auxiliary walking training device for nursing old people |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN205494329U (en) | Intelligence is saved oneself and is led blind walking stick | |
| CN204484697U (en) | Intelligent blind-guiding crutch | |
| CN105058393A (en) | Guest greeting robot | |
| CN109571499A (en) | A kind of intelligent navigation leads robot and its implementation | |
| CN202409427U (en) | Portable intelligent electronic blind guide instrument | |
| CN109998874A (en) | A kind of intelligent blind-guiding device | |
| CN204446522U (en) | Based on hyperacoustic intelligent blind-guiding device | |
| CN205507543U (en) | Intelligent vehicle control system | |
| CN110478206B (en) | Intelligent blind guiding system and equipment | |
| CN105640747A (en) | Intelligent blind guiding system | |
| CN107962573A (en) | Accompany humanoid robot and robot control method | |
| CN103236156A (en) | Blind guide system based on traffic signal indicating lamp | |
| CN108245387A (en) | A kind of intelligent blind guiding system based on wearable device | |
| CN210186001U (en) | Intelligent blind guiding device | |
| CN108446026A (en) | A kind of bootstrap technique, guiding equipment and a kind of medium based on augmented reality | |
| CN107328408A (en) | Machine dog for guiding blind | |
| WO2022095288A1 (en) | Interactive smart white cane and zebra crossing | |
| CN207132903U (en) | Machine dog for guiding blind | |
| CN202459187U (en) | A blind person guide device prompting pedestrians about making way for the blind | |
| CN107765613A (en) | A kind of dcs of intelligent robot | |
| CN205459718U (en) | Multifunctional blind guiding cane | |
| CN213157517U (en) | Intelligent visual identification blind-guiding walking stick based on Internet of things | |
| CN116869408A (en) | Interaction method and electronic equipment | |
| CN106157685A (en) | One seeks parking stall mobile phone software platform | |
| CN105957359A (en) | Face recognition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |