CN109925171B - Multifunctional navigation equipment - Google Patents

Multifunctional navigation equipment Download PDF

Info

Publication number
CN109925171B
CN109925171B CN201910312761.3A CN201910312761A CN109925171B CN 109925171 B CN109925171 B CN 109925171B CN 201910312761 A CN201910312761 A CN 201910312761A CN 109925171 B CN109925171 B CN 109925171B
Authority
CN
China
Prior art keywords
control module
information
module
data
backpack
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201910312761.3A
Other languages
Chinese (zh)
Other versions
CN109925171A (en
Inventor
何迈
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan University
Original Assignee
Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan University filed Critical Sichuan University
Priority to CN201910312761.3A priority Critical patent/CN109925171B/en
Publication of CN109925171A publication Critical patent/CN109925171A/en
Application granted granted Critical
Publication of CN109925171B publication Critical patent/CN109925171B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention discloses multifunctional navigation equipment which comprises a server, a walking stick, a backpack storage box, a walking stick storage column and an RFID active tag, wherein the server is connected with the walking stick storage column; the walking stick is provided with a first battery, a first control module, a first wireless communication module, a first gyroscope, an RFID recognizer, a first loudspeaker, a binocular camera, an ultrasonic detector, a lighting lamp, a light sensor, a physical feedback module, a first wireless charging coil and a first microphone; and a second control module, a second microphone, a GPS module, a second battery, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker, a memory, a wired charger, a second wireless charging coil and a warning lamp are arranged in the backpack. The invention can carry out indoor navigation and outdoor navigation, can warn barriers, can also foretell road information to users in advance, reduce the route exploring time of the users, and can avoid the users from searching for sockets by wireless charging.

Description

Multifunctional navigation equipment
Technical Field
The invention relates to the field of travel auxiliary equipment, in particular to multifunctional navigation equipment.
Background
The blind people are special groups, the blind people lose visual function due to congenital or acquired reasons, and daily life, particularly travel, is greatly restricted. The world health organization estimates that 4000 to 4500 thousands of blind people exist in the world, and nearly 1300 thousands of patients with visual disabilities exist in China, wherein about 550 thousands of blind people exist, and about 750 thousands of low-vision patients exist. In real life, most blind people assist walking through ordinary bamboo, wooden cane or ordinary blind cane, and this kind of blind cane is only an ordinary stock, and it can only improve passive discernment ability through night vision luminous paint etc. to road surface and aerial dangerous obstacle restriction in the scope that the blind cane can touch through surveying local obstacle, even present ordinary blind cane improves passive discernment ability through night vision luminous paint, but the blind person can't know voluntarily whether by other people effective discernment.
The existing walking stick does not have an active searching function, and once a user forgets the position of the walking stick, the walking stick is difficult to search. The existing walking stick with the navigation function usually concentrates electronic equipment on the walking stick, the self volume capacity of the walking stick is limited, the self battery capacity is small, the endurance is poor, the excessive electronic equipment is integrated on the walking stick, the weight of the walking stick is increased, the load of the hand of a user is increased, and the normal use of the user is greatly influenced. In addition, the walking stick with the navigation function depends on the GPS for navigation, and the fact that the common navigation equipment is installed in the walking stick cannot meet the indoor navigation requirement and the short-distance navigation requirement of a fixed place, so that a user still has a lot of inconvenience when using the walking stick.
Disclosure of Invention
Aiming at the defects in the prior art, the multifunctional navigation equipment provided by the invention solves the problem that the existing navigation walking stick is inconvenient to use.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
the multifunctional navigation equipment comprises a server, a walking stick, a backpack, an RFID active tag, a walking stick storage column and a backpack storage box; the walking stick comprises a stick body and a handle; a first battery, a first control module, a first wireless communication module, a first gyroscope, an RFID identifier and a first loudspeaker are arranged in the stick body, and a binocular camera, an ultrasonic detector, an illuminating lamp and a light sensor are arranged at the front end of the stick body; a first wireless charging coil is arranged at the bottom of the crutch body and is connected with a first battery; the handle is provided with a physical feedback module and a first microphone; the first control module is respectively connected with the first battery, the physical feedback module, the first wireless communication module, the first loudspeaker, the binocular camera, the ultrasonic detector, the illuminating lamp, the light sensor, the first gyroscope and the RFID identifier;
the backpack comprises a control box, wherein a second control module, a second microphone, a GPS module, a second battery, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker, a memory and a wired charger are arranged in the control box; a warning lamp is arranged at the rear end of the control box; a second wireless charging coil is arranged at the bottom of the control box and connected with a second battery; the second control module is respectively connected with a second battery, a second microphone, a GPS module, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker, a memory, a warning lamp and a wired charger;
the bottom of the backpack storage box is provided with a second wireless charger for wirelessly charging a second battery when the backpack is stored;
the first wireless charger is arranged at the bottom of the walking stick storage column and used for wirelessly charging the first battery when the walking stick is stored;
the RFID active tag is used for calibrating a placing place;
a first battery for powering the electronics in the hand wand;
the physical feedback module is used for feeding back information to a user through a physical contact mode according to the signal of the first control module;
the first wireless communication module is used for being in wireless connection with the second wireless communication module so as to enable the first control module to be in wireless communication with the second control module;
the RFID recognizer is used for recognizing the information of the RFID active tag and sending the acquired information to the first control module;
the first gyroscope is used for acquiring pose information of the walking stick;
the first loudspeaker is used for broadcasting sound information according to the signal of the first control module;
the binocular camera is used for acquiring the three-dimensional space information and the image information in front of the walking stick and sending the three-dimensional space information and the image information to the first control module;
the ultrasonic detector is used for acquiring distance information between the ultrasonic detector and an obstacle and sending the acquired information to the first control module;
the illuminating lamp is used for illuminating the environment in front of the walking stick according to the signal of the first control module;
the light sensor is used for acquiring light and shade information of the surrounding environment and sending the acquired information to the first control module;
the first microphone is used for acquiring the received sound information and sending the sound information to the first control module;
the first control module is used for analyzing an instruction contained in the information from the first microphone and controlling a device on the walking stick to perform corresponding operation, and when the instruction does not point to the device on the walking stick, the instruction is sent to the second control module; the RFID tag is used for analyzing the information from the RFID identifier, acquiring the distance between the RFID tag and the corresponding RFID active tag and sending the distance information to the second control module; the first loudspeaker is used for controlling the first loudspeaker to broadcast the sound information; the physical feedback module is used for controlling the physical feedback module to work according to the data of the second control module; the first control module is used for sending the data of the binocular camera and the data of the first gyroscope to the second control module after preprocessing; the device is used for acquiring data of the ultrasonic detector and acquiring the distance between the ultrasonic detector and surrounding obstacles; the system is used for acquiring data of the light sensor and controlling the work of the illuminating lamp according to the data; the device is used for acquiring data from the second control module and carrying out corresponding operation on the device on the walking stick according to the data;
the second microphone is used for acquiring the received sound information and sending the sound information to the second control module;
the GPS module is used for acquiring the position of the backpack and sending the position to the second control module;
a second battery for supplying electric power;
the second wireless communication module is used for being in wireless connection with the first wireless communication module so as to enable the second control module to be in wireless communication with the first control module;
the third wireless communication module is used for connecting with the server and accessing the Internet;
the second gyroscope is used for acquiring pose information of the backpack and sending the information to the second control module;
the second loudspeaker is used for broadcasting sound information according to the signal of the second control module;
a memory for storing data;
the warning lamp is used for giving out light warning to surrounding people;
the wired charger is used for accessing commercial power and charging the first battery and the second battery;
the second control module is used for acquiring the data from the first control module and judging whether the instruction information in the data points to the electronic device in the control box or not, if so, corresponding operation is carried out, and if not, the instruction information is sent to the server; the data storage module is used for storing data from the first control module into the memory; the walking stick position early warning system is used for acquiring road sign information, the position of the environment where the walking stick is located and road surface condition information according to data of the binocular camera, correcting the position according to data of the first gyroscope, and performing real-time traffic early warning according to the data after the position correction and the road surface condition information; the RFID-based navigation planning system is used for performing RFID-based navigation planning according to a user instruction, distance information obtained through the RFID identifier and the arrangement position of the RFID active tag; the electronic device is used for analyzing an instruction contained in the information from the second microphone, judging whether the instruction points to the electronic device in the control box or the walking stick, if so, carrying out corresponding operation on the electronic device, and otherwise, sending the instruction to the server; the backpack positioning system is used for positioning the backpack according to the data of the GPS module; the system comprises a server, a backpack and a navigation device, wherein the server is used for acquiring the pose information of the backpack and the working state of the navigation device; the second loudspeaker is used for controlling the second loudspeaker to broadcast the sound information; the data reading device is used for writing data generated when the navigation device works into the memory and reading data prestored in the memory; the warning lamp is used for controlling the warning lamp to work; the voice navigation system is used for carrying out voice navigation according to navigation data from the server;
the server is used for planning a navigation route on line according to the position information and the destination information of the backpack and sending navigation data to the second control module; and analyzing the instruction in the voice data on line, acquiring an audio resource corresponding to the instruction in the internet, sending the audio resource to the second control module, and playing the audio resource through the first loudspeaker and/or the second loudspeaker.
Furthermore, an earphone storage box with wireless charging is arranged on the side surface of the backpack, and a wireless charging type Bluetooth earphone is placed in the wireless charging earphone storage box; the charging component of the earphone storage box with wireless charging is connected with the second battery; and a Bluetooth module is also arranged in the control box and is connected with the second control module.
Furthermore, the side surface of the backpack is also provided with a sound channel switch connected with the second control module, and the sound channel switch is used for controlling the audio to be output by the first loudspeaker and/or the second loudspeaker and/or the wireless charging type Bluetooth headset.
Further, the first wireless communication module and the second wireless communication module are both WiFi modules.
Further, the third wireless communication module is a 5G module and/or a 4G module.
Further, a spring wire is arranged between the walking stick and the backpack.
Furthermore, a clamping sleeve for hanging the walking stick is arranged on the side surface of the backpack.
Furthermore, the control box is arranged at the lower end of the backpack, and a hollow chamber positioned at the upper end of the control box is arranged in the backpack.
Further, the lower extreme of control box is provided with the radiator.
Furthermore, the physical feedback module comprises an m multiplied by n lattice which is composed of at least 9 physical feedback units, each physical feedback unit comprises a movable conduit, and a permanent magnet is movably arranged in the movable conduit; one end of the movable conduit is sealed with a seal made of elastic material, the other end of the movable conduit is provided with an iron block, and the sealed end of the movable conduit is a physical feedback end; the iron block is wound with a coil connected with the first control module, and the coil generates a magnetic field repelling the permanent magnet when being electrified, so that the permanent magnet moves towards the iron block when the physical feedback unit is powered off, and the permanent magnet moves towards the seal and impacts the seal when the physical feedback unit is powered on; wherein m and n are both positive integers and are both greater than or equal to 3.
The invention has the beneficial effects that:
1. the invention can calibrate each position by arranging the RFID active tag in an indoor or short-distance environment, and position the environment position by identifying the RFID tag and the signal strength of the tag, thereby realizing navigation in the indoor environment or the short-distance environment. The backpack storage box and the walking stick storage column can be arranged in a bedroom of a user, when the user has a rest, the backpack and the walking stick can be respectively placed into the backpack storage box and the walking stick storage column to be wirelessly charged, the cruising ability of the walking stick and the backpack is ensured, and the embarrassment that the user performs wired charging on the walking stick or the backpack can be reduced by adopting a wireless charging means.
2. The voice control system can acquire the voice instruction of a user through the first microphone and the second microphone, can control the lighting lamp and the warning lamp to be turned on or off in a voice mode, can control the audio data in the Internet to be acquired in a voice mode, and can receive the voice through the Internet to solve the problem of vexation. The navigation data can also be obtained by inquiring the destination information through voice. In addition, the user can wake up the walking stick or the backpack through voice, and the user is helped to roughly position the walking stick or the backpack through the response of the first loudspeaker or the second loudspeaker, so that the walking stick or the backpack is helped to be found.
3. The invention reduces data processing devices in the walking stick, reduces the volume occupation, and can provide larger space for the first battery, thereby effectively improving the cruising ability of the walking stick, reducing the weight of the walking stick and reducing the load of a user. According to the invention, the electronic device with larger power consumption is arranged in the backpack, and the burden of the self weight of the navigation equipment on the hand of a user in the traveling process is reduced in a back mode.
4. According to the walking stick, image fluctuation caused by shaking of the binocular camera in the using process of the walking stick can be corrected through the data of the first gyroscope, so that an image acquired by the binocular camera is kept normal after correction, the environment where the walking stick is located is subjected to three-dimensional modeling through the image data of the binocular camera, space data in a three-dimensional model is broadcasted in a voice mode, the road condition in front of a user is reminded, and a specific advancing direction is broadcasted to the user in combination with a navigation route.
5. The invention can automatically detect the brightness degree of the environment, when the brightness degree is lower than the threshold value, the illuminating lamp is automatically turned on to supplement light, the ultrasonic detector can also detect the distance of the obstacle in front of the walking stick, and the detection of the binocular camera is assisted to give obstacle avoidance reminding.
6. The signal generated by the second gyroscope continuously changes can indicate that the backpack is worn by a user to walk in the movement process, so that the warning lamp can be lightened when the second control module detects that the signal generated by the second gyroscope continuously changes, the rear personnel of the user can be reminded of avoiding, and the safety of the user and other pedestrians is ensured.
7. The sound channel change-over switch provided by the invention can manually select the output mode, so that the audio source interruption caused by actively cutting off the audio output of the first loudspeaker and/or the second loudspeaker when the wireless charging type Bluetooth earphone is connected with the Bluetooth module without wearing the wireless charging type Bluetooth earphone is avoided, and the selected output mode can continuously output audio data.
8. First wireless communication module and second wireless communication module are the wiFi module, can make the wireless connection distance of knapsack and stick reach tens meters, satisfy in ordinary family that knapsack and stick are placed and still keep data connection in different places, are convenient for look for the general position of stick or knapsack through voice command.
9. The spring wire can connect the walking stick and the backpack, so that the walking stick is prevented from being lost when a user stands outdoors; and the clamping sleeve arranged on the side surface of the backpack can also be used for hanging the walking stick, so that the embarrassment that a user cannot release the walking stick when standing is avoided.
10. The empty chamber arranged at the upper end of the control box in the backpack can also store other articles, and the heat radiator is helpful for heat radiation of electronic devices in the control box.
11. The physical feedback module controls the impact of the iron block on the seal by controlling the power-on time of each physical feedback unit, and when a person holds the physical feedback module, the person can feel the impact. The invention controls different physical feedback units to work at different time points in a dot matrix form, can form a feedback route with a direction, controls the physical feedback module according to the forward direction instruction in navigation, and can transmit the forward direction to a user in a physical perception form, thereby reducing the interference of environmental noise on voice broadcasting.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a block circuit diagram of the present invention;
FIG. 3 is a schematic structural diagram of a physical feedback unit;
fig. 4 is a schematic view of the structure of the case for storing a backpack and the pole for storing a stick.
Wherein: 1. a pole body; 2. a handle; 3. a binocular camera; 4. a light sensor; 5. an ultrasonic detector; 6. an illuminating lamp; 7. an RFID identifier; 8. a first speaker; 9. a physical feedback module; 10. a first microphone; 11. a sound channel changeover switch; 12. a second speaker; 13. a card sleeve; 14. a warning light; 15. a control box; 16. an empty chamber; 17. an earphone storage box with wireless charging function; 18. a movable catheter; 19. a permanent magnet; 20. sealing; 21. an iron block; 22. a coil; 23. a spring wire; 24. a first wireless charging coil; 25. a second wireless charging coil; 26. a backpack storage case; 27. a second wireless charger; 28. a cane storage column; 29. a first wireless charger.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, 2 and 4, the multi-functional navigation device includes a server, a cane, a backpack, an RFID active tag, a cane storage pole 28 and a backpack storage bin 26; the walking stick comprises a stick body 1 and a handle 2; a first battery, a first control module, a first wireless communication module, a first gyroscope, an RFID (radio frequency identification) recognizer 7 and a first loudspeaker 12 are arranged in the stick body 1, and a binocular camera 3, an ultrasonic detector 5, an illuminating lamp 6 and a light sensor 4 are arranged at the front end of the stick body 1; a first wireless charging coil 24 is arranged at the bottom of the crutch body 1, and the first wireless charging coil 24 is connected with a first battery; the handle 2 is provided with a physical feedback module 9 and a first microphone 10; the first control module is respectively connected with a first battery, a physical feedback module 9, a first wireless communication module, a first loudspeaker 8, a binocular camera 3, an ultrasonic detector 5, an illuminating lamp 6, a light sensor 4, a first gyroscope and an RFID (radio frequency identification) recognizer 7;
the backpack comprises a control box 15, wherein a second control module, a second microphone, a GPS module, a second battery, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker 12, a memory and a wired charger are arranged in the control box 15; a warning lamp 14 is arranged at the rear end of the control box 15; (ii) a A second wireless charging coil 25 is arranged at the bottom of the control box 15 and is connected with a second battery; the second control module is respectively connected with a second battery, a second microphone, a GPS module, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker 12, a memory, a warning lamp 14 and a wired charger;
the bottom of the backpack storing box 26 is provided with a second wireless charger 27 for wirelessly charging a second battery when storing the backpack; the bottom of the cane storage column 28 is provided with a first wireless charger 29 for wirelessly charging a first battery when the cane is stored;
the RFID active tag is used for user-defined placement and further calibrating a placement place;
a first battery for powering the electronics in the hand wand;
the physical feedback module 9 is used for feeding back information to a user through a physical contact mode according to the signal of the first control module;
the first wireless communication module is used for being in wireless connection with the second wireless communication module so as to enable the first control module to be in wireless communication with the second control module;
the RFID recognizer 7 is used for recognizing the information of the RFID active tag and sending the acquired information to the first control module;
the first gyroscope is used for acquiring pose information of the walking stick;
the first loudspeaker 8 is used for broadcasting sound information according to the signal of the first control module;
the binocular camera 3 is used for acquiring three-dimensional space information and image information in front of the walking stick and sending the three-dimensional space information and the image information to the first control module;
the ultrasonic detector 5 is used for acquiring distance information between the ultrasonic detector and an obstacle and sending the acquired information to the first control module;
the illuminating lamp 6 is used for illuminating the environment in front of the walking stick according to the signal of the first control module;
the light sensor 4 is used for acquiring light and shade information of the surrounding environment and sending the acquired information to the first control module;
the first microphone 10 is used for acquiring the received sound information and sending the sound information to the first control module;
the first control module is used for analyzing an instruction contained in the information from the first microphone 10 and controlling a device on the walking stick to perform corresponding operation, and when the instruction does not point to the device on the walking stick, the instruction is sent to the second control module; the RFID tag is used for analyzing the information from the RFID identifier 7, acquiring the distance between the RFID tag and the corresponding RFID active tag and sending the distance information to the second control module; for controlling the first loudspeaker 8 to broadcast sound information; the physical feedback module 9 is used for controlling the work according to the data of the second control module; the binocular camera 3 is used for preprocessing data of the binocular camera and data of the first gyroscope and then sending the preprocessed data to the second control module; for acquiring data of the ultrasonic detector 5, acquiring a distance to a surrounding obstacle; the device is used for acquiring data of the light sensor 4 and controlling the work of the illuminating lamp 6 according to the data; the device is used for acquiring data from the second control module and carrying out corresponding operation on the device on the walking stick according to the data;
the second microphone is used for acquiring the received sound information and sending the sound information to the second control module;
the GPS module is used for acquiring the position of the backpack and sending the position to the second control module;
a second battery for supplying electric power;
the second wireless communication module is used for being in wireless connection with the first wireless communication module so as to enable the second control module to be in wireless communication with the first control module;
the third wireless communication module is used for connecting with the server and accessing the Internet;
the second gyroscope is used for acquiring pose information of the backpack and sending the information to the second control module;
the second loudspeaker 12 is used for broadcasting sound information according to the signal of the second control module;
a memory for storing data;
a warning lamp 14 for giving a light warning to surrounding persons;
the wired charger is used for accessing commercial power and charging the first battery and the second battery;
the second control module is used for acquiring the data from the first control module and judging whether the instruction information in the data points to the electronic device in the control box 15 or not, if so, corresponding operation is carried out, and if not, the instruction information is sent to the server; the data storage module is used for storing data from the first control module into the memory; the walking stick position early warning system is used for acquiring road sign information, the position of the environment where the walking stick is located and road surface condition information according to the data of the binocular camera 3, correcting the position according to the data of the first gyroscope, and performing real-time traffic early warning according to the data after position correction and the road surface condition information; the RFID-based navigation planning system is used for performing RFID-based navigation planning according to a user instruction, the distance information obtained by the RFID identifier 7 and the arrangement position of the RFID active tag; the electronic device is used for analyzing an instruction contained in the information from the second microphone, judging whether the instruction points to the electronic device in the control box 15 or the walking stick, if so, carrying out corresponding operation on the electronic device, and otherwise, sending the instruction to the server; the backpack positioning system is used for positioning the backpack according to the data of the GPS module; the system comprises a server, a backpack and a navigation device, wherein the server is used for acquiring the pose information of the backpack and the working state of the navigation device; for controlling the second loudspeaker 12 to broadcast sound information; the data reading device is used for writing data generated when the navigation device works into the memory and reading data prestored in the memory; for controlling the operation of the warning lamp 14; the voice navigation system is used for carrying out voice navigation according to navigation data from the server;
the server is used for planning a navigation route on line according to the position information and the destination information of the backpack and sending navigation data to the second control module; and analyzing the instruction in the voice data on line, acquiring an audio resource corresponding to the instruction in the internet, sending the audio resource to the second control module, and playing the audio resource through the first loudspeaker 8 and/or the second loudspeaker 12.
An earphone storage box 17 with wireless charging is arranged on the side surface of the backpack, and a wireless charging type Bluetooth earphone is placed in the earphone storage box 17 with wireless charging; the charging member of the earphone storage box 17 with wireless charging is connected with the second battery; the control box 15 is also internally provided with a Bluetooth module which is connected with a second control module.
The side of the backpack is further provided with a sound channel switch 11 connected with the second control module, and the sound channel switch 11 is used for controlling the audio to be output by the first loudspeaker 8 and/or the second loudspeaker 12 and/or the wireless charging type Bluetooth headset.
The first wireless communication module and the second wireless communication module are both WiFi modules. The third wireless communication module is a 5G module and/or a 4G module.
A detachable spring wire 23 is arranged between the walking stick and the backpack, and a lead for connecting the first battery and the second battery can be arranged inside the spring wire 23, so that the first battery and the second battery can supplement electric quantity mutually. The side of the backpack is also provided with a cutting sleeve 13 for hanging the walking stick. The control box 15 is provided at the lower end of the backpack, in which a chamber 16 is provided at the upper end of the control box 15. The lower end of the control box 15 is provided with a radiator.
As shown in fig. 3, the physical feedback module 9 includes an m × n lattice composed of at least 9 physical feedback units, each physical feedback unit includes a movable conduit 18, and a permanent magnet 19 is movably disposed in the movable conduit 18; one end of the movable conduit 18 is sealed with a seal 20 made of elastic material, the other end is provided with an iron block 21, and the end of the seal 20 of the movable conduit 18 is a physical feedback end; a coil 22 connected with the first control module is wound on the iron block 21, and the coil 22 generates a magnetic field repelling the permanent magnet 19 when being electrified, so that the permanent magnet 19 moves towards the iron block 21 when the physical feedback unit is powered off, and the permanent magnet 19 moves towards the seal 20 and impacts the seal 20 when the physical feedback unit is powered on; wherein m and n are both positive integers and are both greater than or equal to 3.
In an embodiment of the invention, related personnel can set the RFID active tags in the main living environment of the user, calibrate the positions of different RFID active tags, store the calibrated information into a memory and establish a specific scene navigation model based on the RFID active tags. When the user need navigate the walking in this region, assign the navigation demand through voice command, second control module calls concrete scene navigation model based on RFID initiative label in the memory and establishes the route according to the navigation demand, second control module judges whether the stick is located the data connection limit distance of stick and knapsack probably surpasses when the destination, if then remind the user to take the knapsack through first speaker 8 earlier, otherwise send the navigation circuit to first control module, first control module is after acquireing the navigation route, report the direction of going ahead to the user through first speaker 8. In the navigation process, the RFID recognizer 7 continuously recognizes the information of the RFID active tag, and the second control module updates the position and the navigation route of the walking stick in real time according to the data from the RFID recognizer 7 until the walking stick is located at the destination. In the navigation process, if the connection signal of the walking stick and the backpack is weaker than the threshold value, the first control module reminds the user of the impending disconnection of the walking stick and the backpack through the first loudspeaker 8 and/or the physical feedback module 9 and moves forward cautiously.
When the user needs to take the walking stick and the backpack to go out, the user can issue a voice instruction for searching and waking up the walking stick or the backpack through the first microphone 10 or the second microphone, and when the walking stick or the backpack receives the voice instruction, sound feedback can be carried out, so that the user can conveniently and quickly obtain the walking stick or the backpack. When the user obtains the walking stick but does not receive the sound feedback of the backpack, the user can send a voice instruction for searching the backpack to the walking stick, the first control module sends the searching instruction to the second control module through the first wireless communication module and the second wireless communication module, and the second control module controls the second loudspeaker 12 to send feedback sound. Otherwise, the user can also find the walking stick through the backpack, so that the user can quickly obtain the walking stick and the backpack.
When the user needs to navigate in an environment where no RFID active tag is set up, the navigation apparatus acquires the destination data through the first microphone 10 or the second microphone, and the navigation data is established through the server, when the second control module acquires the navigation data from the server, the first loudspeaker 8, the second loudspeaker 12 and the wireless charging type Bluetooth earphone remind the user to select the output mode of the navigation audio through the sound channel switch 11, and reminds the user of the selected audio output mode by the selected mode voice after using the sound channel change-over switch 11 each time, for example, the audio output mode selected by the user is 'only the wireless charging type bluetooth headset outputs audio', then only through wireless rechargeable bluetooth headset reminds the user that the audio output mode selected is "only wireless rechargeable bluetooth headset outputs audio". The sound channel switch 11 may have a plurality of keys respectively corresponding to at least one audio output mode, or may have a single key for switching the audio output mode by pressing it once. The audio output mode includes seven modes of only outputting the wireless charging type bluetooth headset, only outputting the first loudspeaker 8, only outputting the second loudspeaker 12, outputting the wireless charging type bluetooth headset and the first loudspeaker 8 together, outputting the first loudspeaker 8 and the second loudspeaker 12 together, outputting the wireless charging type bluetooth headset and the second loudspeaker 12 together, and outputting the wireless charging type bluetooth headset, the first loudspeaker 8 and the second loudspeaker 12 together.
In the navigation forward process, the second control module starts the electronic devices in the control box 15 except the wired charger to work, and acquires data of the second gyroscope, data of the GPS module, voice data from the second microphone, data from the server and data from the first control module in real time. The first control module starts the electronic device in the walking stick to work and acquires the data of the first gyroscope, the data of the light sensor 4, the data of the binocular camera 3, the data of the ultrasonic detector 5, the data of the RFID recognizer 7, the voice data from the first microphone 10 and the data from the second control module in real time.
According to the walking stick, image fluctuation caused by shaking of the binocular camera 3 in the walking stick using process is corrected through data of the first gyroscope, so that an image obtained by the binocular camera 3 is kept normal after correction, three-dimensional modeling is conducted on the environment where the walking stick is located through image data of the binocular camera 3, space data in the three-dimensional model is broadcasted in a voice mode, the road conditions in front of a user (such as steps, doorsills, stone piers and the like) are reminded, and the specific advancing direction is broadcasted to the user through a navigation route.
The impact of the iron block 21 on the seal 20 is controlled by controlling the power-on time of each physical feedback unit, and when a person holds the physical feedback module 9, the person can feel the impact. The invention controls different physical feedback units to work at different time points in a dot matrix form, can form a feedback route with a direction, controls the physical feedback module 9 according to the forward direction instruction in navigation, and can transmit the forward direction to a user in a physical perception form, thereby reducing the interference of environmental noise on voice broadcasting. For example, the front intersection is a straight line, when a user arrives at the intersection, the physical feedback units in the middle row of the dot matrix work sequentially from back to front, and a straight-line forward impact line is provided for the user to indicate that the intersection is forward; when turning is needed, the physical feedback units are driven from the rear end of the dot matrix to the left front or the right front together to form an impact path towards the left front or the right front, and the impact path indicates left turning or right turning. Similarly, forming a square-shaped impact path on the lattice may indicate a need for u-turn travel.
When the ultrasonic detector 5 or the second controller detects an obstacle from the image acquired by the binocular camera 3 during traveling, the user is warned in a voice broadcast manner. When a user needs to have a rest and get a stuffy feeling, the audio information in the Internet can be acquired by issuing a voice instruction. The second control module can also identify the road sign information and the bus route information from the image data of the binocular camera 3, and increase the information acquisition amount of the user.
When data fluctuation of the second gyroscope exceeds data of general walking in the navigation process, the accident situation that the user falls down can be shown, the second control module sends an alarm to the server, and the server sends the alarm and the GPS signal to the relatives and friends of the user to give an alarm to the relatives and friends of the user. When the third wireless communication module cannot be connected with the server, the second control module can perform offline navigation according to offline map data prestored in the memory. And when the third wireless communication module is connected with the server, the server online navigation mode is preferentially adopted.
The backpack storing box 26 and the walking stick storing column 28 can be arranged in a bedroom of a user, when the user has a rest, the backpack and the walking stick can be respectively placed into the backpack storing box 26 and the walking stick storing column 28 to be wirelessly charged, the cruising ability of the walking stick and the backpack is ensured, and the embarrassment that the user searches for a socket when the user charges the walking stick or the backpack in a wired mode can be reduced by adopting a wireless charging means.
In conclusion, the invention can carry out indoor navigation and outdoor navigation, and monitor the environment in the navigation process, not only can carry out obstacle early warning, but also can foretell road information to the user in advance, reduce the route exploring time of the user and relieve the load on the hands of the user; the invention can store the walking stick and the backpack through the walking stick storage column 28 and the backpack storage box 26, wirelessly charge the first battery and the second battery, avoid the embarrassment of a user for searching for a plug for charging, and is suitable for blind people and amblyopia groups.

Claims (10)

1. A multi-functional navigation device, characterized by: the system comprises a server, a walking stick, a backpack, an RFID active tag, a walking stick storage column and a backpack storage box; the walking stick comprises a stick body and a handle; the crutch body is internally provided with a first battery, a first control module, a first wireless communication module, a first gyroscope, an RFID (radio frequency identification) recognizer and a first loudspeaker, and the front end of the crutch body is provided with a binocular camera, an ultrasonic detector, an illuminating lamp and a light sensor; a first wireless charging coil is arranged at the bottom of the crutch body and is connected with a first battery; the handle is provided with a physical feedback module and a first microphone; the first control module is respectively connected with a first battery, a physical feedback module, a first wireless communication module, a first loudspeaker, a binocular camera, an ultrasonic detector, a lighting lamp, a light sensor, a first gyroscope and an RFID (radio frequency identification) recognizer;
the backpack comprises a control box, wherein a second control module, a second microphone, a GPS module, a second battery, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker, a memory and a wired charger are arranged in the control box; a warning lamp is arranged at the rear end of the control box; a second wireless charging coil is arranged at the bottom of the control box and connected with a second battery; the second control module is respectively connected with a second battery, a second microphone, a GPS module, a second wireless communication module, a third wireless communication module, a second gyroscope, a second loudspeaker, a memory, a warning lamp and a wired charger;
the bottom of the backpack storage box is provided with a second wireless charger for wirelessly charging a second battery when the backpack is stored;
the first wireless charger is arranged at the bottom of the walking stick storage column and used for wirelessly charging the first battery when the walking stick is stored;
the RFID active tag is used for calibrating a placement place;
the first battery is used for supplying power to the electronic device in the walking stick;
the physical feedback module is used for feeding back information to a user through a physical contact mode according to the signal of the first control module;
the first wireless communication module is used for being in wireless connection with the second wireless communication module so as to enable the first control module to be in wireless communication with the second control module;
the RFID recognizer is used for recognizing the information of the RFID active tag and sending the acquired information to the first control module;
the first gyroscope is used for acquiring pose information of the walking stick;
the first loudspeaker is used for broadcasting sound information according to the signal of the first control module;
the binocular camera is used for acquiring three-dimensional space information and image information in front of the walking stick and sending the three-dimensional space information and the image information to the first control module;
the ultrasonic detector is used for acquiring distance information between the ultrasonic detector and an obstacle and sending the acquired information to the first control module;
the illuminating lamp is used for illuminating the environment in front of the walking stick according to the signal of the first control module;
the light sensor is used for acquiring light and shade information of the surrounding environment and sending the acquired information to the first control module;
the first microphone is used for acquiring the received sound information and sending the sound information to the first control module;
the first control module is used for analyzing an instruction contained in the information from the first microphone and controlling a device on the walking stick to perform corresponding operation, and when the instruction does not point to the device on the walking stick, the instruction is sent to the second control module; the RFID tag is used for analyzing the information from the RFID identifier, acquiring the distance between the RFID tag and the corresponding RFID active tag and sending the distance information to the second control module; the first loudspeaker is used for controlling the first loudspeaker to broadcast the sound information; the physical feedback module is used for controlling the physical feedback module to work according to the data of the second control module; the first control module is used for sending the data of the binocular camera and the data of the first gyroscope to the second control module after preprocessing; the device is used for acquiring data of the ultrasonic detector and acquiring the distance between the ultrasonic detector and surrounding obstacles; the system is used for acquiring data of the light sensor and controlling the work of the illuminating lamp according to the data; the device is used for acquiring data from the second control module and carrying out corresponding operation on the device on the walking stick according to the data;
the second microphone is used for acquiring the received sound information and sending the sound information to the second control module;
the GPS module is used for acquiring the position of the backpack and sending the position to the second control module;
the second battery is used for providing electric energy;
the second wireless communication module is used for being in wireless connection with the first wireless communication module so as to enable the second control module to be in wireless communication with the first control module;
the third wireless communication module is used for connecting with the server and accessing the Internet;
the second gyroscope is used for acquiring pose information of the backpack and sending the information to the second control module;
the second loudspeaker is used for broadcasting sound information according to the signal of the second control module;
the memory is used for storing data;
the warning lamp is used for giving out light warning to people around;
the wired charger is used for accessing commercial power and charging the first battery and the second battery;
the second control module is used for acquiring the data from the first control module and judging whether the instruction information in the data points to the electronic device in the control box or not, if so, corresponding operation is carried out, and if not, the instruction information is sent to the server; the data storage module is used for storing data from the first control module into the memory; the walking stick position early warning system is used for acquiring road sign information, the position of the environment where the walking stick is located and road surface condition information according to data of the binocular camera, correcting the position according to data of the first gyroscope, and performing real-time traffic early warning according to the data after the position correction and the road surface condition information; the RFID-based navigation planning system is used for performing RFID-based navigation planning according to a user instruction, distance information obtained through the RFID identifier and the arrangement position of the RFID active tag; the electronic device is used for analyzing an instruction contained in the information from the second microphone, judging whether the instruction points to the electronic device in the control box or the walking stick, if so, carrying out corresponding operation on the electronic device, and otherwise, sending the instruction to the server; the backpack positioning system is used for positioning the backpack according to the data of the GPS module; the system comprises a server, a backpack and a navigation device, wherein the server is used for acquiring the pose information of the backpack and the working state of the navigation device; the second loudspeaker is used for controlling the second loudspeaker to broadcast the sound information; the data reading device is used for writing data generated when the navigation device works into the memory and reading data prestored in the memory; the warning lamp is used for controlling the warning lamp to work; the voice navigation system is used for carrying out voice navigation according to navigation data from the server;
the server is used for planning a navigation route on line according to the position information and the destination information of the backpack and sending navigation data to the second control module; and analyzing the instruction in the voice data on line, acquiring an audio resource corresponding to the instruction in the internet, sending the audio resource to the second control module, and playing the audio resource through the first loudspeaker and/or the second loudspeaker.
2. Multifunction navigation device according to claim 1, characterized in that: an earphone storage box with wireless charging is arranged on the side face of the backpack, and a wireless charging type Bluetooth earphone is placed in the earphone storage box with wireless charging; the charging component of the earphone storage box with wireless charging is connected with the second battery; and a Bluetooth module is further arranged in the control box and is connected with the second control module.
3. Multifunctional navigation device according to claim 2, characterized in that: the side face of the backpack is also provided with a sound channel change-over switch connected with the second control module, and the sound channel change-over switch is used for controlling audio to be output by the first loudspeaker and/or the second loudspeaker and/or the wireless charging type Bluetooth headset.
4. Multifunction navigation device according to claim 1, characterized in that: the first wireless communication module and the second wireless communication module are both WiFi modules.
5. Multifunction navigation device according to claim 1, characterized in that: the third wireless communication module is a 5G module and/or a 4G module.
6. Multifunction navigation device according to claim 1, characterized in that: and a spring wire is arranged between the walking stick and the backpack.
7. Multifunction navigation device according to claim 1, characterized in that: the side surface of the backpack is also provided with a clamping sleeve for hanging the walking stick.
8. Multifunction navigation device according to claim 1, characterized in that: the control box is arranged at the lower end of the backpack, and a hollow chamber positioned at the upper end of the control box is arranged in the backpack.
9. Multifunctional navigation device according to claim 8, characterized in that: the lower end of the control box is provided with a radiator.
10. Multifunction navigation device according to claim 1, characterized in that: the physical feedback module comprises an m multiplied by n lattice which is at least composed of 9 physical feedback units, each physical feedback unit comprises a movable conduit, and a permanent magnet is movably arranged in each movable conduit; one end of the movable guide pipe is sealed with a seal made of elastic materials, the other end of the movable guide pipe is provided with an iron block, and the seal end of the movable guide pipe is a physical feedback end; the iron block is wound with a coil connected with the first control module, and the coil generates a magnetic field repelling the permanent magnet when being electrified, so that the permanent magnet moves towards the iron block when the physical feedback unit is powered off, and the permanent magnet moves towards the seal and impacts the seal when the physical feedback unit is powered on; wherein m and n are both positive integers and are both greater than or equal to 3.
CN201910312761.3A 2019-04-18 2019-04-18 Multifunctional navigation equipment Expired - Fee Related CN109925171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910312761.3A CN109925171B (en) 2019-04-18 2019-04-18 Multifunctional navigation equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910312761.3A CN109925171B (en) 2019-04-18 2019-04-18 Multifunctional navigation equipment

Publications (2)

Publication Number Publication Date
CN109925171A CN109925171A (en) 2019-06-25
CN109925171B true CN109925171B (en) 2020-12-22

Family

ID=66990414

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910312761.3A Expired - Fee Related CN109925171B (en) 2019-04-18 2019-04-18 Multifunctional navigation equipment

Country Status (1)

Country Link
CN (1) CN109925171B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110246178A (en) * 2019-07-09 2019-09-17 华东师范大学 A kind of modularization intelligent auxiliary system visually impaired
CN111035542A (en) * 2019-12-24 2020-04-21 开放智能机器(上海)有限公司 Intelligent blind guiding system based on image recognition
CN111067767B (en) * 2019-12-30 2022-12-23 西安Tcl软件开发有限公司 Walking assistance device, control method thereof, control device thereof, and readable storage medium
CN113332110B (en) * 2021-06-02 2023-06-27 西京学院 Blind guiding flashlight and blind guiding method based on scenery auditory perception

Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110020566A (en) * 2009-08-24 2011-03-03 성균관대학교산학협력단 Crosswalk guiding system for a blindperson
EP2371339A1 (en) * 2010-04-02 2011-10-05 POZOR 360 d.o.o. Surroundings recognition & describing device for blind people
CN202061036U (en) * 2010-12-30 2011-12-07 北京理工大学珠海学院 Blind person navigation system under Internet of Things environment
EP2262458B1 (en) * 2008-04-14 2013-02-20 Sosvi Method for individual guidance and associated device
CN103340737A (en) * 2013-03-25 2013-10-09 东南大学 Vibration touch feeling expression device capable of sensing navigation information of blind person
CN104306139A (en) * 2014-10-24 2015-01-28 南京理工大学 Blind people going out voice assisting tactile stick system based on RFID and navigation method of blind people going out voice assisting tactile stick system
CN104510597A (en) * 2014-12-26 2015-04-15 南京航空航天大学 Double-protecting blind guiding device
CN204428392U (en) * 2014-12-26 2015-07-01 南京航空航天大学 A kind of duplicate protection blind-guide device
CN104739622A (en) * 2015-02-04 2015-07-01 覃泳睿 Novel wearable blind guiding equipment and working method thereof
CN204581890U (en) * 2015-04-02 2015-08-26 周佳 A kind of blind man navigation knapsack with crutch
CN204709329U (en) * 2015-06-08 2015-10-21 深圳市中科微光医疗器械技术有限公司 The device of ambient condition information is passed on for visual disorder personage based on sense of touch
CN105232296A (en) * 2015-10-28 2016-01-13 沙洲职业工学院 Intelligent blind-guide stick and blind-guide system
JP2016067855A (en) * 2014-09-22 2016-05-09 達鋭 橋本 Walking guide device for sight disabled person
CN205234892U (en) * 2015-12-30 2016-05-18 河南工业大学 Backpack electron dot matrix blind person image touch sensing device
CN106618980A (en) * 2016-11-05 2017-05-10 深圳市前海安测信息技术有限公司 Wearable intelligent blind guiding system
CN106726382A (en) * 2017-02-07 2017-05-31 德阳力久云智知识产权运营有限公司 A kind of intelligent blind guiding system
CN207055641U (en) * 2017-01-18 2018-03-02 昆明理工大学 A kind of blind person's knapsack
CN108245387A (en) * 2018-02-11 2018-07-06 中明博瑞成都科技有限公司 A kind of intelligent blind guiding system based on wearable device
KR20180093753A (en) * 2017-02-14 2018-08-22 숭실대학교산학협력단 A stick for the blind
CN109040973A (en) * 2018-05-31 2018-12-18 武汉科技大学 A kind of blind-guiding method based on base station model, blind-guiding stick and blind guiding system
CN109077901A (en) * 2018-06-26 2018-12-25 北京化工大学 It is a kind of to meet the wearable of ergonomics and help blind machine robot mechanism

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107224393A (en) * 2017-08-10 2017-10-03 李海强 Can programme path blind guiding stick

Patent Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2262458B1 (en) * 2008-04-14 2013-02-20 Sosvi Method for individual guidance and associated device
KR20110020566A (en) * 2009-08-24 2011-03-03 성균관대학교산학협력단 Crosswalk guiding system for a blindperson
EP2371339A1 (en) * 2010-04-02 2011-10-05 POZOR 360 d.o.o. Surroundings recognition & describing device for blind people
CN202061036U (en) * 2010-12-30 2011-12-07 北京理工大学珠海学院 Blind person navigation system under Internet of Things environment
CN103340737A (en) * 2013-03-25 2013-10-09 东南大学 Vibration touch feeling expression device capable of sensing navigation information of blind person
JP2016067855A (en) * 2014-09-22 2016-05-09 達鋭 橋本 Walking guide device for sight disabled person
CN104306139A (en) * 2014-10-24 2015-01-28 南京理工大学 Blind people going out voice assisting tactile stick system based on RFID and navigation method of blind people going out voice assisting tactile stick system
CN104510597A (en) * 2014-12-26 2015-04-15 南京航空航天大学 Double-protecting blind guiding device
CN204428392U (en) * 2014-12-26 2015-07-01 南京航空航天大学 A kind of duplicate protection blind-guide device
CN104739622A (en) * 2015-02-04 2015-07-01 覃泳睿 Novel wearable blind guiding equipment and working method thereof
CN204581890U (en) * 2015-04-02 2015-08-26 周佳 A kind of blind man navigation knapsack with crutch
CN204709329U (en) * 2015-06-08 2015-10-21 深圳市中科微光医疗器械技术有限公司 The device of ambient condition information is passed on for visual disorder personage based on sense of touch
CN105232296A (en) * 2015-10-28 2016-01-13 沙洲职业工学院 Intelligent blind-guide stick and blind-guide system
CN205234892U (en) * 2015-12-30 2016-05-18 河南工业大学 Backpack electron dot matrix blind person image touch sensing device
CN106618980A (en) * 2016-11-05 2017-05-10 深圳市前海安测信息技术有限公司 Wearable intelligent blind guiding system
CN207055641U (en) * 2017-01-18 2018-03-02 昆明理工大学 A kind of blind person's knapsack
CN106726382A (en) * 2017-02-07 2017-05-31 德阳力久云智知识产权运营有限公司 A kind of intelligent blind guiding system
KR20180093753A (en) * 2017-02-14 2018-08-22 숭실대학교산학협력단 A stick for the blind
CN108245387A (en) * 2018-02-11 2018-07-06 中明博瑞成都科技有限公司 A kind of intelligent blind guiding system based on wearable device
CN109040973A (en) * 2018-05-31 2018-12-18 武汉科技大学 A kind of blind-guiding method based on base station model, blind-guiding stick and blind guiding system
CN109077901A (en) * 2018-06-26 2018-12-25 北京化工大学 It is a kind of to meet the wearable of ergonomics and help blind machine robot mechanism

Also Published As

Publication number Publication date
CN109925171A (en) 2019-06-25

Similar Documents

Publication Publication Date Title
CN109925171B (en) Multifunctional navigation equipment
CN204484697U (en) Intelligent blind-guiding crutch
CN111643324A (en) Intelligent glasses for blind people
CN105232295A (en) Intelligent guiding system for obstacle avoidance
EP3141234A1 (en) Spectacles for blind person
CN103720576A (en) Navigation glasses for the blind
CN104367447A (en) Auxiliary walking system and method
CN206275808U (en) Blindmen intelligent guidance system
CN209405201U (en) A kind of intelligent apparatus of assisting blind navigation
CN109925170B (en) Navigation equipment for blind and amblyopia group
CN210078040U (en) Intelligent blind guiding device
CN108785029B (en) Intelligent interconnection type blind guiding vehicle
US11554072B2 (en) Smart sensor cane
CN206730112U (en) A kind of intelligent glasses device of blind safety trip
CN210776078U (en) Real-time navigation bluetooth glasses
CN208448079U (en) A kind of blind guiding system based on intelligent interaction
CN114533503B (en) Glasses system for intelligent blind-assisting travel and interaction method
CN214968674U (en) Be applied to blind person's little assistant of leading
CN215350470U (en) Road blind guiding system
CN205301833U (en) Scenic spot guide's watch
KR20160057886A (en) Location confirmation system of a bicycle using augmented reality
KR200475531Y1 (en) Navigation device for blind men
CN114366582A (en) Intelligent hand-pushed pathfinding blind guiding eye
CN112237533A (en) Blind stick traffic guiding system for blind road
CN210812369U (en) Intelligent-based blind guiding walking stick device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20201222

Termination date: 20210418