TW202007383A - Smart aid system for visually impaired - Google Patents

Abstract

The present invention relates to a smart aid system for visually impaired. It comprises a pair of smart glasses including a first processing module, a first Bluetooth module, a first gravity sensing module and an infrared ray sensing module, wherein the first Bluetooth module, the first gravity sensing module and the infrared ray sensing module are electrically connected to the first processing module; and a smart walking stick including a second processing module, a second Bluetooth module electrically connected to the first Bluetooth module, a second gravity sensing module, a positioning module, a vibration module and a wireless transmission module, wherein the second Bluetooth module, the second gravity sensing module, the positioning module, the vibration module and the wireless transmission module are electrically connected to the second processing module.

Description

Smart assistive system for the visually impaired

The invention relates to a smart assistive device system for visually impaired persons, in particular to a smart assistive device system in which smart glasses and a smart cane are combined, infrared obstacles are used to detect an obstacle in front, and the vibration of the smart cane is used to remind the visual aid The visually impaired must avoid obstacles. When the visually impaired falls, they can also record the location and transmit information to the cloud platform to inform the visually impaired's family.

Pressing, visually impaired people will have a lot of inconveniences in life because of vision problems. They often need to rely on various assistive devices to assist their needs in life. Common aids include guide sticks, sidewalk guide bricks, and braille tools. According to the statistics of the World Health Organization, there are about 253 million visually impaired people in the world. Among the visually impaired people, there are 217 million amblyopic groups and 36 million blind people. Amblyopia and other visually impaired people account for the total visually impaired people. 86%, while the total number of blind people accounted for 14%. In addition, in terms of Taiwan, the Ministry of Health and Welfare statistics in 2017 that the current visually impaired population in Taiwan is approximately 56,830.

Republic of China Patent Announcement No. TW M548990 "Crutches for Visually Impaired Persons" provides a crutches that can sense obstacles in front of them. The crutches are equipped with ultrasonic sensing module, luminous warning device and sound warning device. The sensing module senses the distance between the object in front and the visually impaired, and according to the length of the distance, the luminous warning device displays light sources of different colors, and the sound warning device also emits different scales or sound effects according to the distance to remind the visually impaired The Republic of China Patent Announcement No. TW I474173 "Walking Assist System and Walking Assist Method" uses camera devices to capture realistic scenes, and detects the distance between items and camera devices to create a three-dimensional model of the scene, and then reminds the visually impaired The distance of the object in front of the person.

It can be seen from the previous patent that there are more and more assistive devices and methods to assist the life of visually impaired people. However, for visually impaired people, there are still many variables in the outdoor environment to restrict the movement of visually impaired people. Danger to the visually impaired. Secondly, how to provide a safer combination of assistive devices for the visually impaired to walk outdoors, this is the direction that the inventor has considered.

Now, in view of the fact that the above-mentioned existing visually impaired assistive devices still have many deficiencies in actual use, they are a tireless spirit, and are assisted by their rich professional knowledge and many years of practical experience, and Improve it and research and create the present invention accordingly.

The main purpose of the present invention is to provide a smart assistive system for visually impaired persons, which is mainly composed of smart glasses and a smart cane. The smart glasses and the smart cane are connected to each other through wireless Bluetooth technology, when the smart glasses detect an obstacle in front , You can let the smart cane remind the visually impaired with a vibration effect, and let the visually impaired know that they should avoid the obstacle in front.

In order to achieve the above-mentioned implementation objective, a smart assistive system for visually impaired persons of the present invention includes: a smart glasses with a first processing module, a first Bluetooth module, a first gravity sensing module and An infrared sensing module, wherein the first Bluetooth module, the first gravity sensing module and the infrared sensing module are all electrically connected to the first processing module; and a smart stick with a second processing module , A second Bluetooth module electrically connected to the first Bluetooth module, a second gravity sensing module, a positioning module, a vibration module and a wireless transmission module, wherein the second Bluetooth module, the first The second gravity sensing module, positioning module, vibration module and wireless transmission module are all electrically connected to the second processing module.

In one embodiment of the present invention, the first gravity sensing module and the second gravity sensing module both include a gyroscope and an accelerometer.

In an embodiment of the present invention, the first gravity sensing module and the second gravity sensing module sample the acceleration changes of the smart glasses and the smart cane to determine whether they are in a dumping state.

In one embodiment of the invention, the positioning module is GPS.

In one embodiment of the invention, the wireless transmission module uses LoRa wireless transmission technology.

In an embodiment of the invention, the smart glasses further have a first power supply module electrically connected to the first processing module.

In one embodiment of the present invention, the smart walking stick further has a second power supply module electrically connected to the second processing module.

In one embodiment of the present invention, the wireless transmission module is further wirelessly connected to a cloud platform.

The purpose of the present invention and its structural and functional advantages will be explained based on the structure shown in the following drawings and in conjunction with specific embodiments, so that the reviewing committee can have a more in-depth and specific understanding of the present invention.

Please refer to the first figure to the third figure. The present invention provides a visually impaired smart assistive device system, which includes a smart glasses (1) and a smart cane (2). The smart glasses (1) has a first processing module (11), a first Bluetooth module (12), a first gravity sensing module (13) and an infrared sensing module (14), wherein The first gravity sensing module (13) includes a gyroscope and an accelerometer, and the first Bluetooth module (12), the first gravity sensing module (13), and the infrared sensing module (14) are all powered Is connected to the first processing module (11), and the first processing module (11) is electrically connected to a first power supply module (15);

The smart walking stick (2) has a second processing module (21), a second Bluetooth module (22) electrically connected to the first Bluetooth module (12), and a second gravity sensing module (23) , A positioning module (24), a vibration module (25) and a wireless transmission module (26) using LoRa wireless transmission technology, wherein the second gravity sensing module (23) includes a gyroscope and acceleration And the positioning module (24) is GPS, and the second Bluetooth module (22), the second gravity sensing module (23), the positioning module (24), the vibration module (25) and the wireless transmission module The group (26) is electrically connected to the second processing module (21), and the second processing module (21) is electrically connected to a second power supply module (27);

When in use, the first gravity sensing module (13) and the second gravity sensing module (23) sample the acceleration changes of the smart glasses (1) and the smart cane (2) to determine whether the visually impaired person is in a dumping state If it is judged to be in a dumping state, the wireless transmission module (26) of the smart cane (2) will transmit this fall information and position information to a cloud platform (3) that is wirelessly connected.

In addition, through the following specific embodiments, the scope of the present invention can be further proved to be practical, but it is not intended to limit the scope of the present invention in any form.

Please continue to refer to the first picture to the third picture. The visually impaired smart assistive device system of the present invention mainly includes a smart glasses (1) and a smart cane (2). Before use, the visually impaired can wear smart glasses (1 ), and through the first Bluetooth module (12) and the second Bluetooth module (22) of the smart cane (2) pairing, after the pairing is successful, you can start using the invention;

Please also refer to the fourth figure, the usage of the present invention is roughly as shown in the figure; first, the infrared glasses sensing module (14) is provided on the smart glasses (1), because it is installed above the smart glasses (1), Therefore, obstacles with higher or lower positions can be detected; the infrared sensor module (14) has an infrared emitter (141) and a position sensor (142) (Position Sensing Detector, PSD), which A triangulation method is used to form a triangle from a measurement target point and two reference points with known coordinates, wherein the two reference points with known coordinates are the infrared emitters (14) on the infrared sensor module (14) 141) and the position sensor (142), and the measurement target point is an obstacle; in actual use, the infrared emitter (141) will continuously emit infrared rays, when the infrared rays hit the obstacle, the obstacles will convert the infrared rays Reflected to the position sensor (142), this reflection information will be transmitted to the first processing module (11) for calculation, because the distance between the two reference points of the infrared emitter (141) and the position sensor (142) is known Parameters, therefore, by calculating the length of the two sides of the triangle and measuring the angle formed by the two reference points and the measurement target point, the first processing module (11) can calculate the distance of the obstacle (measurement target point) And coordinates.

When the first processing module (11) determines that the obstacle is quite close to the visually impaired person, it transmits the warning information to the second Bluetooth module (22) of the smart stick (2) through the first Bluetooth module (12), The warning information received by the second Bluetooth module (22) is transmitted to the vibration module (25) through the second processing module (21), so that the vibration module (25) will vibrate in the smart stick (2), And conveyed to the holding part, so that the hand of the visually impaired can feel the vibration, to remind the visually impaired that there is an obstacle in front.

When the infrared sensor module (14) is tested, the smart glasses (1) are worn on the body, and every 1 meter is a unit, and 5 units are measured in sequence to actually record the distance conversion voltage. The selected infrared The actual measured distance is output as a voltage for the first processing module (11) to read the distance. The relationship between the distance and the voltage is shown in Table 1 below.

Table 1

As shown in Table 1, the closer the distance to the obstacle and the higher the voltage, the first processing module (11) uses this data to interpret the distance between the obstacle and the smart glasses (1); during the test The detection distance is not easy to be changed by the appearance of the object and the reflectivity of the object. Based on the walking safety and reaction time of the visually impaired, it is recommended to set the effective detection distance to 1.5 meters, which represents the infrared sensor module ( 14) When an obstacle in front of 1.5 meters is detected and the visually impaired person is warned through the vibration module (25), the visually impaired person is just 1.5 meters away from the obstacle, and the visually impaired person can carry out the obstacle within this distance Dodge action.

In addition, please refer to the fifth figure together. There are often many variables in the outdoor environment facilities, which can cause impediment to the movement of the visually impaired. If the visually impaired person does not immediately find the obstacle, it will cause the visually impaired person to fall. According to this, the smart glasses (1) of the present invention are provided with a first gravity sensing module (13), and the smart cane (2) is also provided with a second gravity sensing module (23), which all include 3 The axis accelerometer and 3-axis gyroscope can be used to sense the state of use of the smart glasses (1) and the smart cane (2), and determine the posture of the visually impaired.

The signals captured by the first gravity sensing module (13) and the second gravity sensing module (23) are transmitted to the first processing module (11) and the second processing module (21) respectively The actual judgment method of calculation is as follows; first, the relationship between the general human posture and the vertical angle of the ground needs to be known in advance. The included angle θ _Acc , θ _Acc senses the angle θ _Gyro by adding a gyroscope, and then uses the Kalman Filter (Kalman Filter) model to calculate the tilt angle θ of the human body; when the actual angle between the human body and the ground is known, It can be classified according to the angle threshold (Dth). When the angle threshold is 30 degrees, if the measured angle is less than 30 degrees, it is classified as a straight action (walking, running, standing), if the measured angle is greater than 30 Degrees, it is classified as a dumping action (sitting, lying, falling);

After setting the angle threshold, in actual use, the first gravity sensing module (13) and the second gravity sensing module (23) sense the smart glasses (1) and the smart cane with a 3-axis accelerometer and gyroscope (2) The tilt angle, the first gravity sensing module (13) will transmit this angle information to the first processing module (11), and the second gravity sensing module (23) will transmit this angle information to The second processing module (21) determines the current posture of the visually impaired person, and then calculates the vertical velocity X(tk+) of the last k samples of the visually impaired person through the fourth-order Runge-Kutta integral equation 1), and use the tilt angle θ and X(tk+1) of the visually impaired to identify the six human postures of standing, sitting, lying, running, walking, and falling;

Subsequently, in order to determine whether the visually impaired person has a posture change, the first processing module (11) and the second processing module (21) will calculate the maximum and minimum vertical acceleration measured during each i sampling period The maximum difference (X MaxDiff), within the i-th sampling period, when the calculated maximum difference is greater than the angle threshold (X MaxDiffx> Dth), the first processing module (11) and the second processing module (21) It is judged as a posture change, and the current action is recognized;

In order to prevent the system from erroneously judging the fall posture, the second processing module (21) of the smart walking stick (2) recognizes the fall event, and if it receives the fall information from the smart glasses (1), Only then can the fall event be determined; if the fall event of the visually impaired person is confirmed, the second processing module (21) will immediately pass the position information of the positioning module (24) through the wireless transmission module (26), And the time of occurrence is uploaded to the cloud platform (3) for storage, and the family members of the visually impaired can view the information of the fall event stored in the cloud platform (3) through their smartphones; thus, the smart glasses (1) and wisdom The walking stick (2) can improve the recognition rate of fall events of visually impaired persons and avoid the occurrence of false alarms.

In addition, the wireless transmission module (26) of the present invention uses the LoRa wireless transmission technology, because the dangerous events of visually impaired people mostly occur outdoors, and the visually impaired people rarely have the habit of carrying a smartphone with them, so when the fall event When this happens, it is not suitable for the visually impaired through the 3G/4G network of the smartphone to the cloud platform (3); and common Internet of Things communication technologies such as WiFi and Bluetooth are all short-range wireless communication technologies, so In the present invention, the LoRA transmission technology of the IoT Wide Area Network (IoT Wide Area Network, IoTWAN) is selected to transmit the fall information and position information of the current fall event to the cloud platform (3).

It can be seen from the above implementation description that the present invention has the following advantages compared with the prior art:

1. The smart assistive system for visually impaired persons of the present invention has a smart glasses and a smart cane. The smart glasses detect obstacles in front of the visually impaired and transmit a warning message to the smart canes. Persons with disabilities, in order to avoid obstacles smoothly, reduce the risk of visually impaired persons in the outdoors.

2. Both the smart glasses and the smart cane of the visually impaired smart assistive system of the present invention can determine whether a visually impaired person has a fall event. If both of the visually impaired persons detect a fall, the smart cane transmits the fall information and position information to The cloud platform allows family members of the visually impaired to be informed of this information, thereby providing immediate assistance to the visually impaired.

In summary, the visually impaired smart assistive device system of the present invention can indeed achieve the expected use effect through the embodiments disclosed above, and the present invention has not been disclosed before the application, and has fully complied with the patent law Regulations and requirements. I filed an application for a patent for invention in accordance with the law, pleaded for the review, and granted the patent.

However, the illustrations and descriptions disclosed above are only preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention; those who are familiar with this skill, according to the characteristic scope of the present invention, do other things Equivalent changes or modifications should be regarded as not departing from the design scope of the present invention.

(1)‧‧‧Smart glasses (11)‧‧‧First processing module

(12)‧‧‧First Bluetooth module(13)‧‧‧First gravity sensing module

(14)‧‧‧Infrared sensing module(141)‧‧‧Infrared emitter

(142)‧‧‧Position sensor(15)‧‧‧‧First power supply module

(2) ‧‧‧ Smart Wand (21) ‧‧‧ second processing module

(22)‧‧‧Second Bluetooth module(23)‧‧‧‧Second gravity sensing module

(24)‧‧‧positioning module(25)‧‧‧‧vibration module

(26) ‧‧‧ wireless transmission module (27) ‧‧‧ second power supply module

(3) ‧‧‧ cloud platform

Figure 1: Block diagram of the system architecture of the preferred embodiment of the present invention.

Figure 2: The overall appearance of smart glasses according to the preferred embodiment of the present invention.

Third figure: The overall appearance of the smart cane according to the preferred embodiment of the present invention.

Figure 4: A schematic diagram of obstacle detection according to a preferred embodiment of the present invention.

Fifth figure: A schematic diagram of a fall event according to the preferred embodiment of the present invention.

Figure 6: A schematic diagram of angle identification of a gravity sensing module according to a preferred embodiment of the present invention.

(1)‧‧‧Smart glasses

(11)‧‧‧First processing module

(12)‧‧‧The first Bluetooth module

(13)‧‧‧First gravity sensing module

(14)‧‧‧Infrared sensing module

(141)‧‧‧Infrared emitter

(142)‧‧‧Position sensor

(15)‧‧‧First power supply module

(2)‧‧‧cane of wisdom

(21)‧‧‧second processing module

(22)‧‧‧Second Bluetooth module

(23)‧‧‧Second gravity sensing module

(24)‧‧‧positioning module

(25)‧‧‧Vibration module

(26)‧‧‧Wireless transmission module

(27)‧‧‧Second power supply module

(3) ‧‧‧ cloud platform

Claims (8)

A smart assistive device system for the visually impaired includes: a smart glasses with a first processing module, a first Bluetooth module, a first gravity sensing module and an infrared sensing module, wherein The first Bluetooth module, the first gravity sensing module and the infrared sensing module are all electrically connected to the first processing module; and a smart walking stick has a second processing module and an electrical A second Bluetooth module connected to the first Bluetooth module, a second gravity sensing module, a positioning module, a vibration module and a wireless transmission module, wherein the second Bluetooth module and the second The gravity sensing module, the positioning module, the vibration module and the wireless transmission module are all electrically connected to the second processing module. As described in item 1 of the patent application scope, a smart assistive device system for visually impaired persons, wherein both the first gravity sensing module and the second gravity sensing module include a gyroscope and an accelerometer. The smart assistive system for visually impaired persons as described in item 1 of the patent scope, wherein the first gravity sensing module and the second gravity sensing module sample the acceleration changes of the smart glasses and the smart cane to determine Whether it is dumping. As described in item 1 of the patent application, the smart assistive system for visually impaired persons, wherein the positioning module is GPS. As described in item 1 of the patent application scope, a smart assistive device system for visually impaired persons, wherein the wireless transmission module uses LoRa wireless transmission technology. As described in the first patent application, the smart assistive device system for visually impaired persons, wherein the smart glasses further has a first power supply module electrically connected to the first processing module. As described in item 1 of the patent application scope, a smart assistive device system for visually impaired persons, wherein the smart cane further has a second power supply module electrically connected to the second processing module. As described in item 1 of the patent application scope, a visually impaired smart assistive device system, wherein the wireless transmission module is further wirelessly connected to a cloud platform.

Images