JP3239665U - Systems to assist visually impaired users - Google Patents

Abstract

The present invention relates to a system for assisting visually impaired users. In particular, the present invention is a visual aid module that answers questions posed by users, gives instructions, and provides warning messages, real-time scanning for obstacle detection, person tracking, object detection and tracking, and optical character recognition. module, the navigation module for outdoor navigation and indoor map tracking, and the data to the visual aid module and the real-time scanning module and the navigation module, and the machine learning algorithms that process the data from the visual aid module and the real-time scanning module and the navigation module It relates to a system for assisting visually impaired users with processing modules for use. [Selection drawing] Fig. 1

Description

The present invention relates to assistive devices, and more particularly to systems for assisting visually impaired users.

Visual impairment affects approximately 2 billion people worldwide, of whom about 39 million are completely blind. Eighty-two percent of visually impaired people are over the age of 50, making them much more likely to be involved in an accident.

Solutions that allow visually impaired users to "feel" and "see" their surroundings range from service dogs to assistive technology devices. Typical service dogs are usually expensive, with breeds such as golden retrievers costing between $500 and $2,000 per dog. Smart canes, like the popular WeWALK Smart Cane, cost at least $500 per cane. Assistive technology devices such as screen readers and Braille all suffer from the inability to interface with the real world. Moreover, Braille suffers from a steep learning curve and is rather difficult to adopt in public and private institutions.

Various systems are currently disclosed in the art for assisting visually impaired users. For example, US Pat. No. 9,508,269 relates to a system for providing assistance from an assistant to a user. The system includes a user device including a visual data collector and an audible output device. The device can detect all objects in the user's surroundings such as objects, obstacles and people. The device also has GPS to determine the user's position. In addition, the device communicates to the user when there are obstacles, communicates search results, provides directions, and reads menus to the user.

US2013/0253818 discloses an indoor guidance system for the blind or visually impaired. The system uses mobile devices to provide maps, visual and audio guidance, and assist individuals in navigating indoor locations. Maps are sent to mobile devices, and the system uses a combination of GPS and other technologies (Bluetooth, infrared, Wi-Fi, RFID, etc.) to provide location information to users.

However, none of them disclose a system using machine learning algorithms such as deep learning to assist visually impaired users as the present invention does.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a faster, more versatile and more reliable system for assisting visually impaired users.

Accordingly, the present invention provides a system for assisting visually impaired users,
a. a visual aid module for answering questions posed by said user, giving instructions and providing warning messages;
b. real-time scanning modules for obstacle detection, person tracking, object detection and tracking, and optical character recognition;
c. a navigation module for outdoor navigation and indoor map tracking, and d. a processing module using machine learning algorithms for processing data to and from the vision aid module, the real-time scanning module and the navigation module;
The machine learning algorithms include regression algorithms, decision trees, association rule learning, artificial neural networks, support vector machines, inductive logic programming, batian networks, instance-based learning, manifold learning, subspace learning, deep learning, dictionary learning, Or selected from the group consisting of these combinations.

FIG. 1 illustrates a system for assisting visually impaired users according to a preferred embodiment of the present invention.

FIG. 2 shows a preferred configuration of the system for assisting visually impaired users.

FIG. 1 illustrates a system for assisting visually impaired users according to a preferred embodiment of the present invention. The system comprises an assistance device 100 further comprising a vision assistance module 102 . The visual aid module 102 is for answering questions posed by the user, giving instructions, and providing warning messages. Visual aid module 102 is further connected to audio device 112 . Audio device 112 is for capturing audio information from a user and for communicating audio information to the user. Audio device 112 is preferably both an input device and an output device such as a microphone, speaker, headphone, earpiece and bone conduction headphone. The input surface of audio device 112 captures voice commands or questions from the user and captures sounds from the surroundings. The output surface of audio device 112 provides audio information to the user, such as the sound from a call, alarm, or warning message.

Additionally, the assistance device 100 comprises a real-time scanning module 104 . Real-time scanning module 104 is for obstacle detection, person tracking, object detection and tracking, and optical character recognition. Additionally, the real-time scanning module 104 is also connected to at least one camera 114 for capturing visual information by the user. A camera 114 may be placed in front of the user to mimic the user's first person view. There may also be other cameras 114 directed to the side and rear of the user. Camera 114 may be any suitable video input device such as a video camera, infrared camera, night vision camera, or thermal camera.

Assistive device 100 also includes a navigation module 106 for outdoor navigation and indoor map tracking.

Finally, assistance device 100 includes processing module 108 for processing data to visual assistance module 102, real-time scanning module 104, and navigation module 106, and data from visual assistance module 102, real-time scanning module 104, and navigation module 106. Prepare. Processing module 108 uses machine learning algorithms, such as deep learning, to process data sent to and received from vision assistance module 102 , real-time scanning module 104 , and navigation module 106 . Processing modules include regression algorithms (linear, nonlinear, or logistic), decision trees or decision graphs, association rule learning, artificial neural networks, support vector machines, inductive logic programming, batian networks, instance-based learning, manifold learning, partial Other machine learning algorithms may be used, such as spatial learning, deep learning, dictionary learning, or combinations thereof.

Processing module 108 is further connected to transceiver 110 for connection to various systems, devices and networks such as the Internet, mobile devices and GPS satellites. Transceiver 110 may use wired or wireless connections such as USB, Bluetooth®, Wi-Fi®, 3G, 4G, and ZigBee®.

FIG. 2 shows a preferred configuration of the system for assisting visually impaired users. Assistive device 202 is preferably enclosed in a housing and connected to contraception 200 so that a user can wear assistive device 202 on the user's head. The contrasts 200 are preferably spectacles or sunglasses. Preferably, the camera of assistive device 202 is pointed in the direction the user is facing.

In another embodiment of the invention, the assistive device may be placed anywhere within the user's vicinity, such as in the user's pocket. The audio device and camera of the assistive device are connected to the sunglasses contraption and remotely connected to the assistive device.

Also, in yet another embodiment of the present invention, the contrast can be worn on various parts of the body. The contrast may be a headband, necklace, shoulder strap, chest strap, wristband, belt, or any other contrast worn on the body.

In an embodiment of the present invention, when the user wants to ask the assistance device for a piece of information, the user can ask the question through an audio device connected to the visual assistance module. For example, a user asks, "What is the capital of Kenya?" A microphone captures the user's verbal question and the visual aid module sends this question to the processing module. A processing module checks the question and searches for the answer in an internet search engine. This answer is then sent to the visual aid module and communicated to the user via earphones.

In another embodiment of the invention, the real-time scanning module is capable of person tracking. A camera captures video of the user's vicinity, and a real-time scanning module can use facial recognition to identify people. The visual aid module can also notify the user if the identified person is one of the user's known contacts. The user can then ask the real-time scanning module to track the identified person.

In a further embodiment of the invention, the real-time scanning module is capable of detecting and tracking objects. For example, a user can ask the assistive device to find a vacuum cleaner. A camera connected to a real-time scanning module captures video and detects the vacuum cleaner. A real-time scanning module then tracks the vacuum until the user picks it up.

The support device's real-time scanning module is also capable of obstacle detection. A camera captures the video and a real-time scanning module detects objects from the video. The real-time scanning module then transmits detected objects to the processing module. The processing module uses deep learning to see if the detected object could be an obstacle for the user. If the detected object is considered an obstacle, the visual aid module warns the user that there is an obstacle near the user. The warning can include information such as the direction of the obstacle, the size of the obstacle, and the distance of the obstacle.

Also, the real-time scanning module is capable of optical character recognition. A camera captures an image or video of an object (eg, menu or signage) containing text. A real-time scanning module then detects and decodes the text. The visual aid module can also use text-to-speech technology to convey text to the user in audio form.

In yet another embodiment of the invention, the navigation module has outdoor navigation capabilities. Preferably, the navigation module can identify the user's location and destination via GPS. For example, a user requests an assistive device to give directions on how to get to a shopping mall. The navigation module identifies the location of the shopping mall as well as identifying the user's current location. The navigation module then identifies the fastest and easiest route to the mall. The visual aid module then provides instructions/instructions to the user. For example, the assistance module tells the user, "After 500 meters, please turn right."

The navigation module is also capable of indoor map tracking. The navigation module uses a variety of indoor positioning technologies including, but not limited to, magnetic positioning, inertial measurements, visual marker positioning, Wi-Fi positioning, and Bluetooth positioning. can be used. For example, upon entering a shopping mall, the navigation module downloads a map of the shopping mall. The navigation module also connects to the mall's Wi-Fi® to determine the user's location.

In another embodiment of the invention, a system for assisting visually impaired users can be implemented in any device having a processor, memory storage, and a power source. Such devices can include, but are not limited to, microcomputers, mobile phones, smart phones, personal digital assistants, smart watches, tablet computers, desktop computers, and laptop computers.

Claims (6)

A system for assisting visually impaired users, comprising:
a. a visual aid module for answering questions posed by said user, giving instructions and providing warning messages;
b. real-time scanning modules for obstacle detection, person tracking, object detection and tracking, and optical character recognition;
c. a navigation module for outdoor navigation and indoor map tracking, and d. a processing module that uses machine learning algorithms for processing data to said visual aid module, said real-time scanning module and said navigation module, and data from said visual aid module, said real-time scanning module and said navigation module;
The machine learning algorithms include regression algorithms, decision trees, association rule learning, artificial neural networks, support vector machines, inductive logic programming, batian networks, instance-based learning, manifold learning, subspace learning, deep learning, dictionary learning, or a system selected from the group consisting of combinations thereof. 3. The system of claim 1, wherein the visual aid module is connected to an audio device. 3. The system of claim 1, wherein said real-time scanning module is connected to at least one camera. 4. The system of claim 3, wherein the at least one camera is any suitable video input device including, but not limited to, video cameras, infrared cameras, night vision cameras, and thermal cameras. 2. The system of claim 1, wherein said outdoor navigation of said navigation module is via GPS. 2. The system of claim 1, wherein said processing module is connected to a transceiver.

Images