KR20150097043A - Smart System for a person who is visually impaired using eyeglasses with camera and a cane with control module - Google Patents

Abstract

The present invention relates to a smart system for assisting a visually impaired person. For the same, a system for assisting a visually impaired person is connected to a glasses module and a cane module with a wireless network. The glasses module comprises: an image sensor which enables the visually impaired person to obtain an image of an obstruction in front of the person according to a gaze direction of glasses while moving; an image signal analysis unit which receives the image obtained from the image sensor and analyzes the image so as to determine the presence of the front obstruction; and a first wireless communication unit which transmits an image analysis signal, analyzed by the image signal analysis unit, to the cane module. The cane module comprises: a second wireless communication unit which receives the image analysis signal transmitted from the first wireless communication unit; a cane control unit which converts the image analysis signal received from the second wireless communication unit into a touch signal which can be recognized by the visually impaired person according to the image analysis signal; and a braille vibration unit which outputs information using the touch signal according to a control signal from the cane control unit.

Description

Technical Field [0001] The present invention relates to a system and a method for a visually impaired person using smart glasses and a cane,

The present invention relates to a smart system for assisting a visually impaired person, and more particularly, to a system for assisting a visually impaired person by connecting a wand equipped with a camera and a smart control device by wireless communication.

Most of the people who visually impaired are aware of the surrounding obstacles by listening to the sound and feeling the touch by tapping and tackling the topographical object of the place where the visually impaired person wants to go using the stick for blind people. Others may be assisted by a guide dog. As such, visual guide dogs and walking sticks are one of the necessities when moving visually impaired.

However, when using a cane, judging by the sound and feeling felt by directly beating is not only effective after being trained to a certain extent by other senses after being blind, but also slow and inconvenient to detect, Even if it is utilized, there are difficulties in acting together in public places, and there are restrictions on understanding the complex signaling system on the road or detecting the directionality.

ETRI's 2012 "Future Top Ten Technologies" presented the 'circumstance recognition technology' as a key technology in the cognitive science field. This indicates that the cognitive technology that enables the system to grasp the user's surroundings has high economic / technological value.

In recent years, the field of manufacturing a medical assistance system has been attracting attention. In particular, there is a growing need to apply a technique of recognizing the user's surroundings to a system for assisting the visually impaired.

The present invention reflects the above-mentioned demand, and it is possible to connect a walking stick equipped with a camera and a stick equipped with a smart control device by wireless communication, thereby enabling a safe walking without being disturbed by an obstacle when a visually impaired person moves The present invention also provides a visual assistance system and method capable of improving the user's perception of the surrounding situation through information converted into a braille signal.

To this end, according to one embodiment of the present invention, there is provided an auxiliary system for a visually impaired person, in which an eyeglass module and a wand module are connected via a wireless network, wherein the eyeglass module includes an image of a front obstacle An image signal analyzer for receiving the image obtained from the image sensor and analyzing the image to determine the presence of the front obstacle; and an image analyzing unit for analyzing the image analysis signal analyzed by the image signal analyzing unit, To the first wireless communication unit; The cane module includes a second wireless communication unit that receives the image analysis signal transmitted from the first wireless communication unit and a tactile signal that can be recognized by the blind person according to the image analysis signal received from the second wireless communication unit And a braille vibration unit for outputting information as a tactile signal according to a control signal from the cane control unit.

The braille vibration unit may include a vibration motor for implementing the tactile sense signal and a motor driver for driving the vibration motor, and the tactile sense signal may be a braille signal or a vibration signal.

In addition, the cane module includes an infrared sensor for detecting a hole in the moving direction of the visually impaired person at a lower portion thereof, and drives the bottom vibration motor when the hole exists.

The mobile terminal further comprises a smart device connected to the eyeglass module and the cane module via the wireless network. The mobile terminal further includes a speaker on the cane module. And outputs a guidance voice to detect the front obstacle through the eyeglass module and the wand module or to output the tactile sense signal to the visually impaired person in the moving direction.

In addition, the image signal analyzing unit applies a face recognition algorithm when the front obstacle is a pedestrian.

According to another embodiment of the present invention, there is provided a method of assisting a visually impaired person in guiding a vision system in a system in which a spectacle module and a cane module are connected via a wireless network, the method comprising the steps of: (a) Obtaining image information on a front obstacle on a moving road; (b) determining whether a front obstacle is present by applying pattern recognition and a machine learning algorithm to the image information obtained in the step (a); (c) transmitting the image analysis signal determined in step (b) to the wand module through the wireless network; (d) the cane module generates a control signal based on the image analysis signal transmitted in the step (c), and implements a vibration signal in the cane module according to the control signal or stores it in advance in correspondence with the image analysis signal Providing tactile information to a visually impaired person by implementing a vibration signal according to a vibration pattern of the braille; And a control unit.

The system further comprises a speaker in the wand module and the smart device connected to the eyeglass module and the wand module via the wireless network, and (e) Providing a route guidance such as an instruction through the voice; (f) sensing the front obstacle through the spectacle module and the wand module or outputting the tactile signal to the visually impaired person in the moving direction; And further comprising:

The assistant system for the visually impaired according to the present invention can assist the user to walk safely without obstructing the obstacle when the visually impaired person moves by connecting the camera equipped glasses and the staff equipped with the smart control device by wireless communication.

In addition, since the visually impaired person recognizes the recognized information through the camera as the braille information, it is possible to improve the user's perception of the surrounding situation.

Particularly, it is possible to determine whether a pedestrian is a person through the face recognition algorithm, thereby doubling the cognitive ability of a visually impaired person.

1 is a block diagram illustrating a configuration of a blind auxiliary system according to a preferred embodiment of the present invention;
Figure 2 shows the wand module in the system of Figure 1,
FIG. 3 is a flowchart illustrating a route guidance process in a system in which a spectacle module, a cane module, and a smart device according to a preferred embodiment of the present invention are connected to a wireless network.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and how to accomplish it, will be described with reference to the embodiments described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. The embodiments are provided so that those skilled in the art can easily carry out the technical idea of the present invention to those skilled in the art.

In the drawings, embodiments of the present invention are not limited to the specific forms shown and are exaggerated for clarity. Also, the same reference numerals denote the same components throughout the specification.

The expression "and / or" is used herein to mean including at least one of the elements listed before and after. Also, singular forms include plural forms unless the context clearly dictates otherwise. Also, components, steps, operations and elements referred to in the specification as " comprises "or" comprising " refer to the presence or addition of one or more other components, steps, operations, elements, and / or devices.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a visually impaired assistant system 300 according to a preferred embodiment of the present invention.

1, an assistant system 100 for a visually impaired person according to the present invention includes an eyeglass module 100 with a camera and a cane module 200 with a smart control device, System. At this time, it is preferable that the glasses and the cane are manufactured in accordance with the international standard for the visually impaired.

The eyeglass module 100 includes an image sensor 110 for acquiring an image of a forward object or a pedestrian (hereinafter referred to as a front obstacle) along the direction of sight of the eyeglasses while the visually impaired person moves, A first wireless communication unit 130 for transmitting the image analysis signal analyzed by the image signal analysis unit 120 to the wand module, and a second wireless communication unit 130 for transmitting the image analysis signal analyzed by the image signal analysis unit 120 to the wand module, Respectively. At this time, the image sensor 110 may be a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The image signal analyzing unit 120 analyzes the image obtained from the image sensor 110 to determine whether there is an object or a pedestrian in front of the visually impaired person. In order to analyze the image, a machine learning algorithm So that pedestrians or objects can be detected. In particular, if the obstacle is a pedestrian, the face recognition algorithm (Adaboost) is performed. That is, the face recognition algorithm extracts features for each part so that the facial features can be clearly displayed to detect the face. Thereafter, each weak classifier is created with the extracted features, and at this time, a weak classifier capable of classifying the non-face and non-face images among the weak classifiers is selected. We use a weight classifier to classify a strong classifier and classify it as a strong classifier.

The cane module 200 of the blind person includes a second wireless communication unit 230 for receiving the image analysis signal transmitted from the first wireless communication unit 130 and a second wireless communication unit 230 for receiving the image (Or tactile sense) signal that can be recognized by the blind person according to the analysis signal, and a braille (or tactile) signal for outputting braille (or tactile) signals according to a control signal from the stick control unit 220. [ And a vibrating part 210. At this time, the braille vibration unit 210 includes a vibration motor 212 for implementing a braille signal and a motor driver 211 for driving the vibration motor. The vibration motor 210 uses a PWM motor control method and a time interruption function Control cycle and motor. As described above, the braille vibration unit 210 is attached to the wand and implements the vibration signal through the vibration motor (driving motor) or implements the vibration signal according to the vibration pattern of braille stored in advance in correspondence with the image analysis signal, This can be detected by the sense of touch on the staff.

2, four vibration motors 241, 242, 243, and 244 may be provided to indicate four directions of up, down, left, and right. In order to inform the blind person of the existence of an obstacle and the direction of movement, To transmit the touch. Thus, the upper vibration motor 244, the left vibration motor 243, the right vibration motor 242, and the lower vibration motor 241 are arranged in a line. That is, with respect to the obstacles on the left and right sides, it is possible to sense the direction of movement of the vibration motor in the corresponding direction in the left or right direction or to tell the moving direction by the touch by the vibration. An obstacle in the lower direction can be detected. In addition, reference numerals 260 and 250 denote the handle and the cane, respectively.

In addition, six circular motors constitute the vibration motor to implement detailed direction sensing and additional functions in addition to the four directions of the top, bottom, left and right, and to convert the visual information from the camera to tactile information To feel the touch of the palm of the hand. In other words, as shown in FIG. 2, an infrared sensor 251 for detecting a hole in the lower part of the wand may be provided, and the bottom vibration motor (not shown) may be driven if there is a hole. In this case, in the case of the infrared sensor 251, the time and power required for the signal processing can be reduced by connecting the infrared sensor 251 with the vibration motor without performing the signal processing using the microprocessor, and an amplifier (Not shown) may be further provided.

The blind person then carries the eyeglass module 100, the cane module 200 and the own smart device 400 to implement a wireless communication between the eyeglass module and the wand module or to perform wireless communication between the eyeglass module, Wireless communication can be implemented. Here, the smart device 400 may be any one of a smart phone, a PDA, and a tablet PC, and may be a mobile system that is portable and easy to communicate with the Internet to provide a personalized learning method and system. In addition, the wireless communication method is based on Wi-Fi communication, and data can be transmitted through short-range wireless communication such as Bluetooth communication, wireless LAN, and ZigBee.

In the smart device 400, a navigation application for a visually impaired person (hereinafter referred to as an app) is installed. The navigation application receives data through Wi-Fi communication, displays the data, And outputs it to the installed speaker (not shown). Accordingly, not only the voice guidance through the speaker but also the glasses module and the cane module can detect a front obstacle when moving according to the navigation application, and can inform the visually impaired person of the moving direction by the touch by the vibration. For example, when the vibration motor is arranged in a line, the vibrating motor is moved in a direction from the upper side to the lower side (top down method) to increase the vibration intensity or the vibration frequency, up method) can be known.

As such, smart sticks can be configured to work in conjunction with smartphones and smart tablets, which are becoming popular nowadays, making them useful for the visually and hearing-impaired.

FIG. 3 is a flowchart illustrating a process of implementing a visually impaired assistant system 300 in conjunction with an eyeglass module, a cane module, and a smart device 400 according to a preferred embodiment of the present invention.

Referring to FIG. 3, first, image information on a front obstacle on a road on which a visually impaired person moves is read through an image sensor mounted on the spectacle module 100 (S210).

Then, the image signal analyzing unit implements an algorithm for recognizing an object of interest through pattern recognition and a machine learning algorithm in the image acquired through the image sensor, and determines whether the obstacle is an obstacle (an object or a pedestrian) in front of the visually impaired (S220).

 In this case, a lane and an external vehicle, a traffic light, a pedestrian, and the like may be considered as objects of interest. In the case where the obstacle is a pedestrian, a face recognition algorithm is performed. That is, the image signal analyzer uses the HOG feature and the MCT feature from the train data composed of positive data, negative data, and neutral data to determine common characteristics of the pedestrian Extract from the image. This is learned by using a machine learning algorithm so that a pedestrian can be detected from an image. At this time, the machine learning algorithm used is the Adaboost algorithm which constructs strong classifiers by using the weights corresponding to several weak classifiers and weak classifiers.

In addition, it is possible to determine whether or not the recognized pedestrian is a person by carrying out face recognition of pupil detection from the detected pedestrian.

Thereafter, it is determined whether an obstacle is present according to the pattern recognition and the machine learning algorithm through the image analysis unit 220, and the determined image analysis signal is transmitted to the stick control unit through the first wireless communication unit at step S230.

Thereafter, the cane control unit generates a control signal based on the transmitted image analysis signal, implements a vibration signal in the cane module according to the control signal, or generates a vibration signal corresponding to the vibration pattern of braille stored in advance in correspondence with the image analysis signal. Signal so that the tactile information can be provided to the visually impaired to be sensed (S240).

In addition, a route guidance application of the smart device may be generated as a voice and a route guidance such as a direction indication may be made through a speaker installed in the staff (S250). For example, when a user inputs a start point and a destination from a smart device by voice, the navigation application for the visually impaired outputs the result to the speaker through voice synthesis according to the navigation application. At this time, the smart device is connected with the eyeglass module and the wand module through short-range wireless communication, and can also inform the direction of movement of the obstacle by vibration due to vibration.

While the invention has been shown and described with respect to the specific embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. Anyone with it will know easily.

100: Glasses module
110: Image sensor
120: a video signal analyzing unit
130, and 230:
200: cane module
210:
220:
300: Visually impaired assistant system
400: Smart Device

Claims (7)

A blindfold auxiliary system in which an eyeglass module and a wand module are connected via a wireless network,
The eyeglass module may further include an image sensor that acquires an image of a front obstacle along the direction of sight of the eyeglasses while the visually impaired person moves and analyzes the image by receiving the image acquired from the image sensor to determine whether or not the front obstacle is present And a first wireless communication unit for transmitting the image analysis signal analyzed by the image signal analysis unit to the wand module,
The cane module includes a second wireless communication unit that receives the image analysis signal transmitted from the first wireless communication unit and a tactile signal that can be recognized by the blind person according to the image analysis signal received from the second wireless communication unit And a braille vibration unit for outputting information as a tactile signal according to a control signal from the cane control unit. The method according to claim 1,
Wherein the braille vibration unit includes a vibration motor for implementing the haptic signal and a motor driver for driving the vibration motor,
Wherein the tactile signal is a braille signal or a vibration signal. The method according to claim 1,
Wherein the cane module includes an infrared sensor for detecting a hole in the moving direction of the visually impaired person at a lower portion thereof, and drives the bottom vibration motor in the presence of the hole. The method according to claim 1,
And a smart device connected to the eyeglass module and the cane module via the wireless network,
The cane module further comprises a speaker,
And a guidance module for outputting a guidance voice through the speaker at the time of movement of the visually impaired according to the guidance application of the smart device and detecting the front obstacle through the spectacle module and the wand module, And outputs the output signal. The method according to claim 1,
Wherein the image signal analyzing unit applies a face recognition algorithm when the front obstacle is a pedestrian. CLAIMS 1. A method of assisting a visually impaired person in guiding a system in which a spectacle module and a cane module are connected via a wireless network,
(a) acquiring image information about a front obstacle on a road on which the visually impaired person moves through an image sensor mounted on the spectacle module;
(b) determining whether a front obstacle is present by applying pattern recognition and a machine learning algorithm to the image information obtained in the step (a);
(c) transmitting the image analysis signal determined in step (b) to the wand module through the wireless network;
(d) the cane module generates a control signal based on the image analysis signal transmitted in the step (c), and implements a vibration signal in the cane module according to the control signal or stores it in advance in correspondence with the image analysis signal Providing tactile information to a visually impaired person by implementing a vibration signal according to a vibration pattern of the braille;
The method comprising the steps < RTI ID = 0.0 > of: < / RTI > The method according to claim 6,
Wherein the system further comprises a speaker in the smart module connected to the eyeglass module and the cane module via the wireless network and the cane module,
(e) generating a voice guidance application of the smart device by voice and providing a voice guidance of a direction indication or the like through the speaker;
(f) sensing the front obstacle through the spectacle module and the wand module or outputting the tactile signal to the visually impaired person in the moving direction;
≪ / RTI >

Images