KR100563193B1 - Signal transformation method and guidance system providing auditory and tactile indoor/outdoor traveling information for the blind - Google Patents

Abstract

The present invention relates to a portable walking guidance method and apparatus for transmitting an audible or tactile sense of obstacle information and global position information required for walking of an visually impaired person using an ultrasonic sensor and a GPS.

In the present invention, the comprehensive information of the user's gaze direction, walking direction and left and right direction obstacles obtained from glasses with two ultrasonic sensors and a cart with four ultrasonic sensors is provided. It transmits by vibration through a small motor, it is characterized in that the location information for the user to reach the destination from the GPS (user's current location, distance to the destination, direction information to go to the destination, etc.) and the like voice.

Therefore, the present invention provides the location information necessary for the visually impaired to reach a desired destination, and the distance and direction information on obstacles in the walking direction and the gaze direction during walking, thereby providing an effect of safely reaching the desired destination.

Blind Guidance System, Portable Walking Guidance Device, Ultrasonic Sensor, GPS, Speech Synthesis, Acoustic Cognitive Training, Destination Guidance Device, Obstacle Detection

Description

Sign and Transformation Method and Guidance System Providing Auditory and Tactile Indoor / Outdoor Traveling Information for the Blind}

1 is a conceptual diagram showing the concept of the overall configuration of the present invention.

FIG. 2 is an eyeglasses sub-system diagram equipped with an ultrasonic sensor for the visually impaired illustrated in FIG. 1.

3 is a two-stage folding cart sub-system diagram equipped with an ultrasonic sensor, a vibration motor, and wheels for the visually impaired shown in FIG. 1.

4 is an explanatory diagram of a principle of calculating the distance and direction of an obstacle using reflected waves from two ultrasonic sensors.

FIG. 5 is a block diagram illustrating an internal configuration of the portable signal processing and conversion device shown in FIG. 1.

6 is a block diagram of a reflection wave integration method for improving the obstacle detection distance accuracy of the present invention.

7 is a flowchart illustrating a process of a variable pulse transmission method for expanding an obstacle detection distance according to the present invention.

8 is a block diagram of an ultrasonic transceiver for processing reflected waves by sharing six ultrasonic sensors of an ultrasonic receiver of two channels of the present invention.

9 is a diagram illustrating a method of expressing three types of visual information of the present invention as auditory information of a signal tone that a user can recognize.

10 is a view showing a sound level leveling method for increasing the spatial recognition rate of the present invention.

FIG. 11 is a diagram illustrating a pedestrian path determining method for location guidance from location information acquired from a GPS of the present invention.

12 is a diagram illustrating a speech synthesis method based on a pronunciation combination method for expressing arbitrary syllables as speech in a limited memory of the portable device of the present invention.

13 is a view showing an operation mode and a method of using the operation mode using the keypad of the present invention.

<Description of Major Symbols in Drawing>

100: walk guide device 101: glasses bush system

102: cart bush system 103: signal processing and conversion device

104: stereo earphone 105: small vibration motor

201: Ultrasonic sensor for both transmission and reception

301: Obstacle Detection Ultrasonic Sensor 302: Distance Detection Ultrasonic Sensor

303: small vibration motor 304: wheels

305 handle

501: transmitting unit 502: receiving unit

503: signal conversion unit 504: keypad

505: microcontroller

The present invention relates to a portable walking guidance device for the visually impaired. In particular, by using an ultrasonic sensor and GPS to detect obstacles in the user's gaze direction and walking direction, and delivers the location information necessary to reach the destination as audible (sound or voice) or tactile (vibration) information to safely reach the desired destination. And a method and apparatus for enabling the same.

Although the white-cane is widely used as an auxiliary device for the visually impaired, the use of the white-cane for the visually impaired requires a long period of training and also frequently causes injuries due to lack of detection information such as obstacles. In order to compensate for this, researches on electronic pedestrian devices have been conducted at home and abroad, and obstacle detection devices are mainly used.

The “obstacle detection alarm device for visually impaired people” of Korean Patent Laid-Open Publication No. 2001-0001677 includes a three-stage alarm system that detects the presence of obstacles in front by carrying a detection alarm device equipped with an ultrasonic sensor on the neck. Techniques are disclosed. However, since the sensing distance is short as 1.5 meters, it is difficult to guarantee the safety when considering the walking speed of the user, and since the user only knows the presence or absence of an obstacle, the user needs to check the location of the obstacle using a cane.

In the "glasses equipped with an ultrasonic sensor for the visually impaired" in Korean Patent Laid-Open Publication No. 2002-0031690, a glasses equipped with an ultrasonic sensor are worn to detect an obstacle on the head to adjust a time interval at which a beep is generated according to the distance. Technology to help you walk safely has been posted. However, there is a problem in that the location of the obstacle must be confirmed for safe walking even when the obstacle is recognized due to the lack of information on the direction of the obstacle.

In Korean Unexamined Patent Publication No. 2000-0054125, “Wearable walking guide device and method for the visually impaired”, two cameras and GPS are used to allow the user to safely walk to a destination by detecting obstacles and estimating the user's location. Technology is posted. However, there is a problem in that a high performance computer is required to process the image information of the camera, and a lot of time is required for image processing, and the use thereof is restricted by the wireless communication distance between the wearable computer and the high performance computer.

In foreign cases, GuideCane (I. Ulrich and J. Borenstein, `` The GuideCane-applying '') can connect a small robot with an ultrasonic sensor developed by the University of Michigan, USA, to the tip of the wand to detect obstacles ahead and automatically steer. mobile robot technologies to assis the visually impaired ', IEEE Trans.on Systems, Man and Cybernetics, Part A, vol. 31, no.2, pp. 131-136, 2001). Nav Belt (S. Shoval, J. Borenstein, and Y. Koren, 'The NavBelt-a computerized travel aid for the blind based on mobile robotics technology', IEEE Trans. On Biomedical Engineering, vol. 45, no. 11, pp. 1376-1386, 1998). In addition, New Zealand's Sonic-Vision ( http://www.sonicvision.co.nz ) has commercialized an audio-visual transducer based on ultrasound worn on its head. However, since the detection distance of the obstacle is short and only the fragmentary and rough information on the local obstacle is provided, the use of the obstacle is extremely limited.

In addition, many electronic walking devices serve as simple obstacle sensing devices and thus do not serve as portable walking guide devices for guiding a user safely to a destination.

The present invention is to compensate for the shortcomings of the prior art, an object of the present invention is to detect the obstacles in the gaze direction and walking path required during the walking of the visually impaired, the position required to reach the destination using GPS Obtains information (such as the user's current location, distance to the destination, direction information to reach the destination), and delivers the information to the user as comprehensive auditory (beep or voice) and tactile (vibration) information, as well as local obstacle information. The present invention provides an indoor and outdoor portable walking guide apparatus and method that can suggest an absolute location and a method of movement.

Hereinafter, with reference to the accompanying drawings, the configuration and operation of the invention for the embodiment of the present invention will be described in detail.

1 is a conceptual diagram showing the concept of the overall configuration of the present invention.

As illustrated in FIG. 1, the portable walk guide apparatus 100 for indoor or outdoor use according to the present invention may present a user's absolute position and movement method as well as local obstacle information.

The present invention provides an eyeglass subsystem 101 equipped with two ultrasonic sensors for detecting obstacles in a gaze direction, and four ultrasonic sensors and wheels for detecting obstacles in a walking direction and distance to a wall required for walking indoors. And processing apparatus 103 for extracting information from a folding cart sub-system 102 having an attached tag, and GPS and sensors providing global location information of the user and converting the information into auditory or tactile information that can be recognized by the user. And a stereo earphone 104 for transmitting auditory information and a small vibration motor 105 for delivering tactile information.

2 is a configuration diagram of a spectacles sub-system in which an ultrasonic sensor 201 for both transmission and reception is attached to both legs in order to detect an obstacle in a gaze direction.

3 shows two ultrasonic sensors 301 for detecting obstacles in the walking direction, two ultrasonic sensors 302 for measuring the distance to the left and right walls required for walking indoors, and a user for the distance and direction of the obstacles. Three small vibration motors 303 attached to the handle 305 so as to recognize the information by the touch of the finger, and the wheel to transmit the state of the walking road surface so that the user can intuitively sense using the touch. It is a block diagram of the cart sub system comprised by 304. As shown in FIG.

4 is an explanatory diagram of a principle of calculating the distance and direction of an obstacle using reflected waves from two ultrasonic sensors.

The distance and direction information to the obstacle in the gaze direction by the eyeglass sub-system and the distance and direction information to the obstacle in the walking direction by the cart sub-system are shown by using the two ultrasonic sensors shown in FIG. 4. Obtained in the same way. As shown in FIG. 4, the two ultrasonic sensors A and B attached to the glasses or the cart each emit an ultrasonic wave to measure the time when the reflected wave from the reflector at the point D arrives to calculate the distance from the sensor to the reflector. . Distance and direction information from this to the reflector can be obtained through the following equation.

,

,

,

는 반사체로부터 A, B의 센서까지의 측정거리,

은 두 센서사이의 기준선(baseline),

,

는 기준선(baseline)의 중심을 원점으로 하는 반사체의 x,y 좌표이다.here

,

Is the measurement distance from the reflector to the sensors A and B,

Is the baseline between two sensors,

,

Is the x, y coordinate of the reflector whose origin is the center of the baseline.

5 is a block diagram of a signal processing and converting apparatus for extracting information from an ultrasonic sensor and a GPS and converting the information into auditory or tactile information that can be perceived by a user.

The signal processing and conversion apparatus recognizes a user's information about an obstacle and a user's position obtained from a transmitter 501 for emitting an ultrasonic wave with low power consumption, a receiver 502 for processing a reflected wave signal returned from an object, an ultrasonic sensor, and a GPS. A signal conversion unit 503 for converting to auditory or tactile information, a keypad 504 of an input function such as a user operating a system state, a microcontroller 505 for all signal processing and conversion, and a power supply It consists of a power supply consisting of a battery for supplying. The microcontroller 505 extracts distance information from the transmitted and received signals of the ultrasonic sensor, calculates distance and direction information to obstacles, converts the user into perceivable auditory or tactile information, converts GPS from the user's current location, It is responsible for the distance to the destination, direction information processing for reaching the destination, and voice synthesis for delivering location information to the user.

FIG. 6 illustrates a 'partial pulse reflected wave integration' method for enlarging the distance of an obstacle that can be detected using an ultrasonic sensor and detecting an accurate distance.

The microcontroller 505 fires 16 pulses, which are basic transmission pulses, every 60 ms via the variable pulse ultrasonic transmitter and receives ultrasonic waves (601). The reflected wave reflected back to the object is subjected to reflected wave amplification (602), waveform rectified by an active rectifier without a voltage drop (603), and the signal is integrated by an integrator having a reset input (604). The integrated reflected wave signal is passed through a double threshold threshold circuit having two threshold voltages (605), and two TOF detections are performed (606) across the low and high threshold voltages. The detected TOF is performed by the Line Fitting algorithm (606), so that more accurate TOF detection at the time when the reflected wave signal starts to be received (608), not at the moment when the reflected wave signal exceeds the threshold voltage. Also, in order to increase the obstacle detection distance, the reflected wave of the ultrasonic sensor increases as the number of transmitted pulses increases, but when the number of pulses exceeds a certain number of pulses, the amplitude does not increase but the saturation is performed. The number of ultrasonic pulses to be fired is controlled by increasing the number of waves only to detect obstacles in the distance or weak in the reflected signal.

FIG. 7 illustrates a pseudo pulse transmission algorithm for increasing this obstacle detection distance. The microcontroller initially transmits 16 basic pulses, and if there is no reflected wave for 60 ms, the number of pulses is increased by 16 pulses to emit ultrasonic waves and the reflected signal is detected by the integral 604 of FIG. 6. The detection distance of the ultrasonic sensor used for this apparatus was extended to 10 m by this. If no obstacles are detected, increase the number of ultrasonic pulses up to 128 by 16, and reduce by 16 to reduce energy consumption when obstacles are detected.

In order to solve the volume increase and the cost increase of the system caused by the same transmitting and receiving unit for each ultrasonic sensor in the process of transmitting and receiving the six ultrasonic sensors, it is preferable that the six ultrasonic sensors share the receiving unit of two channels as shown in FIG. proper. The microcontroller sets the ordered pair of ultrasonic waves (right and left sensor of the cart, front sensor of the cart, and sensor of the glasses) to emit the ultrasonic waves every 60ms, and processes the reflected wave signal from the sensor that fired the ultrasonic waves through the analog switch at the receiving unit. Detect.

In order to transmit the information about the plurality of obstacles detected by the present invention and the distance information between the left and right walls to the user, the auditory information using the beep sound and the tactile information transmitting vibration to the finger can be used together. To replace the tiredness.

FIG. 9 shows obstacle information 901 obtained from the glasses sub-system, obstacle information 902 obtained from the cart sub-system, and the cart sub-system, in order to express visual information about several obstacles through a beep. A method of expressing three pieces of visual information such as distance information 903 and 904 from the left and right walls is represented by stereo tones of both ears.

Obstacle information obtained from the glasses and cart subsystem is represented by distance and direction from the user, and these two pieces of information are represented by the frequency of the beep, the time difference between the two ears, and the difference between the beeps. . The time difference reaching both ears to indicate the direction information is expressed as two short beeps because it is sensitively perceived at the beginning of the beep. On the other hand, since the distance information with the left and right walls is expressed only by the frequency of the beep, the long beep is represented. In addition, the distance information from the wall is 160-320Hz, the obstacle information obtained from the cart is 320-640Hz, and the obstacle is obtained from glasses. The information is represented by three octave beeps at 640-1280 Hz. The direction information can be easily distinguished by quantizing the time difference between signals reaching both ears by dividing an area of +/- 45 degrees based on the front of which an obstacle can be detected by an ultrasonic sensor into 9 regions in units of 10 degrees. To help. In addition, since the size difference of sounds heard by both ears of a human person falls within a considerably small range, the spatial recognition rate is improved by enlarging the difference of the volume of sound obtained by the information obtained as shown in FIG. . In order to generate a beep according to the walking rhythm of the user, the beep for the distance information with the left and right walls is configured according to the walking rhythm of the left and right feet, respectively. To occur.

In Figure 9, the beep generation period T is set to 700ms as a basic period in consideration of the walking speed of the visually impaired, and according to the characteristics of the user by adjusting the value of the T with the keypad 504 to be adjusted according to the walking speed of the user is natural Allow walking.

Three small vibration motors 303 were attached to the handles of the carts in order to transmit information about obstacles to tactile information using vibration, which is an alternative expression of auditory information. Compared to hearing, tactile information is indistinguishable, so the distance of obstacles is divided into three categories: close, normal, distant, or none, and the obstacles are divided into five directions: left, center left, center, and center right. The distance to the obstacle is represented by the vibration intensity of the motor, and the direction of the obstacle is the type of the vibrating motor, that is, the left motor only vibrates when the obstacle is on the left side, and when the obstacle is on the left side, the left and center motors vibrate, If it is in the center, only the central motor vibrates, if the obstacle is in the middle right side, the central motor and the right motor vibrate, and if the obstacle is on the right side, only the right motor vibrates.

In order to allow the visually impaired to walk independently in the outdoors, the apparatus provides a location guide function. The device incorporates a portable GPS receiver, and the microcontroller 505 analyzes the received signal from the GPS through serial communication to determine latitude, longitude, speed, direction of movement, and the like. The following two location guidance functions are provided using location information obtained from GPS. The first is a destination guide function, which provides a moving method for moving from a current location to a desired destination. The moving method is expressed by a distance and a direction to be moved, which is transmitted to the user by synthesized voice. In addition, some frequently visited destinations also provide a function that can be stored in advance in the device through manipulation of the keypad 504. The second is the starting point guidance function, which provides the distance and direction information from the starting point to the current position when the user knows the relative position of the place where the user wants to go based on the current position, thereby providing the same effect as the visually impaired person viewing the directions. .

11 shows a method for determining a walking path for these two location guidance functions, which is determined by the following equation.

와

은 단위 위도, 경도에 해당하는 거리이며, 예를 들면 위도 35도 근방에서는 각각 110,941 m/deg와 91,290 m/deg의 값을 갖는다. 또한 기준위치 Pr은 위치안내 기능에 따라서 현재 위치 혹은 출발지가 될 수 있다.here

Wow

Is the distance corresponding to the unit latitude and longitude, and has a value of 110,941 m / deg and 91,290 m / deg, respectively, in the vicinity of latitude 35 degrees. In addition, the reference position Pr may be the current position or starting point according to the position guidance function.

FIG. 12 illustrates a speech synthesis method for transmitting a location information obtained from the GPS, a state of the system, and the like to the user by voice. To store voice, analog sound is stored in a memory and a chipcoder capable of playing at an arbitrary position or message unit is used. In order to efficiently use the limited memory, a combination of words and pronunciation combinations is used to synthesize speech for a sentence to be expressed. Since all words cannot be stored in the memory of the chip coder, about 60 vocabulary words frequently used in the device are stored in the chip coder as a message unit, and the words are combined to synthesize desired sentences into voice. In order to synthesize any syllable out of the pre-stored vocabulary into voice, as shown in FIG. 12, one syllable is composed of 'first + neutral' and another one consisting of 'neutral + species' according to the phonetic notation. The syllables are divided and the syllables are represented by successive combination of these two pronunciations. For example, the word 'Hangul' is expressed as a combination of four pronunciations, 'ha + an, that +', and it is quickly reproduced to be similar to the actual pronunciation. In order to implement this method, sound sources composed of initial and neutral, neutral and final are analyzed based on pronunciation, and about 550 pronunciations are stored in chip coder to synthesize random syllables into voice.

The device of the present invention provides as input device a keypad 504 consisting of twelve keys, similar to a telephone button which is used by a visually impaired user who is a real user. The user can select the functions shown in FIG. 13 provided by the device through a simple key input, and all operation modes have two hierarchical structures, system options and operation modes. Each time you press a key, you can press the asterisk (*) button to cancel the entry or return to the previous step. You can press the shop (#) button to check the current input status. System options include the ability for the user to turn the sound output on or off (1308), the GPS module to turn on or off (1307), the ability to determine whether an ultrasonic sensor is used individually (1305), and the frequency of signal generation to match walking speed. Adjusting function 1306, and the like. The device operates in RDY (1300, Ready) mode when it first starts operation. In this mode, obstacle detection and distance measurement with the wall are automatically performed. The user may set an operation mode through a keypad operation of a function such as a current location guide 1301, a destination guide 1302, or a departure guide 1303 for outdoor walking. We also added a function 1304 to store multiple locations as required by the user. In addition, through the built-in acoustic cognitive training function 1309, the user can learn about the virtual environment through simple keypad operation without the actual walking, the distance and direction information on the obstacle detected by the glasses or the cart without the actual walking. In the acoustic recognition training function, you can select which of the three types of tones shown in FIG. 9 to be trained, and the beeps generated by moving virtual obstacles left, right, left, and right using buttons 2, 4, 6, and 8 of the keypad. Estimate the distance and direction for, and confirm the distance and direction information of the virtual obstacle by using 5 button.

According to the portable walking guide device for the visually impaired according to the present invention as described above, by providing information about the user's current location and the direction of movement to the destination by voice, the visually impaired can reach the desired destination, to the destination By changing the detection information on the obstacles that the user will encounter in the process of moving into auditory information such as a beeping sound, the user can recognize and have the effect of safely walking without the help of a cane or a guide dog.

In addition, there is an effect that the user can walk more freely by converting the obstacle detection information of the gaze direction, walking direction and left and right directions required for walking to a three-dimensional signal sound to the user. In addition, the user can repeatedly use the device of the present invention in a short time because the user can repeatedly learn the beep sound for the distance and direction information on the virtual obstacle using the acoustic cognitive training function built into the device without actually walking. There is.

Claims (14)

Glasses equipped with a first sensing means for detecting the obstacle in the eye direction; A cart having a wheel and having second sensing means for sensing an obstacle in a walking direction, third sensing means for sensing a distance from a wall, and vibration means for transmitting tactile information; An obstacle including a GPS for guiding user location information and signal processing and converting means for extracting information from the first, second and third sensing means and converting the information from the first, second and third sensing means into auditory or tactile information that the user can recognize Detection and location guidance system. The method according to claim 1, The first detection means is an obstacle detection and location guidance system, characterized in that the ultrasonic sensor is mounted on both sides of the glasses. The method according to claim 1, The cart has a handle that can be held by the user, Is connected to the handle portion and consists of a support with a wheel at the bottom, The vibration means is mounted to the handle portion, obstacle detection and position guidance system, characterized in that the first sensing means and the second sensing means are located at a predetermined position of the support. The method according to claim 1 or 3, The vibration means is a small vibration motor, the first and second detection means obstacle detection and position guidance system, characterized in that the ultrasonic sensor. First sensing means for detecting an obstacle in a gaze direction; Second sensing means for detecting an obstacle in a walking direction, Third sensing means for sensing a distance from the wall, Signal processing means for receiving and processing and converting the detection signal from the first, second and third sensing means, The detection signal is visual information consisting of distance and direction information of various obstacles, and the signal processing means converts the visual information into a change in the frequency of the beeping sound, a time difference of the beeping sound reaching both ears, and a difference in the relative magnitude of the beeping sounding in both ears. Obstacle detection device, characterized in that the conversion. The method according to claim 5, The signal processing means allocates information on the distances of the various obstacles to different octaves based on the spatial position of the detecting means detecting the obstacles, and uses the change of frequency in the same octave to determine the distance and the obstacles. Obstacle detection device characterized in that the processing to recognize the spatial distribution of. The method according to claim 5, And the signal processing unit processes the direction information of the obstacle to output two short beeps using the characteristic that time difference information reaching both ears is sensitively detected at the beginning of the beep. The method according to claim 5, And the signal processing means is configured to process the difference in the volume of sound obtained by the obtained information in order to enhance the spatial cognition effect to be expanded to the whole range that can be expressed using the size leveling method. The method according to claim 5, The signal processing means configures the distance information with the left and right walls according to the walking rhythm of the left and right feet, respectively, according to the walking speed of the user, and generates a signal sound indicating the distance and direction information for the obstacles in the walking direction and the gaze direction therebetween. And processing the signal sound generation period to be adjusted according to the walking speed of the user. The method according to claim 5, The signal processing means detects the distance using a reflected wave integration method by a circuit consisting of a reflected wave amplifier, an active rectifier, an integrator, a double threshold voltage, and a linear fit to increase the accuracy of the distance of the obstacle detected by the sensing means. Obstacle detection device, characterized in that. The method according to claim 5, The first, second and third sensing means is made of an ultrasonic sensor, And a plurality of ultrasonic sensors share one receiver to process the reflected wave signal. The method according to claim 11, The ultrasonic sensor may detect the obstacle actively by adjusting the number of pulses transmitted by the ultrasound in accordance with the presence or absence of the obstacle in order to increase the detection distance of the obstacle. The method according to claim 1, The auditory information processing in the signal processing and conversion means stores one syllable in memory by dividing it into one pronunciation consisting of 'primary + neutral' and another pronunciation consisting of 'neutral + species' according to the phonetic notation. Sudan, And means for synthesizing speech by expressing syllables by successive combinations of the two pronunciations. The method according to claim 1, A control means for moving the position of the virtual obstacle, Obstacle detection further comprising a training means for estimating the visual information from the auditory information by checking the position information of the obstacle through the voice, after estimating the beep to the distance and direction information of the obstacle by the control means; And location guidance system.

Images