KR102351584B1 - System for providing navigation service for visually impaired person - Google Patents

Abstract

The present invention relates to a navigation service system for the visually impaired.
As an example, a first method for receiving RTK-GPS-based current location information from a GPS satellite, generating real-time image information by capturing surrounding images and detecting obstacles while walking, and transmitting the current location information and the real-time image information in real time wearable device; It is connected to the first wearable device unit, receives destination information from the user by voice, transmits it to the first wearable device unit, detects the user's gaze direction and travel direction using at least one sensor, and the first a second wearable device unit that provides route information and obstacle information received from the wearable device unit as navigation information through an auditory method and a tactile method; and the first wearable device unit to receive the current location information, the real-time image information, and the destination information, respectively, and generate the route information based on the current location information and the destination information, and the real-time image and a cloud server unit for generating the obstacle information based on the information, and transmitting the path information and the obstacle information to the second wearable device unit, wherein the first wearable device unit includes the destination information from the cloud server unit Disclosed is a navigation service system for the visually impaired that receives the route information and the obstacle information in response to .

Description

Navigation service system for the visually impaired

An embodiment of the present invention relates to a navigation service system for the visually impaired.

In recent years, assistive equipment for the visually impaired, such as a smart watch with built-in Braille, has been continuously developed, but the reality is that the means to assist the visually impaired in walking remains in the past, such as a walking stick.

The recently developed gait information guidance system for the visually impaired generally provides only the walking direction with voice or provides information on fixed objects around the path, but cannot respond to moving obstacles in an integrated way. is not provided as

In addition, since the user location information through the general GPS has an error of more than 1 to 2 meters, and its accuracy is very low, the risk of a pedestrian accident is inevitably high. Also, in terms of usability of assistive devices for the visually impaired for this purpose, in most cases, they are not worn directly on the body, but are held in the hand, so portability is somewhat cumbersome.

In addition, the conventional gait information guide device and system for the visually impaired provide information necessary for walking through voice. , there is a problem in that visual information necessary for walking cannot be effectively audible and transmitted.

Registered Patent Publication No. 10-1988037 (Registration Date: June 04, 2019)

An embodiment of the present invention provides route guidance service information based on location information of RTK-GPS, image information of a three-dimensional depth camera, a sensor for detecting the moving direction and gaze direction of pedestrians, and various information obtained from them. It provides a wearable route guidance service system for the visually impaired that can provide route guidance information including information about the surrounding environment in a more accurate and intuitive way using a cloud server.

The navigation service system for the visually impaired according to an embodiment of the present invention receives RTK-GPS-based current location information from a GPS satellite, and generates real-time image information by capturing images of surrounding areas and detecting obstacles while walking, and A first wearable device unit for transmitting the current location information and the real-time image information in real time; It is connected to the first wearable device unit, receives destination information from the user by voice, transmits it to the first wearable device unit, detects the user's gaze direction and travel direction using at least one sensor, and the first a second wearable device unit that provides route information and obstacle information received from the wearable device unit as navigation information through an auditory method and a tactile method; and the first wearable device unit to receive the current location information, the real-time image information, and the destination information, respectively, and generate the route information based on the current location information and the destination information, and the real-time image and a cloud server unit for generating the obstacle information based on the information, and transmitting the path information and the obstacle information to the second wearable device unit, wherein the first wearable device unit includes the destination information from the cloud server unit In response to , the path information and the obstacle information are received.

In addition, the first wearable device unit, a location information receiver for receiving the current location information based on RTK-GPS from a GPS satellite; an image information generator including a three-dimensional depth camera for capturing images of surrounding areas while walking, and generating image data of the three-dimensional depth camera as the real-time image information; a control unit generating a control signal for a navigation operation of the second wearable device unit based on the path information and the obstacle information; and a first wireless communication unit for wirelessly communicating with the second wearable device unit and the cloud server unit, respectively.

In addition, the first wearable device unit may include: a first wearing unit in which the image information generating unit is installed and formed in the form of a band that can be worn on the user's upper body; And the location information receiving unit, the control unit, and the first wireless communication unit is installed may further include a second wearable in the form of a backpack that can be worn on the user's shoulder.

In addition, the control unit compares the path information with the traveling direction and the gaze direction in the virtual space, and when the user moves in a state in which the traveling direction matches the path information within a preset range, a preset order A sound data having a scale according to Depending on the degree, the volume of the warning sound data may be adjusted to be increased or increased in stages.

In addition, the control unit compares the path information with the traveling direction and the gaze direction in a virtual space, and when the user moves in a state in which the traveling direction matches the path information within a preset range, a preset vibration Data is generated and included in the control signal, and when the traveling direction deviates from the path information, the vibration pattern and size of the vibration data are adjusted, and the vibration data according to the degree to which the traveling direction deviates from the path information It can be adjusted to increase the vibration pattern in steps rapidly or to increase the vibration amplitude step by step.

In addition, the second wearable device unit, the head wearing structure forming at least one of the glasses and the headset; a sensor unit installed in the head wearing structure and including a gyro sensor and a geomagnetic sensor for detecting a user's gaze direction and a moving direction; a voice input unit including a microphone for receiving a user's voice; a sound output unit including a speaker for providing the control signal received from the first wearable device unit as route guidance information through an auditory method; a vibration generating unit including a vibrating element for providing the control signal received from the first wearable device unit as navigation information through a tactile method; and a second wireless communication unit for wireless communication with the first wearable device unit.

In addition, the cloud server unit is connected to the first wearable device unit to receive the current location information, the real-time image information, and the destination information, respectively, and transmit the route information and the obstacle information to the second wearable device unit. a data transceiver for transmitting; and a machine learning algorithm execution unit for generating the route information according to the current location information and the destination information using a navigation service, and recognizing an object from the real-time image information based on a machine learning algorithm to generate the obstacle information can do.

In addition, in the navigation service system, at least one of the first wearable device unit and the second wearable device unit is one of a route guidance re-request signal and a route guidance emergency call signal according to a push time and a number of pushes and a call switch unit for generating and transmitting to the cloud server unit through the first wireless communication unit, wherein the cloud server unit transmits route change information to the first wearable device unit upon reception of the route guidance re-request signal or , when the route guidance emergency call signal is received, the vehicle may be called to move to the emergency call point.

In addition, the cloud server unit, when receiving the route guidance re-request signal from the first wearable device unit, changes the route information to the destination information based on the current location information, and sets the route change information to the first 1 emergency request processing unit for route guidance to be transmitted to the wearable device; and when receiving the road guidance emergency call signal from the first wearable device unit, search and call a vehicle that can move to the user's current location based on the current location information in conjunction with a call taxi system or a vehicle call service system, , when the corresponding vehicle enters within a preset distance from the user's current location, at least one of acoustic recognition information having a scale according to a preset order according to the approach distance and vibration recognition information adjusted according to a preset vibration pattern and size The first wearable device unit may further include a vehicle call service unit for providing access information of the corresponding vehicle.

According to the present invention, a cloud server that provides route guidance service information based on location information of RTK-GPS, image information of a three-dimensional depth camera, a sensor for detecting a pedestrian's moving direction and gaze direction, and various information obtained therefrom It is possible to provide a wearable way guide service system for the visually impaired that can provide way guide information including information about the surrounding environment in a more accurate and intuitive way by using

1 is a schematic diagram showing the overall configuration of a navigation service system for the visually impaired and their relationship according to an embodiment of the present invention.
2 is a block diagram showing a detailed configuration of the first wearable device unit according to an embodiment of the present invention.
3 is a view showing an example of the appearance of the image information generating unit and the first wearing unit according to an embodiment of the present invention.
4 is a diagram illustrating an example of the appearance of an image information generating unit and a location information receiving unit according to an embodiment of the present invention.
5 is a diagram illustrating a configuration of a control signal including sound data according to an embodiment of the present invention.
6 is a diagram illustrating a configuration of a control signal according to an error between a moving direction of a visually impaired person and path information provided by a cloud server unit according to an embodiment of the present invention.
7 is a block diagram illustrating a detailed configuration of a second wearable device unit according to an embodiment of the present invention.
8 is a view showing an example of the external appearance of the second wearable device according to an embodiment of the present invention.
9 is a block diagram illustrating a detailed configuration of a cloud server unit according to an embodiment of the present invention.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a schematic diagram showing the overall configuration of a navigation service system for the visually impaired according to an embodiment of the present invention and their relationship, and FIG. 2 is a detailed configuration of a first wearable device according to an embodiment of the present invention 3 is a diagram showing an example of the appearance of the image information generating unit and the first wearing unit according to the embodiment of the present invention, and FIG. 4 is the image information generating unit and the location information receiving unit according to the embodiment of the present invention. It is a view showing an example of the external appearance of, Figure 5 is a view to explain the configuration of a control signal including sound data according to an embodiment of the present invention, Figure 6 is a view showing the visually impaired according to an embodiment of the present invention It is a diagram illustrating the configuration of a control signal according to an error between the progress direction and the path information provided by the cloud server unit, and FIG. 7 is a block diagram showing the detailed configuration of the second wearable device unit according to an embodiment of the present invention , FIG. 8 is a view showing an example of an external appearance of a second wearable device unit according to an embodiment of the present invention, and FIG. 9 is a block diagram showing a detailed configuration of a cloud server unit according to an embodiment of the present invention.

Referring to FIG. 1 , a navigation service system 1000 for the visually impaired according to an embodiment of the present invention includes a first wearable device unit 100 , a second wearable device unit 200 , and a cloud server unit 300 . ) is included.

The first wearable device unit 100 receives the current location information based on RTK-GPS (Real-Time 　 Kinematic GPS) from the GPS satellite 10 as shown in FIG. Real-time image information may be generated through image capturing and obstacle detection, and the generated current location information and real-time image information may be transmitted to the cloud server unit 300 in real time. In addition, the first wearable device unit 100 receives the optimal path information and obstacle information about the current location in response to the destination information requested by the visually impaired from the cloud server unit 300 to receive the second wearable device unit In 200, a control signal for path guidance and obstacle guidance may be generated and transmitted.

To this end, the first wearable device unit 100 includes a location information receiving unit 110 , an image information generating unit 120 , a control unit 130 , a first wireless communication unit 140 , and a first wearing unit as shown in FIG. 2 . It may include at least one of the unit 150 , the second wearing unit 160 , and the call switch unit 170 .

The location information receiver 110 may receive the current location information of the visually impaired based on Real-Time Kinematic GPS (RTK-GPS) from the GPS satellite 10 . The location information receiver 110 receives a correction message in the RTCM (Real-Time 　 Differential 　 Correction 　 Maritime) format from the base station installed in a fixed place through the RTK GPS, and grasps and grasps the current location of the visually impaired with an accuracy of centimeters. The location information may be transmitted to the controller 130 through the serial port. To this end, the location information receiver 110 may be configured to include an antenna for receiving satellite signals. Here, the antenna may be configured in the form of a film in order to reduce the weight of the first wearable device unit 100 . On the other hand, the base station installed in a fixed place is self-configured, or a number of GNSS constant observation stations connected to the DMB antenna currently installed nationwide or the GNSS (Global Navigation　Satellite System) data integration center can be connected to the NTRIP (Network Transport Feature of RTCM via Internet Protocol). ) can be configured as a server.

The RTK-GPS antenna of the location information receiving unit 110 protrudes from the second wearing unit 160 (in the form of a backpack or bag) to be described later and the antenna is level with the ground in order to increase the satellite signal reception rate, so that the interference of the head or other body It is preferable to be installed in a position where this is minimized.

The image information generating unit 120 includes a distance detection sensor (infrared sensor, ultrasonic wave sensor) that detects a distance between a three-dimensional depth camera for capturing images of the surrounding area while walking by the visually impaired and an object (obstacle) within a shooting range of the three-dimensional depth camera. sensor, etc.), and may generate image data of a three-dimensional depth camera and distance sensing data of a distance sensor as real-time image information.

In addition, the image information generating unit 120 acquires image information about the surrounding environment when walking through a three-dimensional depth camera and at the same time when walking using a distance sensor including an infrared sensor and/or an ultrasonic sensor included therein. The first wireless communication unit ( 140) through the cloud server unit 300 to determine what kind of object or object included in the real-time image information is through a deep learning-based object recognition algorithm, and how much each identified object or object is from the visually impaired. It can be configured to detect how far the distance is, or how quickly the distance is approaching. A more detailed description of the cloud server unit 300 will be described later.

On the other hand, the image information generating unit 120 may apply a plurality of three-dimensional depth cameras to cover all directions of the front, back, left, and right of the visually impaired, and such a three-dimensional depth camera can be generated from the walking or running movement of the visually impaired. An elastic body such as a rubber band or a band applied to the first wearing part 150, which will be described later, is configured to minimize the adverse effects of left and right shaking on image processing. In addition, a gyro sensor, an acceleration sensor and a motor are combined to operate. An anti-shake device may be additionally configured.

As shown in FIG. 3 , the image information generating unit 120 may be disposed on the front of the first wearing unit 150 , and may be detachably installed from the first wearing unit 150 . The control unit 130 may generate a control signal for a navigation operation of the second wearable device unit 200 based on the path information and the obstacle information received from the cloud server unit 300 .

More specifically, the control unit 130 compares the path information received from the cloud server unit 300 with the moving direction and the gaze direction sensed by the second wearable device unit 200 in the virtual space to determine the moving direction. When a visually impaired person moves in a state consistent with route information within a preset range, sound data having a scale according to a preset order may be generated and included in the control signal. For example, as shown in FIG. 5 , by composing 'do-mi-sol-do' as a single scale (C scale), it is possible to easily distinguish between the front and rear notes, and thus the visually impaired Follow the order of this scale to accurately follow the path and finally arrive at the destination.

The controller 130 may set the virtual sound generating device in the virtual space at intervals of approximately 5 m along the optimal path through software that reflects the geographic space of the real world. When the virtual sound generating device approaches or moves away from the virtual sound generating device as the blind person moves on a path, the second wearable device unit 200 may transmit it by reflecting it in the control signal. At this time, according to the gaze direction measured through the second wearable device unit 120, a virtual sound effect is transmitted through a three-dimensional sound technology using a head transfer function (HRTF), as if it exists on the real path. to be intuitively recognizable. In addition, whenever a visually impaired person passes the virtual sound generating device in close proximity, it provides feedback that he is following the path, so that the output is subdivided into sound 1, sound 2, and sound 3 according to the degree of proximity. You can easily see how well you are following that path.

However, when the moving direction of the visually impaired deviates from the path information, the control unit 300 generates warning sound data as shown in FIG. 6 and includes it in the control signal, and according to the degree to which the moving direction deviates from the path information You can adjust the volume of the beep data to increase in steps or to speed up.

On the other hand, various obstacles encountered while walking can also be expressed as virtual sound generators. In this case, the obstacles are sounds with different textures depending on the type, and the obstacles are sounds that are generated at the place where the object is located in the real space. is expressed as Accordingly, the visually impaired can identify and avoid the type and location of obstacles through the location of the sound and its texture.

The controller 300 compares the corresponding path information with the moving direction and the gaze direction of the visually impaired in the virtual space, and when the user moves in a state that the corresponding moving direction matches the path information within a preset range, preset vibration Data can be generated and included in the control signal. That is, similar to the above, the virtual vibration generating device is set at regular intervals in the virtual space, and the avatar matching the location of the visually impaired by setting the path information in which the moving direction of the visually impaired is set is applied to the virtual vibration generating device. When approaching, a specific pattern of vibration is provided so that the visually impaired can recognize that they are moving in the correct path.

However, when the moving direction of the visually impaired deviates from the path information, the control unit 300 adjusts the vibration pattern and size of the vibration data as shown in FIG. It can be adjusted so that the vibration pattern of the data increases rapidly in steps or the magnitude of the vibration increases in steps.

On the other hand, various obstacles encountered while walking can also be expressed as virtual vibration generating devices. At this time, the obstacles are vibrations with different textures depending on the type, and the obstacles are vibrations that occur at the place where the object is located in the real space. is expressed as Accordingly, the visually impaired can identify and avoid the type and location of obstacles through the location and texture of the vibration.

In addition, when the control unit 300 generates the vibration data and includes it in the control signal to provide it to the second wearable device unit 200, similarly to the above-described sound data, the virtual vibration generating device matches the actual path information. By setting a virtual vibration generating device at regular intervals in the virtual space where the visually impaired approaches, based on the current location information, a specific size or pattern vibration data or increase/decrease in vibration size is adjusted to help the visually impaired through vibration information. Make sure you know that you are moving on the correct path.

The controller 130 generates a virtual 3D space and avatar reflecting the geographic space and user of the real world, together with software for controlling the overall operation of the first and second wearable device units 100 and 200, and It can monitor, and it can include software that creates the auditory information needed by real users in a way that reflects all the collected information in the virtual space, and can be implemented through an embedded PC, etc.

The first wireless communication unit 140 may perform wireless communication with the second wearable device unit 200 and the cloud server unit 300 , respectively. At this time, the first wireless communication unit 140 transmits the auditory and vibration signals (or vibration data) to the sound output unit 240 and the vibration generating unit 250 of the second wearable device unit 200 in a short distance such as Bluetooth. A wireless communication technology may be applied, and a wireless Internet communication technology using 5G technology for stable and fast communication with the cloud server unit 300 may be applied.

Meanwhile, the first wireless communication unit 140 may include a display function for monitoring the state of the control unit 130 itself, and information from the location information receiving unit 110 , the sound output unit 240 and the vibration generating unit 250 . It can be configured to include a serial port function to receive The first wireless communication unit 140 may be manufactured in the form of a PCB board optimized from the circuit stage for miniaturization and weight reduction. The first wearing unit 150, the image information generating unit 120 is installed, may be formed in the form of a band that can be worn on the upper body of the visually impaired. For example, as shown in FIG. 3 , the first wearing part 150 has the same shape as an X-shaped vest or top, and a belt may be added to better fix it to the upper body, and the belt has elastic force. It is preferable that it is made in the form of a band having In addition, it is desirable to minimize discomfort by disposing a support cushion on the lower part of the waist of the vest or top so that it is in close contact with the body. In addition, a waterproof coating is applied to the entire surface to facilitate cleaning with a cleaner or the like in case of contamination.

The second wearing unit 160, the location information receiving unit 110, the control unit 130, and the first wireless communication unit 140 are installed, may be made in the form of a backpack that can be worn on the user's shoulder. For example, the second wearing unit 160 may be formed in the form of a bag structure, but may be implemented in a form that can be stored in the bag structure and carried through the bag as shown in FIG. 6 . The second wearable device unit 160 may be equipped with a removable battery to supply power to the power supply unit.

The power supply unit may be configured with a rechargeable battery so that the user can freely work outdoors for a certain time, and preferably has a capacity that can be used for more than a day after being charged. However, it is not necessarily limited to this configuration, and may be appropriately configured as needed in consideration of the capacity, weight, price, and the like of the battery.

The call switch unit 170 is installed in at least one of the first wearable device unit 100 and the second wearable device unit 200, and according to the push time and the number of pushes, a route guidance re-request signal and directions One of the guidance emergency call signals may be generated and transmitted to the cloud server unit 300 through the first wireless communication unit 140 .

For example, the call switch 170 may be formed in the form of a button, and when pressed for about 3 seconds or more, a route guidance re-request signal is generated to be transmitted to the cloud server unit 300 through the first wireless communication unit 140 and , can be continuously pressed three times to generate a route guidance emergency call signal to be transmitted to the cloud server unit 300 through the first wireless communication unit 140 . The configuration of the service provided by the cloud server unit 300 by generating such a signal will be described later.

The second wearable device unit 200 is connected to the first wearable device unit 100, receives destination information from the visually impaired by voice and transmits it to the first wearable device unit 100, at least one It is possible to detect the gaze direction and the moving direction of the visually impaired by using the sensor of have.

To this end, the second wearable device unit 200 includes a head wearing structure 210 , a sensor unit 220 , a voice input unit 230 , a sound output unit 240 , a vibration generating unit 250 , and a second wireless communication unit ( 260) may include at least one of.

As shown in FIG. 8 , the head wearing structure 210 may take the form of at least one of glasses and a headset.

The sensor unit 220 may include a gyro sensor and a geomagnetic sensor installed in the front of the head wearing structure 210 to detect a gaze direction and a moving direction of the visually impaired. Accordingly, the gyro sensor and the geomagnetic sensor are installed in a position that matches the rotation axis of the neck as much as possible in the head wearing structure 210 , so that the gaze direction of the visually impaired can be grasped as precisely as possible.

In addition, the second wearable device unit 200 includes a gyro sensor and a magnetometer of the sensor unit 220 and a moving direction information providing part comprising a sound output unit 240 and a vibration generating unit 250 to be described later. It is composed of IMU sensors in the form of microboards together, and like a compass, the sensor measures the direction the user's gaze is facing in real time through the method of measuring the angle with the Earth's Magnetic North, and the control unit 130 through the serial port. It can be configured to pass to In addition, in order to correct an error of the geomagnetic sensor that may occur over time, the calibration state of the IMU sensor may also be transmitted to the controller 130 through the serial port at all times. When there is a problem in the correction state of each sensor, the control unit 130 may include a function to detect and guide to perform a series of correction actions.

The voice input unit 230 may include a microphone for receiving a user's voice. The voice input unit 230 may be configured such that the cloud server unit 300 may specify a user of the device by processing the voice input through the microphone like a kind of fingerprint that identifies a user. Accordingly, the cloud server unit 300 checks the current situation information and history information such as the moving route, destination, and current location of a specific user, so that viewing and monitoring are possible if necessary.

The sound output unit 240 may include a speaker for providing a control signal received from the first wearable device unit 100 as navigation information through an auditory method. The sound output unit 240 is installed at a position close to the ear among the head wearing structures 210, and may transmit information about the route to the destination and surrounding obstacles through short-range wireless communication such as Bluetooth in an auditory manner.

The vibration generating unit 250 may include at least one vibration element or a vibration motor for providing a control signal received from the first wearable device unit 100 as navigation information through a tactile method. The vibration element of the vibration generating unit 250 may be installed in the frame structure to transmit information on a route to a destination and surrounding obstacles through short-range wireless communication such as Bluetooth in a tactile manner.

The second wireless communication unit 260 may be configured with devices for short-range wireless communication with the first wearable device unit 100 .

The cloud server unit 300 is connected to the first wearable device unit to receive the current location information, the real-time image information, and the destination information, respectively, and the route information based on the current location information and the destination information to generate, generate the obstacle information based on the real-time image information, and transmit the path information and the obstacle information to the second wearable device unit

To this end, the cloud server unit 300 may include at least one of a data transmission/reception unit 310 and a machine learning algorithm execution unit 320 .

The data transceiver unit 310 is connected to the first wearable device unit 100 and receives current location information, real-time image information (including image + distance data) and destination information from the first wearable device unit 100, respectively. , and may transmit path information and obstacle information to the second wearable device unit 200 . Here, the data transceiver 310 may receive an image from the first wearable device unit 100 in a streaming manner.

The machine learning algorithm execution unit 320 may use a navigation service to generate route information according to the current location information and destination information, and based on the machine learning algorithm, recognize objects from real-time image information to generate obstacle information. have. In particular, by using an object recognition algorithm among machine learning algorithms, the type and location of obstacles in the image can be determined and transmitted to the control unit 130, and the guide block for the blind in the image information is analyzed through a lane recognition algorithm to correct GPS information and It can be used for route departure warning.

On the other hand, the cloud server unit 300 transmits the route change information to the first wearable device unit 100 when receiving the route guidance re-request signal, or when receiving the route guidance emergency call signal, the vehicle moves to the emergency call point can be called to

To this end, the cloud server unit 300 may further include at least one of a route guidance emergency request processing unit 330 and a vehicle call service unit 340 .

The route guidance emergency request processing unit 330, when receiving a route guidance re-request signal from the first wearable device unit 100, changes the route information to the destination information based on the current location information of the visually impaired, and the route The change information may be transmitted to the first wearable device unit 100 .

For example, the visually impaired person is not familiar with the navigation service provided by the system 1000, or the auditory and/or tactile information for navigation is wrongly judged and is located at a point out of the route. When it is confusing to understand the information, when a new route is recommended or when a new destination needs to be changed, the call switch unit 170 is pressed for a certain period of time to receive a route guidance re-request signal, at this time, the current location By receiving the information together, it is possible to generate and provide information for a new route guidance service.

The vehicle call service unit 340, when receiving a road guidance emergency call signal from the first wearable device unit 100, works in conjunction with a call taxi system or a vehicle call service system based on the current location information of the visually impaired. When a vehicle that can move to a location is searched for and called, and the called vehicle enters within a preset distance from the current location of the visually impaired, acoustic recognition information and preset vibration having a scale according to a preset sequence according to the approach distance At least one of the vibration recognition information adjusted according to the pattern and size may be transmitted to the first wearable device unit 100 to provide access information of the corresponding vehicle.

For example, when a visually impaired person moves along a route and has to move quickly to a corresponding destination, or when another destination occurs in a hurry from the current location, the cloud server unit 300 ), the cloud server unit 300 automatically calls a taxi that can move from the current location called by the visually impaired, and the taxi moves to the place where the visually impaired is located. When the distance between the two objects is reduced to within a preset distance by comparing the GPS information of the taxi with the location where the visually impaired requested emergency service using the GPS information of the taxi, the acoustic signal and vibration signal according to the approach distance Changes are made so that the approaching level of the called taxi can be recognized even if it is invisible.

According to the present invention, more accurate and specific navigation in a macroscopic range is made possible by using the centimeter-accuracy RTK-GPS, which replaces the conventional metric-unit-accuracy GPS.

In addition, by using the conventional infrastructure such as the blind guide block as a target for the lane recognition algorithm through image information analysis, it is possible to provide more specific guidance on the path of the visually impaired in a microscopic range without investment in additional infrastructure.

In addition, it serves to correct user location information using RTK-GPS, enabling navigation even indoors where GPS does not work well.

In addition, by solving the object recognition algorithm that requires a lot of computing power by way of a server, more specific real-time information about the type or property of an obstacle can be used for road guidance. In addition, it is possible to provide intuitive, accurate, and safer route guidance by delivering 3D stereophonic sound signals and vibration signals using the head transfer function to the visually impaired.

What has been described above is only one embodiment for implementing the route guidance service system for the visually impaired according to the present invention, and the present invention is not limited to the above embodiment, but as claimed in the claims below Without departing from the gist of the invention, it will be said that the technical spirit of the present invention exists to the extent that various modifications can be made by anyone with ordinary knowledge in the field to which the invention pertains.

1000: navigation service system for the visually impaired
100: first wearable device unit
110: location information receiving unit
120: image information generating unit
130: control unit
140: first wireless communication unit
150: first wearing part
160: second wearing part
170: call switch unit
200: second wearable device unit
210: head wearing structure
220: sensor unit
230: voice input unit
240: sound output unit
250: vibration generating unit
260: second wireless communication unit
300: cloud server unit
310: data transceiver
320: machine learning algorithm execution unit
330: road guidance emergency request processing unit
340: ride-hailing service department
10: GPS satellite

Claims (8)

A first wearable device that receives RTK-GPS-based current location information from a GPS satellite, generates real-time image information through image capture and obstacle detection while walking, and transmits the current location information and the real-time image information in real time wealth;
It is connected to the first wearable device unit, receives destination information from the user by voice, transmits it to the first wearable device unit, detects the user's gaze direction and travel direction using at least one sensor, and the first a second wearable device unit that provides route information and obstacle information received from the wearable device unit as navigation information through an auditory method and a tactile method;
It is connected to the first wearable device unit to receive the current location information, the real-time image information and the destination information, respectively, generate the route information based on the current location information and the destination information, and the real-time image information a cloud server unit for generating the obstacle information based on , and transmitting the path information and the obstacle information to the second wearable device unit; and
It is installed in at least one of the first wearable device unit and the second wearable device unit, and generates one of a route guidance re-request signal and a route guidance emergency call signal according to the push time and the number of pushes to generate the first wearable device. and a call switch unit for transmitting to the cloud server unit through the type device unit,
The first wearable device unit receives the route information and the obstacle information in response to the destination information from the cloud server unit,
The cloud server unit,
a data transceiver connected to the first wearable device unit to receive the current location information, the real-time image information, and the destination information, respectively, and transmit the route information and the obstacle information to the second wearable device unit; and
A machine learning algorithm execution unit for generating the route information according to the current location information and the destination information using a navigation service, and recognizing an object from the real-time image information based on a machine learning algorithm to generate the obstacle information, ,
The cloud server unit,
When the route guidance re-request signal is received, route change information is transmitted to the first wearable device unit, or when the route guidance emergency call signal is received, the vehicle is called to move to an emergency call point,
The cloud server unit,
When receiving the route guidance re-request signal from the first wearable device unit, changing the route information to the destination information based on the current location information, and transmitting the route change information to the first wearable device unit Route guidance emergency request processing unit; and
When receiving the route guidance emergency call signal from the first wearable device unit, search and call a vehicle that can move to the user's current location based on the current location information in conjunction with a call taxi system or a vehicle call service system, When the corresponding vehicle enters within a preset distance from the user's current location, at least one of sound recognition information having a scale according to a preset order according to the approach distance and vibration recognition information adjusted according to a preset vibration pattern and size is said Direction guidance service system for the visually impaired, characterized in that it further comprises a vehicle call service unit that transmits to the first wearable device unit and provides access information of the corresponding vehicle.
According to claim 1,
The first wearable device unit,
a location information receiver for receiving the current location information based on RTK-GPS from a GPS satellite;
an image information generator including a three-dimensional depth camera for capturing images of surrounding areas while walking, and generating image data of the three-dimensional depth camera as the real-time image information;
a control unit generating a control signal for a navigation operation of the second wearable device unit based on the path information and the obstacle information; and
A navigation service system for the visually impaired, characterized in that it comprises a first wireless communication unit for wireless communication with the second wearable device unit and the cloud server unit, respectively.
3. The method of claim 2,
The first wearable device unit,
a first wearing part in which the image information generating part is installed and in the form of a band that can be worn on the user's upper body; and
The navigation service system for the visually impaired, characterized in that it further comprises a second wearable part in the form of a backpack on which the location information receiving unit, the control unit, and the first wireless communication unit are installed.
3. The method of claim 2,
The control unit is
Sound having a scale according to a preset order when the user moves in a state in which the route information, the travel direction, and the gaze direction are compared with the route information in the virtual space and the travel direction matches the route information within a preset range Data is generated and included in the control signal, and when the traveling direction deviates from the path information, alarm sound data is generated and included in the control signal, and the warning sound data according to the degree to which the traveling direction deviates from the path information A navigation service system for the visually impaired, characterized in that the volume is adjusted to be increased or increased in stages.
3. The method of claim 2,
The control unit is
By comparing the path information, the traveling direction, and the gaze direction in the virtual space, when the user moves in a state in which the traveling direction matches the path information within a preset range, preset vibration data is generated and the control is performed. Including in the signal, if the traveling direction deviates from the path information, adjust the vibration pattern and size of the vibration data, but the vibration pattern of the vibration data is gradually changed according to the degree to which the traveling direction deviates from the path information A navigation service system for the visually impaired, characterized in that it increases rapidly or adjusts the vibration amplitude step by step.
3. The method of claim 2,
The second wearable device unit,
a head wearing structure forming at least one of glasses and a headset;
a sensor unit installed in the head wearing structure and including a gyro sensor and a geomagnetic sensor for detecting a user's gaze direction and a moving direction;
a voice input unit including a microphone for receiving a user's voice;
a sound output unit including a speaker for providing the control signal received from the first wearable device unit as route guidance information through an auditory method;
a vibration generating unit including a vibrating element for providing the control signal received from the first wearable device unit as navigation information through a tactile method; and
Navigation service system for the visually impaired, characterized in that it comprises a second wireless communication unit for wireless communication with the first wearable device unit.
delete delete

Images