KR20220073606A - The vision assistance device and method capable of escalator object recognition - Google Patents

Abstract

Matching to generate measurement information of the direction in which the pedestrian proceeds, determine the object appearing from the measurement information based on the object determination model, determine the risk level of the object, and inform the pedestrian's path according to the location and risk level of the object Provided is a visual assistance device that extracts audio information to be used and outputs the audio information.

Description

A visual aid device and method capable of object recognition for an escalator

The present invention relates to a visual assistance apparatus and method, and more particularly, to a visual assistance apparatus and method provided to visually assist a pedestrian in walking.

In general, the visually impaired feel difficulty in recognizing objects, such as obstacles, that exist in a walking direction, and thus, walks with the help of a walking helper or a walking guide dog.

Accordingly, conventionally, a technique for determining the location of an object using a beacon installed on an object has been known, but determining the location of an object using a beacon can only determine the location of an object on which a beacon is installed, and a beacon is not installed. There is a limitation in not being able to determine the location of an object that is not there.

In addition, conventionally, a technique for determining the presence or absence of an object from an image by photographing the front of the pedestrian is known, but this has a limitation in knowing only whether an object exists in front of the pedestrian, and by determining the exact location of the object It is difficult to inform pedestrians of countermeasures.

Accordingly, there is a need for a method for determining the exact location of an object existing in the moving direction of the pedestrian and, accordingly, notifying the pedestrian of a suitable response method.

Domestic Registered Patent No. 10-0734624 (2007.06.26.)

SUMMARY The technical problem to be solved by the present invention is to provide a visual assistance apparatus and method for recognizing an object existing in a moving direction of a pedestrian and notifying a pedestrian of the existence of the object.

In one aspect of the present invention, a sensor unit for generating measurement information by measuring a situation in a direction in which a pedestrian is traveling; a storage unit for storing an object determination model generated by learning to determine an arbitrary object from the measurement information; Determine the object appearing from the measurement information based on the object determination model, and determine the risk level of the object based on a risk criterion set differently according to the type of the object classified into a walking aid object, a fixed object, and a moving object a controller for determining and extracting matching voice information to inform the pedestrian's path according to the location of the object and the level of risk; and an output unit for outputting the voice information.

In addition, when it is determined that the object is a step that is a walking assistance object, the control unit extracts an outer boundary line of the object from the measurement information, sets an outer boundary indicating the appearance of the object, and is located inside the outer boundary. Existing inner boundary lines are extracted, the number of steps of the stairs is determined according to the number of the inner boundary lines, the distance between two adjacent inner boundary lines is calculated, the height of one step of the stairs is determined, and one step of the stairs is determined. The level of risk can be determined according to the height of the stairs and the number of steps.

In addition, when it is determined that the object is a moving walk that is a walking aid object, the control unit extracts an outer boundary line of the object from the measurement information, sets an outer boundary indicating the appearance of the object, and the inner side of the outer boundary By extracting the inner boundary line present in the, the amount of position change of any one inner boundary line is calculated from a plurality of measurement information measured according to a preset time interval, and the movement of the moving walk according to the time interval and the position change amount The speed may be determined, and the risk level may be determined according to the moving speed of the moving walk.

In addition, when it is determined that the object is an escalator that is a walking assistance object, the control unit extracts an outer boundary line of the object from the measurement information, sets an outer boundary indicating the appearance of the object, and the distance between both sides of the outer boundary The interval is calculated, the inner boundary line existing inside the outer boundary is extracted, and the amount of position change of any one inner boundary line is calculated from a plurality of measurement information measured according to a preset time interval, and the time interval and It is possible to determine the moving speed of the escalator according to the amount of change in the position, and determine the risk level according to the distance between the both sides of the escalator and the moving speed.

In addition, the input unit for receiving voice command information from the pedestrian; and a location measurement unit configured to measure current location information of a space in which the pedestrian is currently located.

In addition, the control unit extracts address information and a walking path selection command by analyzing the voice command information, and based on the walking path selection command classified into a shortest distance path, a safe path, and a minimum moving object path, the current location A walking path from the information to the address information may be generated, and voice information matching the walking path and the current location information may be extracted.

In addition, the control unit, when the image in which the shape of the character appears from the measurement information generated by photographing the situation in the direction in which the pedestrian travels among the measurement information is recognized, the control unit rotates the measurement information to an arbitrary rotation angle to the character Adjusts the shape of the forward direction, extracts address information from the adjusted characters in the forward direction, analyzes the voice command information to extract a walking path selection command, and is classified into a shortest distance path, a safe path, and a minimum moving object path. Based on the walking path selection command, a walking path from the current location information to the address information may be generated, and voice information matching the walking path and the current location information may be extracted.

Another aspect of the present invention includes the steps of: generating, by a sensor unit, measurement information by measuring a situation in a direction in which a pedestrian is traveling; storing an object determination model generated by learning to determine an arbitrary object from the measurement information in a storage unit, and determining, by a controller, an object appearing from the measurement information based on the object determination model; determining, by the controller, a risk level of the object based on a risk criterion set differently according to the type of the object classified into a walking aid object, a fixed object, and a moving object; extracting, by the controller, matching voice information to inform the pedestrian's path according to the location of the object and the risk level; and outputting the voice information by an output unit.

In addition, the step of determining the risk level of the object, when the object is determined to be a stairway that is a walking aid object, extracting the outer boundary line of the object from the measurement information, setting the outer boundary indicating the appearance of the object, , by extracting the inner boundary line existing inside the outer boundary, determining the number of steps according to the number of the inner boundary lines, calculating the distance between two adjacent inner boundary lines, and determining the height of one step of the stairs And, it is possible to determine the level of risk according to the height of one stage of the stairs and the number of stages of the stairs.

In addition, the step of determining the risk level of the object, when the object is determined to be a moving walk that is a walking assistance object, by extracting the outer boundary of the object from the measurement information, setting the outer boundary indicating the appearance of the object and extracting the inner boundary line existing inside the outer boundary, calculating the position change amount of any one inner boundary line from a plurality of measurement information measured according to a preset time interval, the time interval and the position change amount It is possible to determine the moving speed of the moving walk in accordance with, and determine the risk level according to the moving speed of the moving walk.

In addition, the step of determining the risk level of the object, when the object is determined to be an escalator that is a walking assistance object, extracting the outer boundary line of the escalator from the measurement information, setting the outer boundary indicating the appearance of the escalator, , Calculate the distance interval on both sides of the outer boundary, extract the inner boundary line existing inside the outer boundary, and determine the amount of position change of any one inner boundary line from a plurality of measurement information measured according to a preset time interval By calculating, it is possible to determine the moving speed of the escalator according to the time interval and the amount of change in the position, and determine the risk level according to the distance between the both sides of the escalator and the moving speed.

In addition, the step of receiving the input unit voice command information from the pedestrian; and measuring, by a location measuring unit, current location information of a space in which the pedestrian is currently located.

According to one aspect of the present invention described above, by providing a visual assistance apparatus and method, it is possible to recognize an object existing in a moving direction of a pedestrian and notify the pedestrian of the existence of the object.

1 is a schematic diagram illustrating an embodiment of a visual aid apparatus according to an embodiment of the present invention.
2 is a control block diagram of a visual aid apparatus according to an embodiment of the present invention.
3 is a block diagram illustrating a process of determining the risk level of an object in the control unit of FIG. 2 .
4 is a block diagram illustrating a process of generating a walking path in the controller of FIG. 2 .
5 is a schematic diagram illustrating an embodiment in which the control unit of FIG. 2 determines the risk level of the stairs, which is a walking assistance object.
6 is a schematic diagram illustrating an embodiment in which the controller of FIG. 2 determines a risk level of a moving walk, which is a walking assistance object.
7 and 8 are flowcharts of a visual assistance method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all scope equivalents as those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a schematic diagram illustrating an embodiment of a visual aid apparatus according to an embodiment of the present invention.

The visual assistance apparatus 100 may be provided to be worn by a pedestrian. For example, the visual assistance apparatus 100 may be provided as glasses, a necklace, a belt, a choker, or the like, and the visual assistance apparatus 100 may be provided as a walking stick, etc., which is provided to be carried by a pedestrian.

Accordingly, the visual assistance apparatus 100 may measure the situation in the direction in which the pedestrian wearing the visual assistance apparatus 100 is traveling, and the visual assistance apparatus 100 may determine the level of risk based on the situation in the traveling direction of the pedestrian. may be arranged to notify pedestrians.

In this case, the visual assistance apparatus 100 may include a camera, a lidar sensor, a radar sensor, an ultrasonic sensor, a Time of Flight (ToF) camera, and the like, wherein the ToF camera is reverse scattering of an infrared beam by a target object. It can be understood as a camera that generates 3D image information of an object using (Back Scattering).

Through this, the visual assistance apparatus 100 can measure the situation in the direction in which the pedestrian proceeds. To this end, the visual assistance apparatus 100 may be provided with a storage unit to store arbitrary information, and 100) may be stored in the storage unit information provided to determine the level of risk based on the situation in the direction in which the pedestrian proceeds.

In this regard, the visual assistance apparatus 100 may be provided to transmit or receive information through a wireless network and an external server 200 in which arbitrary information is stored. In this case, the external server 200 is Information provided to determine the level of risk based on the situation in the direction in which the pedestrian proceeds may be stored.

The visual assistance apparatus 100 may transmit the situation in the direction in which the pedestrian proceeds to the external server 200, and accordingly, the visual assistance apparatus 100 determines the risk level based on the situation in the direction in which the pedestrian proceeds. The risk level determined by the external server 200 may be transmitted according to the information provided so as to be able to do so.

Meanwhile, the visual assistance apparatus 100 may be provided to recognize the pedestrian's voice command, and accordingly, the visual assistance device 100 may analyze the recognized voice command to determine a location to which the pedestrian intends to go.

To this end, the visual assistance apparatus 100 may be provided with a location measuring unit using a location surveying technique such as a Global Position System (GPS), a Global Navigation Satellite System (GLONASS), Galileo, or a Global Navigation Satellite System (GNSS). Such a GNSS technique uses one or more satellites, and may be a technique for calculating a position of a receiver according to a distance between each satellite and a receiver.

Accordingly, the visual assistance apparatus 100 may be provided to generate a walking path from the pedestrian's current location to the pedestrian's desired location.

Hereinafter, the visual assistance apparatus 100 will be described in detail.

2 is a control block diagram of a visual aid apparatus according to an embodiment of the present invention.

The visual assistance apparatus 100 may include a sensor unit 110 , an input unit 120 , a position measurement unit 130 , a storage unit 140 , a control unit 150 , and an output unit 160 .

In addition, the visual assistance apparatus 100 may be implemented by more components than the components shown in FIG. 2 , and may be implemented by fewer components than the components shown in FIG. 2 . Alternatively, in the vision assistance apparatus 100 , at least two components provided in the vision assistance apparatus 100 may be integrated into one component, and one component may perform a complex function. Hereinafter, the above-described components will be described in detail.

The sensor unit 110 may generate measurement information by measuring a situation in a direction in which the pedestrian proceeds. To this end, the sensor unit 110 may include a camera, a lidar sensor, a radar sensor, an ultrasonic sensor, a Time of Flight (ToF) camera, and the like.

Accordingly, the sensor unit 110 may generate an image in which the situation of the pedestrian's traveling direction is captured using a camera, etc., and the sensor unit 110 uses a lidar sensor, a radar sensor, an ultrasonic sensor, and the like for pedestrians. It is possible to measure the distance between the object 300 and the moving direction of the.

The storage unit 140 may store an object determination model generated by learning to determine an arbitrary object 300 from the measurement information.

At this time, the object determination model may be generated to determine the type or type of the object 300 appearing in the image by using a technique such as machine learning, and the object determination model is based on a plurality of images prepared in advance. The type or type of the appearing object 300 may be input from a learner and generated.

Here, machine learning can be understood as a technique of generating a learning model to classify a plurality of information into one or more groups based on a plurality of information, and classifying arbitrary information based on the generated learning model, and In the same way, machine learning is supervised learning that creates a learning model to classify arbitrary information according to a plurality of information classified by an administrator, analyzes a plurality of information itself, or performs a clustering process and performs a learning model Unsupervised learning to create a , semi-supervised learning to create a learning model by mixing supervised learning and unsupervised learning, and rewards generated in the process of performing arbitrary actions on multiple pieces of information It may include reinforcement learning (Reinforcement Learning) that generates a learning model according to the

For example, the object determination model may be generated by learning images of different objects 300 such as beverage cans, Styrofoam boxes, potholes, stairs, boundary stones, jaws, and vehicles.

The control unit 150 may determine the object 300 appearing from the measurement information based on the object determination model, and the control unit 150 according to the type of the object 300 classified into a walking aid object, a fixed object, and a moving object. Based on the risk criteria set differently, the risk level of the object 300 may be determined, and the controller 150 extracts matching voice information to inform the pedestrian's path according to the location and risk level of the object 300 . can do.

Here, the walking assistance object may be provided to represent the object 300 provided for pedestrians to use while walking, such as stairs, escalators, elevators, moving walks, etc. appearing from the measurement information, and the fixed object is a blocking rod appearing from the measurement information. , a power pole, a street tree, a fence, etc. may be provided to represent a fixed object 300 at an arbitrary location, and the moving object may represent a movable object 300 such as a person, vehicle, animal, etc., which appears from measurement information. can be provided.

To this end, the storage unit 140 may store the object 300 , voice information matching the location and risk level of the object 300 , and the output unit 160 outputs the voice information extracted from the control unit 150 . can be printed

In this case, the output unit 160 may extract voice information by providing a speaker, and the output unit 160 may provide voice information to a speaker device such as an earphone connected to the visual assistance device 100 by wire or wirelessly. It can also be passed to be output.

In addition, the output unit 160 may be provided with a vibration element to output vibration according to the level of danger determined by the control unit 150 . In this case, the control unit 150 may control the vibration to be output from the output unit 160 only when the risk level is higher than a predetermined level.

On the other hand, the storage unit 140 stores the object determination model and transmits measurement information through the visual assistance device 100 and the wireless network, or an external server ( 200) may be substituted.

In this case, the visual assistance apparatus 100 may further include a communication unit, and the communication unit may transmit measurement information generated by the sensor unit 110 to the external server 200 .

Accordingly, the external server 200 may determine the object 300 appearing from the measurement information based on the object determination model, and the external server 200 is an object 300 classified into a walking aid object, a fixed object, and a moving object. ), the risk level of the object 300 may be determined based on a risk criterion set differently according to the type.

In this case, the communication unit may receive the risk level from the external server 200, and the control unit 150 may extract matching voice information to inform the pedestrian's path according to the location and risk level of the object 300. have.

In one embodiment, when it is determined that the object 300 is a staircase that is a walking aid object, the controller 150 extracts the outer boundary line of the stairs from the measurement information to set the outer boundary indicating the appearance of the object 300 stairs. The control unit 150 may extract the inner boundary line existing inside the outer boundary line, determine the number of steps according to the number of the inner boundary line, and the control unit 150 calculates the distance between the two adjacent inner boundary lines Thus, it is possible to determine the height of one step of the stairs, and the controller 150 may determine the level of risk according to the height of one step of the stairs and the number of steps.

Here, the outer boundary line of the arbitrary object 300 may mean a boundary line that separates the area recognized as the object 300 from the measurement information and the background area. To this end, the control unit 150 may extract the outer boundary line of the arbitrary object 300 by performing gray scale on the measurement information displayed as an image, or the control unit 150 may perform the measurement information displayed as an image. The outer boundary of the arbitrary object 300 may be extracted by extracting a region showing a certain color tone from .

Also, the inner boundary line of the arbitrary object 300 may mean a boundary line extracted from the inside of the outer boundary line of the object 300 extracted from the measurement information. To this end, the controller 150 may extract the inner boundary line from the inside of the outer boundary line by performing gray scale on the measurement information displayed as an image, or the controller 150 may control a certain color tone from the measurement information displayed as an image. It is also possible to extract a region representing

In this case, the controller 150 may determine the risk level to be higher as the height of one step of the stairs increases, and the controller 150 may determine the risk level to be higher as the number of steps increases.

In this regard, when it is determined that the object 300 is a staircase that is a walking aid object, and the risk level classified as high, medium, or low is determined to be low, the controller 150 determines that there are stairs in the direction in which the pedestrian proceeds. It is possible to extract voice information for notification.

In addition, when it is determined that the object 300 is a staircase that is a walking aid object, and the risk level classified into upper, middle, and lower is determined to be medium, the controller 150 notifies that the stairs existing in the direction in which the pedestrian proceeds are dangerous. Voice information can be extracted.

In addition, when the object 300 is determined to be a stairway that is a walking aid object, and the risk level classified as upper, middle, or lower is determined to be higher, the stairs existing in the direction in which the pedestrian is traveling are dangerous. It is possible to extract voice information that recommends using a different route.

In this case, the controller 150 may further extract voice information indicating a direction in which the outer boundary line of the stairs appearing from the measurement information is closer to the center point of the measurement information.

Meanwhile, when the distance interval from the stairs indicated from the measurement information is less than the threshold distance interval, the controller 150 may extract voice information indicating the number of steps.

Here, the threshold distance interval set for the distance interval with the stairs may be set as a distance interval at which it can be determined that the pedestrian uses the stairs.

In one embodiment, when it is determined that the object 300 is a moving walk that is a walking assistance object, the controller 150 extracts the outer boundary of the moving walk from the measurement information to set the outer boundary indicating the appearance of the moving walk. In addition, the control unit 150 may extract the inner boundary line existing inside the outer boundary, and calculate the amount of change in the position of any one inner boundary line from a plurality of measurement information measured according to a preset time interval, the control unit 150 150 may determine the moving speed of the moving walk according to the time interval and the amount of position change, the control unit 150 may determine the risk level according to the moving speed of the moving walk.

In this case, the controller 150 may determine the higher the risk level as the moving speed of the moving walk increases.

In this regard, the controller 150 determines that the object 300 is a moving walk that is a walking assistance object, and when it is determined that the risk level classified as upper, middle, or lower is lower, the moving walk existing in the direction in which the pedestrian proceeds. It is possible to extract voice information to inform that the is dangerous.

In addition, when it is determined that the object 300 is a moving walk that is a walking aid object, and the risk level classified into upper, middle, and lower is determined to be medium, the moving walk existing in the direction in which the pedestrian proceeds is dangerous. Thus, it is possible to extract voice information that recommends using a different route.

In this case, the control unit 150 may further extract voice information indicating a direction in which the outer boundary line of the moving walk shown from the measurement information is closer to the center point of the measurement information.

In addition, when it is determined that the object 300 is a moving walk that is a walking aid object, and the risk level classified as upper, middle, or lower is determined to be upper, the moving walk existing in the direction in which the pedestrian proceeds is It is possible to extract voice information that is prohibited from being used.

In this case, the control unit 150 may further extract voice information indicating a direction in which the outer boundary line of the moving walk shown from the measurement information is closer to the center point of the measurement information.

In one embodiment, when it is determined that the object 300 is an escalator that is a walking aid object, the controller 150 may extract an outer boundary line of the escalator from the measurement information to set an outer boundary indicating the appearance of the escalator, and the control unit 150 can calculate the distance interval on both sides of the outer boundary, and the control unit 150 extracts the inner boundary line existing inside the outer boundary, By calculating the amount of change in the position of one inner boundary line, the moving speed of the escalator can be determined according to the time interval and the amount of change in position, and the control unit 150 can determine the risk level according to the distance between both sides of the escalator and the moving speed .

In this case, the controller 150 may determine the risk level higher as the moving speed of the escalator increases, and the controller 150 determines that the distance between both sides of the escalator is less than the threshold level, so as to use a different route. Recommended voice information can be extracted.

Here, the threshold level set for the distance interval on both sides of the escalator may mean a distance interval set so that two or more people can stand side by side on the escalator side by side.

In this regard, when it is determined that the object 300 is an escalator that is a walking aid object, and the risk level classified into upper, middle, and lower is determined to be lower, the escalator present in the direction in which the pedestrian proceeds is dangerous. It is possible to extract voice information notifying that

In addition, when it is determined that the object 300 is an escalator that is a walking assistance object, and the risk level classified as high, medium, or low is determined to be medium, the escalator existing in the direction in which the pedestrian proceeds is dangerous and other It is possible to extract voice information that recommends using the route.

In this case, the control unit 150 may further extract voice information informing the direction of the side near the outer boundary of the escalator appearing from the measurement information from the center point of the measurement information.

In addition, when it is determined that the object 300 is an escalator that is a walking aid object, and the risk level classified as high, medium, or low is determined to be high, the control unit 150 controls the use of the escalator in the direction in which the pedestrian proceeds. It is possible to extract prohibited voice information.

In this case, the control unit 150 may further extract voice information informing the direction of the side near the outer boundary of the escalator appearing from the measurement information from the center point of the measurement information.

In an embodiment, when it is determined that the object 300 is a fixed object, the controller 150 controls the number of pixels in the horizontal direction, the number of pixels in the vertical direction, and the distance from the object 300 occupied by the object 300 in the measurement information. The size and height of the object 300 may be calculated according to the interval, and accordingly, the controller 150 may determine the risk level according to the size and height of the object 300 .

In this case, the controller 150 may determine the risk level to be higher as the size of the object 300 increases, and the controller 150 may determine the risk level to be higher as the height of the object 300 increases.

The input unit 120 may receive voice command information from a pedestrian, and for this purpose, the input unit 120 may include a microphone or the like.

Accordingly, the controller 150 may analyze the voice command information using a technique such as speech recognition. Here, the speech recognition technique divides the speech signal into tens of milliseconds, extracts feature vectors appearing in about ten dimensions, divides the speech signal into vowels or consonant units through the extracted feature vectors, and connects the divided vowels or consonants. Thus, it can be understood as a technique for generating a random word and analyzing the generated word to determine the meaning of a voice signal.

The location measurement unit 130 may measure current location information of a space in which the pedestrian is currently located. To this end, the position measuring unit 130 may be provided with a position sensor using a position surveying technique such as a Global Position System (GPS), a Global Navigation Satellite System (GLONASS), Galileo, or a Global Navigation Satellite System (GNSS).

The control unit 150 may extract address information and a walking path selection command by analyzing the voice command information, and the control unit 150 based on the walking path selection command classified into a shortest distance path, a safe path, and a minimum moving object path. , a walking route from the current location information to the address information may be generated, and the controller 150 may extract voice information matching the walking route and the current location information.

Here, the shortest distance path may be provided to indicate a walking path having the shortest distance interval from the current location information to the address information, and the safe path may include one or more walking assistance objects existing in the path from the current location information to the address information. The sum of the risk levels of may be provided to represent the smallest walking path, and the minimum moving object path may be provided to represent the walking path with the fewest moving objects appearing in the path from the current location information to the address information.

To this end, the storage unit 140 may store the risk level determined for the walking assistance object existing on an arbitrary path. Also, the storage unit 140 may store the number of moving objects existing in an arbitrary path at an arbitrary time point.

On the other hand, the storage unit 140 stores the risk level determined for the walking assistance object existing in the arbitrary path or the number of moving objects present in the arbitrary path at any time, and stores the Current location information and address information are transmitted through a wireless network, or generated according to a risk level determined for a walking aid object existing on a random path or the number of moving objects present on a random path at any time. It may be replaced with an external server 200 that is provided to transmit a walking path.

In one embodiment, when it is determined that the time point of generating a walking path from any current location information to any address information is a preset commuting time, the controller 150 is a walking path that bypasses the subway station entrance. can create

On the other hand, when an image in which the shape of a character appears is recognized from the measurement information generated by photographing a situation in the direction in which the pedestrian is traveling among the measurement information, the control unit 150 controls the measurement information to an arbitrary rotation angle. By rotating it, the shape of the character may be adjusted in a forward direction, and the controller 150 may extract address information from the adjusted character in the forward direction.

3 is a block diagram illustrating a process of determining the risk level of an object in the control unit of FIG. 2 .

Referring to FIG. 3 , the sensor unit 110 may generate measurement information by measuring a situation in a direction in which a pedestrian travels. In this case, the sensor unit 110 may generate an image in which the situation in the direction of the pedestrian is photographed using a camera, etc., and the sensor unit 110 uses a lidar sensor, a radar sensor, an ultrasonic sensor, etc. A distance interval with the object 300 existing in the moving direction may be measured.

Accordingly, the controller 150 may determine the object 300 appearing from the measurement information based on the object determination model, and the controller 150 controls the object 300 classified into a walking aid object, a fixed object, and a moving object. Based on the risk criteria set differently depending on the type, it is possible to determine the risk level of the object 300 , and the controller 150 is a voice matched to inform the pedestrian's path according to the location and risk level of the object 300 . information can be extracted.

To this end, the storage unit 140 may store an object determination model generated by learning to determine an arbitrary object 300 from the measurement information. Also, the storage unit 140 may store the object 300 , and voice information matching the location and risk level of the object 300 .

The output unit 160 may output the voice information extracted from the control unit 150 , and the output unit 160 is provided with a vibration element so that vibration is output according to the risk level determined by the control unit 150 . can be

4 is a block diagram illustrating a process of generating a walking path in the controller of FIG. 2 .

Referring to FIG. 4 , the input unit 120 may receive voice command information from a pedestrian, and the location measurement unit 130 may measure current location information of a space in which the pedestrian is currently located.

Accordingly, the control unit 150 may extract the address information and the walking path selection command by analyzing the voice command information, and the control unit 150 is a walking path selection command classified into the shortest distance path, the safe path, and the minimum moving object path. Based on the , a walking route from the current location information to the address information may be generated, and the controller 150 may extract voice information matching the walking route and the current location information.

To this end, the storage unit 140 may store the risk level determined for the walking assistance object existing on an arbitrary path. Also, the storage unit 140 may store the number of moving objects existing in an arbitrary path at an arbitrary time point.

The output unit 160 may output the voice information extracted from the control unit 150 .

On the other hand, when an image in which the shape of a character appears is recognized from the measurement information generated by photographing a situation in the direction in which the pedestrian is traveling among the measurement information, the control unit 150 controls the measurement information to an arbitrary rotation angle. By rotating it, the shape of the character may be adjusted in a forward direction, and the controller 150 may extract address information from the adjusted character in the forward direction.

5 is a schematic diagram illustrating an embodiment in which the control unit of FIG. 2 determines the risk level of the stairs, which is a walking assistance object.

In one embodiment, when it is determined that the object 300 is a stairway that is a walking aid object, the controller 150 extracts the outer boundary line of the object 300 from the measurement information, and the outer boundary indicating the appearance of the object 300 . can be set, and the control unit 150 extracts the inner boundary line existing inside the outer boundary, and determines the number of steps according to the number of the inner boundary line, and the control unit 150 determines the distance between the two adjacent inner boundary lines. By calculating , the height of one step of the staircase may be determined, and the controller 150 may determine the risk level according to the height of one step of the staircase and the number of steps.

Referring to FIG. 5 , the control unit 150 may check the stair corner area set to be recognized as the inner boundary line of the stair.

In this case, the controller 150 may determine the risk level to be higher as the height of one step of the stairs increases, and the controller 150 may determine the risk level to be higher as the number of steps increases.

In this regard, when it is determined that the object 300 is a staircase that is a walking aid object, and the risk level classified as high, medium, or low is determined to be low, the controller 150 determines that there are stairs in the direction in which the pedestrian proceeds. It is possible to extract voice information for notification.

In addition, when it is determined that the object 300 is a staircase that is a walking aid object, and the risk level classified into upper, middle, and lower is determined to be medium, the controller 150 notifies that the stairs existing in the direction in which the pedestrian proceeds are dangerous. Voice information can be extracted.

In addition, when the object 300 is determined to be a stairway that is a walking aid object, and the risk level classified as upper, middle, or lower is determined to be higher, the stairs existing in the direction in which the pedestrian is traveling are dangerous. It is possible to extract voice information that recommends using a different route.

In this case, the controller 150 may further extract voice information indicating a direction in which the outer boundary line of the stairs appearing from the measurement information is closer to the center point of the measurement information.

Meanwhile, when the distance interval from the stairs indicated from the measurement information is less than the threshold distance interval, the controller 150 may extract voice information indicating the number of steps.

Here, the threshold distance interval set for the distance interval with the stairs may be set as a distance interval at which it can be determined that the pedestrian uses the stairs.

6 is a schematic diagram illustrating an embodiment in which the controller of FIG. 2 determines a risk level of a moving walk, which is a walking assistance object.

In one embodiment, when it is determined that the object 300 is a moving walk that is a walking assistance object, the controller 150 extracts an outer boundary line of the object 300 from the measurement information, thereby indicating the external appearance of the object 300 . The boundary can be set, and the control unit 150 extracts the inner boundary line existing inside the outer boundary, and calculates the amount of position change of any one inner boundary line from a plurality of measurement information measured according to a preset time interval. In addition, the controller 150 may determine the moving speed of the moving walk according to the time interval and the amount of position change, and the controller 150 may determine the risk level according to the moving speed of the moving walk.

Referring to FIG. 6 , an area in which two adjacent footrests set to be recognized as an inner boundary line of the moving walk by the controller 150 is separated can be identified.

In this case, the controller 150 may determine the higher the risk level as the moving speed of the moving walk increases.

In this regard, the controller 150 determines that the object 300 is a moving walk that is a walking assistance object, and when it is determined that the risk level classified as upper, middle, or lower is lower, the moving walk existing in the direction in which the pedestrian proceeds. It is possible to extract voice information to inform that the is dangerous.

In addition, when it is determined that the object 300 is a moving walk that is a walking aid object, and the risk level classified into upper, middle, and lower is determined to be medium, the moving walk existing in the direction in which the pedestrian proceeds is dangerous. Thus, it is possible to extract voice information that recommends using a different route.

In this case, the control unit 150 may further extract voice information indicating a direction in which the outer boundary line of the moving walk shown from the measurement information is closer to the center point of the measurement information.

In addition, when it is determined that the object 300 is a moving walk that is a walking aid object, and the risk level classified as upper, middle, or lower is determined to be upper, the moving walk existing in the direction in which the pedestrian proceeds is It is possible to extract voice information that is prohibited from being used.

In this case, the control unit 150 may further extract voice information indicating a direction in which the outer boundary line of the moving walk shown from the measurement information is closer to the center point of the measurement information.

7 and 8 are flowcharts of a visual assistance method according to an embodiment of the present invention.

Since the visual assistance method according to an embodiment of the present invention proceeds in substantially the same configuration as that of the visual assistance apparatus 100 shown in FIG. , and repeated descriptions will be omitted.

The visual assistance method includes generating measurement information (700), determining an object (710), determining a risk level of an object (720), extracting voice information (730), and outputting voice information step 740 .

The step 700 of generating the measurement information may be a step in which the sensor unit 110 measures the situation in the direction in which the pedestrian proceeds to generate the measurement information.

In step 710 of determining the object, an object determination model generated by learning to determine an arbitrary object 300 from measurement information is stored in the storage unit 140 , and the controller 150 measures it based on the object determination model It may be a step of determining the object 300 appearing from the information.

In step 720 of determining the risk level of the object, the control unit 150 based on the risk criterion set differently according to the type of the object 300 classified into a walking aid object, a fixed object, and a moving object, the object 300 It may be a step to determine the risk level of

The step 730 of extracting the voice information may be a step of extracting, by the controller 150 , matching voice information to inform the pedestrian's path according to the location and risk level of the object 300 .

The step 740 of outputting the voice information may be a step in which the output unit 160 outputs the voice information.

Meanwhile, the visual assistance method includes receiving voice command information (800), measuring current location information (810), extracting address information (820), generating a walking path (830), and voice information The step of extracting and outputting 840 may be further included.

The step 800 of receiving voice command information may be a step in which the input unit 120 receives voice command information from a pedestrian.

The step 810 of measuring the current location information may be a step in which the location measuring unit 130 measures the current location information of the space in which the pedestrian is currently located.

The step of extracting the address information 820 may be a step in which the controller 150 analyzes the voice command information to extract the address information and the walking path selection command.

In the step 830 of generating a walking path, the controller 150 generates a walking path from the current location information to the address information based on the walking path selection command classified into the shortest distance path, the safe path, and the minimum moving object path. can be a step.

The step 840 of extracting and outputting the voice information may be a step in which the controller 150 extracts voice information matching the walking path and current location information, and the output unit 160 outputs the voice information.

Although the above has been described with reference to the embodiments, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention as set forth in the following claims. will be able

100: visual aid
200: external server
300: object

Claims (11)

a sensor unit for generating measurement information by measuring a situation in a direction in which a pedestrian is traveling;
a storage unit for storing an object determination model generated by learning to determine an arbitrary object from the measurement information;
Determine the object appearing from the measurement information based on the object determination model, and determine the risk level of the object based on a risk criterion set differently according to the type of the object classified into a walking aid object, a fixed object, and a moving object a controller for determining and extracting matching voice information to inform the pedestrian's path according to the location of the object and the level of risk; and
Including; an output unit for outputting the voice information;
The control unit is
When it is determined that the object is an escalator that is a walking assistance object, the outer boundary line of the escalator is extracted from the measurement information, an outer boundary indicating the appearance of the escalator is set, and the distance between both sides of the outer boundary is calculated, and the Extracting the inner boundary line existing inside the outer boundary, calculating the position change amount of any one inner boundary line from a plurality of measurement information measured according to a preset time interval, according to the time interval and the position change amount Determining the moving speed of the escalator, and determining the level of danger according to the distance between the both sides of the escalator and the moving speed, a visual assistance device capable of object recognition for the escalator.
According to claim 1,
an input unit for receiving voice command information from a pedestrian; and
A visual assistance device capable of object recognition for an escalator, further comprising a; a position measuring unit for measuring current position information of a space in which the pedestrian is currently located.
According to claim 2, wherein the control unit,
Address information and walking path selection command are extracted by analyzing the voice command information, and based on the walking path selection command classified into a shortest distance path, a safe path, and a minimum moving object path, from the current location information to the address information A visual assistance device capable of object recognition for an escalator, which generates a walking path of the escalator and extracts voice information matching the walking path and the current location information.
According to claim 2, wherein the control unit,
When an image in which the shape of a character appears is recognized from the measurement information generated by photographing the situation in the direction in which the pedestrian is traveling among the measurement information, the measurement information is rotated at an arbitrary rotation angle to adjust the shape of the character in the forward direction And, a visual assistance device capable of object recognition for the escalator, extracting address information from the characters adjusted in the forward direction.
According to claim 4, wherein the control unit,
A walking path selection command is extracted by analyzing the voice command information, and based on the walking path selection command classified into a shortest distance path, a safe path, and a minimum moving object path, a walking path from the current location information to the address information A visual aid device capable of object recognition for escalators that creates
According to claim 5, wherein the control unit,
A visual assistance device capable of object recognition for an escalator that extracts voice information matching the walking path and the current location information.
In the visual assistance method using a visual assistance device capable of object recognition for the escalator,
generating, by a sensor unit, measurement information by measuring a situation in a direction in which a pedestrian is traveling;
storing an object determination model generated by learning to determine an arbitrary object from the measurement information in a storage unit, and determining, by a controller, an object appearing from the measurement information based on the object determination model;
determining, by the controller, a risk level of the object based on a risk criterion set differently according to the type of the object classified into a walking aid object, a fixed object, and a moving object;
extracting, by the controller, matching voice information to inform the pedestrian's path according to the location of the object and the risk level; and
Including; outputting the voice information by an output unit;
The step of determining the risk level of the object,
When it is determined that the object is an escalator that is a walking assistance object, the outer boundary line of the escalator is extracted from the measurement information, an outer boundary indicating the appearance of the escalator is set, and the distance between both sides of the outer boundary is calculated, and the Extracting the inner boundary line existing inside the outer boundary, calculating the position change amount of any one inner boundary line from a plurality of measurement information measured according to a preset time interval, according to the time interval and the position change amount Determining the moving speed of the escalator, and determining the level of danger according to the distance between the both sides of the escalator and the moving speed, a visual assistance method.
8. The method of claim 7,
receiving, by an input unit, voice command information from a pedestrian; and
The method further comprising: a position measuring unit measuring current position information of a space in which the pedestrian is currently located; further comprising, a visual assistance method.
9. The method of claim 8,
When an image in which a shape of a character appears is recognized from measurement information generated by photographing a situation in a direction in which the pedestrian is traveling among the measurement information, the control unit rotates the measurement information at an arbitrary rotation angle to change the shape of the character Adjusting in the forward direction, and extracting address information from the character adjusted in the forward direction; further comprising, a visual assistance method.
10. The method of claim 9,
The control unit extracts a walking path selection command by analyzing the voice command information, and based on the walking path selection command classified into a shortest distance path, a safe path, and a minimum moving object path, from the current location information to the address information Creating a walking path of the; further comprising, a visual assistance method.
11. The method of claim 10,
The method further comprising; extracting, by the controller, voice information matching the walking path and the current location information.

Images