KR20210085865A - Crosswalk guide apparatus and method for pedestrians of the visually impaired - Google Patents

Abstract

The present invention relates to a guide device and method for walking in a crosswalk for visually impaired people. The device according to an embodiment of the present invention comprises: an image acquisition unit that collects a front image transmitted from a terminal worn by the visually impaired; a crosswalk detection unit that sets a plurality of first regions of interest for the front image, and estimates first regions of interest corresponding to the crosswalk through at least one of feature-based analysis or machine learning for the first regions of interest; a walking signal detection unit which sets one central region of the front image as second regions of interest, estimates candidate regions including a gait signal in the second regions of interest through at least one of feature-based analysis or machine learning for the second regions of interest, and determines the state of the walking signal included in the candidate areas; a progress direction determination unit for generating projection data on the basis of the horizontal direction on the basis of the first regions of interest corresponding to the crosswalk, and estimating the direction of movement of the visually impaired by analyzing the gradient change of pixel values constituting the projection data; and an information analysis unit that generates walking guide information output to the user interface of the terminal on the basis of the output data of the crosswalk detection unit, the walking signal detection unit, and the traveling direction determination unit.

Description

CROSSWALK GUIDE APPARATUS AND METHOD FOR PEDESTRIANS OF THE VISUALLY IMPAIRED

The present invention relates to a guide apparatus and method for walking in a crosswalk for a visually impaired person, and more particularly, by analyzing a front image of a wearable device worn by a visually impaired person, information for inducing safe walking in a situation of crossing a crosswalk. It relates to an apparatus and method for providing.

It is an important issue in terms of improving the quality of life of the visually impaired to safely move to a desired destination without the help of others. There are braille blocks as road facilities to help the visually impaired walk, and most of the visually impaired use a cane for walking. By sensibly recognizing the braille blocks that exist on the road using a stick, it judges its own direction and obtains information about the stopping situation according to whether it is a linear braille block or a dotted braille block.

However, a problem arises when a visually impaired person crosses a crosswalk that does not have braille blocks installed. First of all, even if the blind person recognizes the stop braille block, he cannot know whether there is a crosswalk in front of him. In addition, although there is a crosswalk that transmits a walking signal to the visually impaired through sound, in most cases, it is impossible to check whether the walking signal is on.

The biggest problem is that there are no braille blocks to guide the walking direction at the crosswalk, so it is impossible to know whether the blind person is crossing in the correct direction. Therefore, in order to secure the mobility of the visually impaired in a crosswalk, which is likely to be exposed to danger due to a driving vehicle, a method of inducing safe walking is required.

Republic of Korea Patent Publication No. 10-1845754 (2018.04.05)

The present invention has been devised by the necessity as described above, and provides information so that the visually impaired can stably recognize walking signals such as crosswalks and traffic lights installed in crosswalks, and at the same time to proceed in the right direction in a situation of crossing the crosswalk. An object of the present invention is to provide an assisting apparatus and method.

A guide device for walking in a crosswalk for a blind person according to an embodiment of the present invention is an image acquisition unit that collects a front image transmitted from a terminal worn by the visually impaired, and sets a plurality of first areas of interest for the front image and a crosswalk detector estimating the first regions of interest corresponding to the crosswalk through at least one of feature-based analysis or machine learning for the first regions of interest, and set one central region of the front image as the second region of interest and estimating candidate regions including a gait signal in the second region of interest through at least one of feature-based analysis or machine learning for the second region of interest, and determining the state of the gait signal included in the candidate regions. The signal detector generates projection data based on the horizontal direction based on the first regions of interest corresponding to the crosswalk, and analyzes the gradient change of pixel values constituting the projection data to estimate the moving direction of the visually impaired. It may include an information analysis unit that generates walking guide information output to the user interface of the terminal based on the output data of the determination unit, the crosswalk detection unit, the walking signal detection unit, and the traveling direction determination unit.

The crosswalk detection unit according to an embodiment of the present invention performs binarization based on a characteristic of a crosswalk pattern identified in each of a plurality of first ROIs, and a plurality of first ROIs are traversed based on the binarization. It can be judged individually whether or not it falls under the report.

The crosswalk detection unit according to an embodiment of the present invention performs binarization using the average value of the characteristics of the crosswalk pattern as a threshold value, and estimates the average value and variance value for the crosswalk area derived based on the binarization, It may be determined whether to perform additional binarization based on the estimated mean and variance values.

The crosswalk detection unit according to an embodiment of the present invention may perform shape correction based on a morphology technique for a crosswalk pattern.

The traveling direction determining unit according to an embodiment of the present invention may estimate a direction in which the crosswalk pattern is inclined by analyzing a change in a section in which pixel values constituting the projection data change from a minimum value to a maximum value.

The information analysis unit according to an embodiment of the present invention determines that a crosswalk exists in front of the visually impaired when the number of first regions of interest corresponding to the crosswalk is equal to or greater than a predetermined reference number, and approaches the crosswalk It is possible to generate walking guide information notifying

The information analysis unit according to an embodiment of the present invention may generate walking guide information for inducing the visually impaired to stop or walk in a crosswalk based on the state of the walking signal included in the candidate areas.

The information analysis unit according to an embodiment of the present invention, when the moving direction of the visually impaired is out of a predetermined angular range based on the virtual vertical line of the crosswalk, a walking guide for guiding the visually impaired within a predetermined angular range information can be generated.

A guide method for walking in a crosswalk for a blind person according to an embodiment of the present invention includes collecting a front image transmitted from a terminal worn by the visually impaired, setting a plurality of first regions of interest for the front image, estimating the first regions of interest corresponding to the crosswalk through at least one of feature-based analysis or machine learning of the first regions of interest; setting one central region of the front image as the second region of interest; Estimating candidate regions including a gait signal among the second region of interest through at least one of feature-based analysis or machine learning of the region of interest, and determining the state of the gait signal included in the candidate regions; estimating the moving direction of the visually impaired by generating projection data based on the horizontal direction based on the corresponding first regions of interest and analyzing the gradient change of pixel values constituting the projection data; and a crosswalk detector and a walking signal The method may include generating walking guide information output to a user interface of the terminal based on the output data of the detection unit and the traveling direction determination unit.

In the step of estimating the first ROIs according to an embodiment of the present invention, binarization is performed based on the characteristics of the crosswalk pattern identified in each of the plurality of first ROIs, and a plurality of second ROIs based on the binarization is performed. 1 It is possible to individually judge whether the areas of interest correspond to crosswalks.

In the step of estimating the first regions of interest according to an embodiment of the present invention, binarization is performed using the average value of the characteristics of the crosswalk pattern as a threshold value, and the average value and variance for the crosswalk area derived based on the binarization It is possible to estimate the value, and determine whether to perform additional binarization based on the estimated average value and variance value.

In the step of estimating the first regions of interest according to an embodiment of the present invention, shape correction based on a morphology technique for the crosswalk pattern may be performed.

In the step of estimating the moving direction of the blind person according to an embodiment of the present invention, the direction in which the crosswalk pattern is inclined is estimated by analyzing the change in the section in which pixel values constituting the projection data change from the minimum value to the maximum value. can

In the step of generating walking guide information according to an embodiment of the present invention, if the number of first regions of interest corresponding to the crosswalk is equal to or greater than a predetermined reference number, it is determined that there is a crosswalk in front of the blind person, It is possible to generate walking guide information notifying the approach of the crosswalk.

In the step of generating the walking guide information according to an embodiment of the present invention, based on the state of the walking signal included in the candidate areas, it is possible to generate walking guide information that induces the visually impaired to stop or walk in a crosswalk. have.

In the step of generating walking guide information according to an embodiment of the present invention, when the moving direction of the visually impaired is out of a predetermined angular range based on the virtual vertical line of the crosswalk, the moving direction of the visually impaired is set within a predetermined angular range. It is possible to generate walking guide information leading to the inside.

Meanwhile, according to an embodiment of the present invention, it is possible to provide a computer-readable recording medium in which a program for executing the above-described method is recorded in a computer.

According to the apparatus and method provided as an embodiment of the present invention, by analyzing the front image acquired through a wearable camera or smart phone worn by the visually impaired and delivering information to induce safe walking for each situation, the visual Assist people with disabilities to safely cross the crosswalk.

1 is a block diagram illustrating a guide device for walking in a crosswalk for a visually impaired person according to an embodiment of the present invention.
2 illustrates a plurality of first ROIs for an anterior image according to an embodiment of the present invention.
3 illustrates a result of binarization of a plurality of first ROIs according to an embodiment of the present invention.
4 shows the binarization problem according to the poor paint condition of the crosswalk.
5 shows (a) the binarization result of the conventional basic method according to the existence of an obstacle, and (b) the binarization result according to an embodiment of the present invention.
6 is a graph showing a change in inclination according to projection data and a moving direction according to an embodiment of the present invention.
7 is a flowchart illustrating a guide method for walking in a crosswalk for the visually impaired according to an embodiment of the present invention.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" described in the specification mean a unit for processing at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a guide device for walking in a crosswalk for a visually impaired person according to an embodiment of the present invention.

Referring to FIG. 1 , a guide device 100 according to an embodiment of the present invention is a device capable of interworking with a terminal 200 worn by a visually impaired (hereinafter referred to as a user), and a front image ( 10) can be analyzed to induce the user's stable walking in the crosswalk. In this case, the guide device 100 may communicate with the terminal 200 worn by the user through wired/wireless network communication, and may output walking guide information through a user interface that can be driven by the terminal 200 worn by the user.

Specifically, the guide device 100 according to an embodiment of the present invention may include an image acquisition unit 110 that collects the front image 10 transmitted from the terminal 200 worn by the user. The front image 10 refers to an image of a direction in which the body in which the terminal 200 is worn faces based on the moving direction of the user. For example, if the user is wearing a head-mounted display (HMD) type terminal, images in the direction of the head based on the moving direction of the user are taken as the front image 10 , which is the image acquisition unit 110 . can be transmitted to At this time, the image acquisition unit 110 analyzes the front and rear changes of the front images 10 collected from the terminal 200 worn by the user, so that the images greatly deviated from the moving direction due to the movement of the head are the front images 10 . It is determined that this is not the case, and these images can be deleted by themselves.

The guide apparatus 100 according to an embodiment of the present invention may include a crosswalk detection unit 120 that detects whether a crosswalk exists in the front image 10 . The crosswalk detector 120 may set a plurality of first regions of interest 20 for the front image 10 . The reason why some regions, not the entire region, of the front image 10 are set as the first regions of interest 20 is to enable rapid analysis while maintaining accuracy in a situation where walking is to be guided in real time. In this case, the size, number, etc. of the plurality of first regions of interest 20 may be automatically determined by the size of the collected front image 10 , or may be set or changed through an external input applied through the user interface. have.

The crosswalk detection unit 120 may estimate the first areas of interest 21 corresponding to the crosswalk through at least one of feature-based analysis or machine learning for each of the first areas of interest 20 . The feature-based analysis refers to analyzing characteristic elements such as the shape, size, and color of the crosswalk pattern included in the first regions of interest 20 . For example, in the feature-based analysis of the crosswalk pattern, whether the white area and the gray area are alternately repeated with a certain thickness, whether the ratio of the area of the white area to the gray area is similar, bi-modal ) may be included in the analysis on whether or not it has a brightness distribution of the shape. In this case, the crosswalk detection unit 120 binarizes the first regions of interest 20 in order to improve the analysis accuracy of the crosswalk pattern in the feature-based analysis process. More details on binarization will be described later with reference to FIG. 3 .

Meanwhile, machine learning includes image-based supervised learning, unsupervised learning, reinforcement learning, and deep learning. For example, the machine learning algorithm may include a support vector machine (SVM), a random tree, a convolutional neural network (CNN), and the like. The crosswalk detection unit 120 learns whether a crosswalk is included in the first regions of interest 20 using a machine learning algorithm as in the above-described example, and a front image ( 10), it is possible to determine whether a crosswalk exists.

The guide device 100 according to an embodiment of the present invention may include a walking signal detection unit 130 for checking the lighting of a walking signal such as a traffic light installed in a crosswalk. The walking signal detector 130 may set one central region of the front image 10 as the second ROI. That is, the walking signal detector 130 sets a region of interest for the front image 10 separately from the crosswalk detector 120 . For example, in the case of a pedestrian traffic light, since it is generally located above the height of a person, the walking signal detection unit 130 sets the 1/2 region based on the center of the front image 10 as the second region of interest. can In this case, the criterion for determining the second ROI may be automatically determined by the size of the collected front image 10 , or may be set or changed through an external input applied through the user interface.

The gait signal detector 130 may estimate candidate regions including the gait signal among the second ROIs through at least one of a feature-based analysis of the second ROI or machine learning. The feature-based analysis refers to analyzing feature elements such as the shape, size, and color of the gait signal included in the second region of interest. For example, the feature-based analysis of the walking signal may include analysis of whether a red or green lighting area exists, and whether the shape of the lighting area is a human shape rather than a circle.

At this time, in the case of a traffic light for pedestrians, unlike a traffic light for a vehicle, there is a high possibility that there are visual obstructions around, and the cases of false detection of a candidate area may increase due to the influence of ambient lighting at night. Accordingly, the gait signal detector 130 estimates a plurality of candidate regions determined to contain the gait signal within the second region of interest, thereby reducing the probability of erroneous detection in determining the gait signal. can That is, the gait signal detector 130 selects all candidate regions extracted within one second region of interest as considerations for determining the state of the gait signal.

The walking signal detector 130 may determine the state of the walking signal included in the candidate regions selected through the above-described process. For example, the walking signal detector 130 may detect a change in lighting of a red signal or a green signal in the walking signal included in the candidate regions. If the green signal is lit, it may be determined whether the green signal is flickering. Also, when there is an area in which time is displayed in an arrow shape or a number shape, the time remaining until the signal is changed may be estimated. Such determination of the state of the walking signal may improve accuracy by comparing a plurality of candidate regions.

The guide apparatus 100 according to an embodiment of the present invention may include a moving direction determining unit 140 for detecting the moving direction of the user in real time in a situation of crossing a crosswalk. Determination of the traveling direction may be performed with respect to the first regions of interest 20 detected as a crosswalk by the crosswalk detection unit 120 . At this time, since obstacles such as surrounding pedestrians may exist in a situation in which the user is walking along the crosswalk, the first regions of interest 20 determined as the crosswalk may change for each frame of the front image 10 .

The traveling direction determiner 140 may use the binarization result 30 for the first regions of interest 21 corresponding to the crosswalk to generate the projection data. That is, the traveling direction determiner 140 may generate projection data for indicating how much the normally binarized first regions of interest 30 have changed based on the horizontal direction of the crosswalk. For example, if a user accurately crosses a crosswalk vertically, a clear boundary between data values and a sharp change in boundary may appear in the projection data according to the crosswalk pattern. If the user is crossing the crosswalk at an angle, the projection data may show an ambiguous boundary between data values and a change in the boundary with a predetermined inclination.

The moving direction determining unit 140 may estimate the moving direction of the visually impaired by analyzing a change in inclination of pixel values constituting the projection data. As described above, depending on the user's moving direction with respect to the crosswalk, the inclination change inevitably occurs in the section where the pixel values change in the projection data. Accordingly, the traveling direction determining unit 140 may determine whether the user is crossing the crosswalk in the correct direction based on the change in inclination of pixel values constituting the projection data. A more detailed description of the slope determination will be described later with reference to FIG. 6 .

The guide device 100 according to an embodiment of the present invention outputs to the user interface of the terminal 200 based on the output data of the crosswalk detection unit 120 , the walking signal detection unit 130 , and the traveling direction determination unit 140 . It may include an information analysis unit 150 that generates the walking guide information to be used. That is, the information analysis unit 150 serves to provide the user with feedback on the result of the analysis based on the front image 10 . At this time, the information analysis unit 150 receives and analyzes the output data of the crosswalk detection unit 120 , the walking signal detection unit 130 , and the traveling direction determination unit 140 individually, and provides walking guide information for each analysis result. are created individually.

If the number of the first regions of interest 21 corresponding to the crosswalk is greater than or equal to a predetermined reference number, the information analysis unit 150 determines that there is a crosswalk in front of the visually impaired and prevents access to the crosswalk. The notification may generate walking guide information. For example, if there are three or more first ROIs corresponding to a crosswalk among the eight first ROIs, and it is determined that the distance between the crosswalk and the user is within a predetermined range, the information analysis unit 150 may generate walking guide information indicating that the crosswalk has been approached. At this time, the walking guide information through the user interface of the terminal 200 "There is a crosswalk in front, please stop once." may be output in the form of a sound of the phrase.

The information analysis unit 150 may generate walking guide information for inducing the visually impaired to stop or walk in a crosswalk based on the state of the walking signal included in the candidate areas. For example, if it is determined that the walking signal is a red signal, the information analysis unit 150 may say "This is a current stop signal. Please stop" through the user interface. It is possible to generate walking guide information for inducing a stop that is output as a phrase. If it is determined that there is not enough time for the signal to flash even if it is a green signal or to cross through arrows and numbers, the information analysis unit 150 displays "Not enough time remaining. Please stop" through the user interface. It is possible to generate walking guide information for inducing a stop that is output as a phrase.

In addition, when it is determined that the walking signal has changed from the red signal to the green signal, the information analysis unit 150 "changes to a walking signal. You can cross the crosswalk" through the user interface. It is possible to generate gait guide information for inducing a gait that is output as a phrase. If it is determined that the red and green signals are lit at the same time, the information analysis unit 150 displays "The traffic light is broken. You cannot use this crosswalk" through the user interface. It is possible to generate walking guide information for inducing a stop that is output as a phrase.

The information analysis unit 150 may generate walking guide information that guides the moving direction of the visually impaired within a predetermined angular range when the moving direction of the visually impaired is out of a predetermined angular range based on the virtual vertical line of the crosswalk. have. In this case, the predetermined angle range may be automatically determined based on the width of the crosswalk, the first entry point of the crosswalk, the current moving point, and the like. For example, when it is determined that the user walks in a crosswalk within a normal range, the information analysis unit 150 does not generate separate walking guide information. However, when walking guide information is requested from the user through the user interface, the information analysis unit 150 "maintains the current direction." It is possible to generate walking guide information output as a phrase. If it is determined that the user's moving direction exceeds a certain angle to the left (or right), the information analysis unit 150 "Please adjust the direction to the right (or left)" through the user interface. It is possible to generate walking guide information output as a phrase.

Since the above-described walking guide information is provided to the user in real time through the user interface, the user can stably walk in the crosswalk based on the walking guide information output in the form of sound.

2 shows a plurality of first ROIs 20 for the anterior image 10 according to an embodiment of the present invention, and FIG. 3 shows a plurality of first ROIs according to an embodiment of the present invention. 20) for the binarization result (30).

The plurality of first regions of interest 20 according to an embodiment of the present invention may be set at predetermined intervals in the front image 10 . In this case, in order to increase the accuracy of determining the region of interest, the first regions of interest 20 may be set in a rectangular shape with respect to the vertical direction of the front image as shown in FIG. 2 . When there is a crosswalk in front, it is highly likely that some or several parts will be obscured by obstacles such as people or vehicles traveling on the road. Accordingly, even when some of the regions of interest on the crosswalk are obscured by obstacles, in order to increase the reliability of detection of the crosswalk, the first regions of interest 20 may be set as plural as shown in FIG. 2 .

Referring to FIG. 3 , the crosswalk detection unit 120 according to an embodiment of the present invention may perform binarization based on the characteristics of the crosswalk pattern identified in each of the plurality of first regions of interest 21 . . If there is only a crosswalk without other obstacles in the region of interest, since the pixels in the region of interest have a bi-modal distribution, binarization clearly expresses the pattern of the crosswalk in the first regions of interest 21. process can be understood. For example, the crosswalk detection unit 120 detects the difference between the average brightness values of the white area and the gray (or black) area, the area ratio of the white area, from the white area to the gray area along the vertical direction, and from the gray area to the white area. A crosswalk pattern in the first region of interest 21 may be determined based on the number of times it is converted to .

Specifically, the crosswalk detection unit 120 may perform primary binarization using the average value of the features of the crosswalk pattern as a threshold value. When the primary binarization is completed, the mean and variance values of the resulting white area (i.e. crosswalk area) can be estimated. At this time, when the variance value is sufficiently small (i.e. less than a predetermined reference value), the crosswalk detection unit 120 may use the first regions of interest 30 that have been first binarized as data for determining the crosswalk. Conversely, when the variance value is large (i.e. greater than or equal to a predetermined reference value), the crosswalk detection unit 120 may additionally perform secondary binarization using the average value of the white area as a threshold value. Through the iterative execution of such binarization, the crosswalk pattern can be determined more accurately and the analysis accuracy can be improved.

4 shows the binarization problem according to the poor paint condition of the crosswalk.

Referring to FIG. 4 , the white area of the crosswalk photographed by the front image 10 may be partially peeled off, such as a portion indicated by a yellow dotted line, or the paint state may be poor. As described above, when the painted state of the white area of the crosswalk photographed with the front image 10 is poor, there is a possibility that the binarization result is not desirable. Looking at the first and last binarization results 40 on the basis of the left side, it can be seen that a part of the red box area that should be completely separated by a white area is omitted and is expressed as the binarization result 40 .

Therefore, in order to improve the above-described problem, the crosswalk detection unit 120 according to an embodiment of the present invention may perform color correction based on a morphology technique for the crosswalk pattern. That is, the crosswalk detection unit 120 may improve the painted state of the white area by performing pre-processing and post-processing through morphology before and after the binarization step. Morphology refers to an operation technique for extracting only a desired part in an image, extracting elements necessary to express the shape of an object structure in an image, or clarifying the structure of an object in an image. Therefore, in the present invention, a morphology technique for clarifying an ambiguous object region generated in the binarization process may be used.

5 shows (a) a binarization result 40 of a conventional basic method, and (b) a binarization result 30 according to an embodiment of the present invention according to the presence or absence of an obstacle.

If an obstacle exists in the first region of interest 21 , a problem as shown in FIG. 5 (a) occurs when normal binarization is performed. The upper image of FIG. 5A is the front image 10 in which the first region of interest 21 is set, and the lower image shows the binarization result 51 according to the OTSU algorithm. Since a vehicle acting as an obstacle exists in the area indicated by the yellow dotted line in the upper image of FIG. 5 (a), when using the conventional binarization method, the white color of the crosswalk as shown in the lower image of FIG. 5 (a) A phenomenon occurs in which the region and the gray region cannot be separated. Therefore, it is necessary to solve this problem by improving the conventional binarization technique.

One of the improvement methods for this is the binarization method according to an embodiment of the present invention described with reference to FIG. 3 . The upper image of FIG. 5B is the front image 10 in which the first region of interest 21 is set, and the lower image shows the binarization result 52 according to an embodiment of the present invention. Even if there is an obstacle, such as an area indicated by a yellow dotted line in the upper image of FIG. 5 (b), if another binarization method is used according to an embodiment of the present invention, as shown in the lower image of FIG. The white area and the gray area can be clearly separated.

6 is a graph showing a change in inclination according to projection data and a moving direction according to an embodiment of the present invention.

When the user accurately walks in the vertical direction of the crosswalk, the normally binarized first region of interest 30 and the projected data in the form of a graph may appear as shown in FIG. 6A . In this case, the X axis of the projection data may represent the vertical length of the binarized first region of interest 30 , and the Y axis may represent pixel values. The pixel value displayed on the Y-axis means the number or size of pixels accumulated on the projection data. Referring to the graph on the right of FIG. 6A , it can be seen that the pixel values of the projection data show a distribution in the form of a square wave that is clearly distinguished according to the pattern of the crosswalk. Since pixel values change rapidly at the boundary generated by the square wave distribution, it can be understood that the user is accurately and accurately walking in the vertical direction of the crosswalk through these changes.

On the other hand, when the user walks at a predetermined angle (eg 40 degrees) based on the horizontal direction of the crosswalk, the normally binarized first region of interest 30 and the projection data in the form of a graph are displayed as shown in FIG. may appear Referring to the graph on the right of FIG. 6B , it can be confirmed that, unlike in FIG. 6A , the boundary at which pixel values of the projection data change changes while having a predetermined inclination. That is, by obtaining the slope or comparing the brightness of the section in which such a change occurs, it can be determined in which direction the user is walking in the crosswalk.

In other words, the progress direction determination unit 140 according to an embodiment of the present invention analyzes a change in a section in which pixel values constituting the projection data change from a minimum value to a maximum value (eg, estimating the slope or comparing the brightness of the section). etc.), estimating the direction in which the crosswalk pattern is inclined, and determining the user's moving direction.

7 is a flowchart illustrating a guide method for walking in a crosswalk for the visually impaired according to an embodiment of the present invention.

Referring to FIG. 7 , the guide method for walking in a crosswalk for the visually impaired according to an embodiment of the present invention includes the steps of collecting the front image 10 transmitted from the terminal 200 worn by the visually impaired (S10) , set a plurality of first regions of interest 20 for the front image 10, and through at least one of feature-based analysis or machine learning for the first regions of interest 20, the first corresponding to the crosswalk 1 Step of estimating the regions of interest 21 ( S20 ), setting one central region of the front image 10 as the second region of interest, and performing at least one of feature-based analysis or machine learning for the second region of interest estimating candidate regions including the gait signal among the second regions of interest through the step of determining the state of the gait signal included in the candidate regions (S30), based on the first regions of interest 21 corresponding to the crosswalk estimating the moving direction of the visually impaired by generating projection data based on the horizontal direction and analyzing the gradient change of pixel values constituting the projection data (S40) and a crosswalk detection unit 120, a walking signal detection unit ( 130) and generating (S50) walking guide information output to the user interface of the terminal 200 based on the output data of the proceeding direction determining unit 140 .

Regarding the method according to an embodiment of the present invention, the contents of the above-described apparatus may be applied. Accordingly, in relation to the method, descriptions of the same contents as those of the above-described apparatus are omitted.

Meanwhile, according to an embodiment of the present invention, it is possible to provide a computer-readable recording medium in which a program for executing the above-described method is recorded in a computer. In other words, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of the data used in the above-described method may be recorded in a computer-readable medium through various means. A recording medium for recording an executable computer program or code for performing various methods of the present invention should not be construed as including temporary objects such as carrier waves or signals. The computer-readable medium may include a storage medium such as a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, a memory card, etc.) and an optically readable medium (eg, a CD-ROM, a DVD, etc.).

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

Terms used in this specification will be briefly described, and the present invention will be described in detail.
The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.
When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" described in the specification mean a unit for processing at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.
Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.
1 is a block diagram illustrating a guide device for walking in a crosswalk for a visually impaired person according to an embodiment of the present invention.
Referring to FIG. 1 , a guide device 100 according to an embodiment of the present invention is a device capable of interworking with a terminal 200 worn by a visually impaired (hereinafter referred to as a user), and a front image ( 10) can be analyzed to induce the user's stable walking in the crosswalk. In this case, the guide device 100 may communicate with the terminal 200 worn by the user through wired/wireless network communication, and may output walking guide information through a user interface that can be driven by the terminal 200 worn by the user.
Specifically, the guide device 100 according to an embodiment of the present invention may include an image acquisition unit 110 that collects the front image 10 transmitted from the terminal 200 worn by the user. The front image 10 refers to an image of a direction in which the body in which the terminal 200 is worn faces based on the moving direction of the user. For example, if the user is wearing a head-mounted display (HMD) type terminal, images in the direction of the head based on the moving direction of the user are taken as the front image 10 , which is the image acquisition unit 110 . can be transmitted to At this time, the image acquisition unit 110 analyzes the front and rear changes of the front images 10 collected from the terminal 200 worn by the user, so that the images greatly deviated from the moving direction due to the movement of the head are the front images 10 . It is determined that this is not the case, and these images can be deleted by themselves.
The guide apparatus 100 according to an embodiment of the present invention may include a crosswalk detection unit 120 that detects whether a crosswalk exists in the front image 10 . The crosswalk detector 120 may set a plurality of first regions of interest 20 for the front image 10 . The reason why some regions, not the entire region, of the front image 10 are set as the first regions of interest 20 is to enable rapid analysis while maintaining accuracy in a situation where walking is to be guided in real time. In this case, the size, number, etc. of the plurality of first regions of interest 20 may be automatically determined by the size of the collected front image 10 , or may be set or changed through an external input applied through the user interface. have.
The crosswalk detection unit 120 may estimate the first areas of interest 21 corresponding to the crosswalk through at least one of feature-based analysis or machine learning for each of the first areas of interest 20 . The feature-based analysis refers to analyzing characteristic elements such as the shape, size, and color of the crosswalk pattern included in the first regions of interest 20 . For example, in the feature-based analysis of the crosswalk pattern, whether the white area and the gray area are alternately repeated with a certain thickness, whether the ratio of the area of the white area to the gray area is similar, bi-modal ) may be included in the analysis on whether or not it has a brightness distribution of the shape. In this case, the crosswalk detection unit 120 binarizes the first regions of interest 20 in order to improve the analysis accuracy of the crosswalk pattern in the feature-based analysis process. More details on binarization will be described later with reference to FIG. 3 .
Meanwhile, machine learning includes image-based supervised learning, unsupervised learning, reinforcement learning, and deep learning. For example, the machine learning algorithm may include a support vector machine (SVM), a random tree, a convolutional neural network (CNN), and the like. The crosswalk detection unit 120 learns whether a crosswalk is included in the first regions of interest 20 using a machine learning algorithm as in the above-described example, and a front image ( 10), it is possible to determine whether a crosswalk exists.
The guide device 100 according to an embodiment of the present invention may include a walking signal detection unit 130 for checking the lighting of a walking signal such as a traffic light installed in a crosswalk. The walking signal detector 130 may set one central region of the front image 10 as the second ROI. That is, the walking signal detector 130 sets a region of interest for the front image 10 separately from the crosswalk detector 120 . For example, in the case of a pedestrian traffic light, since it is generally located above the height of a person, the walking signal detection unit 130 sets the 1/2 region based on the center of the front image 10 as the second region of interest. can In this case, the criterion for determining the second ROI may be automatically determined by the size of the collected front image 10 , or may be set or changed through an external input applied through the user interface.
The gait signal detector 130 may estimate candidate regions including the gait signal among the second ROIs through at least one of a feature-based analysis of the second ROI or machine learning. The feature-based analysis refers to analyzing feature elements such as the shape, size, and color of the gait signal included in the second region of interest. For example, the feature-based analysis of the walking signal may include analysis of whether a red or green lighting area exists, and whether the shape of the lighting area is a human shape rather than a circle.
At this time, in the case of a traffic light for pedestrians, unlike a traffic light for a vehicle, there is a high possibility that there are visual obstructions around, and the cases of false detection of a candidate area may increase due to the influence of ambient lighting at night. Accordingly, the gait signal detector 130 estimates a plurality of candidate regions determined to contain the gait signal within the second region of interest, thereby reducing the probability of erroneous detection in determining the gait signal. can That is, the gait signal detector 130 selects all candidate regions extracted within one second region of interest as considerations for determining the state of the gait signal.
The walking signal detector 130 may determine the state of the walking signal included in the candidate regions selected through the above-described process. For example, the walking signal detector 130 may detect a change in lighting of a red signal or a green signal in the walking signal included in the candidate regions. If the green signal is lit, it may be determined whether the green signal is flickering. Also, when there is an area in which time is displayed in an arrow shape or a number shape, the time remaining until the signal is changed may be estimated. Such determination of the state of the walking signal may improve accuracy by comparing a plurality of candidate regions.
The guide apparatus 100 according to an embodiment of the present invention may include a moving direction determining unit 140 for detecting the user's moving direction in real time in a situation of crossing a crosswalk. Determination of the traveling direction may be performed with respect to the first regions of interest 20 detected as a crosswalk by the crosswalk detection unit 120 . At this time, since obstacles such as surrounding pedestrians may exist in a situation in which the user is walking along the crosswalk, the first regions of interest 20 determined as the crosswalk may change for each frame of the front image 10 .
The traveling direction determiner 140 may use the binarization result 30 for the first regions of interest 21 corresponding to the crosswalk to generate the projection data. That is, the traveling direction determiner 140 may generate projection data for indicating how much the normally binarized first regions of interest 30 have changed based on the horizontal direction of the crosswalk. For example, if a user accurately crosses a crosswalk vertically, a clear boundary between data values and a sharp change in boundary may appear in the projection data according to the crosswalk pattern. If the user is crossing the crosswalk at an angle, the projection data may show an ambiguous boundary between data values and a change in the boundary with a predetermined inclination.
The moving direction determining unit 140 may estimate the moving direction of the visually impaired by analyzing a change in inclination of pixel values constituting the projection data. As described above, depending on the user's moving direction with respect to the crosswalk, the inclination change inevitably occurs in the section where the pixel values change in the projection data. Accordingly, the traveling direction determining unit 140 may determine whether the user is crossing the crosswalk in the correct direction based on the change in inclination of pixel values constituting the projection data. A more detailed description of the slope determination will be described later with reference to FIG. 6 .
The guide device 100 according to an embodiment of the present invention outputs to the user interface of the terminal 200 based on the output data of the crosswalk detection unit 120 , the walking signal detection unit 130 , and the traveling direction determination unit 140 . It may include an information analysis unit 150 that generates the walking guide information to be used. That is, the information analysis unit 150 serves to provide the user with feedback on the result of the analysis based on the front image 10 . At this time, the information analysis unit 150 receives and analyzes the output data of the crosswalk detection unit 120 , the walking signal detection unit 130 , and the traveling direction determination unit 140 individually, and provides walking guide information for each analysis result. are created individually.
If the number of the first regions of interest 21 corresponding to the crosswalk is greater than or equal to a predetermined reference number, the information analysis unit 150 determines that there is a crosswalk in front of the visually impaired and prevents access to the crosswalk. The notification may generate walking guide information. For example, if there are three or more first ROIs corresponding to a crosswalk among the eight first ROIs, and it is determined that the distance between the crosswalk and the user is within a predetermined range, the information analysis unit 150 may generate walking guide information indicating that the crosswalk has been approached. At this time, the walking guide information through the user interface of the terminal 200 "There is a crosswalk in front, please stop once." may be output in the form of a sound of the phrase.
The information analysis unit 150 may generate walking guide information for inducing the visually impaired to stop or walk in a crosswalk based on the state of the walking signal included in the candidate areas. For example, if it is determined that the walking signal is a red signal, the information analysis unit 150 may say "This is a current stop signal. Please stop" through the user interface. It is possible to generate walking guide information for inducing a stop that is output as a phrase. If it is determined that there is not enough time for the signal to flash even if it is a green signal or to cross through arrows and numbers, the information analysis unit 150 displays "Not enough time remaining. Please stop" through the user interface. It is possible to generate walking guide information for inducing a stop that is output as a phrase.
In addition, when it is determined that the walking signal has changed from the red signal to the green signal, the information analysis unit 150 "changes to a walking signal. You can cross the crosswalk" through the user interface. It is possible to generate gait guide information for inducing a gait that is output as a phrase. If it is determined that the red and green signals are lit at the same time, the information analysis unit 150 displays "The traffic light is broken. You cannot use this crosswalk" through the user interface. It is possible to generate walking guide information for inducing a stop that is output as a phrase.
The information analysis unit 150 may generate walking guide information that guides the moving direction of the visually impaired within a predetermined angular range when the moving direction of the visually impaired is out of a predetermined angular range based on the virtual vertical line of the crosswalk. have. In this case, the predetermined angle range may be automatically determined based on the width of the crosswalk, the first entry point of the crosswalk, the current moving point, and the like. For example, when it is determined that the user walks in a crosswalk within a normal range, the information analysis unit 150 does not generate separate walking guide information. However, when walking guide information is requested from the user through the user interface, the information analysis unit 150 "maintains the current direction." It is possible to generate walking guide information output as a phrase. If it is determined that the user's moving direction exceeds a certain angle to the left (or right), the information analysis unit 150 "Please adjust the direction to the right (or left)" through the user interface. It is possible to generate walking guide information output as a phrase.
Since the above-described walking guide information is provided to the user in real time through the user interface, the user can stably walk in the crosswalk based on the walking guide information output in the form of sound.
2 shows a plurality of first ROIs 20 for the anterior image 10 according to an embodiment of the present invention, and FIG. 3 shows a plurality of first ROIs according to an embodiment of the present invention. 20) for the binarization result (30).
The plurality of first regions of interest 20 according to an embodiment of the present invention may be set at predetermined intervals in the front image 10 . In this case, in order to increase the accuracy of determining the region of interest, the first regions of interest 20 may be set in a rectangular shape with respect to the vertical direction of the front image as shown in FIG. 2 . When there is a crosswalk in front, it is highly likely that some or several parts will be obscured by obstacles such as people or vehicles traveling on the road. Accordingly, even when some of the regions of interest on the crosswalk are obscured by obstacles, in order to increase the reliability of detection of the crosswalk, the first regions of interest 20 may be set in plurality as shown in FIG. 2 .
Referring to FIG. 3 , the crosswalk detection unit 120 according to an embodiment of the present invention may perform binarization based on the characteristics of the crosswalk pattern identified in each of the plurality of first regions of interest 21 . . If there is only a crosswalk without other obstacles in the region of interest, since the pixels in the region of interest have a bi-modal distribution, binarization clearly expresses the pattern of the crosswalk in the first regions of interest 21. process can be understood. For example, the crosswalk detection unit 120 detects the difference between the average brightness values of the white area and the gray (or black) area, the area ratio of the white area, from the white area to the gray area along the vertical direction, and from the gray area to the white area A crosswalk pattern in the first region of interest 21 may be determined based on the number of times it is converted to .
Specifically, the crosswalk detection unit 120 may perform primary binarization using the average value of the features of the crosswalk pattern as a threshold value. When the primary binarization is completed, the average value and variance value of the resulting white area (ie the crosswalk area) can be estimated. At this time, when the variance value is sufficiently small (ie, less than a predetermined reference value), the crosswalk detection unit 120 may use the first regions of interest 30 that have been first binarized as data for determining the crosswalk. Conversely, when the variance value is large (ie, greater than or equal to a predetermined reference value), the crosswalk detection unit 120 may additionally perform secondary binarization using the average value of the white area as a threshold value. Through the iterative execution of such binarization, the crosswalk pattern can be determined more accurately and the analysis accuracy can be improved.
4 shows the binarization problem according to the poor paint condition of the crosswalk.
Referring to FIG. 4 , the white area of the crosswalk photographed by the front image 10 may be partially peeled off, such as a portion indicated by a yellow dotted line, or the paint state may be poor. As described above, when the painted state of the white area of the crosswalk photographed with the front image 10 is poor, there is a possibility that the binarization result is not desirable. Looking at the first and last binarization results 40 on the basis of the left side, it can be seen that a part of the red box area that should be completely separated by a white area is omitted and is expressed as the binarization result 40 .
Therefore, in order to improve the above-described problem, the crosswalk detection unit 120 according to an embodiment of the present invention may perform color correction based on a morphology technique for the crosswalk pattern. That is, the crosswalk detection unit 120 may improve the painted state of the white area by performing pre-processing and post-processing through morphology before and after the binarization step. Morphology refers to an operation technique for extracting only a desired part in an image, extracting elements necessary to express the shape of an object structure in an image, or clarifying the structure of an object in an image. Therefore, in the present invention, a morphology technique for clarifying an ambiguous object region generated in the binarization process may be used.
5 shows (a) a binarization result 40 of a conventional basic method, and (b) a binarization result 30 according to an embodiment of the present invention according to the presence or absence of an obstacle.
If an obstacle exists in the first region of interest 21 , a problem as shown in FIG. 5 (a) occurs when normal binarization is performed. The upper image of FIG. 5A is the front image 10 in which the first region of interest 21 is set, and the lower image shows the binarization result 51 according to the OTSU algorithm. Since a vehicle acting as an obstacle exists in the area indicated by the yellow dotted line in the upper image of FIG. 5A, when using the conventional binarization method, the white color of the crosswalk as shown in the lower image of FIG. 5A A phenomenon occurs in which the region and the gray region cannot be separated. Therefore, it is necessary to solve this problem by improving the conventional binarization technique.
One of the improvement methods for this is the binarization method according to an embodiment of the present invention described with reference to FIG. 3 . The upper image of FIG. 5B is the front image 10 in which the first region of interest 21 is set, and the lower image shows the binarization result 52 according to an embodiment of the present invention. Even if there is an obstacle such as the area indicated by the yellow dotted line in the upper image of FIG. 5 (b), if another binarization method is used according to an embodiment of the present invention, as in the lower image of FIG. The white area and the gray area can be clearly separated.
6 is a graph showing a change in inclination according to projection data and a moving direction according to an embodiment of the present invention.
When the user accurately walks in the vertical direction of the crosswalk, the normally binarized first region of interest 30 and the projection data in the form of a graph may appear as shown in FIG. 6( a ). In this case, the X axis of the projection data may represent the vertical length of the binarized first region of interest 30 , and the Y axis may represent pixel values. The pixel value displayed on the Y-axis means the number or size of pixels accumulated on the projection data. Referring to the graph on the right of FIG. 6A , it can be seen that the pixel values of the projection data show a distribution in the form of a square wave that is clearly distinguished according to the pattern of the crosswalk. Since pixel values change rapidly at the boundary generated by the square wave distribution, it can be understood that the user is accurately and accurately walking in the vertical direction of the crosswalk through these changes.
On the other hand, when the user walks at a predetermined angle (eg 40 degrees) based on the horizontal direction of the crosswalk, the normally binarized first region of interest 30 and the projection data in the form of a graph are displayed as shown in FIG. may appear Referring to the graph on the right of FIG. 6B , it can be confirmed that, unlike in FIG. 6A , the boundary at which pixel values of the projection data change changes while having a predetermined inclination. That is, by obtaining the slope or comparing the brightness of the section in which such a change occurs, it can be determined in which direction the user is walking in the crosswalk.
In other words, the progress direction determination unit 140 according to an embodiment of the present invention analyzes a change in a section in which pixel values constituting the projection data change from a minimum value to a maximum value (eg, estimating the slope or comparing the brightness of the section). etc.), estimating the direction in which the crosswalk pattern is inclined, and determining the user's moving direction.
7 is a flowchart illustrating a guide method for walking in a crosswalk for the visually impaired according to an embodiment of the present invention.
Referring to FIG. 7 , the guide method for walking in a crosswalk for the visually impaired according to an embodiment of the present invention includes the steps of collecting the front image 10 transmitted from the terminal 200 worn by the visually impaired (S10) , set a plurality of first regions of interest 20 for the front image 10, and perform a first corresponding to a crosswalk through at least one of feature-based analysis or machine learning for the first regions of interest 20 1 Step of estimating the regions of interest 21 ( S20 ), setting one central region of the front image 10 as the second region of interest, and performing at least one of feature-based analysis or machine learning for the second region of interest estimating candidate regions including the gait signal among the second regions of interest through the step of determining the state of the gait signal included in the candidate regions (S30), based on the first regions of interest 21 corresponding to the crosswalk estimating the moving direction of the visually impaired by generating projection data based on the horizontal direction and analyzing the gradient change of pixel values constituting the projection data (S40) and a crosswalk detection unit 120, a walking signal detection unit ( 130) and generating (S50) walking guide information output to the user interface of the terminal 200 based on the output data of the proceeding direction determining unit 140 .
Regarding the method according to an embodiment of the present invention, the contents of the above-described apparatus may be applied. Accordingly, in relation to the method, descriptions of the same contents as those of the above-described apparatus are omitted.
Meanwhile, according to an embodiment of the present invention, it is possible to provide a computer-readable recording medium in which a program for executing the above-described method in a computer is recorded. In other words, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of the data used in the above-described method may be recorded in a computer-readable medium through various means. A recording medium for recording an executable computer program or code for performing various methods of the present invention should not be construed as including temporary objects such as carrier waves or signals. The computer-readable medium may include a storage medium such as a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, a memory card, etc.) and an optically readable medium (eg, a CD-ROM, a DVD, etc.).
The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.
The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. .

Claims (16)

In the guide device for walking in a crosswalk for the visually impaired,
An image acquisition unit for collecting the front image transmitted from the terminal worn by the visually impaired;
A crosswalk detection unit that sets a plurality of first ROIs for the front image and estimates first ROIs corresponding to a crosswalk through at least one of feature-based analysis or machine learning for the first ROIs ;
A central region of the front image is set as a second region of interest, and candidate regions including a walking signal among the second region of interest are estimated through at least one of a feature-based analysis of the second region of interest or machine learning. and a walking signal detector for determining the state of the walking signal included in the candidate regions;
A moving direction for estimating the moving direction of the visually impaired by generating projection data based on the horizontal direction based on the first regions of interest corresponding to the crosswalk, and analyzing the gradient change of pixel values constituting the projection data judging unit; and
An apparatus including an information analysis unit for generating walking guide information output to a user interface of the terminal based on the output data of the crosswalk detection unit, the walking signal detection unit, and the traveling direction determination unit.
The method of claim 1,
The crosswalk detection unit,
performing binarization based on the characteristics of the crosswalk pattern identified in each of the plurality of first regions of interest, and individually determining whether the plurality of first regions of interest correspond to a crosswalk based on the binarization device to do.
3. The method of claim 2,
The crosswalk detection unit,
Perform binarization using the average value of the characteristics of the crosswalk pattern as a threshold value, estimate the average value and variance value for the crosswalk area derived based on the binarization, and additionally based on the estimated average value and variance value An apparatus, characterized in that it determines whether to perform binarization.
4. The method of claim 3,
The crosswalk detection unit,
Apparatus, characterized in that performing a shape correction based on the morphology technique for the crosswalk pattern.
The method of claim 1,
The progress direction determination unit,
The device, characterized in that by analyzing a change in a section in which pixel values constituting the projection data change from a minimum value to a maximum value, the direction in which the pedestrian crossing pattern is inclined is estimated.
The method of claim 1,
The information analysis unit,
When the number of first areas of interest corresponding to the crosswalk is greater than or equal to a predetermined reference number, it is determined that a crosswalk exists in front of the visually impaired, and generating walking guide information informing the approach to the crosswalk device characterized.
The method of claim 1,
The information analysis unit,
Device characterized in that based on the state of the walking signal included in the candidate areas to generate walking guide information for inducing the visually impaired to stop or walk in the crosswalk.
The method of claim 1,
The information analysis unit,
When the moving direction of the visually impaired is out of a predetermined angular range based on the virtual vertical line of the crosswalk, generating walking guide information for guiding the visually impaired person's moving direction within the predetermined angular range Device.
In the guiding method for the visually impaired to walk in a crosswalk,
Collecting the front image transmitted from the terminal worn by the visually impaired;
setting a plurality of first ROIs for the front image, and estimating first ROIs corresponding to a crosswalk through at least one of feature-based analysis or machine learning of the first ROIs;
A central region of the front image is set as a second region of interest, and candidate regions including a walking signal among the second region of interest are estimated through at least one of a feature-based analysis of the second region of interest or machine learning. and determining the state of the walking signal included in the candidate regions;
generating projection data based on a horizontal direction based on the first regions of interest corresponding to the crosswalk, and estimating the moving direction of the visually impaired by analyzing a change in inclination of pixel values constituting the projection data; and
The method comprising the step of generating the walking guide information output to the user interface of the terminal based on the output data of the crosswalk detection unit, the walking signal detection unit, and the traveling direction determination unit.
10. The method of claim 9,
In the step of estimating the first regions of interest,
performing binarization based on the characteristics of the crosswalk pattern identified in each of the plurality of first regions of interest, and individually determining whether the plurality of first regions of interest correspond to a crosswalk based on the binarization how to do it
11. The method of claim 10,
In the step of estimating the first regions of interest,
Perform binarization using the average value of the characteristics of the crosswalk pattern as a threshold value, estimate the average value and variance value for the crosswalk area derived based on the binarization, and additionally based on the estimated average value and variance value A method comprising determining whether to perform binarization.
12. The method of claim 11,
In the step of estimating the first regions of interest,
Method, characterized in that performing a shape correction based on the morphology technique for the crosswalk pattern.
10. The method of claim 9,
In the step of estimating the progress direction of the visually impaired,
The device, characterized in that by analyzing a change in a section in which pixel values constituting the projection data change from a minimum value to a maximum value, the direction in which the pedestrian crossing pattern is inclined is estimated.
10. The method of claim 9,
In the step of generating the walking guide information,
When the number of first areas of interest corresponding to the crosswalk is greater than or equal to a predetermined reference number, it is determined that a crosswalk exists in front of the visually impaired, and generating walking guide information informing the approach to the crosswalk How to characterize.
10. The method of claim 9,
In the step of generating the walking guide information,
Based on the state of the walking signal included in the candidate areas, the method characterized in that the generation of walking guide information for inducing the visually impaired to stop or walk in the crosswalk.
10. The method of claim 9,
In the step of generating the walking guide information,
When the moving direction of the visually impaired is out of a predetermined angular range based on the virtual vertical line of the crosswalk, generating walking guide information for guiding the visually impaired person's moving direction within the predetermined angular range Way.

Images