KR102644647B1 - Path guidance device for visually impaired based on braille block detection and method of operating the same - Google Patents

Abstract

The route guidance device includes an image acquisition unit that receives a front image, generates a region of interest image, a critical range calculation unit that calculates a critical range corresponding to the region of interest image, receives the critical range, and inputs the region of interest image to the A Braille block detection unit that generates a binarized image by applying a threshold range and generates detection data by detecting a Braille block from the binarized image, and generates object data including shape and size information of the Braille block based on the detection data. It may include a region dividing unit that analyzes the object data, a path determination unit that generates Braille block analysis data including Braille block pattern information and a recommended progress path, and a user interface that provides the Braille block analysis data to the user. You can.

Description

Braille block detection-based route guidance device for the visually impaired and its operating method {PATH GUIDANCE DEVICE FOR VISUALLY IMPAIRED BASED ON BRAILLE BLOCK DETECTION AND METHOD OF OPERATING THE SAME}

The present invention relates to a route guidance device for the visually impaired, and more specifically, to a route guidance device for the visually impaired that detects braille blocks from a front image and provides accurate route information, and to a method of driving the same.

In order to improve the mobility of the visually impaired, various studies are being conducted to provide relevant information by analyzing various situations encountered while walking. Representative examples include attempts to combine smart functions by attaching sensors to walking sticks for the visually impaired, as well as research on detecting obstacles and planning routes using wearable cameras.

Meanwhile, commercial products such as OrCam, which provides text and face recognition functions by attaching a small camera to the glasses of the visually impaired, have been released. When a visually impaired person wears glasses with a communication function and transmits an image, a professional guide Services such as Aira, which provides walking guidance, have also appeared.

Braille blocks are essential facilities for walking for the visually impaired. The visually impaired recognize the direction of movement by recognizing the braille blocks through the senses of their feet, and determine whether the braille blocks are linear or dot-shaped to obtain information about stopping situations. However, there are difficulties in responding preemptively to situations where Braille blocks are temporarily stopped or change direction, or encounter junctions such as three or four-way intersections.

Therefore, in order to encourage safe walking of the visually impaired in indoor and outdoor environments where Braille blocks exist, technology is required to detect the Braille block area in front of the user and accurately determine the progress path based on images obtained from cameras.

Korean Patent Publication No. 10-2016-0029909, “Information provision device for the visually impaired” Korean Patent No. 10-2256721, “Wearable device for independent living of the visually impaired”

Another object of the present invention is to provide a route guidance device that detects a Braille block pattern from a front image.

Another object of the present invention is to provide a route guidance device that analyzes the pattern of a Braille block and guides an accurate route.

Another object of the present invention is to provide a method of driving the route guidance device.

However, the problem to be solved by the present invention is not limited to the above-mentioned problem, and may be expanded in various ways without departing from the spirit and scope of the present invention.

In order to achieve an object of the present invention, a route guidance device according to embodiments of the present invention includes an image acquisition unit that receives a front image and generates a region of interest image, and an image acquisition unit that calculates a critical range corresponding to the region of interest image. A threshold range calculation unit, a Braille block detection unit that receives the threshold range, applies the threshold range to the region of interest image to generate a binarized image, and detects a Braille block from the binarized image to generate detection data, the detection data A region dividing unit that generates object data including the shape and size information of the braille block based on the region division unit, and a path determination unit that generates braille block analysis data including braille block pattern information and a recommended progress path by analyzing the object data. It may include a unit, and a user interface that provides the Braille block analysis data to the user.

In one embodiment, the braille block pattern included in the braille block pattern information may include at least one of straight ahead, left turn, right turn, intersection, left turn, right turn, left and right turn, and stop. The path determination unit may determine the braille block pattern based on the average value and variance value for each region of the projection data included in the object data.

In one embodiment, the path determination unit may determine whether the pedestrian deviates to the left, deviates to the right, slopes to the left, and slopes to the right based on the object data. When the left deviation occurs, the route determination unit determines the recommended progress path to move to the right, when the right deviation occurs, the route determination unit determines the recommended progress path to move to the left, and when the left tilt occurs, the recommended progress path turns right. , and when the right inclination occurs, the recommended proceeding route may be determined to be a left turn.

In one embodiment, the threshold range calculation unit receives a reference threshold range, masks the region of interest image based on the reference threshold range, performs an HSV histogram on the masked region of interest image, and calculates a LOW threshold and a HIGH threshold. The threshold range can be calculated by extracting each threshold value.

In one embodiment, the braille block detector detects the braille block corresponding to the reference information in the binarized image based on pre-stored reference information of the braille block, and detects a braille block including the braille block in the binarized image. Detection data that distinguishes between area 1 and area 2, which does not include the braille block, can be generated.

In one embodiment, the region divider extracts an upper boundary of the Braille block, a lower boundary of the Braille block, and a slope of the Braille block from the detection data, and at least one of the upper boundary, the lower boundary, and the slope The object data containing one can be created.

In one embodiment, the device may further include an update control unit that receives the object data and determines whether to update the threshold range based on a reference variance value.

In one embodiment, the update control unit compares the reference variance value with the variance value of projection data included in the object data, and when the variance value of the projection data is greater than the reference variance value, an update signal is sent to the threshold range calculator. can be output.

In one embodiment, the user interface may include at least one of a smartphone, wireless earphones, a smart watch, and an electric wheelchair.

In order to achieve another object of the present invention, a method of driving a route guidance device according to embodiments of the present invention includes receiving a front image, generating a region of interest image based on the front image, and responding to the region of interest image. calculating a critical range, generating a binarized image by applying the critical range to the region of interest image, detecting a braille block from the binarized image, and object data including shape and size information of the braille block. It may include generating, and analyzing the object data to generate braille block analysis data including braille block pattern information and a recommended progress path. The step of generating the braille block analysis data includes determining a braille block pattern based on the average and variance values for each region of the projection data included in the object data, and whether the pedestrian deviates to the left based on the object data; It may include determining whether the route is deviated to the right, slanted to the left, and slanted to the right, and determining the recommended path corresponding to each of the deviated left, slanted to the right, slanted to the left, and slanted to the right. .

The route guidance device according to embodiments of the present invention can accurately detect a braille block pattern from a front image and analyze the pattern of the braille block to guide pedestrians to an accurate route.

However, the effects of the present invention are not limited to the effects described above, and may be expanded in various ways without departing from the spirit and scope of the present invention.

1 is a block diagram showing the configuration of a route guidance device according to embodiments of the present invention.
Figure 2 is a diagram illustrating an example in which a region of interest image is generated in an image acquisition unit.
Figure 3 is a block diagram showing an algorithm by which a critical range calculation unit calculates a critical range for an image of a region of interest.
Figure 4 is a diagram showing an example in which a binarized image is generated in the Braille block detection unit.
Figure 5 is a table showing the results of comparing the actual image and the binarized image generated by the Braille block detection unit.
Figure 6 is a diagram showing the operation of extracting the upper boundary of a Braille block from the area dividing unit.
Figure 7 is a diagram showing the operation of extracting the bottom boundary of a Braille block from the area dividing unit.
Figure 8 is a diagram showing an example in which the upper and lower boundaries of a Braille block are extracted from a region of interest image.
Figure 9 is a diagram showing the operation of extracting the slope of a Braille block from the area dividing unit.
Figure 10 is a diagram for explaining the operation of the update control unit.
Figure 11 is a diagram showing an example of a Braille block pattern recognized by the path determination unit.
Figure 12 is a diagram showing judgment criteria for determining a Braille block pattern from object data.
Figure 13 is a diagram showing the logical structure by which the path determination unit determines the Braille block pattern.
Figure 14 is a diagram showing an example of a proceeding path determined by the path determination unit.
Figure 15 is a diagram showing the judgment criteria for determining the progress path from object data.

Hereinafter, various embodiments of this document are described with reference to the attached drawings.

The examples and terms used herein are not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or substitutes for the examples.

In the following description of various embodiments, if a detailed description of a related known function or configuration is judged to unnecessarily obscure the gist of the invention, the detailed description will be omitted.

The terms described below are terms defined in consideration of functions in various embodiments, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

In connection with the description of the drawings, similar reference numbers may be used for similar components.

Singular expressions may include plural expressions, unless the context clearly indicates otherwise.

In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of the items listed together.

Expressions such as “first,” “second,” “first,” or “second,” can modify the corresponding components regardless of order or importance, and are used to distinguish one component from another. It is only used and does not limit the corresponding components.

When a component (e.g., a first) component is said to be "connected (functionally or communicatively)" or "connected" to another (e.g., second) component, it means that the component is connected to the other component. It may be connected directly to the element or may be connected through another component (e.g., a third component).

In this specification, “configured to” means “suitable for,” “having the ability to,” or “changed to,” depending on the situation, for example, in terms of hardware or software. ," can be used interchangeably with "made to," "capable of," or "designed to."

In some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components.

For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

Additionally, the term 'or' means 'inclusive or' rather than 'exclusive or'.

That is, unless otherwise stated or clear from the context, the expression 'x uses a or b' means any of the natural inclusive permutations.

In the above-described specific embodiments, components included in the invention are expressed in singular or plural numbers depending on the specific embodiment presented.

However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the above-described embodiments are not limited to singular or plural components, and even if the components expressed in plural are composed of singular or , Even components expressed as singular may be composed of plural elements.

Meanwhile, in the description of the invention, specific embodiments have been described, but of course, various modifications are possible without departing from the scope of the technical idea implied by the various embodiments.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the claims described below as well as equivalents to these claims.

1 is a block diagram showing the configuration of a route guidance device according to embodiments of the present invention.

The route guidance device can accurately detect the braille block pattern from the front image (IMG) and analyze the pattern of the braille block to guide pedestrians to the correct route.

Referring to FIG. 1, the route guidance device includes an image acquisition unit 100, a critical range calculation unit 200, a Braille block detection unit 300, and a region division unit 400. The route guidance device includes a route determination unit 600, and a user interface 700. Additionally, the route guidance device may further include an update control unit 500.

The image acquisition unit 100 may receive an input front image (IMG). The image acquisition unit 100 may generate a region of interest image (ROI) based on the front image (IMG).

The image acquisition unit 100 may output a region of interest image (ROI) to the threshold range calculation unit 200 and the Braille block detection unit 300.

The threshold range calculator 200 may convert the region of interest image (ROI) into the HSV color space.

The critical range calculation unit 200 may calculate a critical range corresponding to the region of interest image (ROI).

For example, the critical range calculation unit 200 may calculate a critical range for each HSV color space component by applying sequential masking to the region of interest image (ROI).

The critical range calculation unit 200 may output the critical range to the Braille block detection unit 300.

The Braille block detection unit 300 may receive the threshold range and apply the threshold range to the region of interest image (ROI) to generate a binarized image.

The Braille block detection unit 300 may detect a Braille block from the binarized image and generate detection data DD.

The Braille block detection unit 300 may output the detection data DD to the region dividing unit 400.

The region dividing unit 400 may generate object data OD including shape and size information of the Braille block based on the detection data DD.

The area division unit 400 may output object data OD to the update control unit 500 and the path determination unit 600.

The update control unit 500 may receive the object data OD and determine whether to update the threshold range based on the reference variance value.

For example, the update control unit 500 may output the update signal US to the threshold range calculation unit 200.

The path determination unit 600 may generate Braille block analysis data (AD) including Braille block pattern information and a recommended progress path by analyzing the object data (OD).

The user interface 700 may provide the Braille block analysis data (AD) to the user.

FIG. 2 is a diagram illustrating an example in which a region of interest image (ROI) is generated in the image acquisition unit 100.

Referring to FIG. 2, the image acquisition unit 100 may receive an input front image (IMG).

For example, the image acquisition unit 100 may be a smartphone, smart glasses, a wearable camera, a black box, etc.

The image acquisition unit 100 may generate a region of interest image (ROI) based on the front image (IMG). For example, the image acquisition unit 100 may normalize front image (IMG) input at various resolutions.

Specifically, the image acquisition unit 100 determines the resolution of the front image (IMG), whether the front image (IMG) is a horizontal image or a vertical image, the number of frames per second of the front image (IMG), and the shooting of the front image (IMG). The front image (IMG) can be normalized by considering angles, etc.

For example, as shown in FIG. 2, the image acquisition unit 100 may generate a region of interest image (ROI) with a size of 144 × 480 from a front image (IMG) with a size of 480 × 720.

The image acquisition unit 100 may output a region of interest image (ROI) to the threshold range calculation unit 200 and the Braille block detection unit 300.

Figure 3 is a block diagram showing an algorithm by which a critical range calculation unit calculates a critical range for an image of a region of interest.

Referring to FIG. 3, the critical range calculation unit 200 may apply masking to the region of interest image (ROI) to calculate the critical range for the HSV color space component.

The threshold range calculation unit 200 may receive a region of interest image (ROI) and convert the region of interest image (ROI) into the HSV color space.

For example, the critical range calculation unit 200 extracts the H component (color component, hue), S component (saturation component, saturation), and V component (brightness component, value) from the region of interest image (ROI). You can.

The critical range calculation unit 200 may calculate a critical range corresponding to the region of interest image (ROI).

For example, each threshold range is a preset color threshold (lower color limit) in the HSV color space. H color upper limit), preset saturation threshold range (saturation lower limit) S Saturation upper limit) and preset brightness threshold range (brightness lower limit) V brightness upper limit) may be included.

In one embodiment, the threshold range calculation unit 200 receives a reference threshold range and applies sequential masking to the region of interest image (ROI) to calculate a threshold range for each of the H component, S component, and V component. can do.

For example, the reference threshold range is the reference color threshold range (10 H 110), reference saturation critical range (30 S 255) and reference brightness critical range (30 V 255) may be included.

*The threshold range calculation unit 200 may mask the region of interest image (ROI) based on the reference threshold range.

The threshold range calculation unit 200 may calculate an HSV histogram for the region of interest image (ROI). The threshold range calculation unit 200 may extract the LOW threshold and the HIGH threshold, respectively, based on the HSV histogram.

The threshold range calculation unit 200 may calculate a threshold range for each HSV component, including a LOW threshold and a HIGH threshold for the V component.

In one embodiment, the critical range calculation unit 200 may calculate a critical range for each HSV color space component by applying sequential masking to the region of interest image (ROI).

For example, the S component critical range calculation unit may calculate the S component critical range based on a reference threshold range.

For example, the V component critical range calculator may calculate the V component critical range based on the S component critical range.

For example, the H component critical range calculation unit may calculate the H component critical range based on the V component critical range.

The critical range calculator 200 may further include a multi-mode critical range calculator.

The multi-mode critical range calculation unit 200 may determine whether the H component is multi-mode based on the H histogram calculation result.

The multi-mode critical range calculator may determine whether it is single mode or multi-mode based on the shape of the histogram.

The multi-mode critical range calculation unit can determine whether or not the multi-mode is multi-mode according to the number of maximum points and minimum points in the form of a histogram. For example, Figure 5(a) may represent a single mode, and Figure 5(b) may represent a multi-mode.

The multi-mode critical range calculator may additionally mask the region of interest image (ROI) when the H component is multi-mode.

The multi-mode critical range calculation unit may recalculate the S histogram and V histogram for the region of interest image (ROI).

The multi-mode threshold range calculation unit may extract LOW thresholds and HIGH thresholds for each of the S component and V component based on the histogram.

The multi-mode threshold range calculation unit additionally calculates the S component threshold range including a LOW threshold and a HIGH threshold for the S component, and the V component threshold range including a LOW threshold and a HIGH threshold for the V component. can do.

The critical range calculation unit 200 may output the critical range including the H component critical range, the S component critical range, and the V component critical range to the Braille block detection unit 300.

Figure 4 is a diagram showing an example of a binarized image generated by the Braille block detection unit 300, and Figure 5 is a diagram showing the results of comparing the actual image and the binarized image generated by the Braille block detection unit 300.

Referring to FIGS. 4 and 5, the Braille block detection unit 300 may receive the region of interest image (ROI) and apply the threshold range to the region of interest image (ROI) to generate a binarized image.

For example, the Braille block detection unit 300 may receive the threshold range and convert the region of interest image (ROI) into the HSV color space. The Braille block detector 300 may generate a binarized image by applying the H component threshold range, S component threshold range, and V component threshold range to the region of interest image (ROI), respectively.

As shown in FIG. 4, the Braille block detection unit 300 may convert the region of interest image (ROI) into a binarized image based on the histogram analysis result and threshold range data performed by the threshold range calculation unit 200.

For example, the Braille block detector 300 may generate a binarized image reflecting the H component critical range, S component critical range, and V component critical range, respectively.

For example, in the case of multi-mode, additional masking and threshold range calculation are performed, so accurate object detection may be possible in the binarized image.

The Braille block detection unit 300 according to embodiments of the present invention can generate an accurate binarized image compared to existing technology.

As shown in Figure 5, as a result of comparing the actual image (GT) in which the region of interest is ideally binarized and the binarized image (Result) of the Braille block detection unit 300 according to the present invention, the actual image (GT) and the binarized image (Result) )'s similarity is over 91% on average, showing very high accuracy.

The Braille block detection unit 300 may detect the Braille block corresponding to the reference information in the binarized image based on pre-stored reference information of the Braille block.

The Braille block detection unit 300 may generate detection data DD that distinguishes a first area including the Braille block from the binarized image and a second area not including the Braille block.

The Braille block detection unit 300 may detect a Braille block from the binarized image and generate detection data DD. The Braille block detection unit 300 may output detection data DD including a binarized image to the region dividing unit 400.

Figure 6 is a diagram showing the operation of extracting the upper border of the Braille block in the region dividing unit 400, Figure 7 is a diagram showing the operation of extracting the lower border of the Braille block in the region dividing unit 400, and Figure 8 is a diagram illustrating an example in which the upper and lower boundaries of a braille block are extracted from a region of interest image (ROI), and FIG. 9 is a diagram illustrating an operation of extracting the slope of a braille block in the region dividing unit 400.

Referring to FIGS. 6 to 9 , the region dividing unit 400 may generate object data OD including shape and size information of the Braille block based on the detection data DD.

For example, the region divider 400 may extract the upper boundary of the Braille block, the lower boundary of the Braille block, and the slope of the Braille block from the detection data DD.

Specifically, the region dividing unit 400 receives detection data DD, extracts the upper boundary from the detection data DD, extracts the lower boundary from the detection data DD, and extracts the slope from the detection data DD. An inspection can be performed and an area can be divided from the detection data (DD).

The region dividing unit 400 may generate one-dimensional projection data by accumulating the binarized images of the detection data DD in the vertical direction.

As shown in FIG. 6, the region dividing unit 400 can extract the upper boundary from projection data. For example, the region divider 400 may detect the location of a point that meets MAX (maximum value)-delta based on projection data to extract the upper boundary.

In an ideal case, the projection data would maintain the same size in the central area, but in actual projection data, there may be fluctuations due to effects such as noise, so the region divider 400 may set delta to accommodate this.

If the binarized image included in the detection data (DD) contains a lot of noise, there may be multiple boundary points in the projection data, so the area divider 400 starts from the right boundary of the left area based on the area already divided in the previous frame. The upper section that exists between the left boundary of the right area can be considered a meaningful boundary point.

For example, the region dividing unit 400 may select the leftmost boundary point (T1) and the rightmost boundary point (T2) in the upper section.

As shown in FIG. 7, the region dividing unit 400 can extract the lower boundary from projection data. The bottom border can be extracted separately by dividing the left and right areas.

For example, in the case of the left area, the minimum value MIN is obtained in the left area based on T1 in FIG. 8, and then the location of the point where MIN+delta meets can be detected as the boundary point (B1).

If multiple points are detected in the left area, the position of the rightmost boundary can be set to B1.

For example, in the case of the right area, after finding the minimum value MIN in the right area based on T2, the location of the point where MIN+delta meets can be detected as the boundary point (B2).

If multiple points are detected in the right area, the position of the leftmost boundary can be set to B2.

Referring to FIG. 8, an example is shown in which the region dividing unit 400 extracts the upper boundary and the lower boundary for the right branch pattern among the Braille blocks.

Projection data can be divided into three areas: BB_L (left area), BB_C (middle area), and BB_R (right area). Transition areas TR_L and TR_R may exist between each area.

Based on the boundary of the projection data, the region dividing unit 400 can collect information on the shape and size of the Braille block.

As shown in FIG. 9, the region dividing unit 400 may perform a slope test from the detection data DD. For example, the front image (IMG) may be tilted to the right at the left border of the object and to the left at the right border, and if the object's orientation is tilted to one side, the tilt direction at the left border or right border may be the same. .

However, since both cases appear similar in the projection data, it is necessary to perform a tilt check to check whether a tilt reversal phenomenon occurs on the left and right sides, respectively.

For example, if the left slope is reversed, the positions of T1 and B1 in the projection data may become closer, and if the right slope is reversed, the positions of T2 and B2 may become closer.

Accordingly, the region dividing unit 400 may obtain the numerical difference between the upper boundary value and the lower boundary value of the binarized image for a certain section in the moving direction, and if a negative number appears, it may be determined that the slope is reversed.

The region divider 400 can determine the location of each region of BB_L (left region), BB_C (middle region), and BB_R (right region) by reflecting the upper border, lower border, and slope on the left and right. .

The region dividing unit 400 may generate the object data OD including at least one of the upper boundary, the lower boundary, and the slope.

For example, the region dividing unit 400 may generate object data OD including shape and size information of a Braille block by extracting the upper boundary, lower boundary, and slope from the projection data.

The area division unit 400 may output object data OD to the update control unit 500 and the path determination unit 600.

FIG. 10 is a diagram for explaining the operation of the update control unit 500.

Referring to FIG. 10, the update control unit 500 may receive the object data OD and determine whether to update the threshold range based on the reference variance value.

The critical range needs to be updated to the optimal range depending on the indoor and outdoor environment, weather, illumination level, etc.

However, since calculating the critical range anew every frame may consume unnecessary calculation data, the update control unit 500 may determine whether to update the critical range based on the reference variance value that serves as a reference for updating.

For example, the update control unit 500 compares the reference variance value with the variance value of projection data included in the object data OD, and when the variance value of the projection data is greater than the reference variance value, the threshold range An update signal (US) may be output to the calculation unit 200.

As shown in FIG. 10(a), when the variance value of the projection data is smaller than the reference variance value, the update control unit 500 may determine that update is unnecessary.

As shown in FIG. 10(b), when the variance value of the projection data is greater than the reference variance value, the update control unit 500 may determine that an update is necessary.

The threshold range calculation unit 200 may receive an update signal (US). The threshold range calculation unit 200 may recalculate the threshold range according to the update signal US.

For example, the threshold range calculation unit 200 may recalculate the threshold range according to the update signal US and output the updated threshold range to the Braille block detection unit 300.

FIG. 11 is a diagram showing an example of a Braille block pattern recognized by the path determination unit 600, FIG. 12 is a diagram showing judgment criteria for determining a Braille block pattern from object data (OD), and FIG. 13 is a path determination unit. (600) is a diagram showing the logical structure for judging the Braille block pattern.

The path determination unit 600 according to embodiments of the present invention may generate Braille block analysis data (AD) including Braille block pattern information and a recommended progress path by analyzing the object data (OD).

Referring to Figures 11 to 13, the braille block pattern may include at least one of straight ahead, left turn, right turn, intersection, left turn, right turn, left and right turn, and stop.

The path determination unit 600 may calculate statistical values for each area for projection data based on the object data (OD).

For example, the path determination unit 600 may calculate the average and variance values for each region for projection data in BB_L (left region), BB_C (middle region), and BB_R (right region) along the horizontal direction.

For example, the path determination unit 600 may calculate the average and variance values for each region for projection data in the transition regions TR_L and TR_R along the horizontal direction.

The path determination unit 600 may divide a partial region of the region of interest image into BB_T along the vertical direction and calculate the average and variance values for each region for the projection data corresponding to this region.

As shown in FIG. 12, when the average value of the projection data is greater than the first reference average value (m1), the path determination unit 600 may determine that a Braille block exists in the upper area.

If the average value for the projection data is greater than the second reference average value (m2) and the variance value for the projection data is less than the first reference variance value (var1), the path determination unit 600 determines that a braille block exists in the left area. You can judge.

If the width of the projection data is greater than the reference width (w1) and the variance value for the projection data is less than the second reference variance value (var2), the path determination unit 600 may determine that a staircase exists in the left transition area. there is.

If the average value of the projection data is greater than the second reference average value (m2), the path determination unit 600 may determine that a Braille block exists in the central area.

If the width of the projection data is greater than the reference width (w1) and the variance value for the projection data is less than the second reference variance value (var2), the path determination unit 600 may determine that a staircase exists in the right transition area. there is.

If the average value for the projection data is greater than the second reference average value (m2) and the variance value for the projection data is less than the first reference variance value (var1), the path determination unit 600 determines that a braille block exists in the right area. You can judge.

As shown in Figure 13, the method of discriminating the braille block pattern for each feature is primarily classified according to whether a braille block is present at the top.

For example, in the general case where feature ⓐ=1, the path determination unit 600 can determine whether braille blocks exist on the left and right sides, respectively, based on BB_C.

The path determination unit 600 determines that a braille block exists when the braille block ratio in the corresponding area exceeds a certain size (feature ⓑ or feature ⓕ) and when a staircase exists in the transition area (feature ⓒ or feature ⓔ). can do.

On the other hand, if the upper braille block does not exist (for example, feature ⓐ = 0), the path determination unit 600 may not consider the presence of stairs in the left transition area and the right transition area. The reason for not considering the presence of steps in the left and right transition areas may be to distinguish between the stationary pattern and the left and right branching pattern.

FIG. 14 is a diagram showing an example of a progress path determined by the path determination unit 600, and FIG. 15 is a diagram showing judgment criteria for determining a proceeding route from object data OD.

Referring to FIGS. 14 and 15 , the path determination unit 600 may determine the distance and slope at which the pedestrian deviates from the Braille block based on the binarized image of the object data (OD) in order to determine the recommended proceeding path.

As shown in FIG. 14, the path determination unit 600 can determine whether the pedestrian deviates to the left, deviates to the right, slopes to the left, and slopes to the right based on the Braille block.

In one embodiment, whether left or right deviation from the Braille block can be determined in a situation where there is no left or right tilt reversal.

*For example, the path determination unit 600 may determine the start and end positions of the BB_C area based on the degree to which they approach the left and right boundaries of the area of interest.

When deviation to the left occurs, the route determination unit 600 may determine the recommended proceeding route as a rightward movement.

When a deviation to the right occurs, the route determination unit 600 may determine the recommended proceeding route as a left movement.

In one embodiment, whether the braille block is tilted left or right can be determined in a situation where the left and right tilts are reversed.

For example, when reversing the left slope, the path determination unit 600 may determine the slope based on the width of the right transition area. When reversing the right slope, the path determination unit 600 may determine the slope based on the width of the left transition area.

When a left slope occurs, the route determination unit 600 may determine the recommended route to take a right turn.

When a right inclination occurs, the route determination unit 600 may determine the recommended proceeding route as a left turn.

The path determination unit 600 may output braille block analysis data (AD) including braille block pattern information and a recommended progress path to the user interface 700.

The user interface 700 may provide the Braille block analysis data (AD) to the user.

For example, the user interface 700 may be a smartphone, wireless earphone, smart watch, electric wheelchair, wearable device, etc.

The user interface 700 can deliver information about the front Braille block to the user in real time, based on the Braille block analysis data (AD).

In this way, the route guidance device according to embodiments of the present invention can accurately detect the braille block pattern from the front image (IMG) and analyze the pattern of the braille block to guide pedestrians to the correct route.

As described above, although the embodiments have been described with limited drawings, those skilled in the art will be able to make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

100: Image acquisition unit 200: Critical range calculation unit
300: Braille block detection unit 400: Area dividing unit
500: update control unit 600: path determination unit
700: User interface

Claims (10)

An image acquisition unit that receives a front image and generates a region of interest image;
a threshold range calculation unit that calculates a threshold range corresponding to the region of interest image;
a Braille block detection unit that receives the threshold range, generates a binarized image by applying the threshold range to the image of the region of interest, and generates detection data by detecting a Braille block from the binarized image;
a region dividing unit that generates object data including shape and size information of the braille block based on the detection data;
a path determination unit that generates braille block analysis data including braille block pattern information and a recommended progress path by analyzing the object data; and
It includes a user interface that provides the Braille block analysis data to the user,
The critical range calculation unit,
Further comprising a multi-mode threshold range calculation unit that determines whether the region of interest image is single-mode or multi-mode based on the HSV histogram, and additionally calculates a threshold range for the multi-mode if it is determined to be multi-mode,
Route guidance device. According to paragraph 1,
The braille block pattern included in the braille block pattern information includes at least one of straight ahead, left turn, right turn, intersection, left turn, right turn, left and right turn, and stop,
The path determination unit,
Characterized in that the braille block pattern is determined based on the average value and variance value for each region for the projection data included in the object data,
Characterized in that,
Route guidance device. According to paragraph 1,
The path determination unit,
Based on the object data, determine whether the pedestrian deviates to the left, deviates to the right, slopes to the left, and slopes to the right,
When the left deviation occurs, the recommended proceeding path is determined to be a right movement,
When the deviation to the right occurs, the recommended proceeding path is determined to move to the left,
When the left slope occurs, the recommended route is decided to turn right,
When the right inclination occurs, the recommended proceeding route is determined to be a left turn,
Route guidance device. According to paragraph 1,
The critical range calculation unit,
Receive a reference threshold range, mask the region of interest image based on the reference threshold range, calculate an HSV histogram for the masked region of interest image, and extract a LOW threshold and a HIGH threshold, respectively, to determine the threshold range. Characterized in calculating,
Route guidance device. According to paragraph 1,
The Braille block detection unit,
Based on pre-stored reference information of the braille block, detecting the braille block corresponding to the reference information in the binarized image,
Characterized in generating detection data that distinguishes between a first area including the braille block and a second area not containing the braille block in the binarized image,
Route guidance device. According to paragraph 1,
The area dividing unit,
Extract the top border of the Braille block, the bottom border of the Braille block, and the slope of the Braille block from the detection data, and generate the object data including at least one of the top border, the bottom border, and the slope. Characterized in that,
Route guidance device. According to paragraph 1,
Characterized in that it further comprises an update control unit that receives the object data and determines whether to update the threshold range based on the reference variance value,
Route guidance device. In clause 7,
The update control unit,
Comparing the reference variance value with the variance value of projection data included in the object data, and outputting an update signal to the threshold range calculation unit when the variance value of the projection data is greater than the reference variance value,
Route guidance device. According to paragraph 1,
The user interface is,
Characterized by including at least one of a smartphone, wireless earphones, smart watch, and electric wheelchair,
Route guidance device. Receiving a front image;
generating a region of interest image based on the front image;
calculating a threshold range corresponding to the region of interest image;
generating a binarized image by applying the threshold range to the region of interest image and detecting a braille block from the binarized image;
Generating object data including shape and size information of the Braille block; and
By analyzing the object data, generating braille block analysis data including braille block pattern information and a recommended progress path,
The step of calculating the critical range is,
Determining whether the region of interest image is single-mode or multi-mode based on the HSV histogram, and if it is determined to be multi-mode, additionally calculating a threshold range for the multi-mode,
The step of generating the braille block analysis data is,
determining a braille block pattern based on the average and variance values for each region of the projection data included in the object data; and
Based on the object data, it is determined whether the pedestrian deviates to the left, deviates to the right, inclines to the left, and inclines to the right, and determines whether the pedestrian deviates to the left, deviates to the right, inclines to the left, and inclines to the right. Characterized by comprising the step of determining a recommended progress path,
How to drive a route guidance device.