KR20190056281A - Apparatus and method of guiding indoor road for visually handicapped based on visible light communication - Google Patents

Abstract

The present invention relates to an indoor road guiding device and method for blind people based on visible light communication. According to an embodiment of the present invention, a user terminal comprises: a moving path determining unit which determines a moving path based on a destination; a light device recognizing unit which recognizes a light device adjacent to a current position of the user terminal among light devices positioned on the determined moving path; a navigation unit which transmits a navigation guiding request to the recognized light device, and receives a current position of the light device from the light device in response to the transmitted navigation guiding request; a moving direction calculating unit which calculates a moving direction by using the predetermined moving path, the current position of the light device, and the current position of the user terminal; and a feedback providing unit which provides at least one of haptic feedback or auditory feedback depending on the calculated moving direction. The present invention can change the intensity of haptic feedback or auditory feedback according to a user′s current position.

Description

FIELD OF THE INVENTION [0001] The present invention relates to an indoor navigation system for visually impaired persons,

TECHNICAL FIELD The present invention relates to a technical idea for guiding a road in a room using a visible light communication technology, and provides guidance information on an indoor road to a visually impaired person by transmitting physical feedback on an obstacle irregularly located in the room using visible light communication And more particularly,

Generally, the market size for services and products related to the handicapped is relatively small, but it is stable in terms of profitability.

More specifically, the market for services and products related to persons with disabilities can be set at relatively high prices due to a small fluctuation in demand due to the nature of demand and supply.

The number of domestic blind people is about 250,000, the US is 1.3 million, and the UK / France is about 100,000 and 200,000 respectively. Considering the global market size, it is a small but long-term function.

Currently, the world market for handicapped persons and elderly people is $ 59.2 billion (as of 2011). As the system for accessibility of household appliances is spreading in developed countries, and it is likely to be institutionalized as a trade technology barrier, It is necessary.

In Korea, the number of people with disabilities and elderly people is about 798,000 (2010: 16.7% of the total population), and the domestic market for handicapped goods is estimated at about 72.7 billion won (2009).

On average, 12.9% of all blind people distinguish the light, but 78% of the low vision people are responding to both light and color.

In some cases, the degree of recognition of light is known to be possible. [Kim, In-cheol, A Study on the Application of Smart Technology for Exhibition Space for the Blind, 2007]

The visually impaired can use the Google Android talkback function, the iOS Voice Over function, or the Samsung TouchWiz color control function to provide coloring pages, the color blindness / color reduction, and the color adjustment process as a wizard.

For example, Apple Inc.'s iOS Voice Over offers a virtual control feature called a rotor, which is selected by flick gestures when the rotor is turned, Using a fast-flipping gesture, you can move from the existing focus to the next focus.

In addition, if you flip quickly to the left or right, a sound may be produced when a 'perfect' sound is produced or a row is changed, and a voice may be generated to indicate to which app each time the focus moves.

In general, the visually impaired can move using a point-freedom block, a cane, or the like.

Factors that make it difficult for the visually impaired to move include stairs, bollards, and geometric braille block placement.

Therefore, the visually handicapped moves according to the changes appearing on the wall when moving the cane, and the movement of the cane at the intersection or turning point of the road increases and the distance between the cane node and the node can be reduced.

The difficulty of the walking process of the visually impaired is understood as the determination of the current position and the setting of the direction.

Therefore, there is a need to develop an indoor route guidance technology based on visible light communication, in which a visually impaired person can determine the current location and set the traveling direction more quickly.

Korean Patent No. 10-1371246, " Guide Device for the Blind Using Visible Light Communication " Korean Patent No. 10-1481378, " Method and Apparatus for Providing a Path for the Visually Impaired in a Subway Station Using Visible Light Communication " Korean Patent No. 10-1371246, " Guide Device for the Blind Using Visible Light Communication " Korean Patent Publication No. 10-2015-0113245, " Exhibition space guidance system and method using a mobile terminal vibration signal " Korean Patent No. 10-1558060, "Indoor Location Recognition System Based on Visible Light Communication, Indoor Navigation Method, Indoor Navigation System, Server and Electronic Device Performing Indoor Navigation"

Kim, In - Chul, A Study on the Application of Smart Technology to the Exhibition Space for Visually Impaired Persons, The Korean Institute of Architectural Architects 2007 Vol.3 No.4, 55p ~ 62p

The present invention provides a function of communicating with a user terminal device when a visually impaired person (hereinafter referred to as a user) approaches a visible light communication-based illuminator device and guiding the user step by step through haptic / auditory feedback through a user terminal device And the like.

It is an object of the present invention to provide a function of predicting a destination from a current position by changing a strength of haptic feedback based on a touch point using a front portion of a user terminal.

The present invention may aim to provide haptic feedback with respect to directional guidance using the back side of the user terminal device.

The present invention may aim to change the intensity of the haptic / auditory feedback according to the current position of the user.

The present invention may aim to change the intensity of the haptic / auditory feedback according to the intensity of the ambient noise.

The present invention can be aimed at delivering the strength of the haptic feedback in a stepwise manner during direction guidance.

The present invention aims to provide auditory feedback with an earphone when the user wears an earphone.

The present invention aims at providing a function of focusing the user on the moving direction by changing the strength of the haptic feedback considering the change of the moving speed and the stride of the user.

The present invention may aim to increase the recognition rate of the user to the destination by gradually providing the intensity of the haptic feedback from the origin to the destination along the movement direction.

According to an embodiment of the present invention, a user terminal includes a travel route determining unit that determines a travel route based on a destination, and an illuminating device that is adjacent to a current location of the user terminal, A navigation unit for transmitting a navigation guidance request to the recognized lighting device and receiving a current position of the lighting device from the lighting device in response to the transmitted navigation guidance request; And a feedback providing unit for providing at least one of haptic feedback or auditory feedback in accordance with the calculated movement direction, and a movement direction calculation unit for calculating a movement direction using the current position of the illumination device and the current position of the user terminal device have.

According to an embodiment of the present invention, the feedback providing unit may provide the haptic feedback through a front portion and a rear portion of the user terminal.

According to an embodiment of the present invention, the feedback providing unit may provide front haptic feedback when sensing a touch input on the front unit, The intensity of the haptic feedback can be adjusted.

According to an embodiment of the present invention, when the drag input in the front portion is moved along the moving direction, the feedback providing portion sets the front haptic to the next position in the moving direction with reference to the current position of the drag input. The strength of the feedback can be increased.

According to an embodiment of the present invention, the feedback providing unit may provide the front haptic feedback in consideration of the moving speed and the stride of the user.

According to an embodiment of the present invention, when the moving speed of the user is smaller than a predetermined moving speed value and the stride is smaller than a predetermined stride size value, the intensity of the front haptic feedback is set to a first magnitude The intensity of the front haptic feedback can be continuously increased by a second magnitude when the moving speed of the user is smaller than the predetermined moving speed value and the stride is greater than the predetermined stride size value .

According to an embodiment of the present invention, when the moving speed of the user is larger than the predetermined moving speed value and the stride is smaller than the predetermined stride size value, 1 magnitude, and when the moving speed of the user is larger than the predetermined moving speed value and the stride is greater than the predetermined stride size value, the intensity of the frontal haptic feedback is increased by the second magnitude Can be continuously increased with a deviation.

According to an embodiment of the present invention, when providing the rear haptic feedback through the rear portion according to the calculated movement direction, the feedback providing portion divides the rear portion into a plurality of portions, The intensity of the rear haptic feedback can be increased from the portion to the end based on the moving direction.

According to an embodiment of the present invention, the start portion corresponds to the current position of the user terminal, and the end portion can be determined based on the destination.

According to an embodiment of the present invention, the feedback providing unit may increase the strength of the rear haptic feedback at the front portion of the starting portion when the moving direction is the leftward turning direction, and then increase the back haptic feedback of the left portion of the front portion The strength can be increased.

According to an embodiment of the present invention, the feedback providing unit may further increase the strength of the rear haptic feedback when the position of the user terminal is adjacent to the destination.

According to an embodiment of the present invention, the feedback providing unit may increase the strength of the haptic feedback and reduce the strength of the auditory feedback when the noise level at the current position of the user terminal is greater than the reference value.

According to an embodiment of the present invention, the feedback providing unit may increase the strength of the haptic feedback and reduce the strength of the auditory feedback when the current position of the user terminal is determined to be a public place.

According to an embodiment of the present invention, the travel route determining unit determines the travel route based on the destination using the map server, and the map server transmits the travel route to the illuminating device located on the determined travel route, Can be transmitted.

According to an embodiment of the present invention, there is provided a method of operating a user terminal, the method comprising the steps of: determining a travel route based on a destination in a travel route determining unit; A navigation apparatus comprising: a recognition unit for recognizing a lighting device adjacent to a current position of a terminal device; a navigation unit for transmitting a navigation guidance request to the recognized lighting device; Calculating a moving direction using the determined moving route, the current position of the lighting device and the current position of the user terminal device in the moving direction calculating section, , The haptic feedback or the audio feedback, depending on the calculated movement direction, Also it may include the step of providing at least.

According to an embodiment of the present invention, the step of providing at least one of haptic feedback or voice feedback according to the calculated movement direction may include providing the haptic feedback through the front and rear portions of the user terminal .

According to an embodiment of the present invention, the step of providing the haptic feedback through the front portion and the rear portion of the user terminal includes: providing front haptic feedback when detecting a touch input in the front portion; And increasing the intensity of the frontal haptic feedback to the next position on the moving direction based on the current position of the drag input when the drag input in the moving direction is moved along the moving direction.

According to an embodiment of the present invention, there is provided a method of operating a route guidance system, the method comprising: transmitting a destination to a map server in a user terminal; determining a travel route based on the destination; Identifying the user terminal device adjacent to the current location of the illuminator based on the transmitted identification code in the illuminator, Receiving, at the device, a navigation guidance request from the identified user terminal device and communicating the current location of the lighting device to the identified user terminal device, at the user terminal device, And the current position of the user terminal device, And in the user terminal device, providing at least one of haptic feedback or auditory feedback according to the calculated movement direction.

According to an embodiment of the present invention, in the user terminal, the step of providing at least one of haptic feedback or voice feedback according to the calculated movement direction may include: receiving the haptic feedback through a front portion and a rear portion of the user terminal; And the like.

According to an embodiment of the present invention, the step of providing the haptic feedback through the front and back portions of the user terminal may include providing front haptic feedback when detecting a touch input on the front portion, And increasing the intensity of the front haptic feedback to the next position on the moving direction based on the current position of the drag input when the drag input on the moving part is moved along the moving direction.

The present invention provides a function of communicating with a user terminal device when a visually impaired person (hereinafter referred to as a user) approaches a visible light communication-based illuminator device and guiding the user step by step through haptic / auditory feedback through a user terminal device .

The present invention can provide a function of predicting a destination from a current position by changing a strength of haptic feedback based on a touch point using a front portion of a user terminal.

The present invention can utilize the back side of the user terminal device to provide haptic feedback related to directional guidance.

The present invention can change the intensity of the haptic / auditory feedback according to the current position of the user.

The present invention can change the intensity of the haptic / auditory feedback according to the intensity of the ambient noise.

The present invention can transmit the intensity of the haptic feedback in a stepwise manner during the guidance.

The present invention can provide auditory feedback with an earphone when the user wears an earphone.

The present invention can provide a function of focusing the user on the moving direction by changing the intensity of the haptic feedback considering the change of the moving speed and the stride of the user.

According to the present invention, the intensity of the haptic feedback from the source to the destination is gradually provided along the moving direction, so that the user can increase the recognition rate to the destination.

1 is a view illustrating components of a route guidance system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating components of a user terminal according to an exemplary embodiment of the present invention. Referring to FIG.
FIGS. 3A to 4 are views for explaining an embodiment in which the intensity of the front haptic feedback is changed by the user terminal according to the embodiment of the present invention.
5A and 5B are views for explaining an embodiment in which a user terminal device according to an embodiment of the present invention changes the intensity of rear haptic feedback.
6 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.
7 is a flowchart illustrating a method of operating a navigation system according to an embodiment of the present invention.
8 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.
9 is a flowchart illustrating a method of operating a navigation system according to an embodiment of the present invention.
10 to 13 are flowcharts illustrating a method of operating a user terminal according to an embodiment of the present invention.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings.

It is to be understood that the embodiments and terminologies used herein are not intended to limit the invention to the particular embodiments described, but to include various modifications, equivalents, and / or alternatives of the embodiments.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

The following terms are defined in consideration of functions in various embodiments and may vary depending on the intention of a user, an operator, or the like. Therefore, the definition should be based on the contents throughout this specification.

In connection with the description of the drawings, like reference numerals may be used for similar components.

The singular expressions may include plural expressions unless the context clearly dictates otherwise.

In this document, the expressions " A or B " or " at least one of A and / or B " and the like may include all possible combinations of the items listed together.

Expressions such as " first, " " second, " " first, " or " second, " But is not limited to those components.

When it is mentioned that some (e.g., first) component is "(functionally or communicatively) connected" or "connected" to another (second) component, May be connected directly to the component, or may be connected through another component (e.g., a third component).

As used herein, the term " configured to " is intended to encompass all types of information, including, but not limited to, " , "" Made to "," can do ", or" designed to ".

In some situations, the expression " a device configured to " may mean that the device can " do " with other devices or components.

For example, a processor configured (or configured) to perform the phrases " A, B, and C " may be implemented by executing one or more software programs stored in a memory device or a dedicated processor (e.g., an embedded processor) , And a general purpose processor (e.g., a CPU or an application processor) capable of performing the corresponding operations.

Also, the term 'or' implies an inclusive or 'inclusive' rather than an exclusive or 'exclusive'.

That is, unless expressly stated otherwise or clear from the context, the expression 'x uses a or b' means any of the natural inclusive permutations.

1 is a view illustrating components of a route guidance system according to an embodiment of the present invention.

Referring to FIG. 1, the navigation system 100 may include a user terminal 110, a map server 120, and a lighting device 130.

According to an embodiment of the present invention, the user terminal 110 may set a destination based on a user input, and may transmit an established destination and an identification code to the map server 120.

For example, the map server 120 may calculate a movement route to be moved by the user based on the destination received from the user terminal device 110, and transmit the navigation route to the lighting device 130 located on the calculated movement route, ) ≪ / RTI >

According to an embodiment of the present invention, the illumination device 130 receives a navigation guidance request from the user terminal device 110 that approaches within a predetermined distance, and transmits the navigation guidance request to the user terminal device 110 Can be identified using an identification code.

In addition, the lighting device 130 may provide the current location of the lighting device 130 to the user terminal device 110 when the user terminal device 110 is identified.

According to an embodiment of the present invention, the user terminal 110 calculates the moving direction based on the moving path, the current position of the lighting apparatus, and the current position of the user terminal 110, Or auditory feedback can be generated.

Therefore, according to the present invention, when a visually impaired person (hereinafter referred to as a user) accesses a visible light communication-based illumination device, communication is performed with the user terminal device, and the user is guided step by step through the haptic / Function can be provided.

Here, the haptic feedback may also be referred to as front feedback, rear feedback, front haptic feedback, and rear haptic feedback.

FIG. 2 is a diagram illustrating components of a user terminal according to an exemplary embodiment of the present invention. Referring to FIG.

2, a user terminal 200 according to an exemplary embodiment of the present invention includes a movement path determination unit 210, a lighting device recognition unit 220, a navigation unit 230, a movement direction calculation unit 240, And a feedback providing unit 250.

For example, the user terminal device 200 may also be referred to as a room light guidance device for a visually impaired person based on visible light communication.

According to an embodiment of the present invention, the movement path determination unit 210 may determine a movement path based on a destination.

For example, the travel route determining unit 210 can determine the travel route based on the destination using the map server.

That is, the travel route determining unit 210 may transmit the destination data to the map server, and may determine the travel route by receiving the travel route determined by the map server.

According to an embodiment of the present invention, the illumination device recognition unit 220 can recognize the illumination device adjacent to the current position of the user terminal device among the illumination devices positioned on the movement path.

That is, the illumination device recognition unit 220 can recognize the illumination device that has received the identification code by the map server.

Here, the recognized lighting device can be located within a reference distance that can smoothly provide light-fidelity (Li-Fi) data.

According to an embodiment of the present invention, the navigation unit 230 may transmit a navigation guidance request to the lighting device.

In one example, the navigation unit 230 may receive the current location of the illuminator from the illuminator in response to the navigation guide request.

For example, the navigation unit 230 may transmit the navigation guide request to the lighting device using the near field communication.

For example, Bluetooth, RFID, IrDA, Ultra Wideband (UWB), ZigBee, Z-WAVE, and the like can be used for short-range communication.

Here, the navigation guidance request may include data requesting the current position of the lighting apparatus and the identification code of the user terminal device 200. [

According to an embodiment of the present invention, the movement direction calculation unit 240 may calculate the movement direction using the movement path, the current position of the illumination device, and the current position of the user terminal device.

For example, the movement direction calculation unit 240 may calculate the movement direction by collating the current position of the lighting device with the current position of the user terminal device.

For example, when the current position of the illumination device located on the movement path is the left front with respect to the current position of the user terminal, the movement direction calculation unit 240 may calculate the movement direction toward the left front direction.

According to one embodiment of the present invention, the feedback unit 250 may provide at least one of haptic feedback or auditory feedback according to the direction of movement.

In one example, the feedback providing unit 250 may provide haptic feedback through the front and back portions of the user terminal device 200.

According to an embodiment of the present invention, the feedback unit 250 may provide front haptic feedback when detecting a touch input on the front part.

For example, the feedback providing unit 250 may adjust the strength of the front haptic feedback with respect to the moving direction according to the drag input change in the front portion.

That is, the feedback providing unit 250 can further increase the intensity of the front haptic feedback as the drag input at the front portion is adjacent to the destination.

According to an embodiment of the present invention, when the drag input in the front portion is moved along the moving direction, the feedback providing portion 250 may perform a front haptic feedback to the next position on the moving direction with respect to the current position of the drag input The strength can be increased.

For example, the feedback providing unit 250 may provide the front haptic feedback in consideration of the moving speed and the stride of the user.

According to an embodiment of the present invention, when the moving speed of the user is smaller than a predetermined moving speed value and the stride is smaller than the predetermined stride size value, the feedback providing unit 250 may set the strength of the front haptic feedback to be constant .

For example, the feedback providing unit 250 may continuously increase the strength of the front haptic feedback by a second magnitude when the user's moving speed is smaller than a predetermined moving speed value and the stride is greater than a predetermined stride size value .

For example, the second size may be relatively large compared to the first size. That is, if the first size corresponds to 10, the second size may correspond to 20.

According to an embodiment of the present invention, when the moving speed of the user is greater than a predetermined moving speed value and the stride is smaller than the predetermined stride size value, the feedback providing unit 250 may change the strength of the front haptic feedback by a first magnitude .

For example, if the user's movement speed is greater than a predetermined movement speed value and the stride is greater than a predetermined stride size value, the feedback providing unit 250 continuously increases the strength of the frontal haptic feedback by a second magnitude .

According to one embodiment of the present invention, when providing the rear haptic feedback through the rear portion according to the calculated movement direction, the feedback providing portion 250 divides the rear portion into a plurality of portions, The intensity of the rear haptic feedback can be increased to the end based on the moving direction.

That is, the feedback providing unit 250 may gradually increase the feedback strength from the start point to the end point of the movement direction.

For example, the beginning may correspond to the current location of the user terminal, and the end may be determined based on the destination.

According to an embodiment of the present invention, when the moving direction is the left-turning direction, the feedback providing unit 250 increases the strength of the rear haptic feedback in the front portion of the starting portion and then increases the rear haptic feedback strength of the left portion of the front portion .

For example, the feedback providing unit 250 may further increase the strength of the rear haptic feedback as the position of the user terminal device is closer to the destination.

According to an embodiment of the present invention, when the current position of the user terminal is determined to be a public place, the feedback providing unit 250 may increase the strength of the haptic feedback and reduce the strength of the auditory feedback.

The present invention can change the intensity of the haptic / auditory feedback according to the intensity of the ambient noise.

For example, the user terminal device 200 may provide frontal haptic feedback using a vibrator positioned below the front panel touch panel display.

In addition, the user terminal device 200 may provide rear haptic feedback using a plurality of vibrators positioned under the cover of the rear portion.

The present invention can provide a function of predicting a destination from a current position by changing a strength of haptic feedback based on a touch point using a front portion of a user terminal.

Further, the present invention can change the intensity of the haptic / auditory feedback according to the current position of the user.

FIGS. 3A to 4 are views for explaining an embodiment in which the intensity of the front haptic feedback is changed by the user terminal according to the embodiment of the present invention.

3A to 3D illustrate changes in the overall feedback strength according to the moving speed and stride of the user. 3A to 3D, the abscissa represents the movement distance, and the ordinate represents the feedback strength.

FIG. 3A illustrates the feedback strength according to the change of the moving distance when the user's stride is small and the moving speed is low.

Referring to FIG. 3A, the user terminal may transmit the front haptic feedback in a form of continuously increasing the feedback strength when the stride is small and the moving speed is low.

For example, the user terminal device may increase the intensity of the front haptic feedback, similar to the feedback intensity variation of Figure 3a, if the user's movement speed is less than a predetermined movement speed value and the stride is smaller than a predetermined stride size value have.

For example, the predetermined movement velocity value may correspond to an average value of movement velocity values calculated when at least three users move using a cane indoors.

In addition, the predetermined stride size value may be calculated based on an average value of the number of steps calculated when at least three users move a certain distance using a cane indoors.

Here, when at least three or more visually impaired persons are walking, the user terminal device can calculate at least three user's steps by using a pedometer function or the like.

FIG. 3B illustrates the change in the feedback strength according to the change of the moving distance when the user's stride is small and the moving speed is high.

Referring to FIG. 3B, the user terminal can transmit the front haptic feedback in a form of continuously increasing the feedback strength with a deviation when the stride is small and the moving speed is high.

For example, if the user's movement speed is greater than a predetermined movement speed value and the stride is smaller than a predetermined stride size value, the user terminal device may increase the strength of the frontal haptic feedback similarly to the feedback strength change of Figure 3b have.

In addition, when the user's moving speed is high, the user terminal apparatus can vary the intensity of the feedback, thereby preventing the user from concentrating on the haptic feedback due to the shock felt on the foot.

In addition, the user terminal can increase the feedback strength with a deviation in case the moving speed is slow when the user's moving speed is high.

For example, the user terminal device can increase the instantaneous concentration of the user as the feedback strength increases with variation.

FIG. 3C illustrates a change in the feedback strength according to the change in the moving distance when the user's stride is large and the moving speed is low.

Referring to FIG. 3C, the user terminal can transmit front haptic feedback in the form of continuously increasing the feedback strength when the stride is large and the moving speed is low.

For example, the user terminal may increase the intensity of the front haptic feedback, similar to the feedback strength change of Figure 3c, if the user's movement speed is less than a predetermined movement speed value and the stride is greater than a predetermined stride size value have.

That is, the moving speed of the user terminal is slow, but the moving distance is increased in case the stride is small when the stride is large, so that the intensity of the feedback is further increased to increase the concentration of the user on the front haptic feedback .

FIG. 3D illustrates a change in the feedback strength according to the change of the moving distance when the user's step width is large and the moving speed is high.

Referring to FIG. 3D, the user terminal can transmit the front haptic feedback in the form of continuously increasing the feedback strength in a case where the stride is large and the moving speed is high.

For example, the user terminal device may increase the intensity of the front haptic feedback, similar to the feedback strength change of Figure 3d, if the user's movement speed is greater than a predetermined movement speed value and the stride is greater than a predetermined stride size value have.

For example, the user terminal may support variation of the feedback intensity to compensate for the user's concentration on the frontal haptic feedback in case the movement speed is slow.

Also, as shown in FIGS. 3A to 3D, the user terminal can maximize the feedback strength at a time point when the user's moving direction is switched, as opposed to the time when the user terminal moves in a predetermined direction.

The present invention can provide a function of focusing the user on the moving direction by changing the intensity of the haptic feedback considering the change of the moving speed and the stride of the user.

Also, according to the present invention, the intensity of the haptic feedback from the start point to the destination is gradually provided along the movement direction, so that the user can increase the recognition rate to the destination.

Specifically, FIG. 4 illustrates an operation of changing the feedback strength according to the change of the touch drag direction in the front portion of the user terminal device.

Referring to FIG. 4, the user terminal may change the position and intensity of providing frontal haptic feedback on the front side in accordance with steps 400 through 540.

The user terminal device may display a screen of the first step 400 when the direction of movement of the user is a predetermined distance and then leftwardly rotated.

The user terminal device recognizes the user's touch input at contact point 401 and provides frontal haptic feedback at point 402 and point 403 based on the recognized contact point 401. [

Here, the intensity of the frontal haptic feedback to the point 403 may be greater than the intensity of the frontal haptic feedback to the point 402.

Here, the contact point 401 may correspond to a point at which the touch input of the user is first recognized.

Then, the user terminal can display the screen of the second step 410 when the direction of movement of the user is a predetermined distance and then leftward.

The user terminal device recognizes the user's touch at point 411 and provides front point haptic feedback at point 412 and point 413 based on the recognized point 411.

Here, the intensity of the front haptic feedback to the point 413 may be greater than the intensity of the front haptic feedback to the point 412.

In addition, the contact point 411 may correspond to a point where the touch input of the user is recognized by some dragging along the moving direction.

Then, the user terminal device can display the screen of the third step 420 when the direction of movement of the user is a certain distance and then leftward.

The user terminal device recognizes the user's touch at point of contact 421 and provides frontal haptic feedback at point 422 and point 423 based on the recognized point of contact 421.

Here, the intensity of the front haptic feedback to the point 423 may be greater than the intensity of the front haptic feedback to the point 422.

Further, the contact point 421 may correspond to a recognized point where the touch input of the user is partially dragged along the moving direction.

Then, the user terminal device can display the screen of the fourth step 430 when the moving direction of the user is a predetermined distance and then leftward.

The user terminal device recognizes the touch of the user at contact point 431 and provides front haptic feedback at point 432 and point 433 based on the recognized contact point 431. [

Here, the intensity of the front haptic feedback to the point 432 can maximize the strength of the feedback to convey the left turn point.

Finally, the user terminal device can display the screen of the fifth step 440 when the moving direction of the user is the leftward direction and then goes straight up a certain distance.

The user terminal device recognizes the touch of the user at contact point 441 and provides front haptic feedback at point 442 and point 443 based on the recognized contact point 441. [

Here, the intensity of the front haptic feedback to the point 443 may be greater than the intensity of the front haptic feedback to the point 442.

5A and 5B are views for explaining an embodiment in which a user terminal device according to an embodiment of the present invention changes the intensity of rear haptic feedback.

Referring to FIG. 5A, the back surface 500 of the user terminal may be divided into nine parts to indicate the direction of movement.

 According to an embodiment of the present invention, the user terminal device can sequentially provide rear haptic feedback at a point 501, a point 502, and a point 503 when the direction of movement is leftward.

Here, the intensity of the backside haptic feedback provided at point 503 may be increased relative to the intensity of the backside haptic feedback provided at point 501.

Referring to FIG. 5B, the back surface 510 of the user terminal device may be divided into nine parts to indicate the direction of movement.

 According to an embodiment of the present invention, the user terminal device can sequentially provide rear haptic feedback at the point 511, the point 512, and the point 513 when the moving direction is the reverse direction.

Here, the intensity of the rear haptic feedback provided at point 513 may be increased relative to the intensity of the rear haptic feedback provided at point 511. [

For example, the user terminal device can determine the number of times of providing the rear haptic feedback according to the degree of movement distance.

More specifically, if the movement distance is longer than a predetermined reference distance, the user terminal apparatus may provide the number of times of providing the rear haptic feedback more than a predetermined number of times.

In addition, if the movement distance is shorter than a predetermined reference distance, the user terminal device may provide the number of times of providing the rear haptic feedback by a predetermined number of times or less.

6 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.

For example, the method of operation of the user terminal may also be referred to as a cabin guidance method for visually impaired persons based on visible light communication.

Referring to FIG. 6, the method of operation of the user terminal device may determine a movement path in step 601. FIG.

That is, the operation method of the user terminal can determine the movement route based on the destination through the map server.

The method of operation of the user terminal device in step 602 may recognize an adjacent lighting device on the path.

That is, the operation method of the user terminal device can recognize the illumination device adjacent to the current position of the user terminal device among the illumination devices located on the movement path.

The method of operation of the user terminal device in step 603 may transmit a navigation guidance request to the lighting device.

That is, the operation method of the user terminal device may transmit the navigation guide request to the nearest illuminating device at the current position of the user terminal device.

The method of operation of the user terminal device in step 604 may receive the current location of the lighting device.

That is, the method of operation of the user terminal device may receive the current position of the illuminating device from the illuminating device in response to the navigation guidance request.

In step 605, the method of operation of the user terminal device may calculate the direction of movement of the user.

That is, the operation method of the user terminal device can calculate the movement direction using the movement path, the current position of the illumination device, and the current position of the user terminal device.

The method of operation of the user terminal device in step 606 may provide haptic feedback or auditory feedback depending on the direction of movement.

That is, the operation method of the user terminal can provide the haptic feedback using the front and rear portions according to the calculated movement direction or provide the auditory feedback through the speaker.

7 is a flowchart illustrating a method of operating a navigation system according to an embodiment of the present invention.

Referring to FIG. 7, the method of operation of the navigation system may determine a destination in step 701. FIG.

That is, the user terminal can determine the destination based on the user's input.

In step 702, the method of operation of the route guidance system may transmit the destination to the map server.

That is, the user terminal device can transmit the destination setting data to the map server.

In operation 703, the operation method of the route guidance system may transmit the movement route to the user after calculating the movement route, and may transmit the user identification code to the illumination device included in the movement route.

That is, the map server may determine a movement route for moving to a destination determined by the user, and may transmit an identification code of the user terminal device that has transmitted the destination setting data to a plurality of illumination devices positioned on the determined movement route.

In step 704, the method of operation of the guidance system determines whether there is a lighting device in the vicinity.

For example, if there is another lighting device in the vicinity of the user terminal device, the method of operation of the navigation system may proceed to step 705. In addition, the operation method of the route guidance system may terminate the procedure when no other illumination device exists.

In step 705, the method of operation of the guidance system may send a navigation guidance request to the illumination device.

That is, the user terminal device may send a navigation guide request to request the current location of the lighting device to the lighting device.

In step 706, the method of operation of the route guidance system may identify the user terminal device.

That is, the lighting device can identify the user terminal device using the identification code of the user terminal device received from the map server.

Here, the lighting apparatus can determine whether the received identification code matches the identification code of the user terminal device.

In step 707, the method of operation of the guidance system may transmit the current location of the illumination device to the user terminal device.

The method of operation of the guidance system in step 708 may calculate the direction of movement by collating the current position of the lighting device with the current position of the user terminal device.

That is, the user terminal device may calculate the moving direction in consideration of the moving path, the current position of the lighting device, and the current position of the user terminal device, and may provide haptic feedback or auditory feedback according to the calculated moving direction.

8 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 8, in operation 801, a method of operating a user terminal device may determine a destination based on a user's input.

That is, the operation method of the user terminal device can set the destination and determine the movement route based on the set destination.

The method of operation of the user terminal device in step 802 may determine whether other lighting devices are present around the lighting device.

That is, if there is another illumination device in the vicinity, the operation method of the user terminal device proceeds to step 803, and if not exists, the corresponding procedure may be terminated.

In operation 803, the operation method of the user terminal may determine whether to search for another illumination included in the movement path.

That is, if there is another lighting apparatus included in the movement path, the operation method of the user terminal apparatus proceeds to step 804, and if not existed, terminates the corresponding procedure.

In operation 804, the operation method of the user terminal device may transmit the movement path of the user identification code user to another lighting device.

That is, the operation method of the user terminal device may transmit the user identification code to another illuminating device corresponding to the next illuminating device on the moving route.

Here, the operation method of the user terminal device can minimize the information exposure of the user terminal device by sequentially transmitting the user identification code.

9 is a flowchart illustrating a method of operating a navigation system according to an embodiment of the present invention.

Referring to FIG. 9, in operation 901, a method of operating the guidance system can recognize a lighting apparatus.

That is, the user terminal device can recognize the lighting device holding the identification code of the user terminal device.

Here, the illumination device may be located on a movement path on which the user moves to the destination.

The method of operation of the navigation system in step 902 may send a navigation guidance request.

That is, the user terminal device may send a navigation guide request to request the current location of the lighting device to the recognized lighting device.

In operation 903, the operation method of the guidance system determines whether the illumination device identifies the user terminal device.

That is, after the illumination device receives the navigation guidance request, it can determine whether or not the identification code of the user terminal device requesting the navigation guidance has been received from the map server.

Here, the operation method of the guidance system may proceed to step 904 if the user terminal device is identified. On the other hand, the operation method of the guidance system can end the procedure if the user terminal device is not identified.

The method of operation of the guidance system in step 904 may transmit the current location while changing the color of the illumination device.

That is, the lighting device can display the delivery status through color change while conveying the current position to the user terminal device.

10 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.

Specifically, FIG. 10 illustrates the operation of adjusting the haptic feedback strength according to the change of the direction of movement of the user after the user terminal device provides haptic feedback in relation to the direction of movement.

Referring to FIG. 10, in operation 1001, an operation method of a user terminal device may generate haptic feedback in a moving direction.

That is, the method of operation of the user terminal device may provide haptic feedback using the front or back of the user terminal device in relation to the movement direction.

In step 1002, the method of operation of the user terminal device may recognize a movement direction error.

That is, the operation method of the user terminal device recognizes the moving direction of the user based on the satellite navigation system and recognizes the inconsistency of the direction of movement indicated by the recognized moving direction and the haptic feedback, thereby recognizing the moving direction error .

The method of operation of the user terminal device in step 1003 may increase the haptic feedback strength in the direction of movement.

That is, the operation method of the user terminal device can increase the strength of the haptic feedback in the normal movement direction to correct the movement direction error.

In step 1004, the method of operation of the user terminal may determine whether the moving direction is corrected.

That is, the operation method of the user terminal apparatus can determine whether the moving direction is corrected by using a method of recognizing the moving direction error.

In addition, the operation method of the user terminal ends the procedure when the moving direction is corrected.

On the other hand, if the moving direction is not corrected, the operation method of the user terminal proceeds to step 1005. The method of operation of the user terminal in step 1005 may further increase the haptic feedback strength in the direction of movement.

11 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.

Specifically, FIG. 11 illustrates the operation of adjusting the strength of haptic feedback and auditory feedback depending on whether the user terminal is in a public place.

Referring to FIG. 11, in operation 1101, a method of operating a user terminal device may set a destination based on a user's input.

In operation 1102, the method of operation of the user terminal device may transmit the destination to the map server.

That is, the operation method of the user terminal apparatus can transmit the determined destination based on the user's input to the map server, and receive the travel route determined by the map server.

In step 1103, the method of operating the user terminal device may determine whether the destination corresponds to a public place.

Here, if the destination corresponds to a public place, the operation proceeds to step 1104, and if the destination does not correspond to a public place, the operation procedure may be terminated.

In step 1104, the method of operation of the user terminal device may schedule settings to increase the strength of the haptic feedback at the destination and lower the auditory feedback.

That is, the operation method of the user terminal device may set the strength of the feedback in advance in case the user terminal device is located in a public place.

In operation 1105, the method of operating the user terminal device may determine whether a public place exists on the movement route.

Here, if there is a public place on the movement route, the operation method of the user terminal proceeds to step 1106, and if the public place does not exist on the movement route, the corresponding procedure may be terminated.

In operation 1104, the method of operating the user terminal device may increase the intensity of the haptic feedback at a location corresponding to a public place and reserve a setting for lowering auditory feedback.

That is, the operation method of the user terminal device may set the strength of the feedback in advance in case the user terminal device is located in a public place.

12 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.

Specifically, FIG. 12 illustrates an operation of determining whether to transmit auditory feedback according to whether a user terminal wears an earphone of a user.

Referring to FIG. 12, in operation 1201, a method of operating a user terminal may set a destination.

That is, the operation method of the user terminal can set the destination based on the user input.

In step 1202, the method of operating the user terminal may determine whether the earphone is worn.

Here, if the user wears the earphone, the operation proceeds to step 1203, and if the user does not wear the earphone, the user terminal can end the procedure.

In step 1203, the method of operation of the user terminal device may transmit audible feedback through the earphone.

For example, auditory feedback may include speech or alarm sound signals.

The present invention can provide auditory feedback with an earphone when the user wears an earphone.

13 is a flowchart illustrating a method of operating a user terminal according to an exemplary embodiment of the present invention.

13 illustrates an operation in which the method of operation of the user terminal device determines front haptic feedback or back haptic feedback.

Referring to FIG. 13, the method of operation of the user terminal determines whether there is another lighting device closer to the lighting device in step 1301.

In one example, the method of operation of the user terminal may terminate the procedure to perform a procedure for recognizing another illuminator if there is another illuminator that is closer to the recognized illuminator.

On the other hand, if the operation method of the user terminal device does not include any other lighting device that is closer to the recognized lighting device, the process may proceed to step 1302.

In operation 1302, the method of operation of the user terminal may determine whether or not the touch input is detected on the front face.

That is, when the touch input is detected at the front portion of the user terminal, the operation proceeds to Step 1303, and if the touch input is not detected, the operation proceeds to Step 1304.

The method of operation of the user terminal device in step 1303 may provide front feedback based on the lighting device.

That is, the operation method of the user terminal device may receive the current position of the illumination device and provide front haptic feedback according to the movement direction in the movement path.

The method of operation of the user terminal device in step 1304 may provide back-feedback based on the lighting device.

That is, the operation method of the user terminal device may receive the current position of the illumination device and provide the rear haptic feedback according to the movement direction in the movement path.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: path guide system 110: user terminal device
120: Map server 130: Lighting device
200: user terminal device 210: movement route determining unit
220: illuminating device recognizing unit 230: navigation unit
240: moving direction calculating section 250: feedback providing section

Claims (20)

A movement path determination unit for determining a movement path based on a destination;
A lighting device recognition unit for recognizing a lighting device adjacent to the current position of the user terminal device among the lighting devices located on the determined traveling path;
A navigation unit for transmitting a navigation guidance request to the recognized lighting device and receiving the current position of the lighting device from the lighting device in response to the transmitted navigation guidance request;
A movement direction calculation unit for calculating a movement direction using the determined movement path, the current position of the illumination device, and the current position of the user terminal device; And
And a feedback providing unit for providing at least one of haptic feedback or auditory feedback according to the calculated movement direction
User terminal.
The method according to claim 1,
The feedback providing unit may provide the haptic feedback through the front and rear portions of the user terminal
User terminal.
3. The method of claim 2,
Wherein the feedback providing unit provides front haptic feedback when the touch input on the front face is sensed and controls the strength of the provided front haptic feedback with respect to the calculated movement direction according to a drag input change on the front face
User terminal.
The method of claim 3,
Wherein the feedback providing unit increases the strength of the front haptic feedback to the next position on the moving direction based on the current position of the drag input when the drag input on the front face is moved along the moving direction
User terminal.
The method of claim 3,
The feedback providing unit may provide the front haptic feedback in consideration of the moving speed and the stride of the user
User terminal.
6. The method of claim 5,
The feedback providing unit,
When the moving speed of the user is smaller than a predetermined moving speed value and the stride is smaller than a predetermined stride size value, the intensity of the front haptic feedback is continuously increased by a first magnitude,
The intensity of the front haptic feedback is continuously increased by a second magnitude when the moving speed of the user is smaller than the predetermined moving speed value and the stride is greater than the predetermined stride size value
User terminal.
The method according to claim 6,
The feedback providing unit,
Increasing the intensity of the frontal haptic feedback by the first magnitude when the moving speed of the user is greater than the predetermined moving speed value and the stride is smaller than the predetermined stride size value,
When the moving speed of the user is larger than the predetermined moving speed value and the stride is greater than the predetermined stride size value, the intensity of the front haptic feedback is continuously increased by the deviation of the second magnitude
User terminal.
3. The method of claim 2,
The feedback providing unit,
Dividing the rear portion into a plurality of portions when the rear haptic feedback is provided through the rear portion in accordance with the calculated movement direction and dividing the rear portion into a plurality of portions by dividing the rear haptic feedback from a start portion of the divided portions to an end portion based on the movement direction To increase the strength of feedback
User terminal.
9. The method of claim 8,
The starting portion corresponding to the current location of the user terminal device,
The ending part is determined based on the destination
User terminal.
9. The method of claim 8,
Wherein the feedback providing unit increases the strength of the rear haptic feedback at the front portion of the starting portion and increases the rear haptic feedback strength of the left portion of the front portion when the moving direction is the left rotation direction
User terminal.
9. The method of claim 8,
Wherein the feedback providing unit is configured to further increase the strength of the rear haptic feedback when the position of the user terminal is adjacent to the destination
User terminal.
The method according to claim 1,
Wherein the feedback providing unit increases the strength of the haptic feedback and decreases the strength of the auditory feedback when the noise level at the current position of the user terminal is greater than the reference value
User terminal.
The method according to claim 1,
Wherein the feedback providing unit increases the strength of the haptic feedback and decreases the strength of the auditory feedback when the current position of the user terminal is determined to be a public place
User terminal.
The method according to claim 1,
Wherein the travel route determining unit determines the travel route based on the destination using the map server,
The map server transmits an identification code of the user terminal device to an illuminating device located on the determined route
User terminal.
Determining a movement path based on a destination in the movement path determination unit;
Recognizing, in the illumination device recognition section, an illumination device adjacent to the current position of the user terminal device among the illumination devices positioned on the determined movement path;
Transmitting, at the navigation unit, a navigation guidance request to the recognized lighting device;
Receiving, at the navigation unit, the current position of the lighting device from the lighting device in response to the transmitted navigation guidance request;
Calculating a moving direction in the moving direction calculating unit using the determined moving path, the current position of the lighting device, and the current position of the user terminal device; And
In feedback providing, providing at least one of haptic feedback or spoken feedback in accordance with the calculated direction of movement
A method of operating a user terminal device.
16. The method of claim 15,
Wherein providing at least one of haptic feedback or spoken feedback in accordance with the calculated direction of movement comprises:
And providing the haptic feedback through the front and back portions of the user terminal device
A method of operating a user terminal device.
17. The method of claim 16,
Wherein the providing of the haptic feedback through the front and back portions of the user terminal comprises:
Providing front haptic feedback when sensing touch input on the front face; And
Increasing the intensity of the frontal haptic feedback to the next position on the moving direction based on the current position of the drag input when the drag input on the front portion is moved along the moving direction
A method of operating a user terminal device.
Transmitting, at the user terminal, a destination to a map server;
Determining a travel route on the map server based on the destination and transmitting an identification code of the user terminal to the illuminating device located on the determined travel route;
Identifying, at the lighting device, the user terminal device adjacent to a current location of the lighting device based on the transmitted identification code;
Receiving, at the lighting device, a navigation guidance request from the identified user terminal device and communicating the current location of the lighting device to the identified user terminal device;
Calculating, in the user terminal device, a moving direction using the determined moving path, the current position of the lighting device, and the current position of the user terminal device; And
In the user terminal, providing at least one of haptic feedback or auditory feedback according to the calculated direction of movement
A method of operating a navigation system.
The method of claim 18,
In the user terminal, the step of providing at least one of haptic feedback or audio feedback according to the calculated movement direction may include:
And providing the haptic feedback through the front and back portions of the user terminal device
A method of operating a navigation system.
20. The method of claim 19,
Wherein the providing of the haptic feedback through the front and back portions of the user terminal comprises:
Providing front haptic feedback when sensing touch input on the front face; And
Increasing the intensity of the frontal haptic feedback to the next position on the moving direction based on the current position of the drag input when the drag input on the front portion is moved along the moving direction
A method of operating a navigation system.

Images