KR102222102B1 - An augment reality navigation system and method of route guidance of an augment reality navigation system - Google Patents

Abstract

The route guidance method of the augmented reality navigation system captures an image of the front of the vehicle in real time through a camera, and based on the destination information set by the user, turn-by-turn for an intersection on the current driving route of the vehicle. -turn; TBT) After checking the information, extracting the first coordinate, which is the coordinate of the center point of the intersection, and selecting the selected structure, which is one of the structures located around the intersection, based on the direction in which the vehicle should turn at the intersection. Extract the second coordinate. Thereafter, the route guidance method of the augmented reality navigation system sets the position at which the rotation instruction image corresponding to the turn-by-turn information is displayed on the image based on the first coordinate and the second coordinate, and then superimposes the rotation instruction image and the image. To indicate.

Description

AR navigation system and route guidance method of augmented reality navigation system {AN AUGMENT REALITY NAVIGATION SYSTEM AND METHOD OF ROUTE GUIDANCE OF AN AUGMENT REALITY NAVIGATION SYSTEM}

The present invention relates to a navigation system, and more particularly, to an augmented reality navigation system that displays turn-by-turn information by applying augmented reality technology.

In general, a navigation system maps its own location, measured using a global positioning system (hereinafter referred to as "GPS"), to map data, and receives a stopover and/or a destination from the user and reaches the destination. It is a system that searches for a route of, maps and displays the searched route on the map data, and performs directions along the route.

In recent years, beyond the general navigation system that displays an electronic map on the screen and provides a route through it, an augmented reality (augmented reality) that provides an image of the current road photographed with a camera on the screen and displays the route to the destination on the screen. Augmented Reality; AR) Research and development of navigation systems are actively underway.

The augmented reality navigation system displays turn-by-turn (TBT) information in the form of arrows in order to inform the left/right direction and distance of the user or the vehicle at an intersection, a turn section, and the like. That is, the augmented reality navigation system displays the turn-by-turn information in order to clearly recognize an instantaneous change of direction without impairing the sense of reality on a screen providing a complex actual road image.

In order to implement the conventional method of displaying the turn-by-turn information in the form of an arrow, an image database of all buildings located around the road must be constructed, and it is implemented using a computer vision technology based on the database. For example, a database in which all building images visible from the driver's driving direction perspective are databased is basically required. In addition, the navigation system recognizes the image of the building from the information stored in the database using computer vision technology for the front image that enters the camera while the vehicle is running, and then creates a curved arrow image corresponding to the turn-by-turn information. It is displayed on the road displayed on the screen. However, the method of displaying the turn-by-turn information requires a large amount of cost because the images of all buildings in the area providing the navigation service must be converted into a database, and the arrow image corresponding to the turn-by-turn information has passed the rotation section. There is a problem in that it is difficult for the driver to intuitively recognize the rotation or driving route by being displayed as

An object of the present invention is to provide a method for providing a route guidance of an augmented reality navigation system in which a driver can intuitively and clearly recognize turn-by-turn information.

Another object of the present invention is to provide an augmented reality navigation system in which a driver can intuitively and clearly recognize turn-by-turn information.

However, the object of the present invention is not limited to the above-described objects, and may be variously extended without departing from the spirit and scope of the present invention.

In order to achieve an object of the present invention, a route guidance method of an augmented reality navigation system according to embodiments of the present invention captures an image in front of a vehicle through a camera in real time, and a vehicle based on destination information set by a user. After checking turn-by-turn (TBT) information about the intersection on the current driving route of the intersection, the first coordinate that is the coordinate of the center point of the intersection is extracted, and the vehicle should rotate at the intersection. The second coordinate may be extracted by selecting a selection structure, which is one of structures located around the intersection, based on the direction. Thereafter, the route guidance method of the augmented reality navigation system sets a position at which a rotation instruction image corresponding to the turn-by-turn information is displayed on the image based on the first coordinate and the second coordinate, and then the rotation The instruction image and the image may be superimposed and displayed.

According to an embodiment, the existing structures correspond to the outermost buildings disposed adjacent to the intersection, and the selection structure is disposed at a position in which the vehicle rotates in the direction in which the vehicle should rotate at the intersection, and the first coordinate It may correspond to the building that is located closest to.

According to an embodiment, the second coordinate may correspond to a coordinate of a corner closest to the first coordinate among corners of the selection structure.

According to an embodiment, the method of setting the position at which the rotation instruction image is displayed on the image generates a virtual first extension line extending in a direction parallel to a direction in which the vehicle is currently traveling from the second coordinate, After generating a virtual second extension line extending in a direction parallel to the direction in which the vehicle will travel after passing through the rotation section of the intersection, the intersection point of the first extension line and the second extension line is calculated from the first coordinate. May be set as the end point of the rotation instruction image.

According to an embodiment, the rotation instruction image may be in the form of a bent arrow pointing in a direction in which the vehicle should rotate.

According to an embodiment, the method of setting the position at which the rotation instruction image is displayed on the image generates a virtual first extension line extending a straight line connecting the third coordinate and the second coordinate, which is the current position of the vehicle. And, after generating a virtual second extension line extending in a direction parallel to the direction in which the vehicle will proceed after passing through the rotation section of the intersection, the intersection of the first extension line and the second extension line is calculated from the first coordinate. The intersection point may be set as the end point of the rotation instruction image.

According to an embodiment, the third coordinate may correspond to a position of a driver's seat of the vehicle.

According to an embodiment, the rotation instruction image may be in the form of a bent arrow pointing in a direction in which the vehicle should rotate.

According to an embodiment, a method of setting a position at which the rotation instruction image is displayed on the image may include setting an end point of the rotation instruction image as the second coordinate.

According to an embodiment, the rotation instruction image may be in the form of a bent arrow pointing in a direction in which the vehicle should rotate.

In order to achieve one object of the present invention, the augmented reality navigation system according to the embodiments of the present invention includes a photographing unit that captures an image of the front of a vehicle in real time through a camera, and each of preset buildings located around an intersection. The database including the coordinates of the corner, the first coordinate that is the coordinate of the center point of the intersection on the optimal driving route of the vehicle based on the destination information set by the user, and the turn-by-turn (TBT) for the intersection. ) A coordinate extracting unit that checks information and extracts a second coordinate that is a coordinate of a selected building that is one of the preset buildings from the database based on the turn-by-turn information, the first coordinate and the second coordinate A setting unit configured to set a position at which the rotation instruction image corresponding to the turn-by-turn information is displayed on the image, and an augmented reality unit that overlaps the rotation instruction image and displays the rotation instruction image on the image.

According to an embodiment, the preset buildings correspond to the outermost buildings disposed adjacent to the intersection, and the selected building is disposed at a position in which the vehicle rotates in the direction in which the vehicle should rotate at the intersection. It corresponds to the building located closest to the coordinates, and the second coordinate may correspond to the coordinates of the corner of the selected building of the selected building.

According to an embodiment, the coordinate extracting unit may extract the corner coordinates of the selected building as the second coordinates based on the moving direction of the vehicle and the direction to be rotated.

According to an embodiment, the setting unit is a virtual first extension line extending from the second coordinate in a direction in which the vehicle is currently moving, and from the first coordinate in a direction in which the vehicle is going after passing through the rotation section of the intersection. The position of the end point of the rotation instruction image may be set at the intersection of the virtual second extension line.

According to an embodiment, the setting unit passes the vehicle through the rotation section of the intersection from a virtual first extension line extending a straight line connecting the third coordinate and the second coordinate, which is the current position of the vehicle, and the first coordinate. The position of the end point of the rotation instruction image may be set at the intersection of the virtual second extension line extending in the direction to be processed later.

According to an embodiment, the third coordinate may correspond to a position of a driver's seat of the vehicle.

According to an embodiment, the setting unit may set the position of the end point of the rotation instruction image as the second coordinate.

The route guidance method of the augmented reality navigation system and the augmented reality navigation system according to embodiments of the present invention includes corner coordinates (ie, second coordinates) of buildings disposed around an intersection and coordinates of a center point of the intersection (ie, first coordinates). ), the position of the direction indication image representing the TBT information (ie, the position of the end point) can be adjusted in real time. That is, in a rotation section or intersection section in which the driver needs to turn around, the end point of the direction indication image is located on a straight line extending from the corner coordinates of the selected building, so that the driver is intuitive to the position, direction, or driving path to be rotated. Can be recognized as. In addition, by displaying TBT information using only the coordinate information of the corners of the pre-set building without using the existing entire building image database and computer vision technology, the cost for building image DB production is reduced, and the amount of navigation computation is reduced. Can be.

However, the effects of the present invention are not limited to the above-described effects, and may be variously extended without departing from the spirit and scope of the present invention.

1 is a flow chart illustrating a method of providing route guidance in an augmented reality navigation system according to embodiments of the present invention.
FIG. 2A is a diagram illustrating an example of a screen displayed by an augmented reality navigation system by the route guidance providing method of FIG. 1.
FIG. 2B is a diagram illustrating another example of a screen displayed by the augmented reality navigation system by the route guidance providing method of FIG. 1.
FIG. 3 is a diagram illustrating an example of extracting first coordinates and second coordinates included in the method of providing a route guide of the augmented reality navigation system of FIG. 1.
4A and 4B are diagrams illustrating an example of a method of outputting a rotation instruction image included in the method of providing a route guide of the augmented reality navigation system of FIG. 1.
5A and 5B are diagrams illustrating another example of a method of outputting a rotation instruction image included in the method for providing path guidance of the augmented reality navigation system of FIG. 1.
6A and 6B are diagrams illustrating still another example of a method of outputting a rotation instruction image included in the method for providing path guidance of the augmented reality navigation system of FIG. 1.
7 is a block diagram illustrating an augmented reality navigation system according to embodiments of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

FIG. 1 is a flow chart illustrating a method of providing a route guidance of an augmented reality navigation system according to embodiments of the present invention, and FIG. 2A is a flowchart illustrating an example of a screen displayed by an augmented reality navigation system by the route guidance providing method of FIG. 1 2B is a diagram illustrating another example of a screen displayed by the augmented reality navigation system by the route guidance providing method of FIG. 1. Referring to FIGS. 1 to 2B, Augmented Reality (AR, hereinafter, Indicated by AR.) In the navigation system's route guidance provision method, an image of the front of the vehicle is captured (S100), and turn-by-turn (TBT, hereinafter referred to as TBT) for an intersection on the vehicle's driving route. .) Check the information (S200), extract the first coordinate that is the coordinate of the center point of the intersection (S300), and select one of the structures located around the intersection to extract the second coordinate (S400). have. The route guide providing method sets a position at which a rotation instruction image corresponding to the TBT information is displayed on an image based on the first coordinate and the second coordinate (S500), and then overlaps the rotation instruction image and the image. Thus, it can be displayed on the screen (S600).

An image of the front of the vehicle may be photographed (S100) through a camera provided in the AR navigation system. The AR navigation system may capture an image of the front of the vehicle in real time. The image may be output in real time through an output unit provided in the AR navigation system and outputting an image. For example, as shown in FIGS. 2A and 2B, a front image currently being captured is displayed in one area or the whole (here, the left area) of the output unit screen. That is, in FIGS. 2A and 2B, the left area of the screen becomes an AR screen.

The AR navigation system may check TBT information for an intersection or turn section on the current driving route of the vehicle based on destination information set by the user (or driver) (S200). That is, the AR navigation system checks information on whether the vehicle should turn left, turn right, or go straight at an oncoming intersection. For example, a driving route set based on the destination information may be compared with a route on which the vehicle is in progress, and the turn-by-turn information at the intersection may be checked based on this. When the vehicle needs to turn (eg, turn left or turn right) at the intersection, the AR navigation system may output the TBT information. Hereinafter, a method of setting and outputting the rotation instruction image to display the TBT information on the screen using the rotation instruction image will be described.

FIG. 2A shows the TBT information displayed on the AR screen when there is no building in a pinched position in the direction in which the vehicle should proceed in an intersection or turn section (in FIG. 2A, a right-turning direction). That is, the direction indication image DI1 indicating TBT information on the path to which the vehicle should enter is proceeded to the right end of the AR screen. In this case, since there are no obstacles such as buildings on the path in which the vehicle should rotate on the AR screen, as shown in FIG. 2A, the direction indication image DI1 can be displayed in the conventional TBT information provision method.

However, when there is a large structure such as a building at a position where the vehicle is rotated, the direction indication image DI1 as shown in FIG. 2A has a disadvantage in that it cannot intuitively show the driver the route information to be rotated. Accordingly, when a building or the like is located in the rotating section, as illustrated in FIG. 2B, a direction indication image including TBT information may be displayed on the AR screen according to embodiments of the present invention.

The route guidance method of the AR navigation system may extract a first coordinate that is a coordinate of a center point of the intersection or rotation section (S300). In an embodiment, the first coordinates may be preset coordinates for a corresponding intersection. For example, the AR navigation system may store coordinates corresponding to intersections in a preset area. Coordinates at a corresponding intersection may be extracted based on a location measured from a GPS unit provided in the AR navigation system or a captured image. In another embodiment, coordinates for the intersection may be extracted in real time by the operation of the GPS unit. However, the first coordinate is not limited to the center point of the intersection, or the meaning of the center point of the intersection is not limited to mean the center of the intersection. For example, the first coordinates may correspond to two-dimensional image coordinates of traffic facilities, for example, traffic lights, signs, etc. arranged in the traveling direction of the vehicle.

The route guidance method of the AR navigation system selects a selected structure (or selected building), which is one of preset structures located around the intersection, based on the direction in which the vehicle should rotate at the intersection, and second coordinates Can be extracted (S400). In one embodiment, the preset structures may correspond to outermost buildings arranged around the intersection. That is, the structures are disposed around an intersection, and may correspond to buildings disposed closest to the intersection (and road). However, the structures are not limited to buildings. The selection structure may correspond to one of the buildings disposed at a position in which the vehicle rotates in a direction in which the vehicle should rotate at the intersection, which is disposed at a position closest to the first coordinate. For example, as shown in FIG. 2B, a building 10 disposed in a section in which a vehicle should turn right may be selected as the selection structure. This will be described in detail with reference to FIG. 3 below.

In an embodiment, the route guidance method of the AR navigation system may select the selection structure based on a driving direction of the vehicle and a direction in which the vehicle should rotate. Here, the second coordinate may correspond to a coordinate of a corner closest to the first coordinate among corners of the selection structure. The coordinates of the corners at the intersection are databaseized. For example, in the case of a crossroad intersection, four buildings arranged at each corner may be stored as candidates for the selected structure, and corner coordinates of each of the buildings may be stored in a database. The AR navigation system selects, as the selection structure, a building disposed at a position in which the vehicle rotates in the direction in which the vehicle should rotate at the intersection, based on the position of the vehicle currently in progress and the direction in which the vehicle should rotate. The corner coordinates of the second coordinates may be extracted from the database. That is, the existing building image database is not required, and the process of displaying TBT information using the building image data and computer vision technology is omitted. Thus, complex algorithms and hardware for implementing computer vision technology may be omitted or simplified.

Thereafter, in the route guidance method of the AR navigation system, the rotation instruction image corresponding to the TBT information (for example, DI2 in FIG. 2B) is displayed on the image based on the first coordinate and the second coordinate. Can be set (S500). The rotation instruction image 120 may have a shape of an arrow curved in the rotation direction to express the TBT information. The route guidance method of the AR navigation system may set the position of the end point of the rotation instruction image 120 based on a simple operation.

In one embodiment, the position of the end point of the rotation instruction image 120 having the arrow shape is a virtual first extension line extending from the second coordinate in a direction parallel to the direction D1 in which the vehicle is currently moving And a second imaginary extension line extending in a direction parallel to the direction D2 in which the vehicle is to proceed after passing through the rotation section of the intersection may be set from the first coordinate. In another embodiment, the position of the end point may be set as an intersection of the second extension line and a virtual extension line extending a straight line connecting the third coordinate and the second coordinate, which is the current position of the vehicle. In another embodiment, the position of the end point may be set to the second coordinate. That is, the end point may be calculated through a simple calculation algorithm based on the coordinates. The position of the end point may be calculated and changed in real time by the movement of the vehicle.

Thereafter, in the route guidance method of the AR navigation system, the rotation instruction image and the image may be superimposed and displayed on the screen (S600). As shown in FIG. 2B, the AR screen of the output unit included in the AR navigation system may output a screen in which a direction indication image DI2 displaying TBT information is superimposed on an image captured in real time. The direction indication image DI2 may be superimposed on the image and output by the methods described above. FIG. 2B shows an AR screen and a direction indication image DI2 when buildings 10 are located at a position where the vehicle is rotated in a rotation section of the vehicle. That is, the end point of the direction indication image DI2 may be located at the edge of the building 10 (that is, the selection structure 10) differently from FIG. 2A. Accordingly, the driver who sees the AR screen can intuitively recognize that it is necessary to rotate around the selected building 10.

As described above, the route guidance method of the AR navigation system according to embodiments of the present invention includes corner coordinates (ie, second coordinates) of buildings arranged around an intersection and coordinates of a center point of the intersection (ie, first coordinates). Based on the operation of and, the position of the direction indication image representing the TBT information (ie, the position of the end point) can be adjusted in real time. That is, in a rotation section or intersection section in which a specific building 10 must be turned around, the end point of the direction indication image is located on a straight line extending from the corner coordinates of the selected building. Can be intuitively recognized. In addition, by displaying the TBT information using only the coordinate information of the corner of the pre-set building without the use of the existing entire building image database and complex computer vision technology, the cost for building image DB production is reduced, and the computational amount of navigation is reduced. Can be reduced.

FIG. 3 is a diagram illustrating an example of extracting first coordinates and second coordinates included in the method of providing a route guide of the augmented reality navigation system of FIG. 1.

Referring to FIG. 3, in the method of providing a route guidance of the AR navigation system, coordinates of an intersection in front of a vehicle and corner coordinates of a structure selected from among structures located at the intersection may be extracted from a storage unit or a database.

In one embodiment, the first coordinate P1 may correspond to the coordinate of the center point of the intersection. Coordinates at a corresponding intersection may be extracted based on a location measured from a GPS unit provided in the navigation system or a captured image. In another embodiment, coordinates for the intersection may be extracted in real time by the operation of the GPS unit. However, the first coordinate is not limited to the center point of the intersection, or the meaning of the center point of the intersection is not limited to mean the center of the intersection. The AR navigation system may extract coordinate information (ie, first coordinate P1) for an intersection located in front of the vehicle. The coordinate information may correspond to two-dimensional coordinates excluding height information.

In addition, in one embodiment, corner coordinates (AE, BE, CE, DE) of each of the buildings (A, B, C, D) arranged around the intersection are stored in the storage unit of the database or the AR navigation system. There may be. That is, the AR navigation system does not need to have a database for all buildings around an intersection or road to display TBT information, and does not apply computer vision technology using the database, and the corner coordinates (AE, BE , CE, DE) can provide intuitive TBT information to the driver. For example, the AR navigation system may have coordinate information for only the outermost buildings (A, B, C, D) arranged adjacent to the intersection among structures arranged around the intersection. As shown in FIG. 3, each of the corner coordinates AE, BE, CE, and DE may be set to correspond to the coordinates of the corner closest to the first coordinate P1 among the corner coordinates of the corresponding buildings. .

In an embodiment, the route guidance method of the AR navigation system may select a selection structure and corner coordinates of the selected structure based on a direction in which a vehicle in progress should rotate at the intersection. The selection structure may correspond to a building located closest to the first coordinate while being disposed at a position where the vehicle rotates in a direction in which the vehicle should rotate at the intersection. For example, when the AR navigation system displays TBT information that causes the vehicle 100 in the first direction D1 to turn right at the intersection, the selected structure corresponds to the building A, and the second coordinate May be determined by the corner coordinates (AE) of the building A. Alternatively, when the AR navigation system displays TBT information that causes the vehicle 100 in the first direction D1 to turn left at the intersection, the selected structure corresponds to the building D, and the second coordinate is the It can be determined by the corner coordinates (DE) of the building D.

4A and 4B are diagrams illustrating an example of a method of outputting a rotation instruction image included in the method of providing a route guide of the augmented reality navigation system of FIG. 1.

4A and 4B, the method of setting the position at which the rotation instruction image 120 is displayed on the image extends from the second coordinate P2 in a direction parallel to the direction D1 in which the vehicle 100 is currently proceeding. A virtual second extension line L1 is generated (S510), and a second virtual extension extending from the first coordinate P1 in a direction parallel to the direction D2 to proceed after passing through the rotation section of the intersection. After generating the extension line L2 (S520), the intersection point CP of the first extension line L1 and the second extension line L2 is calculated, and the position of the end point of the rotation instruction image 120 is used as the intersection point CP. It can be set (S530).

When the first coordinate P1 and the second coordinate P2 are set, an operation for setting the position of the end point of the rotation instruction image 120 may be performed. The 2 coordinate P2 may correspond to the corner coordinate AE closest to the first coordinate P1 among the corners of the selected building (ie, building A).

First, a virtual first extension line L1 extending in a direction parallel to the first direction D1 in which the vehicle 100 is currently proceeding from the second coordinate P2 may be generated (S510). For example, a first equation for the first extension line L1 may be set based on the two-dimensional coordinates for the intersections and buildings A, B, C, and D and the first direction D1. In addition, a virtual second extension line L2 extending in a direction parallel to the second direction D2 to be proceeded after the vehicle 100 passes through the rotation section of the intersection may be generated from the first coordinate P1 (S520). . For example, a second equation for the second extension line L2 may be set based on the two-dimensional coordinates and the second direction D1.

Thereafter, the intersection point CP of the first extension line L1 and the second extension line L2 may be set as an end point of the rotation instruction image 120 (S530). For example, the coordinates of the intersection point CP may be calculated through the first and second equations. The coordinates of the end point of the rotation indication image 120 representing the TBT information may correspond to the coordinates of the intersection point CP. Accordingly, the TBT information can be shown to the driver based on the location of the corner of the selected building (ie, building A). In one embodiment, the rotation instruction image 120 may correspond to a shape of a curved arrow pointing in a direction in which the vehicle 100 should rotate.

5A and 5B are diagrams illustrating another example of a method of outputting a rotation instruction image included in the method for providing path guidance of the augmented reality navigation system of FIG. 1.

5A and 5B, a method of setting a position at which the rotation instruction image 120 is displayed on the image is to connect the third coordinate P3 and the second coordinate P2, which are the current positions of the vehicle 100. A virtual first extension line L1' extending a straight line is generated (S540), and a virtual second extension line extending in a direction parallel to the direction D2 in which the vehicle will proceed after passing through the rotation section of the intersection from 1 coordinate After the (L2) is generated (S550), the intersection point CP of the first extension line L1' and the second extension line L2 may be set as an end point of the rotation instruction image 120 (S560).

When the second coordinate P2 and the third coordinate P3 are set, an operation for setting the position of the end point of the rotation instruction image 120 may be performed. The 2 coordinate P2 may correspond to the corner coordinate AE closest to the first coordinate P1 among the corners of the selected building (ie, building A).

First, a virtual first extension line L1' extending a straight line connecting the third coordinate P3 and the second coordinate P2, which is the current position of the first vehicle 100, may be generated (S540). The third coordinate P3 may be extracted to correspond to the position of the driver's seat of the vehicle 100. For example, the coordinates of the driver's seat of the vehicle 100 currently in progress may be calculated through GPS. A first equation indicating the first extension line L1' may be set based on the second coordinate P2 and the third coordinate P3. In addition, a virtual second extension line L2 extending in a direction parallel to the second direction D2 to be proceeded after the vehicle 100 passes through the rotation section of the intersection may be generated from the first coordinate P1 (S520). . For example, a second equation for the second extension line L2 may be set based on the two-dimensional coordinates and the second direction D2.

Thereafter, the intersection point CP of the first extension line L1 ′ and the second extension line L2 may be set as an end point of the rotation instruction image 120 (S560). For example, the coordinates of the intersection point CP may be calculated through the first and second equations. The coordinates of the end point of the rotation indication image 120 representing the TBT information may correspond to the coordinates of the intersection point CP. Accordingly, the TBT information may be shown to the driver based on the location of the edge of the selected building (ie, building A) and the current location of the vehicle 100. This embodiment can provide more accurate TBT information to the driver than the embodiment of displaying TBT information according to FIGS. 4A and 4B.

6A and 6B are diagrams illustrating still another example of a method of outputting a rotation instruction image included in the method for providing path guidance of the augmented reality navigation system of FIG. 1.

6A and 6B, as a method of setting a position at which the rotation instruction image 120 is displayed on the image, an end point of the rotation instruction image 120 may be set as a second coordinate P2 (S570). .

In one embodiment, when the first coordinate P1 and the second coordinate P2 are calculated, the AR navigation system sets the second coordinate P2 as the end point of the rotation indication image 120 and is an arrow curved in the rotation direction. It is possible to generate the shape of the rotation instruction image 120. The second coordinate P2 may correspond to the corner coordinate AE closest to the first coordinate P1 among the corners of the selected building (ie, building A). Accordingly, in the present embodiment, it is possible to intuitively grasp TBT information from the driver's first-person view, and display the TBT information with a simpler operation and algorithm than the route guidance method of FIGS. 4A to 5B.

As described above, in the route guidance method of the AR navigation system according to the embodiments of the present invention, the coordinates of the corners of buildings arranged around the intersection (i.e., the second coordinate P2) and the coordinates of the center point of the intersection (i.e. 1 The position of the direction indication image representing TBT information (that is, the position of the end point) can be adjusted in real time based on the operation with the coordinate P1). Therefore, by displaying the TBT information using only the coordinate information of the corner of the pre-set building without the use of the existing entire building image database and computer vision technology, the cost for building image DB production is reduced, and the computational amount of navigation is reduced. Can be.

7 is a block diagram illustrating an augmented reality navigation system according to embodiments of the present invention.

Referring to FIG. 7, the AR navigation system 200 includes a GPS unit 210, an input unit 220, a photographing unit 230, a control unit 240, a database 250, an output unit 260, and a communication unit 270. ) And an augmented reality unit 270. In one embodiment, the control unit 240 may include a coordinate extracting unit 244 for extracting first and second coordinates, and a setting unit 244 for setting a position of a rotation instruction image displaying TBT information. .

The GPS unit 210 may acquire location data on the current location of the AR navigation system 200 by using radio waves received from a plurality of artificial satellites. The AR navigation system 200 may provide a route guidance service based on a current location based on the location data acquired by the GPS unit 210.

The input unit 220 may receive an input from a user and convert it into an electric signal. For example, the input unit 220 is an operation button or operation key (i.e., a physical button, etc.) provided outside of the AR navigation system 200, a microphone that receives a user's voice and a sound generated inside and outside the vehicle, as an output device. The display panel and the input device may include a touch screen in which a touch panel is integrally formed.

The photographing unit 230 may capture an image of the front of the vehicle in real time through the camera. A photographing unit 230 is provided to implement the augmented reality of the navigation system. However, the photographing unit 230 is not limited to photographing only the front of the vehicle, and may further photograph an image of at least one of both sides and the rear of the vehicle in real time.

The controller 240 may control various operations of the AR navigation system 200. For example, the control unit 240 may provide a route guidance service, a real-time traffic information providing service, a DMB service, an image playing service, a voice playing service, and the like to the user. 220, the output unit 260, the database 230, and the communication unit 270 may be controlled. The control unit 240 may include a coordinate extracting unit 242 for extracting first and second coordinates, and a setting unit 244 for setting a position of a rotation instruction image displaying TBT information.

The coordinate extracting unit 242 includes first coordinates, which are coordinates of the center point of an intersection on the optimal driving route of the vehicle, and turn-by-turn (TBT) information about the intersection based on destination information set by the user. And, based on the turn-by-turn information, a second coordinate, which is a coordinate of a selected building, which is one of the preset buildings, may be extracted from the database. In one embodiment, the selected building may correspond to a building located closest to the first coordinate while being disposed at a position in which the vehicle rotates in a direction in which the vehicle should rotate at the intersection. In addition, the second coordinates may correspond to coordinates of corners of the selected building of the selected building. In an embodiment, the coordinate extracting unit 242 may extract the corner coordinates of the selected building as the second coordinates based on the moving direction of the vehicle and the direction to be rotated. Since the method and operation of extracting the first coordinate and the second coordinate have been described in detail with reference to FIGS. 1 to 6B, overlapping contents thereof will be omitted.

The setting unit 244 may set a position at which the rotation instruction image corresponding to the turn-by-turn information is displayed on the image based on the first coordinate and the second coordinate. In an embodiment, as shown in FIG. 4B, the setting unit 244 includes a first virtual extension line extending from the second coordinate in a direction in which the vehicle is currently running, and the vehicle from the first coordinate. The position of the end point of the rotation instruction image may be set at the intersection of the virtual second extension line extending in the direction to proceed after passing through the rotation section of the intersection. For example, the setting unit 244 may calculate the intersection point based on equations for the first and second extension lines. In another embodiment, as shown in FIG. 5B, the setting unit 244 includes a virtual first extension line extending a straight line connecting the third coordinate and the second coordinate, which is the current position of the vehicle, and the first From the coordinates, the position of the end point of the rotation instruction image may be set at an intersection of a virtual second extension line extending in a direction in which the vehicle passes through the rotation section of the intersection. In this case, the third coordinate may correspond to the position of the driver's seat of the vehicle. In another embodiment, the setting unit 244 may set the position of the end point of the rotation instruction image as the second coordinate, as illustrated in FIG. 6B. However, since the method of setting the direction indication image indicating TBT information has been described in detail with reference to FIGS. 2 to 6B, overlapping contents thereof will be omitted.

The database 250 (or the storage unit) may store data required for the operation of the AR navigation system 200 and data generated by the operation. In an embodiment, the database 230 may include coordinates of corners of preset buildings located around an intersection. The preset buildings correspond to the outermost buildings arranged adjacent to the intersection, and the coordinates of the corners correspond to the coordinates of the corners closest to the coordinates of the central point of the intersection among the corners of each of the buildings. can do. For example, data necessary for the operation of the AR navigation system 200 may include an OS, a route search application, a map, etc., and data generated by the operation of the navigation system 200 may include searched route data, It may include driving route data, driving pattern data, and received images.

The output unit 260 may provide information generated by the AR navigation system 200 to a user in the form of an image or audio. For example, the output unit 260 may include the display panel for visually displaying various types of information, a speaker for audibly displaying various types of information, and the like. In addition, as described above, the display panel and the touch panel may be integrally formed as the touch screen.

The communication unit 270 may perform predetermined wired or wireless communication with other external devices.

The augmented reality unit 280 may superimpose the rotation instruction image on the image and display it on the output unit. The augmented reality unit may match an image input by the photographing unit 230 with the rotation instruction image, and the rotation instruction image may be superimposed on the image and displayed as the screen of FIG. 2.

As described above, the AR navigation system 200 according to embodiments of the present invention includes a coordinate extracting unit 242 for extracting the first and second coordinates that are the basis for the generation of the direction indication image, and the first And a setting unit 242 that sets the position of the end point of the direction indication image through an operation based on the second coordinates, so that the driver can intuitively recognize the rotation or the driving route. In addition, by displaying TBT information using only the coordinate information of the corners of the pre-set building without using the existing entire building image database and computer vision technology, the cost for building image DB production is reduced, and the amount of navigation computation is reduced. Can be.

The present invention can be applied to a navigation system that displays TBT information by reflecting augmented reality. Although the above has been described with reference to embodiments of the present invention, those skilled in the art will be able to variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can.

100: vehicle 120: direction indication image
200: AR navigation system 210: GPS unit
220: input unit 230: database
240: control unit 242: coordinate extraction unit
244: setting unit 260: output unit
270: communication department

Claims (17)

Capturing an image of the front of the vehicle in real time through a camera;
Checking turn-by-turn (TBT) information for an intersection on a current driving route of the vehicle based on destination information set by the user;
Extracting first coordinates that are coordinates of the center point of the intersection;
Extracting second coordinates by selecting a selection structure, which is one of structures located around the intersection, based on a direction in which the vehicle should rotate at the intersection;
Setting a position at which a rotation instruction image corresponding to the turn-by-turn information is displayed on the image based on the first coordinate and the second coordinate; And
Including the step of superimposing and displaying the rotation instruction image and the image,
The structures correspond to the outermost buildings disposed adjacent to the intersection,
The selection structure is arranged at a position that the vehicle rotates in a direction in which the vehicle is rotated at the intersection, and corresponds to a building located closest to the first coordinate. delete The method of claim 1, wherein the second coordinate corresponds to a coordinate of a corner closest to the first coordinate among corners of the selection structure. The method of claim 1, wherein the setting of a position at which the rotation instruction image is displayed on the image comprises:
Generating a first virtual extension line extending in a direction parallel to a direction in which the vehicle is currently traveling from the second coordinate;
Generating a second virtual extension line extending in a direction parallel to a direction in which the vehicle will proceed after passing through the rotation section of the intersection from the first coordinate; And
And calculating an intersection point of the first extension line and the second extension line and setting the intersection point as an end point of the rotation instruction image. The method of claim 4, wherein the rotation instruction image is in the form of a curved arrow pointing in a direction in which the vehicle should rotate. The method of claim 1, wherein the setting of a position at which the rotation instruction image is displayed on the image comprises:
Generating a first virtual extension line extending a straight line connecting the third coordinate and the second coordinate, which is a current position of the vehicle;
Generating a second virtual extension line extending in a direction parallel to a direction in which the vehicle will proceed after passing through the rotation section of the intersection from the first coordinate; And
And calculating an intersection point of the first extension line and the second extension line and setting the intersection point as an end point of the rotation instruction image. The method of claim 6, wherein the third coordinate corresponds to a position of a driver's seat of the vehicle. The method of claim 6, wherein the rotation instruction image is in the form of a curved arrow pointing in a direction in which the vehicle should rotate. The method of claim 1, wherein the setting of a position at which the rotation instruction image is displayed on the image comprises:
And setting an end point of the rotation instruction image as the second coordinate. The method of claim 9, wherein the rotation instruction image is in the form of a curved arrow pointing in a direction in which the vehicle should rotate. A photographing unit for capturing an image of the front of the vehicle in real time through a camera;
A database including coordinates of corners of preset buildings located around the intersection;
Based on the destination information set by the user, the first coordinate, which is the coordinate of the center point of the intersection on the driving route of the vehicle, and turn-by-turn (TBT) information for the intersection are checked, and the turn-by-turn A coordinate extraction unit for extracting a second coordinate, which is a coordinate of a selected building, which is one of the preset buildings, from the database based on information;
A setting unit configured to set a position at which a rotation instruction image corresponding to the turn-by-turn information is displayed on the image based on the first coordinate and the second coordinate; And
An augmented reality unit that overlaps the rotation instruction image on the image and displays it on an output unit,
The preset buildings correspond to the outermost buildings disposed adjacent to the intersection,
The selected building corresponds to a building located closest to the first coordinate while being disposed at a position that the vehicle rotates in the direction in which the vehicle should rotate at the intersection,
The second coordinate is an augmented reality navigation system, characterized in that corresponding to the coordinates of the corner of the selected building of the selected building. delete The augmented reality navigation system of claim 11, wherein the coordinate extracting unit extracts the corner coordinates of the selected building as the second coordinates based on the moving direction of the vehicle and the direction to be rotated. The method of claim 11, wherein the setting unit extends from the second coordinate to a virtual first extension line extending in a direction in which the vehicle is currently proceeding, and a direction in which the vehicle proceeds after passing through the rotation section of the intersection from the first coordinate. An augmented reality navigation system, characterized in that the position of the end point of the rotation instruction image is set at an intersection of the virtual second extension line. The method of claim 11, wherein the setting unit passes the vehicle through a rotation section of the intersection from a virtual first extension line extending a straight line connecting the third coordinate and the second coordinate, which is the current position of the vehicle, and the first coordinate. An augmented reality navigation system, characterized in that the position of an end point of the rotation instruction image is set at an intersection of a virtual second extension line extending in a direction to be followed. The augmented reality navigation system of claim 15, wherein the third coordinate corresponds to a position of a driver's seat of the vehicle. The augmented reality navigation system of claim 11, wherein the setting unit sets the position of the end point of the rotation instruction image as the second coordinate.

Images