KR101665599B1 - Augmented reality navigation apparatus for providing route guide service and method thereof - Google Patents

Abstract

The present invention relates to an augmented reality navigation device for a route guidance service and a method thereof, and more particularly to an augmented reality navigation device for a route guidance service, which comprises a photographing section for photographing an image in front of a vehicle, a lane recognition section for recognizing a lane of a road under vehicle travel, A route search unit for searching for a route, calculating a direction and a distance of links included in the route, and a route search unit for searching for a route from the current link to the next link And an augmented reality processing unit for overlaying the generated directional marker on the image so that the generated directional directional landmark is displayed on the road of the recognized lane and displaying it on the screen.

Description

Field of the Invention [0001] The present invention relates to an augmented reality navigation device for route guidance service,

[0001] The present invention relates to an augmented reality navigation apparatus and method for a route guidance service, and more particularly to an augmented reality navigation apparatus for a route guidance service for superimposing a direction guide mark including a direction of a route on an actual road image, An augmented reality navigation device and a method thereof.

Recently, the development of electronic devices has dramatically improved the functions and performance of automobiles. For example, various sensors are used to improve the engine performance to realize the optimum engine efficiency to realize the high-fuel-efficiency vehicle, and the safety of the vehicle is improved by electronically controlling the suspension device, the steering device, and the braking device .

Various functions for the convenience of the driver have been developed as the performance and safety functions of the vehicle have been developed, and in recent years, navigation devices for vehicles have been developed.

The navigation device displays the route and current position of the vehicle on the map, and performs the guidance.

However, in the conventional navigation, the direction in which the vehicle travels is displayed on one side of the screen on which the navigation map (electronic map) for guiding the route is displayed. Therefore, the driver has to judge the driving direction by comparing the actual road information with the navigation map. As a result, it has been difficult to judge the driving direction on the starting road or the ambiguous road.

To solve this problem, an augmented reality navigation system that maps navigation information to actual roads has been developed. However, the augmented reality navigation system has a drawback in that it requires mounting of a large and expensive apparatus such as a head-up display.

Korean Patent Laid-Open Publication No. 2013-0040361, entitled " Real-time Traffic Information Providing Method Based on Augmented Reality Navigation, Server, and Recording Medium &

An object of the present invention is to provide an augmented reality navigation device for a route guidance service capable of providing a route guidance service implemented as an augmented reality using a portable device (e.g., smart phone, etc.) .

Another object of the present invention is to provide a navigation system that displays a directional guidance landmark including a direction of a route on a real road image and displays it on the screen, The present invention provides an augmented reality navigation device and a method thereof for a route guidance service that allows a user to clearly recognize a transition.

According to an aspect of the present invention, there is provided a vehicle navigation system including a photographing unit for photographing an image in front of a vehicle, a lane recognition unit for analyzing the image to recognize a lane of a road on which the vehicle is traveling, A route search section for calculating a direction and a distance of the links included in the route, a direction indicating a traveling direction from the link to the next link in consideration of the order of the remaining distance from the current position to each link start point in the route, An augmented reality processing unit for overlaying the image so that the generated directional landmark is displayed on the road of the recognized lane and displaying it on the screen; A navigation device is provided.

The lane recognition unit detects an edge by converting the image into a gray image to remove noise, performing an SOBEL operation on the noise-removed image, and obtaining an intensity and a direction of the edge; A lane candidate detecting module for detecting a lane candidate using a Hough transform to obtain a lane disappearing point by applying a RANSAC algorithm to the detected lane candidates, And a lane recognition module for recognizing the lane candidates to the lane vanishing point as a lane in which the vehicle is running.

The lane recognition unit includes a real-time detection module for determining a region of interest (ROI) in a next frame using lane information and lane-prior information detected in a previous frame, and performing lane detection only on the determined region of interest, It may further include a lane tracking module that tracks lanes using particle filtering in the frame and tracks the lane through location tracking of the preceding vehicle.

The direction indicator marker generating unit generates a first direction marker indicating a direction change from a current link to a next link, a direction guide marker for at least one or more links positioned next to the current link in descending order of the remaining distance from the current position .

The augmented reality processing unit may display the corresponding residual distance in a predetermined area of the directional landmark.

Also, the augmented reality processing unit may change at least one of size, color, lightness, and saturation so that the first direction guide mark for the current link and the direction guide mark for the next link are classified according to the remaining distance.

When the current link is displayed on the screen, the augmented reality processing unit displays a first direction guide mark for the current link overlaid on the center of the recognized lane road, and displays a second direction guide mark for the next link The first directional landmark is displayed overlaid under the first directional landmark marker and the first directional lander marker for the current link is displayed to the next link starting point when the link of the next order with the current link is displayed on the screen. The length can be adjusted, and the second directional guide mark for the next link can be overlaid under the first directional guide mark.

According to another aspect of the present invention, there is provided a method of providing a route guidance service by an augmented reality navigation device, the method comprising: (a) when a route guidance command including a destination is input, (b) analyzing the obtained image to recognize a lane of a road in which the vehicle is traveling, (c) searching for a route to the destination, calculating a direction and a distance of the links included in the route, d) generating a direction guide mark indicating a direction of travel from the link to the next link in consideration of the order of the remaining distance from the current position to each link start point in the path, (e) Displaying the directional landmark on the screen so that the landmark is displayed on the road of the recognized lane; Method is provided.

The step (b) includes the steps of converting the image into a gray image to remove noise, performing an SOBEL operation on the noise-removed image to detect an edge, and obtaining strength and direction of the edge, Detecting a lane candidate using a Hough transform, calculating a lane disappearance point by applying a RANSAC algorithm to the detected lane candidates, and calculating a lane disappearance point based on the strength and direction of the edge, May be recognized as a lane in which the vehicle is running.

The step (d) includes the steps of: calculating a remaining distance from the current position to each link start point in the path; aligning the in-path links in descending order of the remaining distance; And generating a directional landmark for at least one or more links following the current link in the ordered order.

Next, in step (e), when only the current link is displayed on the obtained image, the first directional guidance marker for the current link is overlaid on the center of the recognized lane road, and the second And displays the first directional guide marker for the current link to the next link start point if the current link and the next order link are displayed on the obtained image The length of the first directional landmark may be adjusted and the second directional landmark for the next link may be overlaid under the first directional landmark.

In addition, at least one of size, color, brightness, and saturation may be different so that the first direction guide mark for the current link and the direction guide mark for the next link are classified according to the remaining distance.

According to the present invention, a route guidance service implemented as an augmented reality can be provided by using a portable device (e.g., smart phone or the like) equipped with a photographing unit and a position recognition unit without mounting a heavy and costly additional device, The stability is increased.

In addition, by displaying a directional guide mark including the direction of the route on the actual road image and displaying it on the screen, it is possible to clearly recognize the direction change without hindering the reality by using only the minimum guide interface can do.

1 is a block diagram schematically showing the configuration of an augmented reality navigation device for a route guidance service according to an embodiment of the present invention;
Fig. 2 is a detailed view of the configuration of the lane recognition unit shown in Fig. 1. Fig.
3 is an exemplary diagram for explaining a lane recognition method according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an example of a screen displayed by augmented reality navigation device according to an embodiment of the present invention when a route is guided. FIG.
5 is a diagram illustrating a method for providing a route guidance service by an augmented reality navigation device according to an embodiment of the present invention.

The foregoing and other objects, features, and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

Hereinafter, an augmented reality navigation device and a method thereof for a route guidance service according to the present invention will be described in detail with reference to the accompanying drawings. The embodiments are provided so that those skilled in the art can easily understand the technical spirit of the present invention, and thus the present invention is not limited thereto. In addition, the matters described in the attached drawings may be different from those actually implemented by the schematic drawings to easily describe the embodiments of the present invention.

In the meantime, each constituent unit described below is only an example for implementing the present invention. Thus, in other implementations of the present invention, other components may be used without departing from the spirit and scope of the present invention. In addition, each component may be implemented solely by hardware or software configuration, but may be implemented by a combination of various hardware and software configurations performing the same function. Also, two or more components may be implemented together by one hardware or software.

Also, the expression " comprising " is intended to merely denote that such elements are present as an expression of " open ", and should not be understood to exclude additional elements.

FIG. 1 is a block diagram schematically showing the configuration of an augmented reality navigation device for a route guidance service according to an embodiment of the present invention. FIG. 2 is a detailed view of the configuration of the lane recognition unit shown in FIG. FIG. 4 is a diagram illustrating a screen displayed by augmented reality navigation device according to an embodiment of the present invention when a route guidance is performed. FIG.

1, an augmented reality navigation device 100 for a route guidance service includes a communication unit 110, an input unit 120, an output unit 130, a storage unit 140, a photographing unit 150, A position measuring unit 170, a path search unit 180, a directional guide marker generating unit 190, an augmented reality processing unit 200, and a control unit 210.

The communication unit 110 is communication means for interconnecting the augmented reality navigation device 100 with the electronic map providing device through a communication network so as to be provided with an electronic map for route guidance, A wireless communication module, a wired communication module such as the Internet, and a short-range wireless communication module such as a Wi-Fi modem.

The input unit 120 is a means for receiving a user request for controlling the operation of the augmented reality navigation device 100. The input unit 120 converts a user's request into an electric signal according to a user's operation. For example, the input unit 120 receives a route guidance command including a destination from a user. When the input unit 120 is implemented as a touch screen, the input unit 120 and the output unit 130 may be the same.

The output unit 130 is a means for providing a user with an operation result or a status of the augmented reality navigation device 100. The output unit 130 may be a form of a visually recognizable display such as an LCD (Liquid Crystal Display) Organic Light Emitting Diodes), and the like. In particular, the output unit 130 according to the present invention displays a route guidance service implemented as an augmented reality in the augmented reality processing unit 200 on the screen.

The storage unit 140 stores a program necessary for the operation of the augmented reality navigation service and data generated during the execution of the program. Basically, the operating program of the augmented reality navigation device 100 is stored. Also, Data, lane recognition result information, and information necessary for lane recognition.

The storage unit 140 stores navigation map data (for example, coordinates of a road located on a traveling path of a vehicle, coordinates of an intersection, road connection status, road lane information), road facilities (traffic lights, And stores electronic map information including coordinate data of the building, attribute information (height, name, color, etc.), and the like. At this time, it is preferable that the storage unit 140 is stored and managed in the form of an electronic map having a minimum capacity to confirm the data.

The photographing unit 150 is configured to photograph an image according to a user operation, and may include, for example, a camera or the like. The photographing unit 150 can photograph an image in front of the vehicle using a CCD or a CMOS device. In the present embodiment, only the photographing of the road image using the CCD or CMOS device provided in the photographing unit 150 has been described. Alternatively, the photographing unit 150 may be configured to receive the image from the external camera.

The lane recognition unit 160 analyzes the image photographed by the photographing unit 150 and recognizes the lane of the road in which the vehicle is traveling. At this time, the recognized lane includes lane information including lane slope, lane offset, road curvature, and road curvature change rate.

The lane recognition unit 160 includes an edge detection module 161, a lane candidate detection module 162, a lane recognition module 163, a real time detection module 164, and a lane tracking module 165 as shown in FIG. 2 .

The edge detection module 161 detects an edge by converting an image photographed by the photographing unit 150 into a gray image to remove noise, performing a SOBEL operation on the image from which the noise is removed, Find the strength and direction of the edge. That is, the edge detection module 161 converts the image into a gray image using the cvCvtColor () function of OpenCV (Open Computer Vision), convolves the gray image with a gaussian kernel Remove noise. Of the functions provided by OpenCV, cvCvtColor () is a function used to modulate a color gamut to gray (such as another color gamut). Then, the edge detection module 161 obtains an edge by differentiating the noise-removed image in the x and y directions with the Sobel operation (cvSobel () function), and calculates the edge magnitude and the edge with the cvCartToPolar Find the direction. Here, the Sobel operation (cvSobel () function) is a function provided by OpenCV, which is a function used to obtain an edge (boundary line), and a function cvCartToPolar () is a function provided by OpenCV. It is a function to use.

 For example, if the edge strength of an image as shown in FIG. 3A is obtained, it can be as shown in FIG. 3B.

The lane candidate detection module 162 finds lane candidate points based on the strength and direction of the edge obtained by the edge detection module 161 and detects lane candidates using a Hough transform. That is, the lane candidate detecting module 162 generates a mask image as shown in FIG. 3C. Here, among the edges belonging to the black area, those near the vertical direction have a high probability of not being a lane, so first check the direction of the edges in this area, and those near the vertical are not considered as lane candidate points. Then, the lane candidate detection module 162 finds a point where the y value is fixed and the x value is increased in the image as shown in Fig. 3B, and the intensity of the edge exceeds the threshold value and becomes the maximum. The lane candidate detection module 162 selects two points having the same y-axis value among the found points and checks whether the distance between the two points satisfies the lane width. If the distance between two points satisfies the width of the lane, compare the strength and direction of the edge to the two points. Since the probability of a lane is high when the strengths of the edges are similar and the directions are opposite to each other, the lane marker candidate module 162 obtains a center point for two points having similar edge strengths and opposite directions and stores the center points as candidate points of the lane . Then, the lane candidate detection module 162 detects a lane candidate using a Hough transform. Huff transformation is a suitable algorithm for finding the direction and position of a lane in an image and is a method of finding a line from an outline existing in an image of the (x, y) coordinate system. Therefore, by performing the Hough transform, all the linear information appearing in the image can be obtained. In this case, not only the straight line appearing from the lane but also the noise due to the vehicle or the background from the front side to the side are included. Among these, a straight line without a lane candidate point is regarded as noise and removed.

The lane recognition module 162 obtains the lane disappearance point by applying the RANSAC algorithm to the lane candidate detected by the lane marker candidate detection module 162 and recognizes the lane candidate for the lane disappearing point as a lane in which the vehicle is running.

The real-time detection module 164 uses the lane information detected in the previous frame and the lane advance information to determine a ROI in which a lane may exist in the next frame, and performs lane detection only on the determined ROI. Here, the lane dictionary information includes the vanishing point of the lane, the lane width, and the thickness.

For example, when the left lane is found, it is possible to estimate the area where the right lane is present according to the prior information, so that the detection speed and the accuracy can be greatly improved, and the lane advance information is converted into the Hough transform And can be used for lane verification.

On the other hand, it is not effective in terms of time and accuracy to independently detect a lane on a frame-by-frame basis, and a lane-tracking technique that uses the lane detection result of the previous frame for the next frame lane detection should be applied.

The lane tracking module 165 tracks the lane using particle filtering in successive frames and tracks the lane through tracking the location of the preceding vehicle. This makes it possible to track lanes effectively when there are a lot of vehicles on the city center or on the roads, or when the lane is not visible to the naked eye due to weather deterioration.

In the present invention, as described above, the lane detection technology and the tracking technology are used together so that the lane can be effectively detected even when the lane is lost or appears during the driving.

Lane detection is performed every frame, and lane tracking is performed using particle filtering for newly detected lanes. Real-time lane detection is implemented by setting the lane detection area of interest (ROI) of the next frame based on the lane information currently being tracked do.

The position measuring unit 170 may measure the position of the augmented reality navigation device 100 and may be a position measuring device such as GPS (Global Positioning System), WIPI positioning, or the like. That is, the position measuring unit 160 may be a GPS (Global Positioning System) sensor that receives a signal from a GPS satellite and detects a current position, a wireless LAN (WiFi) base station for detecting a current position in an area where reception of GPS satellite signals is impossible, And a wireless LAN device that receives a radio wave of an access point and detects an approximate current position. The two types of sensors may be used in combination to improve the accuracy of position detection.

The position measuring unit 170 transmits the measured current position information of the vehicle to the path searching unit 180.

The path search unit 180 searches for a path to a destination and calculates the directions and distances of the links included in the path. Here, the link refers to a road section in which the running direction changes in the path. That is, when a route guidance command including a destination is inputted from the user, the route search unit 180 reads the electronic map information from the electronic map unit stored in the storage unit 140 and searches for a route to the destination (for example, do. Then, the route search unit 180 divides the road section whose direction is changed in the searched route into links, and obtains the distance between the directions of the links and the road section by the corresponding link.

The direction guide marker generating unit 190 generates a direction guide mark indicating the direction of travel from the link to the next link in consideration of the order according to the remaining distance from the current position to each link start point in the route searched in the route search unit 180 Create a marker. That is, the direction guide landmark creating unit 190 creates a first direction guide landmark indicating the direction change from the current link to the next first link, a second direction guide landmark indicating the direction change from the first link to the next second link, etc. . The first link and the second link are in the order of the shortest remaining distance from the current position, and the directional guide marker generating unit 190 can generate the directional guide mark for at least one or more links located next to the current link. Here, the directional sign is a signal indicating which direction to go, and may be a graphic form such as an arrow, a direction change arrow, and the like. Since the direction guide mark provides a turn-by-turn direction, the driver can concentrate on the road while following the route detected by the route search unit 180, thereby providing safety. Directional signs prevent drivers from making mistakes when they are close to each other and do not know where to go when there are bifurcations.

The augmented reality processing unit 200 overlays the image photographed by the photographing unit 150 such that the directional guide mark generated by the directional guide mark generating unit 190 is displayed on the lane of the lane recognized by the lane recognizing unit 160 Display it on the screen. At this time, the augmented reality processing unit 200 may change the size, color, brightness, saturation, and the like so that the first direction guide mark for the current link and the direction guide mark for the next link are classified according to the remaining distance. For example, the size of each directional guide mark can be displayed gradually as the remaining distance becomes smaller. Thus, not only the driver can estimate the approximate distance of the next route guidance but also the direct recognition ability of the direction change is increased as the distance of the direction change becomes closer.

In addition, the augmented reality processing unit 200 can display a remaining distance to a start point of a next link, a current traveling speed of the vehicle, and the like in a certain area of the direction guide landmark. For example, the augmented reality processing unit 200 preferably displays the remaining distance on the lower, lower right, lower left, and the like of the directional navigation marker so that the driver recognizes the time of the next direction change in advance.

The augmented reality processing unit 200 overlays the first direction guide mark 410 for the current link on the center of the recognized lane road as shown in FIG. 4A if the link following the current link is not displayed on the screen at a long distance And overlaying the second direction indicator mark 420 for the next link under the first direction landmark mark 410. [

4A, a first directional guide mark 410 and a remaining distance 55m for a current link are displayed on a road in which the vehicle is traveling, and a second directional landmark mark 420 for indicating the direction of a next link is displayed. And the remaining distance 200m may be displayed in the form of icons on the lower right of the first direction indicator mark 410. [ This means that the current vehicle is traveling at 15km / h, it must be turned 55m straight from the current position and turn left after 200m from the current position.

4B, the augmented reality processing unit 200 searches the augmented < Desc / Clms Page number 18 > direction for the current link so that the first directional guide mark 430 for the current link is displayed up to the next link start point, The length of the guide mark 430 is adjusted and the second direction mark 440 for the next link is overlaid under the first direction mark 430. [ That is, the augmented reality processing unit 200 displays the arrow direction of the first directional landmark 430 so that the arrow length of the first directional landmark mark 430 matches the distance remained until the direction change.

Referring to FIG. 4B, the arrow length of the first direction guide mark 430 for the current link is aligned with the next link start point to display on the road during which the vehicle is traveling along with the remaining distance 5m, The second direction guide mark 440 for display and the remaining distance 200m may be displayed in an icon form on the lower right of the first direction guide mark 430. [ This means that the current vehicle is traveling at 8km / h, it must go straight after 5m from the current position and turn left after 150m from the current position.

Since the directional guidance marker overlaid on the road image by the augmented reality processing unit 200 provides a clear turn-by-turn direction, To increase safety. Directional signs prevent drivers from making mistakes when they are close to each other and do not know where to go when there are bifurcations.

The control unit 210 controls the overall function of the augmented reality navigation device 100 by executing a driving program stored in the storage unit 140 according to a user's instruction inputted through the input unit 120. [

The control unit 210 includes a communication unit 110, an input unit 120, an output unit 130, a storage unit 140, a photographing unit 150, a lane recognition unit 160, a position measurement unit 170, The directional marker generating unit 190, and the augmented reality processing unit 200 according to an embodiment of the present invention.

The control unit 210 may include at least one computing device, which may be a general purpose central processing unit (CPU), a programmable device device (CPLD, FPGA) that is suitably implemented for a particular purpose, Device (ASIC) or a microcontroller chip.

These components, which the augmented reality navigation device 100 may include, may be implemented by hardware, software, or a combination thereof, and two or more components may be simultaneously implemented by one hardware or software.

On the other hand, the augmented reality navigation device 100 implements real-time photographing of a road image on the front face of the vehicle while driving through a photographing unit 150 and displays it on the screen. By adjusting the angle of the cradle, the angle of the actual road visible on the screen and the direction of the displayed direction indicator are appropriately adjusted so that the corresponding direction indicator appears as if it is located on the road. The path of the position or the straight line can be easily identified.

The augmented reality navigation device 100 includes a navigation dedicated terminal, a mobile communication terminal, a smart phone, a portable media player (PMP), and a personal digital assistant (PDA) Digital Assistant), Tablet PC (Tablet PC) and various other devices that can implement the augmented reality navigation function.

5 is a diagram illustrating a method of providing a route guidance service by an augmented reality navigation device according to an embodiment of the present invention.

Referring to FIG. 5, when a user inputs a route guidance command including a destination (S502), the augmented reality navigation device drives the photographing section to acquire an image in front of the vehicle (S504).

The augmented reality navigation device analyzes the acquired image to recognize a lane of a road in which the vehicle is traveling (S506), searches for a route to the destination (S508), and calculates directions and distances of links included in the searched route (S510).

The augmented reality navigation device generates a direction guide landmark indicating the traveling direction from the link to the next link in accordance with the remaining distance from the current position to each link start point in the path in operation S512. At this time, the augmented reality navigation device calculates the remaining distance from the current position to each link start point in the searched route, and arranges the links in the route in descending order of the remaining distance. Then, the augmented reality navigation device generates a first directional landmark indicating the direction change from the current link to the next link, and a directional landmark for at least one or more links positioned next to the current link in the ordered order.

Then, the augmented reality navigation device displays the generated directional marker on the screen so that the generated directional marker is displayed on the recognized lane of the road (S514). At this time, the augmented reality navigation device displays the first directional guide mark for the current link overlaid on the center of the recognized lane road, and overlaid the second directional guide mark for the next link under the first directional guide mark .

An augmented reality navigation device that provides a screen on which an actual running road image is displayed is an environment in which the horizon line is visible on the screen. Therefore, a direction guide mark is displayed on the running road displayed on the screen, The method of displaying the right turn can be naturally combined with the actual road image to provide very intuitive and practical road guidance information without impairing the sense of reality.

On the other hand, the various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 210.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the storage unit 140 and executed by the control unit 210. [

Thus, those skilled in the art will appreciate that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the embodiments described above are to be considered in all respects only as illustrative and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: augmented reality navigation device 110:
120: input unit 130: output unit
140: storage unit 150: photographing unit
160: lane recognition unit 161: edge detection module
162: lane candidate detection module 163: lane recognition module
164: Real-time detection module 165: Lane tracking module
170: Position measurement unit 180: Path search unit
190: direction indicator marker generation unit 200: augmented reality processing unit
210:

Claims (12)

A photographing unit for photographing a video in front of the vehicle;
A lane recognition unit for analyzing the image and recognizing a lane of a road on which the vehicle is traveling;
A route search unit searching for a route to a destination and calculating directions and distances of links included in the route;
A direction guide landmark generating unit for generating a direction guide landmark indicating the traveling direction from the link to the next link in consideration of the order of the remaining distance from the current position to each link start point in the path; And
And an augmented reality processing unit for overlaying the image so that the generated directional landmark is displayed on a road of the recognized lane,
The lane recognition unit detects an edge by performing a SOBEL operation on the image, calculates the strength and direction of the edge using a predefined function, and determines a lane candidate point based on the strength and direction of the edge The lane candidate candidate is detected by removing a straight line having no lane candidate point among the straight lines obtained by using the Hough transform, and a RANSAC algorithm is applied to the detected lane candidates to obtain a lane disappearance point, And recognizes the route as a lane in which the vehicle is traveling.
The method according to claim 1,
The lane recognizing unit,
An edge detecting unit for detecting an edge by converting the image into a gray image to remove noise, performing an SOBEL operation on the noise-removed image, and detecting an edge strength and direction using a predefined function module;
Wherein a y-value is fixed and an x-value is increased in an image in which the strength and direction of the edge are determined, and a point at which the strength of the edge exceeds the threshold value and becomes the maximum is searched for and two points having the same y-axis value among the searched points are selected. If the distance between the two selected points satisfies the predefined width of the lane, the strength and direction of the edge are compared with each other, and if the intensity of the edge is greater than or equal to a predetermined value, A lane marker candidate detecting module for detecting a lane marker candidate using a Hough transform, And
And a lane recognition module for finding a lane disappearance point by applying the RANSAC algorithm to the detected lane candidates and recognizing the lane candidate for the lane disappearing point as a lane in which the vehicle is running. Device.
3. The method of claim 2,
The lane recognition unit may include a real-time detection module that determines a region of interest (ROI) in a next frame using lane information and lane-prior information detected in a previous frame, and performs lane detection only on the determined region of interest; And
Further comprising a lane tracking module for tracking lanes by using particle filtering in successive frames and for tracking lanes through position tracking of the preceding vehicle.
The method according to claim 1,
The direction guide landmark generating unit generates a direction guide landmark for at least one or more links located next to the current link in the order of the remaining distance from the current position, Wherein the augmented reality navigation device comprises:
The method according to claim 1,
Wherein the augmented reality processing unit displays the remaining distance in a predetermined region of the directional landmark.
The method according to claim 1,
Wherein the augmented reality processing unit changes at least one of size, color, brightness, and saturation so that the first direction guide mark for the current link and the direction guide mark for the next link are classified according to the remaining distance. Augmented reality navigation device.
The method according to claim 1,
The augmented reality processing unit displays a first direction guide mark for the current link overlaid on the center of the recognized lane road and displays a second direction guide mark for the next link on the screen when the current link is displayed on the screen, Overlayed under one-way guide mark,
The length of the first directional landmark is adjusted so that the first directional lander mark for the current link is displayed up to the next link start point when the current link and the next order link are displayed on the screen, Wherein the marker is overlaid under the first direction guide mark and displayed.
A method for providing a route guidance service by an augmented reality navigation device,
(a) when a route guidance command including a destination is input, driving the photographing unit to acquire an image in front of the vehicle;
(b) analyzing the obtained image to recognize a lane of a road in which the vehicle is traveling;
(c) searching for a route to the destination, calculating directions and distances of the links included in the route;
(d) generating a direction guide landmark indicating a traveling direction from the link to the next link in consideration of the order according to the remaining distance from the current position to each link start point in the route; And
(e) overlaying the directional landmark on the image so that the generated directional landmark is displayed on the road of the recognized lane, and displaying on the screen,
In the step (b), an edge is detected by performing an SOBEL operation on the image, and the intensity and direction of the edge are determined using a predefined function. Based on the strength and direction of the edge, A lane candidate candidate is detected by removing a straight line having no lane candidate point out of the straight lines obtained by using Hough transform, a lane disappearance point is obtained by applying the RANSAC algorithm to the detected lane candidates, and lane candidate candidates directed to the lane disappearing point are obtained And recognizes the route as a lane in which the vehicle is traveling.
9. The method of claim 8,
The step (b)
Converting the image into a gray image to remove noise, performing an SOBEL operation on the noise-removed image to detect an edge, and obtaining a strength and a direction of the edge using a predefined function;
Wherein a y-value is fixed and an x-value is increased in an image in which the strength and direction of the edge are determined, and a point at which the strength of the edge exceeds the threshold value and becomes the maximum is searched for and two points having the same y-axis value among the searched points are selected. If the distance between the two selected points satisfies the predefined width of the lane, the strength and direction of the edge are compared with each other, and if the intensity of the edge is greater than or equal to a predetermined value, Determining a lane candidate by using a Hough transform; And
Applying the RANSAC algorithm to the detected lane candidates to obtain a lane disappearance point and recognizing the lane candidate for the lane disappearing point as a lane in which the vehicle is driving.
9. The method of claim 8,
The step (d)
Calculating a remaining distance from a current position to each link start point in the path;
Arranging the links in the path in descending order of the remaining distance; And
A first directional landmark indicating a direction change from a current link to a next link, and a direction guide mark for at least one or more links positioned next to the current link in the ordered order. Service method.
9. The method of claim 8,
In the step (e)
When the current link is displayed on the acquired image, a first direction guide mark for the current link is overlaid on the center of the recognized lane road and a second direction guide mark for the next link is displayed on the first direction guide Overlayed under the markers,
The length of the first directional landmark is adjusted so that the first directional lander mark for the current link is displayed up to the next link start point when the current link and the next order link are displayed on the acquired image, And the direction guide mark is overlaid under the first direction guide mark.
12. The method of claim 11,
Wherein at least one of size, color, brightness, and saturation is made different so that the first directional indicator for the current link and the directional indicator of the next link are classified according to the remaining distance.

Images