KR102544640B1 - Lane detection and prediction system, apparatus and method - Google Patents

Abstract

The present invention relates to a lane detection and prediction system, apparatus, and method, and more specifically, to a lane detection and prediction system, apparatus, and method which acquire and preprocess road images, detect and predict lanes through the preprocessed road images, and display AR images. The lane detection and prediction apparatus of the present invention comprises a driving data collecting unit, an image acquiring unit, an image quality converting unit, a labeling and grouping unit, a lane detecting unit, an eccentricity information generating unit, and an AR image generating unit.

Description

Lane detection and prediction system, apparatus and method {Lane detection and prediction system, apparatus and method}

The present invention relates to a lane detection and prediction system, apparatus and method, and more particularly, a lane detection and prediction system that acquires and preprocesses road images, detects and predicts lanes through the preprocessed road images, and displays them as AR images. , device and method.

Due to the development of automobile technology, technologies capable of ensuring the safety of drivers and pedestrians and improving driver convenience have been applied to automobiles. An example of such technology is an adaptive cruise control system and a lane departure warning system, and by integrating and controlling these vehicle safety enhancement technologies, the vehicle recognizes the environment around the vehicle and the vehicle autonomously performs all safety measures. Autonomous driving technology that controls and drives related functions is developing. Therefore, technology for recognizing and detecting objects on the road using vision sensors to detect pedestrians, vehicles, traffic lights, road/traffic safety signs, road markings, and lanes for autonomous driving technology is being developed.

However, object detection technology using conventional vision sensors still has limitations in replacing human eyes due to limitations in sensor performance. Lane detection, which is an essential technology for preventing collisions and maintaining the vehicle's position within the driving lane, is not fully functional in environments where the lane cannot be detected by vision sensors, partial or total loss due to lane deterioration, night driving and lighting problems, etc. There is a problem with not being able to do it.

Registered Patent No. 10-21672191, Registration Date October 13, 2020, 'Road Condition Information Collection System and Method' Registered Patent No. 10-2119033, Registration Date May 29, 2020, 'Apparatus and method for providing road marking damage information using video'

The present invention was created to solve the above problems, and an object of the present invention is a lane detection and prediction system that obtains and preprocesses road images, detects and predicts lanes through the preprocessed road images, and displays them as AR images, It is to provide an apparatus and method.

An apparatus for detecting and predicting lanes according to the present invention for achieving the above object includes a driving data collection unit that collects driving data including a current location of a vehicle, a driving speed, and a driving route to a final destination; an image acquiring unit acquiring an image of a road in front of the vehicle; a picture quality conversion unit generating a final image by upscaling the road image; Using a deep learning-based object extraction model, a plurality of objects are detected from the final image, outlines are set, and object types are labeled and grouped based on the coordinates of each object for which the outlines are set and the size of the outlines. a labeling grouping unit to; Based on the grouped object group, a first section within a predetermined distance from the vehicle in the traveling direction of the vehicle is detected, and present in a second section within a predetermined distance from a point exceeding the predetermined distance in the traveling direction of the vehicle. a lane detection unit for predicting a lane in a second section; a one-sided information generating unit generating current one-sided information or optimized one-sided information according to the gradients of lanes in the first section and the second section; and an AR image generator generating an AR image based on the current lateral information or optimized lateral information and displaying the AR image.

The grouped object group may include at least one of a lane marking object group, a guide rail object group, and a sound barrier object group.

The lane detecting unit may detect the lane in the first section based on the lane marking object group and predict the shape of the lane in the second section based on the guide rail object group and the sound barrier object group.

In addition, when the slopes of the lanes of the first section and the second section are the same, the one-side information generation unit compares the center line of the lane of the first section and the center line of the vehicle in advance to indicate the degree of sideways of the vehicle in the lane of the first section. information is generated, and when the slopes of the lanes of the first section and the second section are different, the lanes of the first section and the lanes of the second section are based on the change in slope of the lanes of the first section and the slope of the second section and the driving speed of the vehicle. It is characterized in that the optimized one-sided information is generated to minimize the centrifugal force applied to the vehicle running.

In addition, the AR image generation unit displays the center line as a dotted line on the road image, displays the center line of the vehicle as a solid line having a predetermined width based on the dotted line, and displays the center line of the vehicle as a solid line having a predetermined width, at a boundary point between the lanes of the first section and the lanes of the second section. The point where the center point of the vehicle should be located within the lane is displayed as a point, or the path where the center line of the vehicle should be located within the first section lane and the second section lane is displayed in a predetermined color, and the width of the lane is smaller than the actual lane width. It is characterized by generating an AR image that is reduced by a predetermined ratio and displayed.

On the other hand, a lane detection and prediction method for achieving the above object includes collecting driving data including a current location, driving speed, and a driving route to a final destination by a driving data collector; Acquiring a road image in front of the vehicle by an image acquisition unit; Up-scaling the road image by an image quality conversion unit to generate a final image; A labeling grouping unit detects a plurality of objects from the final image using a deep learning-based object extraction model, sets an outline, and determines the type of the object based on the coordinates of each object for which the outline is set and the size of the outline. labeling and grouping; A lane detecting unit detects a first section within a predetermined distance from the vehicle based on the grouped object group, and a second section within a predetermined distance from the point exceeding the predetermined distance in the traveling direction of the vehicle. predicting a lane of a second section existing in the section; generating, by a one-sided information generator, current one-sided information or optimized one-sided information according to the gradients of lanes in the first section and the second section; and generating an AR image based on the current one-sided information or the optimized one-sided information by an AR image generating unit and displaying the AR image.

The grouped object group may include at least one of a lane marking object group, a guide rail object group, and a sound barrier object group.

The lane detecting unit may detect the lane in the first section based on the lane marking object group and predict the shape of the lane in the second section based on the guide rail object group and the sound barrier object group.

In addition, when the slopes of the lanes of the first section and the second section are the same, the one-side information generation unit compares the center line of the lane of the first section and the center line of the vehicle in advance to indicate the degree of sideways of the vehicle in the lane of the first section. information is generated, and when the slopes of the lanes of the first section and the second section are different, the lanes of the first section and the lanes of the second section are based on the change in slope of the lanes of the first section and the slope of the second section and the driving speed of the vehicle. It is characterized in that the optimized one-sided information is generated to minimize the centrifugal force applied to the vehicle running.

In addition, the AR image generation unit displays the center line as a dotted line on the road image, displays the center line of the vehicle as a solid line having a predetermined width based on the dotted line, and displays the center line of the vehicle as a solid line having a predetermined width, at a boundary point between the lanes of the first section and the lanes of the second section. The point where the center point of the vehicle should be located within the lane is displayed as a point, or the path where the center line of the vehicle should be located within the lanes of the first section and the second section is displayed in a predetermined color, and the width of the lane is the actual lane width It is characterized by generating an AR image that is reduced by a predetermined ratio and displayed.

On the other hand, the lane detection and prediction system according to the present invention for achieving the above object acquires a road image where the vehicle is located, detects and predicts a lane in the direction of the vehicle through pre-processing of the road image, and displays it as an AR image. lane detection and prediction devices; and a user terminal equipped with an application capable of receiving and storing driving data from the lane detection and prediction device and displaying an AR image.

In addition, the sensing and predicting device may include: a driving data collection unit that collects driving data including a current location of the vehicle, a driving speed, and a driving route to a final destination; an image acquiring unit acquiring an image of a road in front of the vehicle; a picture quality conversion unit generating a final image by upscaling the road image; Using a deep learning-based object extraction model, a plurality of objects are detected from the final image, outlines are set, and object types are labeled and grouped based on the coordinates of each object for which the outlines are set and the size of the outlines. a labeling grouping unit to; Based on the grouped object group, a first section within a predetermined distance from the vehicle in the traveling direction of the vehicle is detected, and present in a second section within a predetermined distance from a point exceeding the predetermined distance in the traveling direction of the vehicle. a lane detection unit for predicting a lane in a second section; a one-sided information generating unit generating current one-sided information or optimized one-sided information according to the gradients of lanes in the first section and the second section; and an AR image generator generating an AR image based on the current lateral information or optimized lateral information and displaying the AR image.

A lane detection and prediction system, apparatus, and method according to the present invention detects and predicts a lane and displays the degree of deflection of the vehicle within the lane, thereby enabling safe driving of the vehicle and providing the driver with optimization within the lane. Driving convenience of the driver can be improved by displaying the selected driving route.

In addition, by reducing the width of the detected lane according to a predetermined condition and displaying the lane, it is possible to induce the driver's attention as well as induce safe driving.

1 is a block diagram of a lane detection and prediction system according to an embodiment of the present invention.
2 is a block diagram of an apparatus for detecting and predicting lanes according to an embodiment of the present invention;
3 is a flowchart of a lane detection and prediction method according to an embodiment of the present invention
4 is an exemplary diagram of lane detection and prediction according to an embodiment of the present invention;
5 is an example of an AR image display according to an embodiment of the present invention;

The above objects, other objects, features and advantages of the present invention will be easily understood through the following preferred embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed content will be thorough and complete and the spirit of the present invention will be sufficiently conveyed to those skilled in the art.

In some cases, it is mentioned in advance that parts that are commonly known in describing the invention and are not greatly related to the invention are not described in order to prevent confusion for no particular reason in explaining the present invention.

In this specification, when a first component is referred to as being operated or executed on a second component (ON), the first component is operated or executed in an environment in which the second component operates or is executed, or the second component is operated or executed. It should be understood that it is operated or executed through direct or indirect interaction with.

When a component, device, or system is referred to as comprising a component consisting of a program or software, even if not explicitly stated otherwise, the component, device, or system refers to hardware (necessary for the program or software to execute or operate). For example, memory, CPU, etc.) or other programs or software (eg, operating system or driver required to drive hardware) should be understood as including.

In addition, it should be understood that, unless otherwise specified, the component may be implemented in any form of software, hardware, or both software and hardware.

In addition, terms used in this specification are for describing embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. The terms 'comprises' and/or 'comprising' used in the specification do not exclude the presence or addition of one or more other elements.

Also, in this specification, terms such as 'unit' and 'device' may be intended to refer to functional and structural combinations of hardware and software driven by the corresponding hardware or for driving the hardware. For example, the hardware herein may be a data processing device including a CPU or other processor. Also, software driven by hardware may refer to a running process, an object, an executable file, a thread of execution, a program, and the like.

In addition, the terms may mean a predetermined code and a logical unit of hardware resources for executing the predetermined code, and do not necessarily mean physically connected codes or one type of hardware in the present invention. can be easily deduced to the average expert in the art.

Hereinafter, with reference to the accompanying drawings for the specific technical content to be carried out in the present invention will be described in detail.

Each component shown in FIGS. 1 and 2 indicates that it may be functionally and/or logically separated, and does not necessarily mean that each component is divided into a separate physical device or written in a separate code according to the present invention. The average expert in a technical field of .

1 is a block diagram of a lane detection and prediction system according to an embodiment of the present invention. As shown in FIG. 1, the lane detection and prediction system according to the present invention obtains a road image where the vehicle is located, detects and predicts a lane in the direction of the vehicle through pre-processing of the road image, and displays the lane as an AR image. It includes a sensing and predicting device 100 and a user terminal 200 equipped with an application capable of receiving and storing driving data from the lane detecting and predicting device 100 and displaying an AR image.

2 is a block diagram of an apparatus for detecting and predicting lanes according to an embodiment of the present invention. As shown in FIG. 2 , the lane detection and prediction device 100 includes a driving data collection unit 110 that collects driving data including a current location of the vehicle, a driving speed, and a driving route to a final destination, and a front of the vehicle. A plurality of objects are obtained from the final image by using an image acquisition unit 120 that acquires a road image for , an image quality conversion unit 130 that generates a final image by upscaling the road image, and a deep learning-based object extraction model. Based on the labeling grouping unit 140 that detects and sets an outline, labels and groups the type of object based on the coordinates of each object for which the outline is set and the size of the outline, and the grouped object group. Detecting a first section within a predetermined distance in the traveling direction of the vehicle from the vehicle, and predicting a lane of the second section existing in the second section within a predetermined distance from a point exceeding the predetermined distance in the traveling direction of the vehicle AR based on the lane detection unit 150 and the current one-side information generation unit 160 that generates current one-side information or optimized one-side information according to the slope of the lanes of the first and second sections, and the current one-side information or optimized one-side information It includes an AR image generating unit 170 that generates an image and displays it.

At this time, although the lane detection and prediction device 100 is not shown in the drawing, a display unit capable of displaying an AR image on the screen and a speaker unit capable of expressing a warning sound when the vehicle is too biased to one side of the lane. , a communication unit capable of network communication may be further included.

Hereinafter, a method for detecting and predicting a lane through the apparatus for detecting and predicting a lane configured as described above will be described in detail with reference to FIGS. 3 to 5 .

3 is a flow chart of a method for detecting and predicting lanes according to an embodiment of the present invention. As shown in FIG. 3 , in the lane detection and prediction method through the lane detection and prediction device according to the present invention, the driving data collection unit 110 includes the current location of the vehicle, the driving speed, and the driving route to the final destination. Collecting driving data (S110), acquiring a road image in front of the vehicle by the image acquisition unit 120 (S120), and generating a final image by upscaling the road image by the image quality conversion unit 130 (S130) and the labeling grouping unit 140 detects a plurality of objects from the final image using a deep learning-based object extraction model and sets an outline, and sets the coordinates of each object for which the outline is set and the size of the outline Labeling and grouping the types of objects based on (S140), and the lane detecting unit 150 detects a first section within a predetermined distance from the vehicle based on the grouped object groups in the traveling direction of the vehicle. and predicting a lane of a second section existing in a second section within a predetermined distance from a point exceeding a predetermined distance in the traveling direction of the vehicle (S150) and generating current hemilateral information or optimized hemilateral information according to the slope of the lane in the second section (S160). The AR image generating unit 170 generates an AR image based on the current hemilateral information or optimized hemilateral information and displays it. It consists of a step (S170) to do.

First, in the step S110, the driving data collection unit 110 collects driving data including the current location, driving speed, and driving route to the final destination of the vehicle. Vehicle condition information that can be collected through the systems can be collected in real time.

Next, in the step S120, the image acquisition unit 120 captures a road image of the front of the vehicle in real time. The image acquisition unit 120 may preferably be a black box device installed in the vehicle.

More specifically, the image acquisition unit 120 may capture road images in real time and collect road images by dividing the collected road images for each frame.

Next, in the step S130, the quality conversion unit 130 upscales the road image to generate a final image. Specifically, the quality conversion unit 130 uses a pre-learned deep learning-based model or algorithm interpolation method. A plurality of road images may be converted into a high-quality road image by upscaling each of a plurality of road images using .

At this time, the image quality conversion unit 130 uses any one of the GAN-based Real-ESRGAN model, SRGAN model, SinGAN model, EDSR model, MDSR model, and SRCNN model as a deep learning-based model, or a Bicubic algorithm Upscaling may be performed using any one interpolation method among interpolation, Lanczcos interpolation, and bilinear interpolation.

Further, in the present invention, the use of the Real-ESRGAN model is described, but since upscaling can be performed using any one of the above-described deep learning-based model or algorithm interpolation, it is not limited thereto.

Also, the image quality conversion unit 130 is provided to include a noise removal module, and the acquiring of the final image may include removing noise.

In the step of removing noise, the image quality conversion unit 130 may remove fine dust or fog from the upscaled road image. Such a noise removal module may be a module that removes noise caused by air scattered light reflected from ambient light sources in an image, or noise caused by aging of a camera lens, or noise caused by fine dust, fog, or the like.

Therefore, in the step of removing noise, the image quality conversion unit 130 analyzes the noise pixels in each upscaled road image using the noise removal module and converts the R, G, and B pixels into HSI (Hue (color), Saturation (Chroma) and Intensity (brightness), the value is extracted, and DCP (Dark Channel Prior) is applied to the extracted value to remove noise.

More specifically, in the step of removing noise, the pre-processing unit applies DCP formed as a transmission map to the upscaled road image, maps the upscaled road image to which DCP is not applied, and then applies the denoising image to the upscaled road image. Noise can be removed from the scaling road image.

To explain this in more detail, DCP represents information of an image corresponding to an existing up-scaled road image including a noise component, and based on DCP values corresponding to area pixels in the existing up-scaled road image, the existing up-scaled road image It may be to estimate a noise value of an upscaled road image of , and remove noise from an existing upscaled road image based on the estimated noise value.

Thus, in the step of acquiring the final image including the step of removing noise, the image quality conversion unit 130 upscales a plurality of road images and converts them into high-quality road images, and furthermore, the upscaled road image. It is possible to obtain a final image by removing noise such as atmospheric scattering light, blurring due to aging of the lens, fine dust, and fog.

Next, in the step S140, the labeling grouping unit 140 detects a plurality of objects from the final image using a deep learning-based object extraction model, sets the outline, and sets the coordinates and Based on the size of the outline, the types of objects are labeled and grouped. The labeling grouping unit 140 detects all objects such as lanes, vehicles, trees, sidewalk blocks, sky, guide rails, and sound barriers in the final image. can

In addition, the labeling grouping unit 140 sets the outline of each detected object using the polygon labeling technique, and classifies the type of object based on the size of the outline and the coordinates in the final image of each object for which the outline is set. You can label and group them.

In this case, the grouped object group may include at least one of a lane marking object group, a guide rail object group, and a sound barrier object group.

In the step of extracting and grouping the objects, a step of assigning a pre-set first weight to each group of grouped objects may be included.

Specifically, in the step of assigning the first weight, the labeling grouping unit 140 assigns the highest weight among the first weights to the lane marking object group, and assigns a higher weight than the weight given to the lane marking object group to groups other than the lane marking object group. A low weight may be assigned. For example, the first weight given to the lane marking object group may be 8 to 10 points based on 10 points.

When an outline is set for each object, the labeling grouping unit 140 may calculate x-axis and y-axis values of the outline. Accordingly, lanes marked on the road may be determined based on the x-axis and y-axis values.

Next, in the step S150, the lane detecting unit 150 detects a first section within a predetermined distance from the vehicle based on the grouped object group in the traveling direction of the vehicle, and is constant in the traveling direction of the vehicle. A lane of a second section existing in a second section within a predetermined distance from a point exceeding the distance is predicted, which may be the same as in FIG. 4 . 4 is an exemplary diagram of lane detection and prediction according to an embodiment of the present invention.

The lane detecting unit 150 may detect a first section lane existing in a first section within a predetermined distance from the vehicle in the traveling direction of the vehicle based on the grouped object group. Preferably, the first section may be a section corresponding to within 100 m from the front end center of the vehicle.

In the step of detecting the lanes of the first section, the lane detecting unit 150 may detect the lanes of the first section based on the lane marking object group to which the highest weight is assigned among the first weights.

In addition, the lane detecting unit 150 predicts the shape of a lane in the second section existing in the second section within a predetermined distance from a point exceeding a predetermined distance in the traveling direction of the vehicle based on the grouped object group. steps can be performed.

Here, the second section may be, for example, a section that is difficult to check with the naked eye due to congestion on the road. In the step of predicting the shape of the lane in the second section, a learned machine learning model may be used.

In the step of estimating the shape of the lane in the second section, the shape of the lane in the second section may be predicted based on the adjacent vehicle object group, the guide rail object group, and the sound barrier object group.

Specifically, for example, the lane detecting unit 150 sees more than a predetermined number of left or right ends of a plurality of adjacent vehicle outlines located in front of a vehicle traveling in the same lane among object groups, but determines the degree to which each end is visible. If it is uniform, it can be predicted that the lane of the second section is a curve. Through this, the lane detecting unit may predict the shape of the lane in the second section and generate the lane in the second section based on the predicted shape of the lane.

Next, in step S160, the one-sided information generation unit 160 generates current one-sided information or optimized one-sided information according to the gradient of the lanes in the first section and the second section.

In order to generate lateral information, current lateral information indicating the degree of lateralization of the vehicle in the lane of the first section may be generated by comparing the center line of the lane in the first section with the center line of the preset vehicle.

To this end, a reference point for detecting the center line of the vehicle may be set in advance based on the angle of view and the pixel value of the image acquisition unit 120 that collects the road image.

If the slope of the lane in the first section and the slope of the predicted lane in the second section do not match, the slope of the lane in the first section and the slope of the lane in the second section are changed based on the change in slope of the lane in the first section and the slope of the lane in the second section and the driving speed of the vehicle. Optimization one-sided information for minimizing centrifugal force applied to a vehicle traveling in a section lane may be generated.

Finally, in the step S160, the AR image generation unit 160 generates an AR image based on the current one-side information or the optimized one-side information and displays it, which may be as shown in FIG. 5. 5 is an exemplary diagram of an AR image display according to an embodiment of the present invention.

The AR image generating unit 160 displays the center line as a dotted line on the road image, and displays the center line of the vehicle as a solid line having a predetermined width based on the dotted line so that the driver or occupant can visually check the degree of one side of the vehicle. can do. In addition, an AR image displaying numerical deflection information such as left (-) 0.0 m and right (+) 0.0 m as a ratio of how much the center line of the vehicle is biased based on the dotted line may be generated.

In addition, the AR image generating unit 160 displays a point within the lane where the center point of the vehicle should be located at the boundary point between the first section lane and the second section lane, or within the first section lane and the second section lane. An AR image in which a path along which the center line of the vehicle should be located may be displayed in a predetermined color may be generated.

In addition, the AR image generating unit 160 sets the widths of the lanes of the first section and the second section to a predetermined value greater than the actual lane width based on the driving data including the current location of the vehicle, the driving speed, and the driving route to the final destination. An AR image reduced by the ratio can be created. For example, if the vehicle's current location is in a child protection area, an AR image that is reduced by 5 to 10% from the actual lane width may be generated.

Therefore, as described above, the lane detection and prediction system, apparatus, and method according to the present invention detect and predict the lane and display the degree of deviation of the vehicle within the lane, thereby enabling safe driving of the vehicle as well as the driver. Driver's driving convenience can be improved by displaying an optimized driving path within the corresponding lane to the driver.

In addition, by reducing the width of the detected lane according to a predetermined condition and displaying the lane, it is possible to induce the driver's attention as well as induce safe driving.

In the above, even though all the components constituting the embodiment of the present invention have been described as being combined or operated as one, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all of the components may be selectively combined with one or more to operate. In addition, although all of the components may be implemented as a single independent piece of hardware, some or all of the components are selectively combined to perform some or all of the combined functions in one or a plurality of pieces of hardware. It may be implemented as a computer program having. Codes and code segments constituting the computer program may be easily inferred by a person skilled in the art. Such a computer program may implement an embodiment of the present invention by being stored in a computer readable storage medium, read and executed by a computer.

On the other hand, although the above has been described and illustrated in relation to preferred embodiments for illustrating the technical idea of the present invention, the present invention is not limited to the configuration and operation as shown and described in this way, and departs from the scope of the technical idea. It will be apparent to those skilled in the art that many changes and modifications can be made to the present invention without modification. Accordingly, all such appropriate alterations and modifications and equivalents are to be regarded as falling within the scope of the present invention. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the attached claims.

100: lane detection and prediction device
110: driving data collection unit
120: image acquisition unit
130: quality conversion unit
140: labeling grouping unit
150: lane detection unit
160: one-sided information generation unit
170: AR image generation unit
200: user terminal

Claims (12)

a driving data collection unit that collects driving data including a current location of the vehicle, a driving speed, and a driving route to a final destination;
an image acquiring unit acquiring an image of a road in front of the vehicle;
a picture quality conversion unit generating a final image by upscaling the road image;
A plurality of objects are detected from the final image using a deep learning-based object extraction model, an outline is set using a polygon labeling technique, and the object is detected based on the coordinates of each object for which the outline is set and the size of the outline. a labeling grouping unit for labeling and grouping types of;
Based on the grouped object group, a first section within a predetermined distance from the vehicle in the traveling direction of the vehicle is detected, and present in a second section within a predetermined distance from a point exceeding the predetermined distance in the traveling direction of the vehicle. a lane detection unit for predicting a lane in a second section;
a one-sided information generating unit generating current one-sided information or optimized one-sided information according to the gradients of lanes in the first section and the second section; and
an AR image generating unit generating an AR image based on the current lateral information or optimized lateral information and displaying the AR image;
Including,
The lane detection unit,
Detecting a lane in the first section based on a lane marking object group;
Based on the guide rail object group and the sound barrier object group, the shape of the lane in the second section is predicted,
Among the object groups, if the left or right ends of a plurality of adjacent vehicle outlines located in front of a vehicle running in the same lane are visible at a predetermined number or more, but the degree of visibility of each end is uniform, the shape of the lane in the second section is predicted as a curve. characterized in that,
The AR image generating unit,
The center line is displayed as a dotted line on the road image, and the center line of the vehicle is displayed as a solid line having a predetermined width based on the dotted line. ,
At the boundary point between the first section lane and the second section lane, the point within the lane where the center point of the vehicle should be located is indicated by a point, or the route where the center line of the vehicle should be located within the first section lane and the second section lane is specified marked with color,
An apparatus for detecting and predicting lanes characterized by generating an AR image displayed by reducing a width of a lane by a predetermined ratio from an actual lane width.
According to claim 1,
The grouped object group,
A lane detection and prediction device comprising at least one of a lane marking object group, a guide rail object group, and a sound barrier object group.
delete According to claim 1,
The one-sided information generating unit,
If the slopes of the lanes of Section 1 and Section 2 are the same,
Comparing the center line of a lane in the first section with a center line of a preset vehicle to generate current side information indicating the degree of lateralization of the vehicle in the lane in the first section;
If the slopes of the lanes of section 1 and section 2 are different,
Based on the change in slope of the lane in section 1, the slope of the lane in section 2, and the driving speed of the vehicle, optimization one-sided information that minimizes the centrifugal force applied to the vehicle traveling in the lane in section 1 and section 2 is generated. Lane detection and prediction device, characterized in that for.
delete Collecting driving data including a current location of the vehicle, a driving speed, and a driving route to a final destination by a driving data collection unit;
Acquiring a road image in front of the vehicle by an image acquisition unit;
Up-scaling the road image by an image quality conversion unit to generate a final image;
A labeling grouping unit detects a plurality of objects from the final image using a deep learning-based object extraction model, sets an outline using a polygon labeling technique, and based on the coordinates of each object for which the outline is set and the size of the outline labeling and grouping the types of the objects by using a method;
A lane detecting unit detects a first section within a predetermined distance from the vehicle based on the grouped object group, and a second section within a predetermined distance from the point exceeding the predetermined distance in the traveling direction of the vehicle. predicting a lane of a second section existing in the section;
generating, by a one-sided information generator, current one-sided information or optimized one-sided information according to the gradients of lanes in the first section and the second section; and
generating an AR image based on the current one-side information or the optimized one-side information and displaying the AR image by an AR image generating unit;
Including,
The lane detection unit,
Detecting a lane in the first section based on a lane marking object group;
Based on the guide rail object group and the sound barrier object group, the shape of the lane in the second section is predicted,
Among the object groups, if the left or right ends of a plurality of adjacent vehicle outlines located in front of a vehicle running in the same lane are visible at a predetermined number or more, but the degree of visibility of each end is uniform, the shape of the lane in the second section is predicted as a curve. characterized in that,
The AR image generating unit,
The center line is displayed as a dotted line on the road image, and the center line of the vehicle is displayed as a solid line having a predetermined width based on the dotted line. ,
At the boundary point between the first section lane and the second section lane, the point within the lane where the center point of the vehicle should be located is indicated by a point, or the route where the center line of the vehicle should be located within the first section lane and the second section lane is specified marked with color,
A lane detection and prediction method characterized by generating an AR image displayed by reducing a lane width by a predetermined ratio from an actual lane width.
According to claim 6,
The grouped object group,
A lane detection and prediction method comprising at least one of a lane marking object group, a guide rail object group, and a sound barrier object group.
delete According to claim 6,
The one-sided information generating unit,
If the slopes of the lanes of Section 1 and Section 2 are the same,
Comparing the center line of a lane in the first section with a center line of a preset vehicle to generate current side information indicating the degree of lateralization of the vehicle in the lane in the first section;
If the slopes of the lanes of section 1 and section 2 are different,
Based on the change in slope of the lane in section 1, the slope of the lane in section 2, and the driving speed of the vehicle, optimization one-sided information that minimizes the centrifugal force applied to the vehicle traveling in the lane in section 1 and section 2 is generated. Lane detection and prediction method characterized in that.
delete A lane detection and prediction device that obtains a road image on which the vehicle is located, detects and predicts a lane in the direction of the vehicle through pre-processing of the road image, and displays the AR image; and
a user terminal equipped with an application capable of receiving and storing driving data from the lane detection and prediction device and displaying an AR image;
Including,
The sensing and predicting device,
a driving data collection unit that collects driving data including a current location of the vehicle, a driving speed, and a driving route to a final destination;
an image acquiring unit acquiring an image of a road in front of the vehicle;
a picture quality conversion unit generating a final image by upscaling the road image;
A plurality of objects are detected from the final image using a deep learning-based object extraction model, an outline is set using a polygon labeling technique, and the object is detected based on the coordinates of each object for which the outline is set and the size of the outline. a labeling grouping unit for labeling and grouping types of;
Based on the grouped object group, a first section within a predetermined distance from the vehicle in the traveling direction of the vehicle is detected, and present in a second section within a predetermined distance from a point exceeding the predetermined distance in the traveling direction of the vehicle. a lane detection unit for predicting a lane in a second section;
a one-sided information generating unit generating current one-sided information or optimized one-sided information according to the gradients of lanes in the first section and the second section; and
an AR image generating unit generating an AR image based on the current lateral information or optimized lateral information and displaying the AR image;
Including,
The lane detection unit,
Detecting a lane in the first section based on a lane marking object group;
Based on the guide rail object group and the sound barrier object group, the shape of the lane in the second section is predicted,
Among the object groups, if the left or right ends of a plurality of adjacent vehicle outlines located in front of a vehicle running in the same lane are visible at a predetermined number or more, but the degree of visibility of each end is uniform, the shape of the lane in the second section is predicted as a curve. characterized in that,
The AR image generating unit,
The center line is displayed as a dotted line on the road image, and the center line of the vehicle is displayed as a solid line having a predetermined width based on the dotted line. ,
At the boundary point between the first section lane and the second section lane, the point within the lane where the center point of the vehicle should be located is indicated by a point, or the route where the center line of the vehicle should be located within the first section lane and the second section lane is specified marked with color,
A lane detection and prediction system, characterized in that for generating an AR image displayed by reducing the width of the lane by a predetermined ratio from the actual lane width.
delete

Images