KR101974602B1 - Lane repainting apparatus and method - Google Patents

Abstract

An apparatus and method for redrawing lanes of a road are provided. A lane re-coloring apparatus mounted on a vehicle according to an embodiment of the present invention includes a first photographing unit arranged to face a front surface of the vehicle and photographing a road surface in a direction in which the vehicle travels, A first lane detection unit configured to detect a lane from a first image having a photographed perspective, a second lane detection unit arranged to face the road surface, and a second lane detecting unit A second lane detection unit for detecting a lane from a second image photographed by the second photographing unit, and a control unit for controlling the X and Y axis movement of the injection nozzle based on the lane detected by the second lane detection unit, And a control unit for causing the nozzle unit to be painted, wherein a jetting pressure value of the jetting nozzle and a distance between the Z axis are determined in advance so as to correspond to the width of the lane to be painted .

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a LANE REPAINTING APPARATUS AND METHOD,

The present invention relates to a technique for redrawing lanes of a road.

Generally, pavement roads such as highways, national roads, and local roads are painted with a center line dividing the center of a vehicle in the opposite lane or a lane dividing the progress section of the vehicle.

The lane of the center line or the driving lane is a baseline that induces the driver to identify the driving direction and induces the safe driving of the vehicle. Therefore, visibility is visually perceived as very important because it is a guideline for driving the vehicle safely have.

When these lanes are first painted on the road, the sketches are sketched with chalk and the worker is working directly with a roller or paint machine.

Such a working method has a problem in that the result of the lane painting is different even on the road with the same width of the lane according to the skill or the sketch of the painting operator.

Currently, all lanes on domestic roads are not standardized because they are made by hand, and the sketch for painting must be sketched directly on the road, which not only causes inconvenience but also requires a considerable amount of work time have.

When the lane is damaged and red coloring is required, all the processes are performed manually as described above, so that the width of the lane does not coincide with the lane of the lane according to the skill of the operator (for example, the lane becomes wider) do.

The present invention is to provide a re-coloring scheme capable of achieving a constant color quality and a working speed without being greatly affected by the skill level of the operator.

According to one aspect of the present invention, there is provided a lane re-coloring apparatus mounted on a vehicle, comprising: a first photographing unit arranged to face a front surface of the vehicle and photographing a road surface in a direction in which the vehicle travels; A first lane detection unit configured to detect a lane from a first image having a photographed perspective, a second lane detection unit arranged to face the road surface, and a second lane detecting unit A second lane detection unit for detecting a lane from a second image photographed by the second photographing unit, and a control unit for controlling the X and Y axis movement of the injection nozzle based on the lane detected by the second lane detection unit, And a control unit for causing the nozzle unit to be painted, wherein a jetting pressure value of the jetting nozzle and a distance between the Z axis are determined in advance so as to correspond to the width of the lane to be painted .

According to an aspect of the present invention, there is provided a method for redrawing a lane by a lane re-coloring apparatus mounted on a vehicle, the method comprising the steps of: (a) (B) detecting a lane from the first image having a perspective; (c) when the vehicle is moving along the detected lane, the lane is detected so as to face the road surface (D) detecting a lane from the second image; and (e) detecting a lane from the second image based on the lane detected from the second image, And the distance between the Z-axis and the jetting pressure value of the jetting nozzle is determined in advance so as to correspond to the width of the lane to be painted.

According to an embodiment of the present invention, it is possible to achieve a constant color quality and an operation speed without being greatly affected by the skill level of the operator.

In addition, it is possible to drastically reduce work time due to manual operation.

1 is a diagram showing a configuration of a lane re-coloring apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a lane detection process according to an exemplary embodiment of the present invention.
3 is a flowchart illustrating a lane detection process according to another embodiment of the present invention.
4 is a diagram illustrating a method of removing unnecessary line segments in a lane detecting process according to an exemplary embodiment of the present invention.
5 is a diagram illustrating a method of connecting lane candidate lines in a lane detecting process according to an exemplary embodiment of the present invention.
6 is a diagram illustrating a method of detecting a final lane in a lane detection process according to an embodiment of the present invention.
Fig. 7 is a view showing the type of the damaged lane.
8 and 9 are experimental results comparing the accuracy of the lane detection method according to an embodiment of the present invention.
10 is an image of an actual injection nozzle according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals (e.g., first, second, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another.

Also, throughout the specification, when an element is referred to as being " connected " or " connected " with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

Also, throughout the specification, when an element is referred to as " including " an element, it is understood that the element may include other elements as well, without departing from the other elements unless specifically stated otherwise. Also, the terms " a ", " module ", and the like in the description mean a unit for processing at least one function or operation, which means that it can be implemented by one or more hardware or software or a combination of hardware and software .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing a configuration of a lane re-coloring apparatus according to an embodiment of the present invention.

A lane re-coloring apparatus 100 according to an embodiment of the present invention includes a first photographing unit 111, a first lane detecting unit 112, a second photographing unit 113, a second lane detecting unit 114, A control unit 115, a control unit 116, and a memory 117. [

The first photographing unit 111 can be installed so as to face the front of the vehicle, and can photograph a road existing in the traveling direction of the vehicle.

The image captured by the first image capturing unit 111 may be output through the display unit in real time.

The first photographing unit 111 may include a CCD, a CMOS, a black box camera, or the like.

On the other hand, the first lane detecting unit 112 detects a background (e.g., sky, tree, etc.) that is unnecessary areas besides the lane from the image photographed by the first photographing unit 111 (hereinafter referred to as a first input image) Can be removed.

At this time, the first lane detecting unit 112 may designate a region of interest (ROI) region in the input image to remove the unnecessary regions.

In addition, the first lane detecting unit 112 can detect lanes in the ROI area, and various algorithms such as Hough transform, Canny Edge Detection, Sobel Edge, and Line Segment Detection can be applied.

In addition, the first lane detecting unit 112 may extract the lane candidate lines by removing unnecessary lanes based on the lane angle of the lane detected in the ROI area.

For this, the first lane detecting unit 112 may divide the left area and the right area based on the input image, and remove unnecessary lines.

This will be described later with reference to FIG.

In addition, the first lane detecting unit 112 can determine the lane by connecting the extracted lane candidate lines.

Since the line segment is composed of the starting point and the ending point, it is possible to connect the lane candidate lines by applying at least one of the distance threshold and the angle threshold between the extracted lane candidate lines.

This will be described later with reference to FIG.

The first lane detecting unit 112 can use lane characteristics having symmetry when determining lanes.

Here, 'the characteristic of the lane having symmetry' can mean the left boundary and the right boundary of the lane.

That is, when the result of connecting the lane candidate lines (hereinafter referred to as "boundary") is located on the left side, there is a symmetrical boundary on the right side.

When looking at the lane of the road from the driver's line of sight, the distance between the two lanes is large in the lane near the driver or the vehicle. On the other hand, the distance between the two lanes in the far lane is small. The final lane can be detected by considering the same vanishing point.

For reference, in the case of a curved lane, the final lane can be detected adaptively considering the symmetry between lanes at each boundary.

This will be described later with reference to FIG.

The lane determined by the first lane detecting unit 112 may overlap the lane in the image photographed by the first photographing unit 111 and displayed on the display unit 115 described later.

In one embodiment, the lane can be guided in the form of a route guidance application (navigation).

On the other hand, the second photographing unit 113 may be installed so as to face the road surface, and may capture an image including the redrawing target lane drawn on the road surface.

The image captured by the second photographing unit 113 may be output through the display unit 115 in real time.

The second photographing unit 113 may include a CCD, a CMOS, a black box camera, or the like.

On the other hand, the second lane detecting section 114 can designate a region of interest (ROI) region in the input image photographed by the second photographing section 113, and can detect lanes in the ROI region.

For this, various algorithms such as Hough transform, Canny Edge Detection, Sobel Edge, and Line Segment Detection can be applied to the second lane detection unit 114.

Further, the secondary lane detecting section 114 can detect the center line between the two boundary lines of the lane in the detected lane.

As will be described later, the control unit 116 can redraw the lane while controlling the injection nozzle to move along the X and Y axes based on the lane information detected by the second lane detecting unit 114. [

For reference, in order to paint the lane of the same width, the pressure for spraying the paint for the first time and the Z-axis height of the injection nozzle can be fixed to a constant value.

On the other hand, the display unit 115 can display the lanes detected by the first lane detecting unit 112 and the second lane detecting unit 114.

At this time, the display unit 115 may divide the area of the screen and display the lanes detected by the first lane detecting unit 112 and the second lane detecting unit 114 in the respective areas, The lane detected by the first lane detecting section 112 or the second lane detecting section 114 can be displayed on the screen.

The display unit 115 can display the lane detected by the first lane detecting unit 112 or the second lane detecting unit 114 according to the driver's selection.

In addition, the display unit 115 may display a reference line at the center of the screen when displaying the lane (including the center line) detected by the second lane detecting unit 114. [

That is, when the reference line displayed at the center of the screen coincides with the center line of the lane detected by the second lane detecting section 114 or does not deviate from the predetermined range, the injection nozzle is controlled by the control section 116, It is possible to perform the re-coloring with the same width as the width of the lane indicated in Fig.

If the center line of the lane does not coincide with the reference line, the controller 116 may output a warning message or a notification sound to inform the driver of the warning. Alternatively, the driver may display the center line of the lane The running direction of the vehicle can be adjusted.

In another embodiment, the controller 116 calculates the difference between the center line of the reference line and the lane and the X and Y values of the injection nozzle corresponding to the difference to move the position of the injection nozzle, .

The display unit 115 may include at least one of a touch screen mounted on the lane replenishing apparatus 100, a driver's smartphone, a PDA, a PMP, a tablet computer, and a notebook computer.

The control unit 116 controls the components of the lane replenishing apparatus 100 such as the first photographing unit 111, the first lane detecting unit 112, the second photographing unit 113, The control section 114 and the display section 115 perform the above-described operation, and the memory 117 can also be controlled.

Meanwhile, the memory 117 may store various information derived from an algorithm for the control unit 116 to control the respective components and a control unit 116 for controlling the respective components.

1, the lane remodeling apparatus 100 may include X, Y, and Z-axis driving motors (not shown) for driving in respective axial directions of the injection nozzles, and may include a paint storage tank, a heater, Fuel tanks, bead storage tanks, compressors, and the like.

2 is a flowchart illustrating a lane detection process according to an exemplary embodiment of the present invention.

2 is a process for detecting a lane by using the photographed image of the first photographing unit 111 as an input image by the first lane detecting unit 112.

The lane renewal apparatus 100 designates an ROI in the input image and removes unnecessary objects (sky, trees, etc.) other than the lane in the ROI (S201).

After S201, the lane renewal apparatus 100 detects a plurality of line segments in the image by applying various algorithms such as Hough transform, Canny Edge Detection, Sobel Edge, and Line Segment Detection (S202).

After S202, the lane remodeling apparatus 100 selects a lane candidate line segment by removing unnecessary lines based on the angles of the detected line segments (S203).

For this purpose, the lane remodeling apparatus 100 may divide the input image into left and right regions, and then remove lines at unnecessary angles that may deviate from the lane.

After S203, the lane renewal apparatus 100 connects the lane candidate lines (S204).

For reference, since the line segments are composed of the starting point and the ending point, the lane candidate lines can be connected by using the distance threshold between the line segments and the angle threshold between the line segments.

After S204, the lane renewal apparatus 100 detects the final lane (S205).

At this time, the lane remodeling apparatus 100 can detect the final lane in consideration of the vanishing point in the case of a straight lane, and can detect the final lane adaptively in consideration of the symmetry between the line lanes in a curved lane.

After S205, the lane renewal apparatus 100 provides the detected lane to the driver through the screen (S206).

3 is a flowchart illustrating a lane detection process according to another embodiment of the present invention.

3 is a process for detecting a lane by using the photographed image of the second photographing unit 113 as an input image by the second lane detecting unit 114. [

The lane re-coloring apparatus 100 designates an ROI in the input image and removes information that may adversely affect lane detection in the input image through preprocessing to remove unnecessary objects other than the lane in the ROI (S301).

Through this process, it is possible to increase the recognition accuracy of the damaged lane included in the input image.

After S301, the lane renewal apparatus 100 detects a plurality of line segments in the image by applying various algorithms such as Hough transform, Canny Edge Detection, Sobel Edge, and Line Segment Detection (S302).

After S302, the lane renewal apparatus 100 connects the detected line segments by one line segment and removes unnecessary lines to detect the last lane (S303).

2, an unnecessary line segment is removed based on a predetermined angle because the perspective is applied to the input image. However, since the image input in FIG. 3 is an image taken facing the road surface, it is highly likely that the perspective is not applied . Accordingly, line segments having an angle at which an angle of a line segment is out of a certain range based on 90 중 among the line segments detected in S302 are regarded as unnecessary line segments and can be removed.

After S303, the lane remodeling apparatus 100 controls the injection nozzle based on the information of the detected lane to redraw the lane (S304).

4 is a diagram illustrating a method of removing unnecessary line segments in a lane detecting process according to an exemplary embodiment of the present invention.

Referring to FIG. 4A, the lane detection algorithm basically detects a line segment, and since an image captured by the first photographing unit 111 has a perspective, line segments at various angles can be detected.

In order to remove unnecessary line segments among the various angle line segments, the image may be divided into a left area and a right area as shown in FIG. 4, and line segments at an angle deviating from a specific angle range may be determined to be unnecessary and removed.

4 (b) and 4 (c) show the lane detection result in which the preprocessing is not performed as shown in FIG. 4 (a) and the lane detection result in the preprocessing as shown in FIG. 4 (a) .

In FIG. 4B, unnecessary line segments other than the lane are not removed, but FIG. 4C shows that the unnecessary line segments other than the lane line are removed.

5 is a diagram illustrating a method of connecting lane candidate lines in a lane detecting process according to an exemplary embodiment of the present invention.

As shown in FIG. 5, line segments are composed of a start point and an end point, and when different line segment distances and angles are within a predetermined threshold value, the line segments can be determined as line segments that can be connected to each other.

That is, when the distance between the start point of the first line segment and the end point of the second line segment is within a predetermined threshold value, the two line segments are determined as line segments that can form the same lane that can be connected to each other. The angle is the same.

In FIG. 5, cases 1 and 2 are examples of connecting straight line segments, and case 3 is an example of connecting curves.

For reference, FIG. 5 illustrates an embodiment in which two line segments are connected to detect lanes. However, when considering various lane conditions such as a damaged lane or a shadow, a plurality of line segments may be connected to form a lane.

6 is a diagram illustrating a method of detecting a final lane in a lane detection process according to an embodiment of the present invention.

As shown in FIG. 6, the present invention can detect a lane using symmetry, which is a characteristic that a lane of a road has a left boundary and a corresponding right boundary.

When looking at the lane of the road from the driver's eyes, the lane that is close to the driver or the vehicle has a large gap between the two boundaries. Conversely, the lane that is far away is detected as the vanishing point where the distance between the two boundaries is small. can do.

For reference, in the case of a curved lane, the final lane can be detected adaptively considering the symmetry between lanes at each boundary.

Fig. 7 is a view showing the type of the damaged lane.

As shown in FIG. 7, there may be various types of corrupted lanes such as corrosion, abrasion or discoloration, and the images photographed by the second photographing unit 113 may include the corrupted lanes.

The present invention can improve the lane recognition accuracy even if the lane is damaged.

Experimental results comparing the accuracy of the lane detecting method according to an embodiment of the present invention will be described below with reference to FIGS. 8 and 9. FIG.

Table 1 compares the lane detection method of the present invention with the Aly method using Caltech Open Database.

First, the Caltech Open Database consists of four roads, Cordova 1, Cordova 2, Washington 1, and Washington 2.

In order to measure whether or not the lane is accurately detected, the following Equations (1), (2), and (3) are used.

TP (True Positive) indicates that the lane is correctly detected in the lane, FN (false negative) indicates that the lane is the background, and FP (false positive) indicates that the background is the lane.

Referring to Fig. 8, (a) in Fig. 8 is an example in which a lane is correctly detected in a lane in Cordova1, and Fig. 8 (b) is an example in which a lane is correctly detected in a lane in Cordova2.

8 (c) is an example of correctly detecting the lane in the lane in Washington1. FIG. 8 (d) shows an example in which a lane is correctly detected in a lane in Washington2.

On the other hand, FIG. 9A shows a result of applying the Aly method, and FIG. 9B shows an example of a result of using the lane detecting method of the present invention.

As shown in FIG. 9 (b), it can be confirmed that the lane detecting method of the present invention is more accurate than the Aly method, and the lane detecting method of the present invention also has the accuracy It can be seen that it is much higher.

[Table 2] shows the execution time of the lane detection method of the present invention in the Caltech Open Database, and it can be confirmed that there is no problem in using the lane detection method in real time on the average of 33.09 msec.

10 is an image of an actual injection nozzle according to an embodiment of the present invention.

10 is connected to a solenoid valve capable of moving in the X, Y, and Z axes under the control of the controller 116, and having an ON / OFF function to supply paint to the spray nozzles.

The lane re-coloring method according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media storing data that can be decoded by a computer system. For example, it may be a ROM (Read Only Memory), a RAM (Random Access Memory), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, or the like. In addition, the computer-readable recording medium may be distributed and executed in a computer system connected to a computer network, and may be stored and executed as a code readable in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the following claims And changes may be made without departing from the spirit and scope of the invention.

100: Lane remover
111: First shooting section
112: first lane detecting section
113: Second shooting section
114: Second lane detection section
115:
116:
117: Memory

Claims (13)

A lane re-coloring apparatus mounted on a vehicle,
A first photographing unit arranged to face a front surface of the vehicle and photographing a road surface in a direction in which the vehicle travels; A first lane detecting unit detecting a lane from a first image having a perspective sensed by the first photographing unit; A second photographing unit arranged to face the road surface and photographing the road surface when the vehicle moves along a lane detected by the first lane detecting unit; A second lane detection unit detecting a lane from a second image photographed by the second photographing unit; And a control unit for controlling the movement of the injection nozzle in X and Y axes based on the lane detected by the second lane detecting unit so as to paint the lane,
Wherein the first lane detecting unit divides the first image into a left area and a right area and removes lines out of a predetermined angle range from a plurality of line segments detected in the first image, A plurality of line segments satisfying the distance threshold and the angle threshold among the plurality of lane candidate line segments are connected as a single line segment using a distance threshold value and an angle threshold value for the starting point and the end point between the selected lane candidate line segments, And a lane is detected by mapping each connected single line segment to either the left boundary or the right boundary in the lane based on a symmetry criterion in which symmetry between both boundary lines of the single lane becomes symmetrical.
delete delete The method according to claim 1,
The first lane detection unit
Wherein a straight lane is detected from the single line segment in consideration of the symmetry of the lane including the left boundary and the right boundary and the vanishing point according to the perspective.
The method according to claim 1,
The first lane detection unit
Wherein a curved lane is detected from the single line segment in consideration of the symmetry of the lane including the left boundary and the right boundary.
The method according to claim 1,
A display unit for displaying a lane detected by the second lane detecting unit on a screen,
Further comprising:
The second lane detection unit
Detecting center lines connecting the centers of both sides of the lane detected from the second image,
The display unit
When a lane detected from the second image is displayed on a screen, a reference line is displayed together with the center of the screen,
The control unit
Wherein the control unit controls the movement of the injection nozzle so that the reference line and the center line coincide with each other or do not deviate from a predetermined range.
The method according to claim 6,
The control unit
If the baseline and the center line do not match or fall outside a predetermined range,
Calculates a distance difference between the reference line and the center line,
Wherein the control unit controls the movement of the injection nozzle in the X and Y axes according to the calculated distance difference to perform redrawing.
A method for redrawing a lane by a lane re-coloring apparatus mounted on a vehicle,
(a) receiving a first image obtained by photographing a road surface in a direction in which the vehicle travels from a first photographing unit arranged so as to face the front of the vehicle; (b) detecting a lane from the first image having a perspective; (c) receiving a second image of a road surface taken from a second photographing unit arranged so as to face the road surface when the vehicle moves along the detected lane; (d) detecting a lane from the second image; And (e) painting the lane by controlling the X and Y axis movement of the injection nozzle based on the lane detected from the second image,
The step (b) includes: dividing the first image into a left area and a right area, removing one or more lane candidate line segments from a plurality of line segments that are out of a predetermined angle range, A plurality of line segments satisfying the distance threshold and the angle threshold among the plurality of lane candidate line segments are connected as a single line segment using a distance threshold value and an angle threshold value for the starting point and the end point between the selected lane candidate line segments, And detecting a lane by mapping each of the connected single line segments to one of a left boundary and a right boundary in a lane based on a symmetry criterion wherein symmetry between both boundary lines of a single lane becomes symmetrical, Recoloring method.
delete delete 9. The method of claim 8,
The step (b)
Detecting a straight lane from the single line segment in consideration of the symmetry of the lane including the left boundary and the right boundary and the vanishing point according to the perspective;
Wherein the lane re-coloring method comprises the steps of:
9. The method of claim 8,
The step (b)
Detecting a curved lane from the single line segment considering the symmetry of the lane including the left boundary and the right boundary;
Wherein the lane re-coloring method comprises the steps of:
delete

Images