KR20220077370A - system and method for recognizing and extracting traffic safety signs for generating precision road maps - Google Patents

Abstract

A traffic safety sign recognition/extraction system for generating a map with precision according to an embodiment of the present invention includes: an input unit for receiving a road driving image; a lane detection unit that detects a lane in an image frame of a road driving image using a deep learning algorithm, and then analyzes and classifies information on the detected lane; An image frame in which the lane is detected according to a preset detection area condition value after generating an integrated object detection area by merging and expanding each detection area of at least two or more deep learning algorithms that detect an image object in different detection methods a road safety sign object detection unit for detecting a road safety sign object located in the integrated object detection area within; and an object analysis unit that analyzes a detailed shape based on the feature points extracted according to the color, saturation, and brightness of the detected road safety sign object.

Description

{system and method for recognizing and extracting traffic safety signs for generating precision road maps}

The present invention relates to a method and system for recognizing/extracting traffic safety signs for map generation with precision.

In recent years, with the rapid development of information processing and communication technology, a method of constructing geographic information as map information data and utilizing it for transportation has been widely used. This map information data is produced based on the actual photographed geographic information and is used for navigation, safe driving, and automatic driving.

A navigation system that utilizes such map data is an automatic vehicle navigation system, which provides directions to a destination through voice and monitor screens using GPS, which uses satellites to determine the current location. In such a navigation device, a GPS receiver that receives the vehicle's current location information from a GPS satellite, a map DB in which map information such as road safety signs are registered, and map information are mapped according to the current location identified through the GPS receiver. An operating program to be displayed on the screen is provided.

The map information built in the map DB of the navigation system is generated through measurement of actual geographic information. Since this map information changes according to new roads or changes in the traffic system, the map information is updated regularly or irregularly. shall. The update of map information in such navigation is usually done manually. In particular, since traffic signs are constantly changing, the map manager frequently moves to a location on the map and checks road safety signs one by one, and then the road safety included in the map information. There was a cumbersome problem of having to update the information on the sign.

Registered Patent Publication No. 10-1199959

An object of the present invention is to provide a method and system for recognizing/extracting traffic safety signs for map generation with precision that can solve the problems of the prior art.

In accordance with an embodiment of the present invention for solving the above problems, a traffic safety sign recognition/extraction system for generating a map with precision includes an input unit for receiving a road driving image; a lane detection unit that detects a lane in an image frame of a road driving image using a deep learning algorithm, and then analyzes and classifies information on the detected lane; An image frame in which the lane is detected according to a preset detection area condition value after generating an integrated object detection area by merging and expanding each detection area of at least two or more deep learning algorithms that detect an image object in different detection methods a road safety sign object detection unit for detecting a road safety sign object located in the integrated object detection area within; and an object analysis unit that analyzes a detailed shape based on the feature points extracted according to the color, saturation, and brightness of the detected road safety sign object.

A method for recognizing/extracting a traffic safety sign for generating a map with precision according to an embodiment of the present invention for solving the above problems includes the steps of pre-processing a plurality of image frames of a road driving image; After detecting a lane in each preprocessed image frame using a deep learning algorithm, analyzing and classifying the type of the detected lane; generating an integrated bounding box by overlapping the detected two detection bounding boxes after detecting a detection bounding box of an object image in each image frame of the preprocessed image using two different deep learning algorithms; detecting the detailed information of the road safety sign object by filtering the position, color, and shape of the object in the overlapping area when the overlap area in the integrated bounding box exceeds a preset value; and providing the ID and detailed information assigned to the road safety sign object detected in the plurality of image frames in the form of a queue.

Therefore, using the traffic safety sign recognition/extraction system and method for creating a map with precision according to an embodiment of the present invention, the detection area is expanded using two deep learning object detection algorithms to detect the accuracy of road safety signs It provides the advantage of being able to improve the accuracy of the detection position of the road safety sign in the road driving image by improving

1 is a block diagram of a system for recognizing/extracting traffic safety signs for map generation with precision according to an embodiment of the present invention.
FIG. 2 is an exemplary view illustrating an operation of the lane detection unit shown in FIG. 1 .
FIG. 3 is a detailed block diagram of the object detection unit shown in FIG. 1 .
4 is a flowchart illustrating a process in which the object analysis unit analyzes an object located in the road surface mark area, and FIG. 5 is a flowchart illustrating the process in which the object analyzer analyzes an image of an object located in an area other than the road surface mark area. .
FIG. 6 is an exemplary view illustrating an operation of the object tracking unit shown in FIG. 1 .
7 is a flowchart illustrating a method for recognizing/extracting a traffic safety sign for generating a map with precision according to an embodiment of the present invention.
8 is a detailed flowchart of the process S720 shown in FIG.
9 is a detailed flowchart of the process S730 shown in FIG.
10 and 11 are detailed flowcharts of the process S740 shown in FIG.
12 is a detailed flowchart of the process S760 shown in FIG.
13 illustrates an example computing environment in which one or more embodiments disclosed herein may be implemented.

Specific structural or functional descriptions of the embodiments of the present invention disclosed in the present specification or application are only exemplified for the purpose of describing the embodiments according to the present invention, and the embodiments according to the present invention may be implemented in various forms. and should not be construed as being limited to the embodiments described in the present specification or application.

Since the embodiment according to the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the present specification or application. However, this is not intended to limit the embodiment according to the concept of the present invention with respect to a specific disclosed form, and should be understood to include all changes, equivalents or substitutes included in the spirit and scope of the present invention.

Terms such as first and/or second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one element from another element, for example, without departing from the scope of the present invention, a first element may be called a second element, and similarly The second component may also be referred to as the first component.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Other expressions describing the relationship between elements, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The terminology used herein is used only to describe specific embodiments, and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. As used herein, terms such as “comprise” or “have” are intended to designate that the specified feature, number, step, operation, component, part, or combination thereof is present, and includes one or more other features or numbers, It should be understood that the existence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, a method and system for recognizing/extracting a traffic safety sign for creating a map with precision according to an embodiment of the present invention will be described in more detail based on the accompanying drawings.

1 is a block diagram of a system for recognizing/extracting traffic safety signs for map generation with precision according to an embodiment of the present invention, FIG. 2 is an exemplary diagram illustrating the operation of the lane detection unit shown in FIG. 1, and FIG. 3 is It is a detailed block diagram of the object detection unit shown in FIG. 1 , FIG. 4 is a flowchart illustrating a process of analyzing an object located in the road surface mark area by the object analysis unit, and FIG. 5 is an image of an object located in an area other than the road surface mark area is a flowchart illustrating a process of analyzing by the object analysis unit, and FIG. 6 is an exemplary diagram illustrating the operation of the object tracking unit shown in FIG. 1 .

First, as shown in FIG. 1 , the system 100 for recognizing/extracting traffic safety signs for map generation with precision according to an embodiment of the present invention includes an input unit 110 , a lane detection unit 120 , and a road safety sign object. It includes a detection unit 130 , an object analysis unit 140 , an ID allocation unit 150 , and an object tracking unit 160 .

On the other hand, the present invention may include a pre-processing configuration for performing a pre-processing process on the image frame of the road driving image, and the pre-processing configuration is a histogram smoothing, For example, it may be a configuration in which a gray level value is mapped by calculating the frequency of the brightness value, calculating the accumulated histogram value using the frequency, normalizing the accumulated histogram value, and applying a gray scale mapping function to the normalized accumulated histogram.

The pre-processing process referred to in the present invention is only a representative example of a method of pre-processing an image frame, and other pre-processing processes may also be applied in the present invention.

The input unit 110 is configured to receive a road driving image.

Next, referring to FIG. 2 , the lane detection unit 120 may be configured to detect lanes in each preprocessed image frame using a deep learning algorithm, and then analyze and classify information on the detected lanes.

More specifically, the lane detection unit 120 detects a lane by using a deep learning algorithm, and then corrects the road sign and the average width between the detected lanes through a morphology filtering (operation) process, and then After removing the area other than the lane area and the overlapping road markings, the corrected lane may be classified into any one of a dotted line, a solid line, an exclusive lane, and a center line. As a deep learning algorithm, a convolutional long short-term memory (LSTM) may be used, and it goes without saying that various deep learning algorithms may be used.

For reference, Morphology is a broad set of image processing operations that process images according to shapes. In Morphology operations, each pixel of an image is adjusted according to the values of other pixels in its neighborhood, and the size and shape of the neighbor are selected. Thus, it means that the operation is processed in a specific form of the target image.

Basically, the morphology operation is an erode and dilate operation process, which applies a filter based on the current pixel, checks the value in the filter area, and modifies the value of the current pixel.

Here, erosion means substituting the minimum pixel value among pixels in the filter area to the current pixel value, dilation means substituting the maximum pixel value among pixels in the filter area into the current pixel value, and the morphological operation is a known technique. Optionally, a more detailed description will be omitted.

Next, referring to FIG. 3, the road safety sign object detection unit 130 merges and expands the object detection bounding box of two different deep learning object detection algorithms to create an integrated object detection area, and then It may be configured to detect a road safety sign object located in the integrated object detection area within the image frame in which the lane is detected according to a set detection area condition value.

More specifically, the road safety sign object detection unit 130 includes an object detection area adjustment unit 131 and an object detection area filtering unit 132 .

The object detection area adjustment unit 131 overlaps the object detection area (bounding box) of each of the first deep learning object detection algorithm and the second deep learning object detection algorithm to a preset overlap value, preferably exceeding 60% It may be configured to create the integrated object detection area.

For reference, the first deep learning object detection algorithm has a fast object detection speed but low recognition accuracy, and the second deep learning object detection algorithm has a slow object detection speed but high recognition accuracy. Gaussian YOLO v3 may be used as the first deep learning object detection algorithm, and MASK R-CNN may be used as the second deep learning object detection algorithm. The Gaussian YOLO v3 and MASK R-CNN algorithms are examples of the first and second deep learning object algorithms, and the present invention is not limited thereto, and of course, other suitable deep learning object algorithms suitable for the characteristics of the algorithm may be used.

Accordingly, in the present invention, an integrated object detection area is created and used so that the advantages of the two deep learning object detection algorithms can be derived by integrating and expanding the detection bounding boxes of the two deep learning object detection algorithms.

On the other hand, when the overlapping integrated object detection area is smaller than a preset overlap value (60%), the object detection area adjustment unit 131 may detect the road safety sign object with the second deep learning object detection algorithm.

The object detection area filtering unit 132 is configured to filter the position of the road safety sign object detected in the integrated object detection area with a preset detection zone condition value, wherein the preset detection zone condition value is the road safety This is an area value for the detection position of the mark object, and the area value includes a road marking area, a center line left area, and a curb right area.

More specifically, the object detection area filtering unit 132 first determines whether the location of the integrated object detection area is a road marking area, and if it is not a road marking area, determines whether it is on the left side of the center line, and determines whether the location of the integrated object detection area is not on the left side of the center line. In this case, it is possible to reduce the error in the detection rate of the road safety sign object and improve the accuracy by specifying the area where the integrated object detection area is generated by tertiarily determining whether it is on the right side of the curb.

Next, referring to FIGS. 4 and 5, the object analysis unit 140 analyzes (determines) the detailed shape based on the feature points extracted according to the color, saturation, and brightness of the road safety sign object in the filtered integrated object detection area. It can be configuration.

More specifically, referring to FIG. 5 , the object analysis unit 140 detects the integrated object detection area (bounding box) of the integrated object detection area (bounding box) when the location of the integrated object detection area (bounding box) in which the road safety sign object is detected is located in the road surface marking area After converting the pixel RGB value to HSV (color, saturation, brightness), the threshold value of the road safety sign object is generated through a preset threshold value algorithm, and the object and the road sign area are calculated based on the generated threshold value. After classifying into white and black, noise is removed through morphology filtering, contours of the detected road safety sign object are extracted and fitted, and the road surface marking area is based on the feature points of the fitted contour. Determine the shape of the road sign object located in Of course, the Otsu Algorithm may be used as the threshold algorithm described above, and other suitable algorithms may be used in addition to it.

In addition, referring to FIG. 4 , the object analysis unit 140 provides a color histogram for the object detection area when the location of the object detection area (bounding box) in which the road safety sign object is detected is located in an area other than the road surface mark area. Calculating (shade value distribution for pixels), converting pixel RGB values of the object detection area into HSV (color, saturation, brightness), and converting the object detection area into CIE (Commission Internationale d'Eclairage) color space coordinates After conversion, the red, blue, and green color bands of the object detection area are counted using the color histogram and the HSV value, and L (brightness), A on the CIE (Commission Internationale d'Eclairage) color space coordinates. (Complementary color from green to red), B (complementary color from yellow to blue) values are extracted, and the object detection area including the LAB value of the color space coordinates and the count value of the RGB band is a gray scale image After conversion, the threshold value of the road safety sign object is generated through a preset threshold value algorithm (Otsu Algorithm), the object and the foreground area are divided based on the generated threshold value, and then morphology filtering is performed. After removing noise and extracting contours of the detected road safety sign object, the shape (including color) of the road safety sign object is determined based on the position of the feature point of the outline.

Next, the ID allocator 160 may be configured to allocate a detection ID for identifying the road safety sign object detected in the image frame analyzed by the object analysis unit 150 to the image frame.

Next, referring to FIG. 6 , the object tracking unit 170 compares and determines whether or not the road safety sign object detected in the before/after image frame of the road driving image is matched, and if it is matched, the ID assigned to the road safety sign object , it may be a configuration that provides detailed information and lane information in the form of a queue.

More specifically, the object tracking unit 170 checks whether a road sign object is detected and whether an ID is assigned in the n+1th image frame from the object analysis unit 150, and then the center of the nth image frame and the n+1th image After calculating the optimal value of the offset vector of the center of the frame, it is determined whether the road sign object matches or not based on the optimal value of the offset vector.

In addition, the object tracking unit 170 stacks the image frames in which the same road sign object is detected, and when the stacking value exceeds the reference value, the road safety sign object ID, segment, and object detection area (bounding box) of the stacked image frames ) and lane information are stored in the form of a queue and provided to an external server.

7 is a flowchart illustrating a method for recognizing/extracting a traffic safety sign for generating a map with precision according to an embodiment of the present invention, FIG. 8 is a detailed flowchart of the process S720 shown in FIG. 7, and FIG. 9 is shown in FIG. The illustrated process S730 is a detailed flowchart, FIGS. 10 and 11 are detailed flowcharts of the process S740 illustrated in FIG. 7 , and FIG. 12 is a detailed flowchart of the process S760 illustrated in FIG. 7 .

First, referring to FIG. 7 , the method (S700) for recognizing/extracting traffic safety signs for map generation with precision according to an embodiment of the present invention is first, The image frame of the road driving image input from the input unit is pre-processed by the pre-processing unit 120 ( S710 ).

Here, the S710 process is a histogram smoothing, for example, calculating the frequency of the brightness value, calculating the accumulated histogram value using the frequency, and then normalizing the and mapping gray level values by applying a gray scale mapping function to the normalized accumulation histogram.

When the process S710 is completed, lanes in each preprocessed image frame are detected using a deep learning algorithm, and then the types of the detected lanes are analyzed and classified (S720).

More specifically, in the S720 process, after detecting lanes using a deep learning algorithm, the road markings and average widths between the detected lanes are corrected through a morphology filtering (calculation) process, and then the corrected lane area It may be a process of classifying the corrected lane into any one of a dotted line, a solid line, an exclusive lane, and a center line after removing the other areas and overlapping road markings. As described above, as a deep learning algorithm, a convolutional long short-term memory (LSTM) may be used, and it goes without saying that various deep learning algorithms may be used in addition to it.

For reference, Morphology is a broad set of image processing operations that process images according to shapes. In Morphology operations, each pixel of an image is adjusted according to the values of other pixels in its neighborhood, and the size and shape of the neighbor are selected. Thus, it means that the operation is processed in a specific form of the target image.

Basically, the morphology operation is an erode and dilate operation process, which applies a filter based on the current pixel, checks the value in the filter area, and modifies the value of the current pixel.

Here, erosion means substituting the minimum pixel value among pixels in the filter area to the current pixel value, dilation means substituting the maximum pixel value among pixels in the filter area into the current pixel value, and the morphological operation is a known technique. Optionally, a more detailed description will be omitted.

When the S720 process is completed, the object detection areas of two different deep learning object detection algorithms are merged and expanded to create an integrated object detection area, and then within the image frame in which the lane is detected according to the preset detection area condition value. Detects a road safety sign object located in the integrated object detection area (S730).

More specifically, the S730 process overlaps the object detection area (bounding box) of each of the first deep learning object detection algorithm and the second deep learning object detection algorithm to a preset overlap value, preferably exceeding 60% and generating an integrated object detection area (S731).

For reference, the first deep learning object detection algorithm has a fast object detection speed but low recognition accuracy, and the second deep learning object detection algorithm has a slow object detection speed but high recognition accuracy. Gaussian YOLO v3 may be used as the first deep learning object detection algorithm, and MASK R-CNN may be used as the second deep learning object detection algorithm. The Gaussian YOLO v3 and MASK R-CNN algorithms are examples of the first and second deep learning object algorithms, and the present invention is not limited thereto, and of course, other suitable deep learning object algorithms suitable for the characteristics of the algorithm may be used.

Therefore, in the present invention, an integrated object detection area is created and used so that the advantages of the two deep learning object detection algorithms can be derived by integrating and expanding the detection bounding boxes of the two deep learning object detection algorithms, and overlapped When the integrated object detection area is smaller than the preset overlap value (60%), the process of detecting the road safety sign object is performed by the second deep learning object detection algorithm.

Meanwhile, the process S730 includes filtering the object detection area using a preset detection area condition value based on the location of the road safety sign object detected in the integrated object detection area (S732).

In step S732, it is first determined whether the location of the integrated object detection area is a road marking area, if it is not a road marking area, it is secondarily determined whether it is on the left side of the center line, and when it is not on the left side of the center line, it is thirdly determined whether it is on the right side of the curb , it is possible to reduce the error in the detection rate of the road safety sign object and improve the accuracy by specifying the area where the integrated object detection area (bounding box) is generated through the area filtering process described above.

On the other hand, when the S730 process is completed, the detailed shape based on the feature points extracted according to the color, saturation, and brightness of the road safety sign object in the integrated object detection area filtered by the object analysis unit 150 is analyzed in the object analysis unit 150 . Analyze (judgment) (S740).

Here, in the S740 process, when the location of the object detection area (bounding box) where the road safety sign object is detected is located in the road surface area, the pixel RGB value of the object detection area (bounding box) is converted to HSV (color, saturation, brightness). ), create a threshold value of the road safety sign object through a preset threshold algorithm, divide the object and the road sign area into white and black based on the generated threshold value, and then through morphological filtering After removing noise, extracting and fitting the contours of the detected road safety sign object, and judging the shape of the road sign object located in the road sign area based on the feature points of the fitted contour. do. As described above, the Otsu Algorithm may be used as the threshold algorithm, and of course, other suitable algorithms may also be used.

In addition, in the S740 process, when the location of the object detection area (bounding box) in which the road safety sign object is detected is located in an area other than the road surface mark area, a color histogram (contrast value distribution for pixels) for the object detection area ) calculation, converting the pixel RGB values of the object detection area into HSV (hue, saturation, brightness) and converting the object detection area into CIE (Commission Internationale d'Eclairage) color space coordinates, the color histogram and the HSV counting the red, blue, and green color bands of the object detection area using the value, L (brightness), A (green to red complementary colors), and B (yellow to blue complementary colors) values are extracted from the CIE (Commission Internationale d'Eclairage) color space coordinates, and LAB values and the RGB values of the color space coordinates After converting the object detection area including the count value of the band into a gray scale image, the threshold value of the road safety sign object is generated through a preset threshold algorithm (Otsu Algorithm), and then the generated threshold value After classifying the object and the foreground region based on It includes processes for determining the shape (including color) of an object.

Thereafter, when the S740 process is completed, the ID allocating unit 150 assigns a detection ID for identifying the road safety sign object detected in the image frame analyzed in the S740 process to the image frame (S750).

Next, when the S750 process is completed, the object tracking unit 160 compares and determines whether the road safety sign object detected in the before/after image frame of the road driving image is matched, and if it is matched, it is assigned to the road safety sign object ID and detailed information lane information are provided in the form of a queue (S760).

More specifically, in the step S760, the object analyzer 140 confirms whether a road sign object is detected in the n+1th image frame and whether an ID is assigned, and then the center of the nth image frame and the center of the n+1th image frame. After calculating the optimal value of the offset vector of It includes a series of processes of storing the road safety sign object ID, segment, object detection area (bounding box) and lane information of the stacked image frames in a queue form and providing it to an external server.

Therefore, by using the method and system for recognizing/extracting traffic safety signs for map generation with precision according to an embodiment of the present invention, the detection error is minimized by subdividing the detection area of the road safety sign object detected in the road driving image. , it provides the advantage of improving the identification of the photographed traffic safety signs and early detection of detection error information.

13 is a diagram illustrating an example computing environment in which one or more embodiments disclosed herein may be implemented, and is an illustration of a system 1000 including a computing device 1100 configured to implement one or more embodiments described above. shows For example, computing device 1100 may be mounted on a vehicle, and may include personal computers, server computers, handheld or laptop devices, mobile devices (mobile phones, PDAs, media players, etc.), multiprocessor systems, consumer electronics, including, but not limited to, minicomputers, mainframe computers, distributed computing environments including any of the aforementioned systems or devices, and the like.

The computing device 1100 may include at least one processing unit 1110 and a memory 1120 . Here, the processing unit 1110 may include, for example, a central processing unit (CPU), a graphic processing unit (GPU), a microprocessor, an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Arrays (FPGA), etc. and may have a plurality of cores. The memory 1120 may be a volatile memory (eg, RAM, etc.), a non-volatile memory (eg, ROM, flash memory, etc.), or a combination thereof. Additionally, computing device 1100 may include additional storage 1130 . Storage 1130 includes, but is not limited to, magnetic storage, optical storage, and the like. The storage 1130 may store computer readable instructions for implementing one or more embodiments disclosed herein, and other computer readable instructions for implementing an operating system, an application program, and the like. Computer readable instructions stored in storage 1130 may be loaded into memory 1120 for execution by processing unit 1110 . Computing device 1100 may also include input device(s) 1140 and output device(s) 1150 .

Here, the input device(s) 1140 may include, for example, a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, or any other input device, or the like. Further, the output device(s) 1150 may include, for example, one or more displays, speakers, printers, or any other output device, or the like. Also, the computing device 1100 may use an input device or an output device included in another computing device as the input device(s) 1140 or the output device(s) 1150 . Computing device 1100 may also include communication connection(s) 1160 that enable computing device 1100 to communicate with another device (eg, computing device 1300 ).

Here, communication connection(s) 1160 may be a modem, network interface card (NIC), integrated network interface, radio frequency transmitter/receiver, infrared port, USB connection, or other for connecting computing device 1100 to another computing device. It may include interfaces. Also, the communication connection(s) 1160 may include a wired connection or a wireless connection. Each component of the above-described computing device 1100 may be connected by various interconnections such as a bus (eg, peripheral component interconnection (PCI), USB, firmware (IEEE 1394), optical bus structure, etc.) and may be interconnected by the network 1200 . As used herein, terms such as "component," "system," and the like, generally refer to a computer-related entity that is hardware, a combination of hardware and software, software, or software in execution.

For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. For example, both an application running on a controller and a controller may be a component. One or more components may reside within a process and/or thread of execution, and components may be localized on one computer or distributed between two or more computers.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

100: Traffic safety sign recognition/extraction system for map generation with precision
110: input unit
120: lane detection unit
130: road safety sign object detection unit
140: object analysis unit
150: ID allocator
160: object tracking unit

Claims (20)

an input unit for receiving a road driving image;
a lane detection unit that detects a lane in an image frame of a road driving image using a deep learning algorithm, and then analyzes and classifies information on the detected lane;
An image frame in which the lane is detected according to a preset detection area condition value after generating an integrated object detection area by merging and expanding respective detection areas of at least two or more deep learning algorithms that detect image objects in different detection methods a road safety sign object detection unit for detecting a road safety sign object located in the integrated object detection area within; and
A traffic safety sign recognition/extraction system for creating maps with precision including an object analysis unit that analyzes detailed shapes based on the feature points extracted according to the color, saturation, and brightness of the detected road safety sign object. According to claim 1,
Further comprising a detection ID allocation unit for allocating a detection ID for identifying the road safety sign object detected in the image frame to the image frame According to claim 1,
After comparing and judging whether the road safety sign object detected in the before/after image frame is matched, if it is matched, the ID assigned to the road safety sign object, detailed information and lane information are provided in the form of a queue. Traffic safety sign recognition/extraction system for generation. According to claim 1,
The lane detection unit
After detecting a lane using a deep learning algorithm, the traffic safety sign recognition / extraction system. According to claim 1,
The object detection unit
A traffic safety sign recognition/extraction system for map creation with precision, including an object detection area adjustment unit that overlaps each object detection area of the deep learning object detection algorithm and creates the integrated object detection area exceeding a preset overlap value. 6. The method of claim 5,
The object detection unit
A traffic safety sign recognition/extraction system for map generation with precision, including an object detection area filtering unit that filters the identification of road safety sign objects detected in the integrated object detection area with a preset detection area condition value. 7. The method of claim 6,
The preset detection area condition value is
A traffic safety sign recognition/extraction system for creating a map with precision including a road surface sign area, a center line left area, and a curb right area. 6. The method of claim 5,
The object detection unit
When the overlapping value of each object detection area of the deep learning object detection algorithm is smaller than the preset overlapping value,
A traffic safety sign recognition/extraction system for map generation with precision that detects the road safety sign object with a deep learning object detection algorithm with high detection accuracy among the deep learning object detection algorithms. According to claim 1,
The object analysis unit
After extracting the outline of the road safety sign object detected by the object detection unit, the road safety sign for map creation with precision to analyze at least one of the shape and color of the road safety sign object based on the position of the feature point of the outline Recognition/extraction system. 4. The method of claim 3,
The object tracking unit
After confirming whether a road sign object is detected in the n+1th image frame and whether an ID is assigned from the object analysis unit, the optimum value of the offset vector of the center of the nth image frame and the center of the n+1th image frame is calculated , a traffic safety sign recognition/extraction system for map generation with precision that determines whether the road sign object matches or not based on the optimal value of the offset vector. 11. The method of claim 10,
The object tracking unit
After stacking the image frames in which the same road safety sign object is detected, if the stacking value exceeds the standard value, the precision of storing and transmitting the road safety sign object ID, segment, object detection area and lane information of the stacked image frames in the form of a queue Traffic safety sign recognition/extraction system for road map generation. After detecting a lane in the image frame of the road driving image using a deep learning algorithm, analyzing and classifying information on the detected lane;
After creating an integrated object detection area by merging and expanding the object detection areas of two different deep learning object detection algorithms, the integrated object detection area is located in the image frame in which the lane is detected according to a preset detection area condition value. Detecting a road safety sign object located;
analyzing the image of the road safety sign object based on the feature points extracted according to the color, saturation, and brightness of the detected road safety sign object;
allocating a detection ID for identifying a road safety sign object detected in the image frame to an image frame; and
After comparing and judging whether the road safety sign object detected in the before/after image frame is matched, if it is matched, the ID assigned to the road safety sign object, detailed information, and lane information are provided in the form of a queue. A method of recognizing/extracting traffic safety signs for 13. The method of claim 12,
The step of analyzing and classifying the detected lane information includes:
After detecting a lane using a deep learning algorithm, classifying the lane into any one of a dotted line, a solid line, a dedicated lane, and a center line from the road markings and average width between the detected lanes, traffic safety for map generation with precision Cover recognition/extraction method. 13. The method of claim 12,
The step of detecting the road safety sign object is
Traffic safety for map creation with precision comprising the step of overlapping each object detection area of the first deep learning object detection algorithm and the second deep learning object detection algorithm to create the integrated object detection area exceeding a preset overlap value Cover recognition/extraction method. 15. The method of claim 14,
The step of detecting the road safety sign object is
A method of recognizing/extracting a road safety sign for map creation with precision, comprising the step of filtering the identification of the road safety sign object detected in the integrated object detection area with a preset detection area condition value. 16. The method of claim 15,
The preset detection area condition value is
A method of recognizing/extracting a road safety sign for map generation with precision including a road sign area, a center line left area, and a curb right area. 13. The method of claim 12,
The step of detecting the road safety sign object is
When the overlapping value of each object detection area of the deep learning object detection algorithm is smaller than the preset overlapping value,
A method of recognizing/extracting a road safety sign for map creation with precision, comprising the step of detecting the road safety sign object with a deep learning object detection algorithm with high detection accuracy among the deep learning object detection algorithms. 13. The method of claim 12,
The step of analyzing the image of the road safety sign object is
After extracting the contour of the detected road safety sign object, based on the position of the feature point of the contour, analyzing at least one or more of the shape and color of the road safety sign object Traffic safety sign for map creation with precision Recognition/extraction method. 13. The method of claim 12,
The step of comparing and judging whether the road safety sign object is matched
After confirming whether a road sign object is detected in the n+1th image frame and whether an ID is assigned from the object analysis unit, the optimum value of the offset vector of the center of the nth image frame and the center of the n+1th image frame is calculated , a method of recognizing/extracting a traffic safety sign for map generation with precision, which is a step of determining whether the road sign object is matched or not based on the optimal value of the offset vector. 11. The method of claim 10,
The step of providing in the form of the queue is
After stacking the image frames in which the same road safety sign object is detected, if the stacking value exceeds the reference value, storing and transmitting the road safety sign object ID, segment, object detection area and lane information of the stacked image frames in the form of a queue A method of recognizing/extracting traffic safety signs for map generation with high precision.

Images