KR20100064140A - Method for detecting traffic sign board in intelligent vehicle and system for executing the method - Google Patents

Abstract

PURPOSE: A method for detecting traffic sign board and a system for executing the method are provided to control a background region and to reinforce the color contrast feature of an image. CONSTITUTION: A traffic sign board detection system inputs a sign board shape and a symbol shape of a traffic sign board installed on a real road into a database(S101). A color-to-feature map and an outline feature map are calculated in an input image(S105). A color tone contrast feature map and contour line feature map are used. An image enhanced map is generated using the color-to-feature map and outline feature map(S106). A traffic sign board within the input image is detected using the image enhanced map(S109).

Description

TECHNICAL FOR DETECTING TRAFFIC SIGN BOARD IN INTELLIGENT VEHICLE AND SYSTEM FOR EXECUTING THE METHOD}

The present invention relates to a traffic sign detection system, and more particularly to a traffic sign detection method for realizing the development of a safe driving assistance system for providing traffic sign information and an intelligent vehicle system capable of unmanned driving. It is about the system.

As vehicle movements become more common, road signs are installed on the roads to provide information to drivers of vehicles for smooth driving and safe driving.

In general, traffic signs may be provided with traffic information signs, milestone signs, weather information boards, air pollution boards, traffic accident boards, municipal information boards and billboards.

Recently, as the number of drivers with low vision and older drivers increases, the necessity of developing a safe driving assistance system that accurately transmits traffic sign information is increasing.

In addition, in order to develop an intelligent driverless vehicle, the development of a vision-based traffic sign recognition system is required together, and accordingly, attention has been focused on a traffic sign detection and recognition model.

However, the existing traffic sign detection model uses traffic due to changes in weather and illumination, blurring of images due to camera movement and sudden change in color contrast, and changes in shape due to occlusion and tilting of traffic signs. Poor detection of signs.

The present invention provides a traffic sign detection method and system capable of providing a more enhanced traffic sign detection and recognition model in preparation for a change in the shape of the traffic sign.

Traffic sign detection method according to an embodiment of the present invention comprises the steps of calculating a color contrast feature map including the color information and the contour feature map including the contour information in the input image; Generating an image enhancement map using the color contrast feature map and the contour feature map; And detecting a traffic sign in the input image by using the image enhancement map.

The calculating of the color contrast feature map may include extracting at least one color information of red information, green information, blue information, and yellow information from the input image, and sampling at least one resolution of the extracted color information. Generating a color pyramid image of the camera; and generating the color contrast feature map based on the generated color pyramid image.

The calculating of the contour feature map may include extracting at least one color information of red information, green information, blue information, and yellow information from the input image, and comparing the color by comparing the extracted color information. Calculating information, extracting contour information using the calculated color contrast information, generating a contour pyramid image by sampling a resolution of the extracted contour information, and generating the contour pyramid image. It may include the step of generating the contour feature map.

The traffic sign detection method according to an embodiment of the present invention may further include configuring a sign shape and a sign shape of the traffic sign as a database, and detecting the traffic sign in the input image comprises: Selecting a sign candidate region from an image reinforcement map, extracting shape information of the selected sign candidate region using the contour pyramid image, and extracting the image using a sign shape and a symbol form configured in the database Recognizing a traffic sign that matches the shape information may be included.

The traffic sign detection system according to an embodiment of the present invention includes a feature map calculator for calculating a feature map including color information and a contour feature map including contour information in an input image; An enhancement map generator configured to generate an image enhancement map using the color contrast feature map and the contour feature map; And a sign detector configured to detect a traffic sign in the input image by using the image reinforcement map.

According to the present invention, it is possible to provide a traffic sign detection model for changing the image due to various factors by enhancing the color contrast characteristics of the image and suppressing the background area.

Therefore, by suggesting a traffic sign detection model with excellent detection performance, it is possible to realize the development of an intelligent driverless vehicle capable of real technical support in the development of a safe driving assistance system and recognizing traffic signs.

Hereinafter, a traffic sign detection method and a system for executing the method according to an embodiment of the present invention with reference to the accompanying drawings will be described in detail.

The present invention can provide a traffic sign detection model for more accurately detecting a traffic sign in an image by enhancing a color contrast characteristic of the image and at the same time suppressing a background area of the image.

1 is a flowchart illustrating the entire process of a traffic sign detection method according to an embodiment of the present invention.

Referring to Figure 1, the traffic sign detection method of the present invention may include the following process.

In step S101, the traffic sign detection system may first configure a sign shape and a sign form of a traffic sign installed on an actual road in order to recognize a traffic sign in an input image. General traffic signs may be classified into regulatory signs, indication signs, caution signs, and the like, and may express text to be conveyed by text or graphic symbols. The step S101 of configuring the database may store and maintain a sign shape of the traffic sign, classify the traffic sign according to the sign shape, and store the sign shape of the traffic sign corresponding to each sign shape. I can keep it.

In operation S102, the traffic sign detection system may extract color information of the input image.

2 is a diagram for describing a process of extracting color information of an input image.

Referring to FIG. 2, in the extracting of the color information (S102), the input image 201 is reduced to a predetermined size (eg, 160 * 120) in order to reduce the processing time. In operation 202, initial red information R, initial green information G, and initial blue information B may be extracted. Subsequently, extracting the color information (S102) normalizes the initial red information (R), the initial green information (G), and the initial blue information (B) 203 to reduce noise of the input image 201. By doing so, the normalized red information (r), green information (g), blue information (b), and yellow information (y) 204 can be extracted.

The normalized red information (r), green information (g), blue information (b), and yellow information (y) may be defined as in Equation (1).

In step S103, the traffic sign detection system compares the normalized red information r, green information g, blue information b, and yellow information y to compare color contrast information rg_by 205 with each other. The contour information 206 of the calculated color contrast information rg_by 205 may be extracted.

The color contrast information (rg_by) 205 may be defined as in Equation 2.

In addition, the contour information 206 may be calculated by applying a Sobel operator or a Canny operator to the color contrast information rg_by.

In step S104, the traffic sign detection system performs a seven-level Gaussian pyramid image (hereinafter, referred to as 204) for the red information (r), green information (g), blue information (b), and yellow information (y) (204). , A 'color pyramid image' may be generated, and a 7-level Gaussian pyramid image (hereinafter, referred to as an “contour pyramid image”) may be generated with respect to the contour information 206.

3 is a diagram for describing a process of constructing a Gaussian pyramid image for contour information.

Referring to FIG. 3, after generating a first layer having level 1 through the Gaussian filtering on the contour information 301, the first layer is Gaussian filtered along with zoom-out to have a level 2 of 2. Create the first layer. As described above, the seventh layer having the level 7 may be generated while repeatedly reducing and Gaussian filtering. That is, the contour pyramid image 302 composed of seven layers may be generated from the contour information 301, and the red information r, green information g, blue information b, and yellow may be generated in the same manner. The color pyramid image composed of seven layers may be generated for the information y.

In operation S105, the traffic sign detection system may calculate a contour feature map representing contour characteristics of the input image using the contour pyramid image, and use the color pyramid image to display a color contrast characteristic of the input image. Contrast feature maps can be calculated.

First, the calculating of the contour feature map may include selecting a third layer to a seventh layer from the contour pyramid image generated in step S104, and then performing zoom-in and Gaussian filtering on the selected layer. While repeating as described above, the third to seventh layers are restored to a predetermined size (for example, 160 * 120).

)을 산출할 수 있다.The contour feature map (Equation 3 and Equation 4) using the reconstructed third to seventh layers 303 may be used.

) Can be calculated.

는 상위 레벨 레이어의 윤곽선 정보,

는 하위 레벨 레이어의 윤곽선 정보,

는 상위 레벨 레이어와 하위 레벨 레이어 간 윤곽선 정보를 의미한다.Where s is the upper level layer, c is the lower level layer,

Is the outline information of the top level layer,

Is the outline information of the lower level layer,

Denotes contour information between the upper level layer and the lower level layer.

는 레이어 간의 픽셀 별 덧셈을 의미한다.here,

Means pixel-by-pixel addition between layers.

)은 3번째에서 7번째까지의 레이어(303) 간 윤곽선 정보를 모두 합산함(

)으로써 산출할 수 있다.In other words, the contour feature map (

) Sums up all the contour information between the 3rd to 7th layers 303 (

Can be calculated as

)을 산출할 수 있다.On the other hand, the step of calculating the color contrast feature map is similar to the process of calculating the contour feature map to select a layer from the third to the seventh layer from the color pyramid image generated in the step (S104) to a predetermined size After reconstructing the color contrast feature map (Equation 5 through Equation 7)

) Can be calculated.

는 하위 레벨 레이어의 적색 정보,

는 상위 레벨 레이어의 녹색 정보,

는 하위 레벨 레이어의 녹색 정보,

는 상위 레벨 레이어의 적색 정보,

는 상위 레벨 레이어와 하위 레벨 레이어 간 적색 정보와 녹색 정보의 대비 값을 의미한다.Where s is the upper level layer, c is the lower level layer,

Is the red information for the lower level layer,

Is the green information for the top level layer,

Is the green information for the lower level layer,

Is the red information for the top level layer,

Denotes a contrast value of red information and green information between a higher level layer and a lower level layer.

는 하위 레벨 레이어의 청색 정보,

는 상위 레벨 레이어의 황색 정보,

는 하위 레벨 레이어의 황색 정보,

는 상위 레벨 레이어의 청색 정보,

는 상위 레벨 레이어와 하위 레벨 레이어 간 청색 정보와 황색 정보의 대비 값을 의미한다.At this time,

Is the blue information of the lower level layer,

Is the yellow information for the top level layer,

Is the yellow information for the lower level layer,

Is the blue information of the higher level layer,

Denotes a contrast value of blue information and yellow information between a higher level layer and a lower level layer.

는 레이어 간의 픽셀 별 덧셈을 의미한다.here,

Means pixel-by-pixel addition between layers.

)은 3번째에서 7번째까지의 레이어(303) 간 색상 대비 정보 즉, 적색 정보와 녹색 정보의 대비 값과 청색 정보와 황색 정보의 대비 값을 모두 합산함(

)으로써 산출할 수 있다.In other words, the color contrast feature map (

) Sums the color contrast information between the third to seventh layers 303, that is, the contrast values of the red and green information and the contrast values of the blue and yellow information (

Can be calculated as

In operation S106, the traffic sign detection system may generate an image enhancement map using the color contrast feature map and the contour feature map.

)과 윤곽선 특징 맵(

)에 대해 수학식 8을 적용함으로써 상기 영상 강화 맵(

)을 생성할 수 있다.Referring to FIG. 4, the image enhancement map 403 may be generated by synthesizing the color contrast feature map 401 and the contour feature map 402. The color contrast feature map (

) And contour feature maps (

By applying Equation 8 to the image enhancement map (

) Can be created.

는 상기 색상 대비 특징 맵에 대한 가중치,

는 상기 윤곽선 특징 맵에 대한 가중치를 의미할 수 있다.At this time,

Is a weight for the color contrast feature map,

May mean a weight for the contour feature map.

In step S107, the traffic sign detection system may select the sign candidate region 404 from the image enhancement map 403 generated in step S106.

In the selecting of the sign candidate region 404 (S107), a candidate region is searched by first searching a pixel corresponding to the maximum energy of the region in the window while moving a window having a predetermined size in the image enhancement map. Can be selected. Subsequently, in the step S107 of selecting the sign candidate region 404, an area size of the sign candidate region is performed by performing an entropy maximization algorithm based on region growing for each selected sign candidate region. (Ie, a sign candidate area) can be calculated.

The area size s (scale (X)) for the sign candidate site X may be defined as shown in Equation (9).

)와 너비(

)를 구함으로써 산출할 수 있다. 이때, 상기 영역 크기(S)는 수학식 10의 조건(

)을 만족시켜야 한다.The area size S for the sign candidate site X is the height with maximum entropy

) And width (

Can be calculated by At this time, the area size (S) is a condition (10)

) Must be satisfied.

Here, D means a set of descriptor values.

)는 수학식 11과 같이 정의하며, 상기 너 비(

)는 수학식 12와 같이 정의할 수 있다.And the height (

) Is defined as in Equation 11, and the width (

) May be defined as in Equation 12.

는 상기 영역 크기(S)를 나타내는 크기(s)와, 상기 표지판 후보지(X)를 나타내는 위치(x)와, 상기 디스크립터 값(D)을 나타내는 값(d)으로 이루어진 확률 질량 함수(probability mass function)를 의미할 수 있다.here,

Is a probability mass function consisting of a size (s) representing the area size (S), a position (x) representing the sign candidate site (X), and a value (d) representing the descriptor value (D). May mean.

In step S108, the traffic sign detection system may extract shape information on the sign candidate region using the contour pyramid image generated in step S104.

5 is a view for explaining a traffic sign recognition process.

Referring to FIG. 5, in the extracting of the shape information (S108), the first to third layers are selected from the contour pyramid image, and then, in each of the selected layers, the region 501 corresponding to the sign candidate region is selected. The contour information 502 may be extracted and an external shape of the extracted contour information 502 may be determined, that is, the shape information. For example, the external shape of the contour information 502 may be extracted by filling the inner region of the contour information 502 with a black color.

In operation S109, the traffic sign detection system may recognize a traffic sign that matches the shape information extracted in the step 108 by using the shape of the sign and the shape of the sign stored in the database.

Recognizing the traffic sign (S109) may perform template matching with the sign shape and the sign shape stored in the database on the extracted shape information to determine whether the sign candidate area corresponds to the actual traffic sign. Can be recognized.

Here, when the matching value is greater than a predetermined threshold in the contour information for each layer of the first to third layers, and the matching value of the shape information is larger than other matching values, it is determined that the traffic sign is a traffic sign and the template matching is terminated.

At this time, if the matching value of all the signs of the database and the shape information extracted from a specific layer is less than or equal to the threshold, another layer is selected and template matching is performed again. On the other hand, if the matching value of all the signs of the database and the shape information extracted from all layers is less than the threshold value, it can be determined that the traffic signs are not.

Therefore, based on the shape information stored in the database, it is possible to recognize whether the sign candidate area corresponds to a traffic sign through template matching with the shape information corresponding to the sign candidate area.

The above traffic sign detection method can be executed by the traffic sign detection system shown in FIG.

6 is a block diagram showing the internal configuration of a traffic sign detection system according to an embodiment of the present invention.

Referring to FIG. 6, the traffic sign detection system 600 of the present invention may include a database 604, a feature map calculator 601, a reinforcement map generator 602, and a sign detector 603. have.

The database 604 stores and maintains a sign shape and a sign shape of a traffic sign installed on an actual road.

The feature map calculator 601 may calculate a color contrast feature map including color information and an outline feature map including outline information in an input image.

7 is a diagram illustrating an example of the configuration of the feature map calculator 601.

Referring to FIG. 7, the feature map calculator 601 includes a color extractor 701, an outline extractor 702, an outline pyramid generator 703, an outline map generator 704, and a color. The pyramid generator 705 and the color map generator 706 may be configured.

The color extractor 701 extracts initial red information (R), initial green information (G), and initial blue information (B) from the input image, and then the initial red information (R), initial green information (G), The normalized blue information (B) is normalized to extract the normalized red information (r), green information (g), blue information (b), and yellow information (y).

The contour extracting unit 702 may calculate color contrast information rg_by by comparing the normalized red information r, green information g, blue information b, and yellow information y with each other. Contour information may be extracted using the calculated color contrast information rg_by. In this case, the contour extractor 702 may be configured as a filter using a Sobel operator or a Canny operator.

The contour pyramid generator 703 may generate a contour pyramid image composed of seven layers by gradually sampling the resolution of the contour information extracted by the contour extractor 702.

The contour map generator 704 may restore the third to seventh layers of the contour pyramid image to a predetermined size, and then generate the contour feature map using the restored third to seventh layers. have.

The color pyramid generating unit 705 samples the resolutions of the red information r, the green information g, the blue information b, and the yellow information y extracted by the color extraction unit 701 in steps, respectively. A color pyramid image composed of seven layers can be generated.

The color map generator 706 selects a third to seventh layer in each color pyramid image for the red information r, green information g, blue information b, and yellow information y. After reconstructing to a predetermined size by comparing the red information (r), green information (g), blue information (b) and yellow information (y) of the layer corresponding to the same level to calculate the color contrast feature map have.

6, the enhancement map generator 602 may serve to generate an image enhancement map using the color contrast feature map and the contour feature map.

The sign detector 603 may serve to detect a traffic sign in the input image by using the image enhancement map.

8 is a diagram illustrating an example of the configuration of the sign detector 603.

Referring to FIG. 8, the sign detector 603 may include a candidate selector 801, a shape extractor 802, and a matching performer 803.

The candidate selecting unit 801 may select a sign candidate region expected as a traffic sign from the image reinforcement map generated by the reinforcing map generating unit 602.

The shape extractor 802 may extract shape information of the sign candidate region using the contour pyramid image generated by the contour pyramid generator 703. The shape extractor 802 selects a first layer to a third layer in the contour pyramid image, extracts contour information of a region corresponding to the sign candidate region from each of the selected layers, and corresponds to the extracted contour information. Shape information can be determined.

The matching performer 803 may recognize a traffic sign that matches the shape information extracted through the shape extractor 802 using the sign shape and the symbol shape stored in the database 604. The matching performer 803 may recognize whether the sign candidate area corresponds to a real traffic sign by matching the extracted shape information with a sign shape and a sign shape stored in the database 604.

Accordingly, the present invention generates an image enhancement map through the contour feature map representing the contour characteristics and the color contrast feature map representing the color contrast characteristics, and recognizes traffic signs from the image enhancement map, thereby blurring the climate and illumination, blurring the image, and the like. Traffic sign detection models can be provided for changes in color contrast, occlusion and skew of traffic signs.

The traffic sign detection method according to the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

1 is a flowchart illustrating the entire process of a traffic sign detection method according to an embodiment of the present invention.

2 is a diagram for describing a process of extracting color information and contour information of an input image.

3 is a diagram for describing a process of constructing a Gaussian pyramid image for contour information.

4 is a diagram for describing a process of generating an image enhancement map.

5 is a view for explaining a traffic sign recognition process.

6 is a block diagram showing the internal configuration of a traffic sign detection system according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating an example of a configuration of the feature map calculator of FIG. 6.

8 is a diagram illustrating an example of a configuration of a sign detector of FIG. 6.

<Explanation of symbols for the main parts of the drawings>

601: feature map calculator

602: reinforcement map generation unit

603: sign detection unit

604: database

Claims (13)

Calculating a color contrast feature map including color information and an outline feature map including contour information in the input image; Generating an image enhancement map using the color contrast feature map and the contour feature map; And, Detecting a traffic sign in the input image using the image enhancement map Traffic sign detection method comprising a. The method of claim 1, Computing the color contrast feature map, Extracting at least one color information of red information, green information, blue information, and yellow information from the input image; Generating at least one color pyramid image by sampling a resolution of the extracted color information; Generating the color contrast feature map using the generated color pyramid image Including, traffic sign detection method. The method of claim 1, Computing the contour feature map, Extracting at least one color information of red information, green information, blue information, and yellow information from the input image; Calculating color contrast information by comparing the extracted color information; Extracting contour information using the calculated color contrast information; Generating a contour pyramid image by sampling the resolution of the extracted contour information; Generating the contour feature map using the generated contour pyramid image Including, traffic sign detection method. The method of claim 1, Generating the image enhancement map, And synthesizing the color contrast feature map and the contour feature map to generate the image enhancement map. The method of claim 3, Steps to construct a sign shape and symbol form for a traffic sign into a database Traffic sign detection method further comprising. The method of claim 5, Detecting a traffic sign in the input image, Selecting a candidate candidate area in the image enhancement map; Extracting shape information on the selected sign candidate region using the contour pyramid image; Recognizing a traffic sign that matches the extracted shape information by using a sign shape and a sign shape configured in the database; Including, traffic sign detection method. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 1 to 6. A feature map calculator configured to calculate a color contrast feature map including color information and an outline feature map including outline information in an input image; An enhancement map generator configured to generate an image enhancement map using the color contrast feature map and the contour feature map; And, Sign detection unit for detecting a traffic sign in the input image using the image enhancement map Traffic sign detection system comprising a. The method of claim 8, The feature map calculator, A color extraction unit configured to extract at least one color information of red information, green information, blue information, and yellow information from the input image; A color pyramid generator for generating at least one color pyramid image by sampling the resolution of the extracted color information; A color map generator for generating the color contrast feature map using the generated color pyramid image Including, traffic sign detection system. The method of claim 8, The feature map calculator, A color extraction unit configured to extract at least one color information of red information, green information, blue information, and yellow information from the input image; An outline extraction unit for calculating color contrast information through comparison between the extracted color information and extracting outline information using the calculated color contrast information; A contour pyramid generator for generating a contour pyramid image by sampling the resolution of the extracted contour information; Contour map generator for generating the contour feature map through the generated contour pyramid image Including, traffic sign detection system. The method of claim 8, The reinforcement map generation unit, And generating the image enhancement map by synthesizing the color contrast feature map and the contour feature map. The method of claim 10, Database that stores the signs and symbol shapes for traffic signs Traffic sign detection system further comprising. The method of claim 12, The sign detection unit, A candidate selecting unit which selects a sign candidate region from the image enhancement map; A shape extracting unit extracting shape information on the selected sign candidate region using the contour pyramid image; Matching performing unit for recognizing the traffic sign that matches the extracted shape information by using the sign shape and symbol shape stored in the database Including, traffic sign detection system.

Images