KR20200002257A - Corner detection-based road sign detecting method and apparatus - Google Patents

Abstract

According to the present invention, disclosed are a method for detecting a road sign for measuring a precise location of a vehicle and an apparatus thereof. According to one embodiment of the present invention, the method for detecting a road sign comprises the steps of: detecting a plurality of vertices in an image obtained from a camera; generating a candidate road sign through the geometric relationship between the plurality of vertices; removing a candidate road sign based on mis-detected vertices by verifying the detected vertices; removing a mis-detected candidate road sign by verifying the remaining candidate road sign; and incorporating adjacent candidate road signs to determine a road sign.

Description

Road sign detection method and apparatus based on vertex detection {CORNER DETECTION-BASED ROAD SIGN DETECTING METHOD AND APPARATUS}

The present invention relates to an apparatus and method for detecting a road sign, and more particularly, to an apparatus and a method for detecting a road sign using vertex detection in order to measure a precise position of a vehicle.

Accurate estimation of vehicle position is essential for advanced driver assistance systems and autonomous driving. In particular, finding out exactly which lane a car is traveling on is very important because it can be used in a variety of applications, such as accurate road guidance, precise traffic analysis, and forward alerts via communication.

Satellite navigation-based methods, such as GPS, are difficult to precisely measure the position because the satellite radio waves are distorted by various causes before reaching the positioning terminal. Among other causes, cooperative positioning (DGNSS) and Satellite Based Augmentation Systems (SBAS) have been developed to minimize the effects of local distortion by sharing and compensating local distortion information to overcome distortion caused by the influence of the atmosphere. , GBAS (Ground Based Augmentation System), RTK (Real Time Kinematic). On the other hand, GNSS / INS was introduced to fuse satellite navigation and inertial navigation system (INS) to overcome the reflection of satellite radio waves by tall buildings and the blocking of underground roads. However, the GNSS / INS terminal, which can secure the level of accuracy by car in the city center where the skyscrapers, underground roads and elevated roads are larger and more numerous, is very expensive, and it is practically impossible to apply them to mass production. For this reason, there is a growing interest in low-cost sensor fusion-based precision positioning system that utilizes landmarks that can provide location information without relying on satellite radio waves.

By detecting a landmark and estimating three-dimensional information with an environmental sensor mounted on the vehicle, the relative position of the vehicle can be known, and when matched with the landmark on a precise map, the global position of the landmark can be known. have. At this time, it can be divided into distance information based method, image feature point based method, and road facility based method according to the type of sensor and landmark used.

Here, the distance-based method and the image feature point-based method have a disadvantage in that a lot of information must be stored and matched in a precise map in common. The road facility-based method detects road facilities such as road markings and road signs from images acquired by cameras and compares them with previously registered road facility information. As road facilities are managed by the transportation authorities, they are relatively small and the amount of information to be stored on precise maps is much smaller. When using video, it is preferable to monocular cameras rather than stereo cameras because of the cost, and generally prefers to be embedded in a multifunctional front camera module.

It is possible to estimate the relative positions of cars and road signs by taking advantage of the fact that road signs on highways are installed 5m above the ground in accordance with applicable laws. At this time, there are some problems when the road sign detection method is applied to a highway. Since highways have much more various types of road signs than downtown roads, the performance of the road sign detection method that detects all road signs at once can be greatly degraded. In particular, since the aspect ratio of the road sign varies, it is difficult to set the reference point as well as the detection. In addition, the method of detecting the entire road sign has a problem that it is not possible to use the road sign at a close range.

According to an embodiment of the present invention, a vertex is detected in a camera image to detect a road sign using a combination of vertices, thereby providing a method for detecting road signs having various aspect ratios.

According to another embodiment of the present invention, a method for verifying detected vertices and road signs is provided in order to minimize false detection causing serious problems in positioning accuracy.

According to another embodiment of the present invention, even if the distance between the road sign and the car is narrowed, there is provided a method for detecting the road sign in close proximity.

According to one embodiment, a road sign detection apparatus for a vehicle includes a processor; And a memory coupled to the processor. At this time, the processor detects the vertices of the plurality of road signs from the image obtained from the camera, generates a candidate road sign through the geometric relationship between the plurality of vertices, and based on the vertices detected incorrectly by verifying the detected vertices. By removing the road signs, verifying the remaining candidate road signs, removing the misdetected candidate road signs, integrating adjacent candidate road signs to determine the road signs, and applying Kalman filter to the vertices of the determined road signs. Can be traced

According to an embodiment of the present disclosure, the processor may continuously track the bottom vertex even when the top vertex of the road sign disappears.

According to one embodiment, when generating a candidate road sign through the geometric relationship between the plurality of vertices, the processor is a non-detection vertex of the intersection of two straight lines that are parallel to the two sides of the three vertices and cross both ends vertex. Assuming that a candidate road sign can be generated.

According to an embodiment of the present disclosure, when determining a road sign by integrating adjacent candidate road signs, the processor may determine an overlapping road sign when an IOU (Intersection over Union) between two candidate road signs is greater than or equal to a predetermined threshold.

According to one embodiment, the processor, HOG-SVM score when the HOG-SVM score used in the step of verifying the width or candidate road signs between the two candidate road signs of the overlapping signs is more than a predetermined value, HOG-SVM score Can remove small signs.

According to an embodiment of the present disclosure, if the width of the two candidate road signs and the HOG-SVM score used in the verification of the candidate road signs are similar among the overlapping signs, the processor may select one of the two candidate road signs with the smaller height. Can be removed.

According to an embodiment, when the processor applies a Kalman filter to track the vertex of the determined road sign, when only some of the four vertices are detected, the distance between the predicted vertex and the detected vertex is less than or equal to a predetermined value. The vertices not detected may be corrected by utilizing the nearest vertex.

According to another embodiment, a method for detecting road signs of a vehicle may include detecting a plurality of vertices from an image obtained from a camera, generating a candidate road sign through a geometric relationship between the plurality of vertices, and verifying the detected vertices. Removing candidate road signs based on erroneously detected vertices, removing false detected candidate road signs by verifying remaining candidate road signs, and incorporating adjacent candidate road signs to determine road signs. It may include.

According to an embodiment, the road sign detection method may further include tracking by applying a Kalman filter to a vertex of the determined road sign.

According to an embodiment of the present disclosure, the tracking may continuously track the bottom vertex even when the top vertex of the road sign disappears.

According to an embodiment, the generating of the candidate road sign may include generating a candidate road sign by assuming that an intersection of two straight lines parallel to two sides of three vertices and passing through both ends of the vertex is an undetected vertex.

According to an embodiment, the determining of the road sign by integrating the adjacent candidate road signs may be determined as an overlapping road sign when the intersection over union (IOU) between two candidate road signs is greater than or equal to a predetermined threshold.

According to one embodiment, the step of determining the road sign by integrating the adjacent candidate road signs, the HOG-SVM score used in the step of verifying the width or candidate road signs between two candidate road signs of the overlapping signs is predetermined If the difference is more than the value, a sign with a small HOG-SVM score can be removed.

According to one embodiment, the step of determining the road sign by integrating the adjacent candidate road signs, if the HOG-SVM score used in the width of the two candidate road signs and the verification step for the candidate road signs of the overlapping signs are similar In addition, one of the two candidate road signs may be removed.

According to the embodiments of the present invention, it is possible to precisely measure the position of a vehicle regardless of the presence of skyscrapers and overpasses in the presence of road signs. In addition, there is no limit of accumulating errors in the measurement position with time, and a large-capacity precision map is not required as compared with the precision map based position measurement method.

According to embodiments of the present invention, since more road signs of various aspect ratios can be detected, more road signs can be utilized to calculate the precise position of the vehicle.

According to embodiments of the present invention, even if the road signs are not close to the full road signs can be used to track the vertices, it is possible to use the road signs at close range.

According to embodiments of the present invention, the processing speed of the road sign detection and tracking is very fast, real-time implementation is possible.

The present invention can be easily applied to an existing system using a pre-mounted lane keeping front camera or a black box camera.

1 shows an example of a road sign on a highway.
FIG. 2 is a diagram for describing a method of measuring a longitudinal vertical distance between a vehicle and a road sign.
3 is a view for explaining a method of measuring the horizontal vertical distance between the vehicle and the road sign.
4 is a flowchart illustrating a road sign detection method according to an embodiment of the present invention.
5 is a view for explaining an example of applying a road sign detection method according to an embodiment of the present invention.
6A is a diagram for describing a method of estimating a vertex when there is an undetected vertex.
6B is a diagram for explaining an example of an undetected vertex.
7 is a view for explaining a method of verifying a candidate road sign according to an embodiment of the present invention.
8 is a diagram illustrating a method of determining a road sign by integrating candidate road signs according to an embodiment of the present invention.
9 is a view for explaining a vertex tracking method according to an embodiment of the present invention.
10 is a view for explaining an embodiment of a vertex tracking method according to an embodiment of the present invention.
11 is a block diagram illustrating a road sign detection apparatus according to an embodiment of the present invention.

Specific structural or functional descriptions of the embodiments according to the inventive concept disclosed herein are merely illustrated for the purpose of describing the embodiments according to the inventive concept, and the embodiments according to the inventive concept. These may be embodied in various forms and are not limited to the embodiments described herein.

Embodiments according to the inventive concept may be variously modified and have various forms, so embodiments are illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments in accordance with the concept of the present invention to specific embodiments, it includes all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are only for the purpose of distinguishing one component from another, for example, without departing from the scope of the rights according to the inventive concept, the first component may be called a second component, Similarly, the second component may also be referred to as the first component.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and are not construed in ideal or excessively formal meanings unless expressly defined herein. Do not.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

1 shows an example of a road sign on a highway.

Referring to Figure 1, it can be seen that there are various types of road signs on the highway. Road signs on highways have varying aspect ratios, making it difficult to set reference points as well as detection. In this case, the relative position between the road sign and the car can be estimated by using the fact that the road sign of the highway is installed 5 m above the ground in accordance with the related laws.

FIG. 2 is a diagram for describing a method of measuring a longitudinal vertical distance between a vehicle and a road sign.

When the bottom vertex of a road sign is detected, the relative position between the car and the road sign can be calculated using pre-calculated camera internal and external parameters and road sign installation rules.

Referring to FIG. 2A, a side view of the camera 210 and the road sign 220 may be confirmed. Here, the optical axis of the camera 210 may be assumed to be parallel to the ground. If the optical axis and the ground are not parallel, it can be changed in parallel using the camera external variables. Referring to FIG. 2 (b), it can be seen that the triangle indicated by the dotted line in FIG. 2 (a) is enlarged.

In this case, by using Equations 1 and 2, the vertical vertical distance Z from the camera to the vertex of the road sign can be calculated.

Where Hp is the height of the sign, Hc is the height of the camera, and Z is the longitudinal distance from the camera to the sign. In addition, v means the vertical coordinate of the lower end point of the sign taken in the image, ov means the vertical coordinate of the camera principal point, f is the focal length in pixels.

In FIG. 2 (b), Hp is fixed to 5m by definition, Hc, ov, and f can be measured in advance through a camera calibration process, and v can be known by detecting the lower corner of the road sign.

3 is a view for explaining a method of measuring the horizontal vertical distance between the vehicle and the road sign. 3 (a) and 3 (b) are views showing a bird's eye view of the road sign 320 and the camera 310 from the sky.

In FIG. 3 (a), X represents the lateral distance from the camera to the bottom vertex of the road sign. Figure 3 (b) shows in detail the triangle shown by the dotted line of Figure 3 (a). In FIG. 3 (b), u denotes the horizontal coordinate of the lower end point of the road sign captured in the image, and ou denotes the horizontal coordinate of the camera pub. Therefore, the lateral distance from the road sign end point to the own vehicle to which the camera is attached can be calculated by the following Equations 3 and 4 below.

Here, the effect ΔZ of the error Δv of the v-coordinate on the longitudinal distance Z can be obtained through Equation 5 below. At this time, when Δv is constant, it can be seen that the size of ΔZ is proportional to the size of Z. Therefore, it can be seen that detecting the vertex when the road sign is as close as possible can improve the accuracy of the relative position between the car and the road sign.

4 is a flowchart illustrating a road sign detection method according to an embodiment of the present invention.

Referring to FIG. 4, in operation S410, the road sign detection apparatus may detect a plurality of vertices from an image obtained from a camera. In this case, the camera may be included in the road sign detection apparatus, or may be a black box camera or a front camera mounted on a vehicle.

In operation S420, the road sign detection apparatus may generate a plurality of candidate road signs through the geometric relationship between the plurality of vertices.

In this case, even when one of the four vertices is not detected, the candidate road sign may be generated by predicting the undetected vertex. According to an embodiment, the road sign detection apparatus may generate a candidate road sign by assuming that an intersection of two straight lines that are parallel to two sides forming three vertices and passing both vertices is an undetected vertex.

The process of generating the candidate road sign will be described in more detail with reference to FIGS. 6 and 7.

In operation S430, the apparatus for detecting road signs may remove candidate road signs based on erroneously detected vertices by verifying the detected vertices. This can be applied to the HOG-SVM-based verification method on the image around the vertices to detect the erroneously detected vertex, and can remove the candidate road sign based on the erroneously detected vertex.

In operation S440, the road sign detection apparatus may remove the false road candidate road sign through verification of the remaining candidate road signs. At this time, the entire candidate road sign may be made into a small image, and HOG-SVM-based verification may be applied thereto.

In operation S450, the road sign detection apparatus may determine adjacent road signs by integrating adjacent candidate road signs.

In this case, the apparatus for detecting road signs according to an embodiment may determine that the road signs are overlapping when the intersection over union (IOU) between two candidate road signs is greater than or equal to a predetermined threshold.

According to another embodiment, the road sign detection apparatus, HOG-SVM when the HOG-SVM score used in the verification step for the width or the candidate road sign between the two candidate road signs of the overlapping signs differ by more than a predetermined value, HOG-SVM Signs with a small score can be removed.

According to another embodiment, the road sign detection apparatus, if the width of the two candidate road signs among overlapping signs and the HOG-SVM score used in the verification step for the candidate road signs are similar, the smaller of the two candidate road signs Can be removed.

The process of determining the road sign will be described in more detail with reference to FIG. 8.

In operation S460, the road sign detection apparatus may apply the Kalman filter to track the vertex of the determined road sign. In this case, the road sign detection apparatus may continuously track the bottom vertex even when the top vertex of the road sign disappears.

According to an exemplary embodiment, when the road sign detection apparatus tracks a vertex of the determined road sign by applying a Kalman filter, when only some of four vertices of the road sign are detected, the distance between the predicted vertex and the detected vertex is predetermined. When the value is less than or equal to the value, the undetected vertex may correct the vertex using the nearest vertex.

The process of determining the road sign will be described in more detail with reference to FIG. 9.

5 is a view for explaining an example of applying a road sign detection method according to an embodiment of the present invention.

Referring to FIG. 5, the vertex detection step (corresponding to FIG. 5A and step S410 of FIG. 4) may detect vertices of a camera image with a vertex detector.

According to an exemplary embodiment, the apparatus for detecting a road sign finds a case in which a geometrical relationship between vertices detected in a sign candidate generation step (corresponding to step S420 of FIG. 5 (b) and FIG. 4) satisfies the candidate road sign. Can be generated. Referring to FIG. 5 (b), it can be seen that a lot of road signs which are still misdetected remain.

According to an embodiment, the road sign detection apparatus vertices that are incorrectly detected by applying HOG-SVM based verification to the image around the vertices in the vertex verification step (corresponding to FIG. 5 (c) and step S430 of FIG. 4) and You can remove candidate road signs based on this. At this time, since the candidate road sign generation method is simple and can be performed at high speed, the method of verifying the vertex after generating the candidate road sign is more efficient than when generating the candidate road sign after verifying the vertex.

According to an embodiment, the road sign detection apparatus makes the entire candidate road sign in a relatively small image in the sign verification step (corresponding to step S440 of FIG. 5 (d) and FIG. 4), and the HOG-SVM-based verification Can be applied. The reason for using the low resolution image is that the contours of the road signs are common but the contents are different from each other, so that only the contour-oriented image is used for verification.

According to an embodiment, the road sign detection apparatus may integrate sign detection results adjacent to each other through NMS (Non-Maximum Suppression) in a sign integration step (corresponding to step S450 of FIG. 5 (e) and 4). have. At this time, in consideration of the case where there is another sign at the bottom of the sign, it is possible to select the lowest vertex of the useful bottom vertex.

According to an embodiment, the road sign detection apparatus may track by applying a Kalman filter to the vertex in the vertex tracking step (corresponding to step S460 of FIG. 5 (f) and FIG. 4). In this case, even if the top vertex of the sign is not visible, the bottom vertex is continuously tracked so that the relative position of the bottom vertex may be estimated even when the road sign is very close to the camera.

6A is a diagram for describing a method of estimating a vertex when there is an undetected vertex.

In the road sign detection apparatus, the candidate road sign generation step may generate candidate road signs by combining three or four vertices. The method using four vertices may generate a rectangle formed of the detected four vertices as candidate road signs. The method of using three vertices may select one of the three types of vertices detected and generate parallelograms formed as candidate road signs. That is, as shown in FIG. 6A, two intersections 621 and 622 formed by three detected vertices 611, 612, and 613 are parallel (623, 624) and push the intersection 614 of two straight lines passing through the vertices of both ends. Assume the detection vertex. In this way, candidate road signs can be generated even when one of the four vertices is undetected.

6B is a diagram for explaining an example of an undetected vertex.

Referring to FIG. 6B, although there is one undetected vertex 630 in the image of the road sign, an example of generating a candidate road sign by predicting the undetected vertex 630 using the remaining three vertices can be seen.

7 is a view for explaining a method of verifying a candidate road sign according to an embodiment of the present invention.

According to an exemplary embodiment, the road sign detection apparatus may remove strange shapes among candidate road signs by utilizing geometric relationships between vertices. Referring to FIG. 7, candidates whose eight angles (α1, α2, α3, α4, β1, β2, β3, β4) and aspect ratio of the rectangle are measured and whose values are not within the range of the training data described in Table 1 below. Can be removed.

In addition, three-dimensional information about the four vertices of the signs can be restored, and candidates whose distance between the two bottom vertices or the sign height are less than 1.0 m can be removed.

8 is a diagram illustrating a method of determining a road sign by integrating candidate road signs according to an embodiment of the present invention.

Referring to FIG. 8, in the case where the signs passing through the road sign verification step overlap each other, the road sign detection device may leave only the optimal sign through the NMS and remove the rest. First, it may be determined that an overlapping sign that has an IOU (Intersection over Union) between two detected signs is greater than or equal to a threshold. If the overlapping signs do not have similar widths or HOG-SVM scores used in the sign verification step, it may be determined that they are overlapped by misdetection, and thus the signs with small HOG-SVM scores may be removed. If the widths of the two signs are similar and the HOG-SVM scores are similar, it may be determined that the overlap occurs due to a small sign attached to the bottom of the sign as shown in FIG. This is to accurately find the lowest coordinate of the sign and to accurately measure the relative distance to the sign based on this lower coordinate.

9 is a view for explaining a method for tracking a vertex according to an embodiment of the present invention.

Referring to FIG. 8, the road sign detection apparatus may track vertices constituting the road sign based on the Kalman filter after the detection of the road sign. When a road sign approaches a car, the sign is not detected because two or more vertices are not included in the image. However, accurate distance estimation between cars and road signs requires detection of the vertices of the road signs as close as possible, so that road signs must be accurately tracked even when only some vertices are detected in the image.

According to an exemplary embodiment, the road sign detection apparatus may track four vertices constituting the road sign with a Kalman filter using a general constant velocity model. If the quadrangle consisting of the prediction of each vertex tracking and the IOU of the newly detected sign is more than a predetermined threshold, it may be determined that the tracked road sign and the detected road sign are associated. In this case, the road sign tracking may be corrected by utilizing the vertex of the detected road sign. Even if the road sign association fails, if the number of failures is within a certain number of times, the vertex and the vertex tracking can be updated with the prediction alone.

At this time, when the sign comes in close proximity, the top vertex prediction value of the road sign disappears out of the image. In this situation, the road sign level association is unsuccessful, so we can directly associate the tracked vertex with the detected vertex result. That is, when the distance between the predicted value of the tracked vertex and the detected vertex is less than or equal to the threshold, it may be determined that it is associated with the nearest vertex, and the vertex track may be corrected by using the associated vertex.

If you rely only on the detection of road signs, you could detect the vertices only when the distance between the car and the road sign is about 13 m or more.However, if you use the tracking of road signs, the distance between the car and the road sign is about 8 m. Could detect the vertex of the road sign. At this time, considering that the three-dimensional restoration error of the vertex is proportional to the distance, it can be seen that the positioning error can be significantly reduced.

10 is a view for explaining an embodiment of a vertex tracking method according to an embodiment of the present invention.

Referring to FIG. 10, it can be seen that the road sign detection apparatus according to an embodiment may detect signs having various aspect ratios, colors, and guide phrases. According to an embodiment, the road sign detection apparatus may detect a sign even in a situation where the contrast between the road sign and the background is reduced by a background such as a soundproof wall or a tree.

11 is a block diagram illustrating a road sign detection apparatus according to an embodiment of the present invention.

Referring to FIG. 11, the road sign detection apparatus 1100 may include a camera 1110, a processor 1120, and a memory 1130. In this case, the camera 1110 is not an essential component of the road sign detection apparatus 1100, and may receive and use a camera image acquired from the outside. In this case, the processor 1120 may include a central processing unit (CPU) or other virtual machine capable of executing a computer program. The memory 1130 may include a nonvolatile storage device such as a fixed hard drive or a removable storage device. The removable storage device may include a compact flash unit, a USB memory stick, or the like. The memory 1130 may also include volatile memory such as various random access memories. Program instructions executable by the processor 1120 may be stored in the memory 1130.

According to an embodiment, the processor 1120 of the road sign detection apparatus of an autonomous vehicle detects vertices of a plurality of road signs in an image obtained from a camera, and detects candidate road signs through a geometric relationship between the plurality of vertices. Generate candidate road signs based on erroneously detected vertices by verifying the detected vertices, remove candidate road signs detected by verifying remaining candidate road signs, and integrate adjacent candidate road signs. Road signs can be determined.

The processor 1120 of the road sign detection apparatus 1100 according to an embodiment may track by applying a Kalman filter to a vertex of the road sign.

In this case, the processor 1120 may continuously track the bottom vertex even when the top vertex of the road sign disappears.

When the processor 1120 of the road sign detecting apparatus 1100 according to an embodiment generates a candidate road sign through the geometric relationship between the plurality of vertices, the processor 1120 may be parallel to two sides of three vertices and cross two vertices. A candidate road sign can be generated by assuming the intersection of a straight line as an undetected vertex.

When the processor 1120 of the road sign detection apparatus 1100 according to an embodiment determines a road sign by integrating adjacent candidate road signs, when the intersection over union (IOU) between two candidate road signs is greater than or equal to a predetermined threshold, the processor 1120 overlaps. Judging by road signs.

The processor 1120 of the road sign detection apparatus 1100 according to an exemplary embodiment may have a HOG-SVM score used in a verification step for a candidate road sign or a width between two candidate road signs among the overlapping signs more than a predetermined value. If they are present, signs with a small HOG-SVM score can be removed.

The processor 1120 of the road sign detection apparatus 1100 according to an embodiment may determine that the two candidates are similar when the width of the two candidate road signs and the HOG-SVM score used in the verification of the candidate road signs are similar. Small road signs can be removed.

When the processor 1120 of the road sign detection apparatus 1100 according to an embodiment tracks by applying a Kalman filter to the determined vertex of the road sign, when only some of the four vertices are detected, the predicted vertex and the detected vertex are detected. When the distance is less than or equal to a predetermined value, the undetected vertex may be corrected by utilizing the nearest vertex.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

1100: road sign detection device
1110: camera
1120: Processor
1130: memory

Claims (14)

In the road sign detection apparatus of an automobile,
A processor; And
A memory coupled to the processor,
The processor,
Detecting the vertices of the plurality of road signs in the image obtained from the camera,
Generating a candidate road sign through a geometric relationship between the plurality of vertices,
Removing candidate road signs based on erroneously detected vertices by verifying the detected vertices;
Validate the remaining candidate road signs to remove misdetected candidate road signs,
Integrate adjacent candidate road signs to determine road signs,
Tracking by applying the Kalman filter to the vertex of the determined road sign
Road sign detection device. The method of claim 1,
And the processor keeps track of the bottom vertex when the top vertex of the road sign disappears. The method of claim 1,
When generating a candidate road sign through the geometric relationship between the plurality of vertices, the processor assumes an intersection of two straight lines that are parallel to two sides of three vertices and passes through both ends of the vertex as the undetected vertex. Road sign detection device characterized in that the production. The method of claim 1,
The processor may determine an overlapping road sign when an intersection over union (IOU) between two candidate road signs is greater than or equal to a predetermined threshold when determining a road sign by integrating adjacent candidate road signs. The method of claim 4, wherein
The processor may be configured to remove a sign having a small HOG-SVM score when the HOG-SVM score used in the verification of the width or candidate road sign between two candidate road signs differs by more than a predetermined value. Road sign detection device, characterized in that. The method of claim 4, wherein
If the width of the two candidate road signs of the overlapping signs and the HOG-SVM score used in the verification step for the candidate road signs are similar, the processor removes the smaller ones of the two candidate road signs. Road sign detection device. The method of claim 1,
The processor,
When tracking by applying the Kalman filter to the vertex of the determined road sign, when only a part of the four vertices are detected, when the distance between the predicted vertex and the detected vertex is less than a predetermined value, the undetected vertex is the nearest vertex Road sign detection device, characterized in that for correcting the vertices using. Detecting a plurality of vertices in an image obtained from a camera;
Generating a candidate road sign through the geometric relationship between the plurality of vertices;
Removing candidate road signs based on erroneously detected vertices by verifying the detected vertices;
Removing the misdetected candidate road signs by verifying the remaining candidate road signs; And
Incorporating adjacent candidate road signs to determine a road sign. The method of claim 8,
The road sign detection method further comprises the step of tracking by applying a Kalman filter to the vertex of the determined road sign. The method of claim 9,
The tracking may include continuously tracking the bottom vertex when the top vertex of the road sign disappears. The method of claim 8,
Generating the candidate road sign,
And a candidate road sign is generated by assuming an intersection of two straight lines parallel to two sides of three vertices and passing both vertices as undetected vertices. The method of claim 8,
Determining the road sign by integrating the adjacent candidate road signs,
If the intersection over union (IOU) between two candidate road signs is more than a predetermined threshold, the road sign detection method characterized in that it is determined as overlapping road signs. The method of claim 12,
Determining the road sign by integrating the adjacent candidate road signs,
When the HOG-SVM score used in the verification step for the width or candidate road signs between the two candidate road signs among the overlapping signs differs by more than a predetermined value, the HOG-SVM score is characterized by removing the signs with small How to detect road signs. The method of claim 12,
Determining the road sign by integrating the adjacent candidate road signs,
If the width of the two candidate road signs of the overlapping signs and the HOG-SVM score used in the verification step for the candidate road signs are similar, the road sign detection method is characterized in that to remove the smaller of the two candidate road signs. .

Images