CN105981042A - Vehicle detection system and method - Google Patents

Abstract

The present invention describes a vehicle detection system and method for detecting one or more vehicles in a dynamic varying region of interest, ROI. The system comprises a scene recognition module (101), a road topology estimation module (102), and a vehicle detecting module (103). The scene recognition module is configured for receiving either high exposure image or low exposure image for identifying condition of one or more scenes in a dynamically varying region of interest. The road topology estimation module configured for receiving either high exposure image or low exposure image for determining at least one of a curve, slope and vanishing point of a road in the dynamically varying region of interest. The vehicle detecting module is coupled with the scene recognition module and road topology module for detecting one or more vehicles on the road at night time.

Description

Vehicle detection system and method

field of invention

The present invention relates generally to vehicle detection, and more particularly to a vehicle detection system in low light conditions, such as at night.

Background of the invention

Advanced driver assistance solutions are gradually gaining market share. Forward Collision Warning is one of the applications that warns the driver when the host vehicle is about to collide with the target vehicle ahead. Vision applications detect vehicles ahead during the day and at night and generate warnings based on the calculated time of imminent collision. Forward collision warning systems and other automotive vision applications use different algorithms to detect vehicles in day and night conditions.

However, existing vehicle detection systems are inefficient, inconvenient, and expensive. This requires an efficient and economical vision system and method based on nighttime vehicle detection. Therefore, there is a need for a system that can provide robust vehicle detection, even in low-light conditions, and eliminate false objects in a variety of real-time scenarios.

Conventional vision processing systems lack a wide range of visibility conditions, including rural conditions (dark) and urban conditions (bright). Additionally, the reasons why detection of this class of vehicles is particularly challenging include:

·There are various differences from different positions of the vehicle to the shape of the vehicle lights.

· Vehicles have different lights. For example, brake lights, side lights turned on.

·The lights of the vehicle are not turned on.

· Vehicle detection in urban conditions with so much ambient light mixed in.

• Detection and distance estimation of two-wheelers.

Therefore, it is necessary to provide a vehicle detection system for detecting one or more vehicles on the road at night. There is a need for a robust system that can identify and eliminate false objects, including lights such as street lamps, traffic cones, etc. that closely resemble the shape of vehicle lights, and other confounding light sources; thus providing a high level of accuracy.

overview

One embodiment of the present invention describes a vehicle detection system. The vehicle detection system includes a scene recognition module configured to receive one of a high-exposure image and a low-exposure image for identifying conditions of one or more scenes in a dynamically changing region of interest (ROI), a a road topology estimation module configured to receive one of the high exposure and low exposure images to determine at least one of the curve, slope and vanishing point of the road in a dynamically changing region of interest (ROI), and a vehicle detection module , which is combined with the scene recognition module and the road topology module to detect one or more vehicles on the road.

Another embodiment of the present invention describes a method of detecting one or more vehicles by a vehicle detection system. The method includes: receiving one of the high-exposure image and the low-exposure image by at least one of the scene recognition module and the road topology estimation module, and identifying one or one of the dynamically changing regions of interest (ROI) by the scene recognition module In the case of the above image scene, at least one of the curve, slope and vanishing point of the road in the dynamically changing region of interest (ROI) is determined, and one or more images are processed to detect the dynamically changing region of interest (ROI). ) of one or more vehicles.

Processing one or more images for detecting one or more vehicles in a dynamically changing region of interest (ROI), the steps include: obtaining possible light sources by a segmentation module, removing noise and unwanted light sources by a filtering module information, identify one or more plaques in the filtered image, determine the characteristics of each identified plaque, use at least one pairing logic among the one or more identified plaques in the dynamically changing ROI One or more objects are identified, and one or more pairs of identified plaques are identified and verified.

In one embodiment, one or more identified plaques in two or more pairs of identified plaques are verified by performing at least one method step comprising performing a test on two pairs of identified plaques that share the same patch. Eliminate a pair of identified patches with a smaller width in a block where the column overlap of the two pairs is high or low, and eliminate a pair of identified patches with a larger width in two pairs of identified patches that share the same patch , where the column overlap of the two pairs is neither very high nor low and the middle patches are distributed asymmetrically, eliminating a pair of identification patches whose width and height are smaller than the other pair of identification patches, where the two pairs have columns Overlap without row overlap, eliminate a pair of identified patches with lower intensity, height, and wider width than the other pair of identified patches, eliminate a pair of identified patches with lower intensity than the other pair of identified patches , where two pairs have the same width and height and the column overlap is high, eliminate a pair of identified patches that has a larger width than the other pair of identified patches, where the two pairs have column overlap and row overlap and are asymmetric , eliminating a pair of identification patches having a smaller width than the other pair of identification patches, wherein the two pairs have columns overlapping and symmetrical, eliminating a pair of identification patches placed within the other pair of identification patches, Where two pairs have very little column overlap, a pair of identified patches placed below another pair of identified patches where two pairs have very little column overlap is eliminated.

Brief description of the drawings

The present invention will be explained as follows with reference to the above-described aspects and other features in conjunction with the accompanying drawings, wherein:

Fig. 1 is a block diagram of a vehicle detection system according to an embodiment of the present invention.

Fig. 2 is a block diagram of a vehicle detection system according to an embodiment of the present invention.

Fig. 3 is a captured image of a vehicle in a dynamically changing ROI according to an exemplary embodiment of the present invention.

Fig. 4 is an input frame provided to a scene recognition module or a road topology estimation module according to an embodiment of the present invention.

Fig. 5 is a dynamically changing ROI image collected according to an embodiment of the present invention.

Fig. 6 is a block diagram of a road topology estimation module according to an embodiment of the present invention.

Fig. 7 is a description of a dynamically changing ROI according to an embodiment of the present invention that changes as the slope and curve of the road change.

Fig. 8 is a captured image for segmentation described according to an exemplary embodiment of the present invention.

Fig. 9 is an output image obtained after segmentation described according to an exemplary embodiment of the present invention.

FIG. 10 is a 3×3 matrix for segmenting a color image described according to an exemplary embodiment of the present invention.

Fig. 11 is a filtered output image of a segmented image according to an exemplary embodiment of the present invention.

Figure 12 shows the separation of two merged plaques according to an exemplary embodiment of the present invention.

Fig. 13 is an image according to an exemplary embodiment of the present invention, and each plaque in the image is assigned a different label.

Fig. 14 is a flowchart of a method for preparing a final patch list from dark frames and bright frames according to an embodiment of the present invention.

Fig. 15 is an illustration of multiple images in which blobs are classified to identify headlights, taillights or any other light according to an exemplary embodiment of the present invention.

Fig. 16 is an image shown according to an exemplary embodiment of the present invention, and the blobs in the image are all classified as merged blobs.

Fig. 17 shows a method for identifying valid pairing based on pairing logic according to an embodiment of the present invention.

Fig. 18 shows a method for confirming and verifying plaques according to an embodiment of the present invention.

Fig. 19 is an image of plaque pairs before and after confirmation and verification according to an exemplary embodiment of the present invention.

Fig. 20 illustrates merged lights and/or patches according to an exemplary embodiment of the present invention.

Fig. 21 shows a method for identifying effective plaques in a dynamically changing ROI for detecting two-wheelers according to an embodiment of the present invention.

Fig. 22 is a state mode conversion cycle of a tracking module according to an embodiment of the present invention.

Fig. 23 is a specific example of estimating the distance between the detected vehicle and the host vehicle through the distance estimating module according to the present invention.

FIG. 24 shows estimated distances of detected vehicles according to an exemplary embodiment of the present invention.

FIG. 25 is a flowchart of a method for detecting one or more vehicles through a vehicle detection system according to an embodiment of the present invention.

Detailed Description of the Invention

Embodiments of the present invention will be described in detail with reference to the accompanying drawings. Also, the present invention is not limited to the embodiments described in the present invention. The size, shape, position, quantity and combination of various elements of the device in the present invention are only exemplary, and those skilled in the art can make various modifications without departing from the scope of the present invention. Therefore, the embodiments of the present invention are merely used to more clearly explain the present invention to those skilled in the art to which the present invention pertains. In the drawings, similar components are labeled with similar reference numerals.

Various places in the specification may refer to "a", "an" or "some" embodiments. This does not necessarily mean that each such reference refers to the same embodiment (or embodiments), or that the feature applies to only a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

As used in this application, the singular forms "a", "an" and "the" should also be read to include the plural forms unless expressly stated otherwise. It should be further understood that the terms "comprising", "consisting of", "comprising" and/or "consisting" used in this application enumerate set features, integers, steps, operations, elements and/or components, but Existing or adding one or more other features, integers, step operations, elements, components, and/or combinations thereof is not excluded. It will be understood that when it is described that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or more than one element may be present between the two elements. In addition, "connected" or "coupled" as used in this application may include real-time connected or coupled. The term "and/or" used in this application includes any and all combinations and arrangements of one or more of the associated listed items.

Unless otherwise defined, all terms (including technical terms and scientific terms) used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application discloses. It is further understood that these terms, as defined in commonly used dictionaries, should be understood to have meanings that are consistent with the context of the relevant field, and will not be understood in an idealized or overly formal sense, unless otherwise specified in the application .

The present invention describes a vehicle detection system and a method for detecting vehicles at night under low light conditions. Vehicle detection requires various applications such as forward collision warning system, dynamic high beam assist, high beam automatic control system, etc. The system detects vehicles at night using vehicle lights, such as taillights, as a primary feature. The system provides a robust vehicle detection using a variety of different classifiers and identifies and eliminates false objects.

Fig. 1 is a block diagram of a vehicle detection system according to an embodiment of the present invention. The vehicle detection system 100 includes a scene recognition module 101 , a road topology estimation module 102 and a vehicle detection module 103 . The scene identification module 101 is configured to receive one of the high-exposure image and the low-exposure image to identify the condition of one or more scenes/images in a dynamically changing region of interest (ROI). The scene recognition module determines saturated pixels, brightness, and regional changes in a dynamically changing region of interest (ROI). The road topology estimation module 102 is configured to receive high exposure images or low exposure images to determine one or more road features in a dynamically changing region of interest (ROI), such as road curves, slopes, and vanishing points. In one embodiment, the road topology estimation module is coupled to the scene recognition module for receiving instances of one or more scenes/images in a dynamically changing region of interest (ROI). The vehicle detection module 103 is connected with the scene recognition module 101 and the road topology estimation module 102, and is configured to detect one or more vehicles on the road at night, ie, the vehicle in front or the oncoming vehicle driving in front of the host vehicle. Here, the system works in nighttime conditions on highways and city roads. The system uses vehicle lights as a main feature for vehicle detection and vehicle classifiers.

The image acquisition unit acquires images in dynamically changing regions of interest for providing the same (eg high and low exposure/channels with different gains/exposures) to the system 100 . The image acquisition unit may be a vehicle camera. The input image is then converted to a grayscale image and a color image, and input is provided to the system 100 . The system 100 operates on a region of interest (ROI). The scene recognition module 101 determines scene conditions, such as day and night, dark versus bright night, fog, and rain. The road topology estimation module 102 determines road scenes, such as curves, slopes, and the like. In one embodiment, a dynamic ROI is an ROI that changes with respect to the curve and gradient of the road. The road topology estimation module 102 and scene recognition module 101 receive and process alternating sets of high exposure/gain frames and low exposure/gain frames. The system 100 is connected to an electronic control unit 104 for providing output signals processed by a vehicle detection module 103 . The electronic control unit 104 processes the received signals for displaying or warning the user or driver of the vehicle.

In one embodiment, the road topology estimation module 102 calculates the dynamically changing ROI based on the following parameters:

Saturated pixels

·brightness

· Regional change

·color

In one embodiment, scenes/images in dynamically changing regions are classified as bright if the parameters described above are high through the entire ROI.

Fig. 2 is a block diagram of a vehicle detection system according to an embodiment of the present invention. The vehicle detection module 103 includes an image segmentation module 201 , a filtering module 202 , a blob recognition module 203 , an object recognition module 204 , a pairing confirmation and verification module 205 , a tracking module 207 , and a distance estimation module 208 . In addition, the vehicle detection module 103 also includes a two-wheeler identification module 206 .

The image segmentation module 201 is configured to receive input data from the scene recognition module 101 or the road topology estimation module 102 to provide binary image data. This binary image data includes taillights, headlights, noise and unnecessary information.

The filtering module 202 is coupled to the image segmentation module 201 and is configured to remove noise and unwanted information, such as very small objects, false positives, and similar other information.

The blob identification module 203 is coupled to the filtering module 202 and is configured to identify one or more blobs in the filtered image and then determine the characteristics of each identified blob. The plaque identification module 203 is configured to perform steps including assigning a unique marker to each of the one or more plaques, determining characteristics of each of the one or more labeled plaques, the characteristics including plaque Origin, width, height, box area, pixel area, number of red pixels, aspect ratio, and plaque outline, one or More than one fusion, and classifying the one or more marker patches within at least one of headlight, taillight, merged light, and null light.

The object identification module 204 is coupled to the plaque identification module 203 and is configured to identify objects based on one or more pairing logics. The object recognition module 204 is configured to perform at least one of the following steps including determining a horizontal overlap of one or more blobs, determining an aspect ratio of one or more blobs, determining a pixel area of one or more blobs ratio, determine a width ratio of one or more patches, and determine a pixel frame area ratio of one or more patches.

Pairing verification and verification module 205 is coupled to object recognition module 204 and is configured to verify and verify one or more identified plaque pairs. The pairing confirmation and verification module 205 is configured to perform one or more steps including identifying a pair of identified patches, identifying one or more identified patches between a row of lights, identifying between two or more pairs of Verifying one or more identified patches between pairs of identified patches, and confirming merged light by identifying one or more merged patches. Identifying plaque pairs was determined by performing the following steps, including determining the width and aspect ratio of the pair, determining the number of unpaired plaques in the pair, and determining the paired and unpaired plaque area as a percentage of the pair width.

In one embodiment, identifying one or more identified patches between light rows in the ROI is accomplished by performing a step comprising performing a line matching algorithm on all light rows in the ROI.

Tracking module 207 is coupled to pair confirmation and verification module 205 and is configured to track one or more confirmed and verified plaque pairs in one or more phases. The one or more phases include an idle phase, a pre-tracking phase, a tracking phase, and an untracing phase.

The two-wheeled vehicle identification module 206 is configured to determine that the identified object in the ROI is a two-wheeled vehicle, based on the information of one or more plaques received from the plaque identification module 203 and the pairing confirmation and verification module 205 . The information of one or more blobs includes blobs classified as headlights or taillights in the blob recognition module and blobs passed through the close range rider shape profile classifier. In one embodiment, the blob identification module 203 identifies a single blob in a region of interest (ROI) to determine a two-wheeler.

The tracking module 207 is coupled to the pairing confirmation and verification module 205 and the two-wheeler identification module 206 and is configured to track one or more confirmed and verified plaque pairs/plaques in one or more stages . The one or more phases include an idle phase, a pre-tracking phase, a tracking phase, and/or an untracing phase.

The distance estimation module 208 is configured to calculate the distance between one or more detected vehicles and a host vehicle, based on the product ratio of the actual width of the detected vehicle size and the focal length of the lens, and the width of the detected vehicle in the image and the camera The product of the coefficients for converting pixels to meters.

Fig. 3 is a captured image of a vehicle in a dynamically changing ROI according to an exemplary embodiment of the present invention. The system utilizes vehicle lights as the main feature for vehicle detection and classification. The image acquisition unit is attached to a predetermined position of the vehicle, acquires an image, and provides the acquired image to the system 100 . The system 100 uses a high exposure image (as shown in Figure 3a) and a low exposure image (as shown in Figure 3b). The image acquisition unit provides acquired images as input frames, which include high exposure/gain and low exposure/gain. As shown in Fig. 4, this input frame is provided to the scene recognition module or the road topology estimation module.

Fig. 5 is an image collected according to an exemplary embodiment. The image acquisition unit acquires images and provides them to the scene recognition module. The scene recognition module 101 processes the received image and classifies the image/scene as light/dark/fog/rain if these parameters such as saturated pixels, brightness, color and area variation by ROI are high. The image/scene classifications are accumulated over time and lags are added to determine classification changes.

Fig. 6 is a block diagram of the road topology estimation module 102 according to an embodiment of the present invention. In this embodiment, the road topology estimation module 102 calculates the predicted vanishing point of the determined region of interest. A ROI (Region of Interest) in an image is the area where the user looks for potential vehicles—a road scene ahead without sky areas. The road topology estimation module 102 receives inputs such as offset estimates, vanishing point estimates, pitch estimates, scene recognition, and vehicle extrinsic parameters to determine/estimate road vanishing points.

Dynamic ROIs are ROIs that change with changes in slope and curve about the road, as shown in Fig. 7(a) and (b).

For curve estimation, the road topology estimation module 102 uses the yaw rate and speed of the host vehicle to estimate the curvature ahead.

For slope estimation, the vehicle detection system uses the following cues to determine the slope ahead

Track-based matching/registration to determine offsets between consecutive frames

· Input from LDWS (Lane Departure Warning System)

• Pitch estimation from feature tracking module.

Scene recognition output, such as day or night, etc.

External parameters of the vehicle, such as yaw rate, speed, etc.

The advantages of using a dynamic ROI are as follows:

Vehicles can be detected even on curved roads

· Reduce false positives

Can avoid unnecessary processing, thereby improving system performance

Image segmentation

Fig. 8 is an image collected for segmentation according to an exemplary embodiment of the present invention. In this example, the lights shown in the input low exposure/gain image and high exposure/gain image are segmented using a sliding window based on dual thresholding. The threshold of a 1D fixed/variable-length local window is calculated from the mean of the window pixel values, a predefined minimum value, and a predefined maximum value. The predefined minimum values can be adjusted according to the brightness of the image. For brighter cases, the minimum value of the threshold is raised further, while for darker cases, the threshold is moved to a predefined value. Either a fixed window or a variable-sized window is used to calculate the threshold of the ROI. The pixel values of an image can be corrected based on a threshold. For example, a seven-segmented image is formed by seven different thresholds calculated from different predefined minimum and maximum values. The segmented output is a binary image, as shown in Figure 9.

For color image segmentation, the color image and the segmented input image (obtained as a grayscale image) are used as input. According to the color information of the color image, the size of the region is increased around the segmented light region. The determination of pixel values near the segmented light pixels is based on thresholding the 8-neighborhood red hue, so that the magnitude of the tail light will increase in low light conditions or in far regions. For example, for a 3×3 matrix as shown in FIG. 10, the value of the middle pixel is determined based on the segmented pixel (ie "1") and the color image. Color images should have a red hue, for tail light conditions. Process the bi-level adaptive segmented image and the color image to get the final segmented image.

filtering

The segmented binary image shown in Figure 9 consists of taillights, noise and unwanted information. Use filtering to remove noise and unwanted information. Filtering can be done using morphological operations or median filtering. Morphological operations such as erosion and dilation are used with structuring elements of size three, so it eliminates patches of size smaller than 3×3. Median filtering is designed to eliminate blobs with sizes smaller than 2×3 and 3×2. Scene-based - erosion for brighter scenes and median filtering for darker scenes - filtering is applied to the segmented image. All segmented images based on different thresholds of the scene are filtered. The output of the filtering module is the filtered image, as shown in Figure 11. After filtering the image, prominent groups of segmented pixels (plaques) are identified.

Separate merged plaques:

The system 100 also includes sub-modules for separating merged patches - two taillights/headlights, taillights/headlights and other lights, taillights/headlights and reflected light, etc. - in the filtered image. The system 100 uses a two-stage erosion with a 3x3 kernel on the segmented image to identify and separate the two merged patches. The filtered image undergoes the following process to separate the two merged patches, as shown in Figure 12.

If the filtered image has one patch and the two-stage erosion image has two patches at the same location, the patches in the filtered image are broken by vertically cutting the overlapping region in the two patch pairs.

If the filtered image has one patch and the two-level erosion image has no or one patch at the same location, then the patch in the filtered image is kept.

Avoid any alterations if the filtered image has one patch and the two-stage erosion image has more than two patches at the same location.

Plaque Identification

The plaque identification module 203 identifies different types of plaques in the filtered image and computes their features. Perform the following steps to identify plaques:

· Plaque marking

· Plaque feature calculation

· Plaque fusion

· Plaque classification

plaque markers

Fig. 13 is an image according to an exemplary embodiment of the present invention, and each plaque in the image is assigned a different label. Labeling is done using 4-connectivity, where the same label is assigned to a group of pixels if they are connected by 4-connectivity. After assigning tags to each patch, information such as start row, end row, start column, end column, assigned tag, and pixel area are all stored in an array.

plaque characteristics

After assigning labels to each patch, the following features are computed:

• The origin of the patch, indicating whether the patch is from a dark frame or a light frame.

· Width, including the gap between the end column and the start column.

· Height, including the gap between the end row and the start row.

· Box area, which is the product of width and height (ie, width x height)

Pixel area, including the total number of white pixels within the box

The number of red pixels, including the total number of red pixels based on the hue value

Aspect ratio, including min(width,height)/maximum(width,height)

Plaque outline, including plaque shape

plaque fusion

Fig. 14 is a flowchart of a method for preparing a final patch list from dark frames and bright frames according to an embodiment of the present invention. In step 1401, patches of low exposure/gain frames and patches of high exposure/gain frames are received to determine overlap of the patches. If there is no overlap, in step 1402 the blob of the high exposure/gain frame is checked to be a potential taillight blob. If there is no overlap, in step 1402 the blob that passes/allows the low exposure/gain frame is included in the blob list. Patches in low exposure/gain frames can occur due to reflection imaging. If the blob of the high exposure/gain frame in step 1402 is not a potential tail light, the blob is discarded in step 1404 . If the blob of high exposure/gain frame in step 1402 is a potential tail light, then the blob of passed/allowed high exposure/gain frame is included in the blob list at step 1405 . If at step 1401 there is more than one patch that overlaps, at step 1406 the patch that passes/allows the low exposure/gain frame is included in the patch list. In step 1407, a final plaque list is prepared.

The plaque list was prepared based on the following criteria:

• A plaque should be within the estimated field of view area.

• A patch should not have any other overlapping patches below it.

• A blob should not have areas of the same brightness between its adjacent blobs.

• Horizontal ROIs are defined from 35-65% of the total width of potential candidates for determining bright frames.

• Plaques that pass primarily from bright frames are all in pairs. If it is a merged patch (high beam/low beam), it will go through the dark frame. Horizontal overlap between two plaques for pairing. The probability of overlapping the threshold due to the movement of consecutive frames is very low.

· A patch is considered to be from a dark frame if a large patch of a light frame is more than 1 patch of a dark frame.

Plaque classification

Figure 15 is an image showing blobs classified to identify headlights, taillights or any other lights according to an exemplary embodiment of the present invention. Once the final patch list is obtained, the patches have been categorized into headlights, taillights, combined lights and other lights. If the red score (red pixel number), since the blob is larger than a predetermined threshold, then the blob is a taillight. The tail light classification is shown in blue in Fig. 15(a). Plaques are classified as headlamps based on the following criteria:

a) any plaque underlying the plaque due to reflection; and/or

b) when there is horizontal overlap between multiple patches, there is also a height ratio between two patches that is less than half the maximum vehicle width; and/or

c) The patch has a minimum and two maxima near the patch, where the pattern of minimum and maximum values is determined by taking a vertical profile of the particular patch.

After taillight and headlight classification, all headlights with low red scores and small size headlights are removed from the list by marking them as invalid patches. In addition, if any one patch has more than one patch below it, it is also marked as an invalid patch.

To classify the above blobs as merged blobs, patterns such as 101 and 111 are checked, where 0 corresponds to the minimum value position and 1 corresponds to the maximum value position. To determine the pattern, a plaque was divided into three segments, and the minimum and maximum positions of each segment were determined using the filtered image. By using these values, the ratio of the center segment to the left and right segments is determined to check the 101 pattern. For the 111 pattern, the ratio of the left and right segments to the center segment is determined.

In one embodiment, the blob identification module 203 identifies a single blob in a region of interest (ROI) to determine a two-wheeler.

Fig. 16 is an image according to an exemplary embodiment of the present invention, wherein more than one blob is classified as a merged blob, and if a blob is classified as merged and its size is small, the The plaques are considered taillights or dipped beams, based on their respective earlier classifications as taillights or headlights.

pairing logic

Fig. 17 shows the process of identifying valid pairing based on pairing logic according to an embodiment of the present invention. The system 100 also includes a pairing logic module that follows a heuristic-based method to determine a pair of taillights from a plurality of blobs. The pairing process is based on the criteria listed below:

1. Check the horizontal overlap of multiple plaques,

2. Check the aspect ratio of multiple plaques,

3. Check the pixel area ratio of multiple plaques,

4. Check the width ratio of multiple plaques,

5. Check the pixel frame area ratio of multiple plaques,

6. For larger patches, a collation is performed to match the shape of multiple patches. The shapes of the plurality of plaques can be obtained by subtracting the original plaque image from the erosion image. Here, erosion is done with structuring elements of size 3. Cosine similarity is used to check the shape of the plaques.

Cosine Similarity: It measures the similarity between vectors of multiple patches.

Cosine similarity = A.B/||A|| ||B||.

in,

||A||The size of the vector A

||B|| size of vector B

A final confidence score is calculated based on the weighted scores obtained from the above checks. Pairing is performed using scores above a threshold according to the score matrix. The pairing is very basic, using a very low threshold to allow for unbalanced tail lights, side lights turned on, and slightly disproportionate patch pairings.

Pairing as determined above allows dynamic ROI based vehicle width collation. The core of the whole logic is the pre-calculated and loaded dynamic triangle when the system is initialized (the ROI is kept updated according to the camera and vehicle parameters).

The row width of the paired geometric centers should lie between the row widths of the smallest and largest triangles (dynamic ROIs), as shown in Fig. 17(a). The output of the pairing logic (shown in Figure 17(b)) is the pairing of possible vehicles.

Pairing Verification and Verification (V&V)

Pairing confirmation and verification module 205 confirms and verifies paired and merged plaques. The input to module 205 is all possible pairings, as shown in Figure 19(a). The module 205 is divided into two sub-modules, including a pairing confirmation module and a combined optical confirmation module

pairing confirmation

Fig. 18 shows a method for confirming and verifying plaques according to an embodiment of the present invention. Pair confirmation for single pairing, pairing between a row of lights, and pairing between multiple pairs.

Verification of a single pair:

Fig. 18(a) is a method for verifying a single pair of plaques according to an embodiment. In order to verify a single pairing of plaques, the following conditions need to be met:

1. Pair width and aspect ratio

2. Number of unpaired plaques between pairs

3. Area of paired and unpaired plaques as a percentage of paired width

Verification of pairings between a row of lights:

In one embodiment, for the case of a row of lights such as reflected lights or street lights, a pair of patches is required to be verified. In most cases, both the reflected light and the street lights are lined up. Check the line matching algorithm for a row of lights. If a row of lights is aligned and the intersection ratio between consecutive patch pairs is the same, the pairing formed by those lights is invalid.

Verification between pairs:

Figures 18(b) to 18(j) illustrate a method for verifying plaque pairing according to an embodiment. The actual pairing of plaques was determined from the two pairs of plaques in the ROI using the following rules:

1. If two pairs share the same patch and the column overlap is very high or very low, eliminate the pair with smaller width, as shown in Fig. 18(b) and (c).

2. If two pairs share the same patch and the column overlap is not very high or low, and the intermediate patches are distributed asymmetrically, the pair with larger width is eliminated, as shown in Fig. 18(d) and (e).

3. If two pairs have column overlap but no row overlap, and the width and height of the lower pair are smaller than the upper pair, then eliminate the lower pair, while assuming that the lower pair has greater height and strength than the upper pair and a width less than For the upper pair, the upper pair is eliminated, and if the column overlap is high and the width and height are the same, the pair with less strength is eliminated, as shown in Figure 18(f), (g), and (h)

4. If the two pairs have columns and rows overlapping and are asymmetrical, eliminate the wider pair, and assume that the column overlap has good symmetry, then eliminate the smaller width pair, as shown in Figure 18(i) and ( as shown in j)

5. If two pairs of columns overlap very little, and one pair is inside another pair, then eliminate the pairing inside, as shown in Figure 18(k),

6. If two pairs of columns overlap very little, and one pair is below the other, then eliminate the pairing below, as shown in Figure 18(l).

Fig. 19 is an image of plaque pairs before and after confirmation and verification according to an exemplary embodiment of the present invention. Figure 19(a) shows the four identified plaque pairs before validation and verification, while Figure 19(b) depicts three pairs after validation and validation, based on the method described in Figure 18 Validated plaque pairs. After pairing confirmation in Figure 19, pairings that meet the four-wheeler criteria are allowed by the tracking system and the remaining pairings are allowed to be eliminated.

Merge light confirmation

Fig. 20 is a diagram illustrating merged lights/patches according to an exemplary embodiment of the present invention. In one embodiment, the merged lights are the headlights of the distant vehicle and require confirmation. To confirm merging lights, the following criteria are performed:

1. If the merging light has a longitudinal overlap with the four-wheeled vehicle ahead and is below it, it is invalid.

2. If the merged light has a longitudinal overlap with oncoming traffic, the merged light is classified as taillight, noise, or unwanted information, and if the merged light is lower than the oncoming vehicle, the merged light is invalid.

3. If the merged light has longitudinal overlap with a pair of blobs that have a shape matching degree above a first shape matching predetermined threshold, and the merged light score is below a second predetermined threshold. If the merged light has longitudinal overlap with a pair of blobs having a shape match below a first predetermined threshold, the merged light is invalid.

4. If the merged light has vertical overlap and horizontal overlap with the four-wheeled vehicle previously paired, the merged light is invalid.

5. If the merged light has vertical overlap, no horizontal overlap, shape match degree is above a predetermined threshold and the merged patch score is below a predetermined threshold, the merged light is invalid.

6. If in the above case the smaller merged lights fade away, then check the tracking of the merged lights. A merged patch is valid if the merged light has vertical overlap, lateral overlap, area ratio, height ratio and is within a predetermined threshold.

two wheeler detection

Fig. 21 shows a method for identifying effective plaques for detecting two-wheeled vehicles in a dynamically changing ROI according to an embodiment of the present invention. The two-round detection module 206 uses the plaque classification information. Module 206 detects vehicles ahead and oncoming vehicles. Unpaired patches, patches not classified as merged lights, patches classified as headlights or taillights are all considered as possible two-wheelers, and additional checks are performed, such as road slope, Cyclist outlines, plaque moves to confirm that the plaque is a two-wheeler. Checks should also be performed for the validity of events within the space, e.g. LHD's oncoming headlights, the area to the right of the host vehicle has blobs, while for RHD these blobs should be in left area. Additionally, the patches should not have any longitudinal overlap with the pair. These plaques meeting the above conditions are considered a two-wheeler. Fig. 21 lists two examples, where the recognized plaque does not meet the above conditions, therefore, it is an invalid plaque.

tracking system

Fig. 22 is a state mode conversion cycle of a tracking module according to an embodiment of the present invention. The function of the tracking module 207 is divided into four stages, including idle 2201 , pre-tracking 2202 , tracking 2203 , and untracking 2204 . By default, the tracking module/tracking system is in an idle state 2201 . Once the object is identified as a pair of patches (in the case of a four-wheeler)/a patch (in the case of a two-wheeler), a new track is initiated (if no previous matching active track exists), The state changes from idle state 2201 to pre-trace state 2202 . The pre-tracking state 2202 is used to reconfirm the existence of the plaque pair/plaque. The conditions for verifying that a pre-tracking object is a patch pair/plaque and transitioning it to the tracking state 2203 are listed below. A pre-tracked object that has been verified as a patch pair/plaque wants to move to tracked state 2203 only if:

• Detected in "N" frames, which has a good confidence score. The confidence of the patch pair/patch in each frame is obtained from the detection confidence returned by the pairing logic/two-wheeler detection

・It has a high frequency of occurrence

· It has a good mobility score (for four-wheeled vehicles only). A tracking system keeps track of the behavior of the plaque pairs. Here, it is believed that the two plaques are in lock step. Movement in the opposite direction is only permitted in front of the vehicle, ie the vehicle is traveling towards the host vehicle, or away from the host vehicle.

In the tracking state 2203, continuous prediction and updating of the tracked object occurs using the Kalman filter. Any other suitable filter known in the art may be used. If the tracked object is found to be missing in a particular frame, Kalman prediction is used to display the bounding box. And at the same moment, the tracking system changes from tracking state 2203 to untracking state 2204 . In the untrack state 2204, the object is verified as an "M" frame. In addition, the untracking state 2204 attempts to improve the continuity of a good tracking system (i.e. a very high number of frames being tracked) by

Large area search for free-form conformance to motion constraints

· If the environment is very dark, search in high gain/high exposure frames

· If close to vehicle pairing, try to match a patch to extend its life

· If approaching a two-wheeler or four-wheeler, use a classifier to search in the neighborhood

For the cancel tracking state 2204, the pairing confidence of "M" frames is obtained to decide whether to move the tracking system back to the tracking state 2203 or to the idle state 2201 (not a valid pairing).

Therefore, during tracking, pre-tracking and multi-frame confirmation discard false detections, while tracking and untracking states tend to fill the detection gaps of the corresponding objects.

In the pre-tracking state, the observation time window of 'N' frames and in the de-tracking state of 'M' frames is a variable. Different situations make tracking state change decision dynamics, such as patch pair/patch category, patch pair/patch score and movement over time, pair width, intermittent patch pair/patch, curve/slope situation.

distance estimation

FIG. 23 is an example of using the distance estimation module 208 to estimate the distance between the detected vehicle and the host vehicle according to the present invention. The distance estimation module 208 is configured to calculate the distance between at least one detected vehicle and a host vehicle, based on the product ratio of the actual width of the detected vehicle size and the focal length of the lens, and the detected vehicle's distance in the image. The product of the width and the factor that converts the camera's pixels to meters.

In one embodiment, perspective geometry is used to estimate distance. If three pairs of vertices corresponding to vehicles are connected to form three straight lines intersecting at one vertex, two perspective triangles starting from one vertex are formed.

In the perspective method, the distance between the detected vehicle and the host vehicle is estimated using the following formula, and its diagrammatic method is shown in FIG. 23 .

$\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}\mathrm{<span\; class="notranslate">\; k</span>}}$

in,

f: focal length of the lens (mm)

W: the actual width of the vehicle size (meters)

w: the width of the vehicle in the image (pixels)

k: the coefficient to convert the pixels of the CCD camera to meters, and

D: Distance to the target vehicle (meters)

Fig. 24 shows the estimated distance of the final detected vehicle according to an exemplary embodiment of the present invention. The vehicle detection system 100 has detected three vehicles, which are marked with rectangular boxes.

FIG. 25 shows a method for detecting one or more vehicles using a vehicle detection system according to an embodiment of the present invention. In step 2501, a high exposure image or a low exposure image is received in a scene recognition module or a road topology estimation module. In step 2502, the scene identification module identifies one or more scene conditions in a dynamically changing region of interest (ROI). In step 2503, within the dynamically changing region of interest (ROI), at least one of the curve, slope and vanishing point of the road is determined. In step 2504, a filtering module removes noise and unwanted information. In step 2505, one or more blobs in the filtered image are identified. In step 2506, characteristics of each identified plaque are determined. In step 2507, at least one pairing logic is used to identify one or more objects from the one or more patches identified in the dynamically changing ROI. In step 2508, one or more identified plaque pairs are identified and verified.

Although the specific embodiments of the present invention have described the system and method of the present invention in detail with reference to the accompanying drawings, the present invention is not limited thereto. Various substitutions, modifications and variations in plaque identification, plaque classification, pairing logic, and pairing validation and verification will be apparent to those skilled in the art without departing from the scope and spirit of the invention.

Claims (17)

1. Vehicle detection system, including: A scene identification module, configured to receive a high-exposure image or a low-exposure image to identify one or more scenes in a dynamically changing region of interest (ROI); a road topology estimation module for receiving high exposure images or low exposure images to determine at least one of road curves, slopes, and vanishing points in a dynamically changing region of interest (ROI); and The vehicle detection module is combined with the scene recognition module and the road topology module to detect one or more vehicles on the road. 2. The system of claim 1, wherein the vehicle detection module comprises: an image segmentation module for receiving input data from the scene recognition module or the road topology estimation module to provide binary image data; a filtering module, coupled to the image segmentation module, configured to remove noise and unwanted information; a plaque identification module, coupled to the filtering module, configured to identify one or more plaques in the filtered image and determine characteristics of each identified plaque; an object recognition module, coupled to the plaque recognition module, configured to recognize an object based on at least one pairing logic; a pair verification and verification module, coupled to the object recognition module, configured to verify and verify one or more pairs of identified plaques; and A tracking module, coupled to the pair validation and verification module, configured to track one or more confirmed and verified plaque pairs in one or more phases. 3. The system according to claim 1, wherein, further comprising a two-wheeled vehicle identification module configured to determine that the identification object in the ROI is a two-wheeled vehicle, based on the recognition and verification module from the plaque identification module and the pairing confirmation and verification module received information about one or more plaques, The information of the one or more patches includes a headlight patch or a taillight patch. 4. The system of claim 2, wherein the plaque identification module is configured to identify one or more plaques in the filtered image and determine the characteristics of each identified plaque, comprising the steps of: assigning a unique label to each of the one or more plaques; determining characteristics of each of the one or more marked plaques, the characteristics including origin of the plaque, width, height, box area, pixel area, number of red pixels, aspect ratio, and plaque outline; determining one or more fusions of one or more labeled plaques in a dark or light frame based on the determined features; and The one or more marker patches are classified within at least one of headlight, taillight, merged light, and dead light. 5. The system of claim 2, wherein the object recognition module is configured to recognize objects based on at least one pairing logic comprising at least one of the following steps: determining the horizontal overlap of one or more plaques; determining the aspect ratio of one or more plaques; determining a pixel area ratio of one or more plaques; determining the width ratio of one or more plaques; and Determine the pixel box area ratio for one or more patches. 6. The system of claim 2, wherein the pairing confirmation and verification module is configured to confirm and verify one or more identification plaque pairs, comprising at least one of the following steps: Confirmation of a pair of identification plaques; Identify one or more identifying patches across a row of lights; and One or more identified plaques are identified between two or more pairs of identified plaques. 7. The system of claim 6, further comprising identifying merged light by identifying one or more merged blobs. 8. The system of claim 6, wherein identifying a pair of identified plaques comprises Determine the width and aspect ratio of the pair, determine the number of unpaired plaques between pairs, and Determine the area of paired and unpaired plaques as a percentage of the paired width. 9. The system of claim 6, wherein identifying one or more identified patches between light rows in the ROI comprises performing a line matching algorithm on all light rows in the ROI. 10. The system of claim 6, wherein verifying one or more identified plaques in two or more pairs of identified plaques comprises at least once: Eliminate a pair of identified patches with a smaller width among two pairs of identified patches that share the same patch, where the column overlap of the two pairs is very high or very low; Eliminate a pair of identified patches with a larger width among two pairs of identified patches that share the same patch, where the column overlap of the two pairs is not very high or low and the middle patches are distributed asymmetrically; eliminating a pair of identified patches that is smaller in width and height than another pair of identified patches, wherein the two pairs have column overlap and do not have row overlap; eliminating a pair of identified plaques that has a lower intensity, height, and wider width than the other pair of identified plaques; Eliminate a pair of identified patches with lower intensity than another pair of identified patches, where both pairs have the same width and height and the column overlap is high; eliminating a pair of identified patches having a greater width than another pair of identified patches, wherein the two pairs have column overlap and row overlap and are asymmetric; eliminating a pair of identified patches having a smaller width than another pair of identified patches, wherein the two pairs have column overlap and are symmetrical; eliminating a pair of identified patches placed within another pair of identified patches, wherein the two pairs have very little column overlap; and Eliminates a pair of identification patches placed below another pair of identification patches where the two pairs have very little column overlap. 11. The system of claim 1, wherein the scene recognition module determines saturated pixels, brightness, and region changes in a dynamically changing region of interest (ROI). 12. The system of claim 1, wherein the road topology estimation module is coupled to the scene recognition module for receiving instances of one or more scenes in a dynamically changing region of interest (ROI). 13. The system of claim 1, further comprising a distance estimation module configured to calculate a distance between at least one detected vehicle and a host vehicle based on a product ratio of the actual width of the detected vehicle size and the focal length of the lens , and the product of the width of the detected vehicle in the image and the coefficient that converts the pixels of the camera to meters. 14. A method of detecting one or more vehicles by a vehicle detection system, comprising: receiving a high-exposure image or a low-exposure image through at least one of a scene recognition module and a road topology estimation module; Identify the situation of one or more images in a dynamically changing region of interest (ROI) by a scene recognition module; determining at least one of a curve, a slope, and a vanishing point of a road in a dynamically changing region of interest (ROI); and processing one or more images to detect one or more of the road in the dynamically changing region of interest (ROI) above vehicles. 15. The method of claim 14, wherein processing one or more images comprises: Eliminate noise and unwanted information through the filtering module; identifying one or more plaques in the filtered image; determining the characteristics of each identified plaque; identifying one or more objects in the one or more identified patches in the dynamically changing ROI using at least one pairing logic; and Confirmation and verification of one or more pairs of identified plaques. 16. The method according to claim 14, wherein, further comprising determining that the recognition object in the ROI is a two-wheeled vehicle, based on receiving information of one or more plaques from the plaque identification module and the pairing confirmation and verification module , The information of one or more patches includes a headlight patch or a taillight patch. 17. The method of claim 14, further comprising tracking one or more identified and validated plaque pairs/plaques in one or more stages by a tracking module, The one or more phases include an idle phase, a pre-tracking phase, a tracking phase, and an untracing phase.