CN116071713A

CN116071713A - Zebra crossing determination method, device, electronic equipment and medium

Info

Publication number: CN116071713A
Application number: CN202211655518.XA
Authority: CN
Inventors: 马东辉; 潘湖杰; 于乾坤
Original assignee: SAIC Motor Corp Ltd; Shanghai Automotive Industry Corp Group
Current assignee: SAIC Motor Corp Ltd; Shanghai Automotive Industry Corp Group
Priority date: 2022-12-22
Filing date: 2022-12-22
Publication date: 2023-05-05

Abstract

The application discloses a zebra crossing determination method, a zebra crossing determination device, electronic equipment and a medium. The method comprises the following steps: acquiring a first image; wherein the first image comprises zebra crossings; obtaining an interested image containing the zebra stripes based on the road lane lines and the road edge information in the first image; performing contour detection on zebra crossings in the interested image to obtain contour areas of a plurality of contours; clustering the first contour and the second contour to obtain at least one cluster under the condition that the contour area is larger than a preset area threshold value according to each contour area; wherein the plurality of contours includes a first contour and the second contour; performing feature extraction on the gray level image corresponding to each class cluster to obtain a first feature vector corresponding to the class cluster; the zebra stripes are determined based on the first feature vector. The effect of accurately detecting the zebra stripes is realized.

Description

Zebra crossing determination method, device, electronic equipment and medium

Technical Field

The application relates to the technical field of image processing, in particular to a zebra crossing determining method, a zebra crossing determining device, electronic equipment and a zebra crossing determining medium.

Background

Road traffic safety is a problem which cannot be ignored in the development process of the automatic driving technology, and the zebra crossing plays a great role in the aspect of road traffic safety, is mainly used for helping pedestrians to pass through intersections, and is one of traffic signs which need special attention in the driving process of an automatic driving automobile. Therefore, accurate and efficient detection of zebra crossings is critical to automatically driving automobiles. However, the current zebra stripes identification method has the problem of low detection precision.

Disclosure of Invention

The embodiment of the application aims to provide a zebra crossing determining method, a zebra crossing determining device, electronic equipment and a medium, so as to achieve the effect of accurately detecting the zebra crossing.

The technical scheme of the application is as follows:

in a first aspect, a zebra stripes determination method is provided, the method comprising:

acquiring a first image; wherein the first image comprises zebra crossings;

obtaining an interested image containing the zebra stripes based on the road lane lines and the road edge information in the first image;

performing contour detection on zebra crossings in the interested image to obtain contour areas of a plurality of contours;

clustering the first contour and the second contour to obtain at least one cluster under the condition that the contour area is larger than a preset area threshold value according to each contour area; wherein the plurality of contours includes a first contour and the second contour;

Performing feature extraction on the gray level image corresponding to each class cluster to obtain a first feature vector corresponding to the class cluster;

the zebra stripes are determined based on the first feature vector.

In a second aspect, there is provided a zebra crossing determination device, the device comprising:

the first acquisition module is used for acquiring a first image; wherein the first image comprises zebra crossings;

the first determining module is used for obtaining an interested image containing the zebra stripes based on the road lane lines and the road edge information in the first image;

the second determining module is used for carrying out contour detection on zebra crossings in the interested image to obtain contour areas of a plurality of contours;

the third determining module is used for clustering the first contour and the second contour to obtain at least one class cluster under the condition that the contour area is determined to be larger than a preset area threshold value; wherein the plurality of contours includes a first contour and the second contour;

a fourth determining module, configured to perform feature extraction on a gray image corresponding to each class cluster to obtain a first feature vector corresponding to the class cluster;

And a fifth determining module, configured to determine the zebra crossing based on the first feature vector.

In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor, a memory, and a program or an instruction stored on the memory and capable of running on the processor, where the program or the instruction is executed by the processor to implement the steps of the zebra crossing determination method in any one of the embodiments of the present application.

In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions that when executed by a processor perform the steps of the zebra crossing determination method of any of the embodiments of the present application.

In a fifth aspect, embodiments of the present application provide a computer program product, the instructions in which, when executed by a processor of an electronic device, enable the electronic device to perform the steps of the zebra crossing determination method of any of the embodiments of the present application.

The technical scheme provided by the embodiment of the application at least brings the following beneficial effects:

according to the method, the device and the system, the interested image comprising the zebra stripes is obtained based on the road lane line and the road edge information in the obtained first image, then contour detection is carried out on the interested image to obtain contour areas of a plurality of contours, the first contour and the second contour are clustered to obtain at least one class cluster under the condition that the contour areas are larger than the preset area threshold value, the gray level image corresponding to the class cluster is subjected to feature extraction to obtain a first feature vector corresponding to the class cluster, the zebra stripes are determined based on the first feature vector, interference of surrounding environments on the zebra stripes is eliminated through determining the interested image, the rest of vehicles are detected through a clustering algorithm, the influence of the problems of zebra stripes damage, paint dropping and the like on detection accuracy is reduced, the influence of the zebra stripes on detection accuracy is reduced, and the zebra stripes detection accuracy is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application and do not constitute an undue limitation on the application.

Fig. 1 is a schematic flow chart of a zebra stripes determining method according to an embodiment of the first aspect of the present application;

FIG. 2 is a second flow chart of a zebra stripes determination method according to an embodiment of the first aspect of the present application;

fig. 3 is a schematic structural diagram of a zebra crossing determining device according to an embodiment of the second aspect of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of a third aspect of the present application.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are intended to be illustrative of the application and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples consistent with some aspects of the present application as detailed in the accompanying claims.

At present, when the zebra stripes are detected, the characteristics of the zebra stripes such as equal interval, black-white alternation and the like can be utilized, the degree of gray level change of the zebra stripes is represented by a bipolar coefficient, and the areas with obvious gray level change are screened out for detection, but the size of an area search frame in the method directly influences the final detection effect, the phenomenon of zebra stripe omission is easy to occur, and the zebra stripes with insignificant black-white alternation characteristics are difficult to identify.

The method can also be that an original image is firstly converted into a bird's eye view image, then the possible occurrence area of the zebra stripes is judged based on the methods of linear detection of a line segment segmentation detector, histogram peak statistics and the like, and finally the detection is carried out by utilizing the unique black-white jump characteristics of the zebra stripes, but the detection precision of the method is relatively low because the fact that more objects with similar texture characteristics to the zebra stripes exist in the image and are easy to interfere the detection of the zebra stripes is considered.

In summary, the conventional zebra crossing detection method based on the image processing technology relies on the characteristics of manual design and the image preprocessing parameters, is easily affected by the interference of surrounding environment and the shielding of other vehicles, and the problems of zebra crossing self damage, paint dropping and the like, and causes the problem of low detection precision.

In order to solve the problems, the embodiment of the application provides a zebra crossing determining method, device electronic equipment and medium, which are used for obtaining an interested image including a zebra crossing based on road lane and road edge information in an obtained first image, then conducting contour detection on the interested image to obtain contour areas of a plurality of contours, clustering the first contour and the second contour to obtain at least one class cluster under the condition that the determined contour area is larger than a preset area threshold value aiming at each contour area, conducting feature extraction on gray images corresponding to the class cluster aiming at each class cluster to obtain a first feature vector corresponding to the class cluster, determining the zebra crossing based on the first feature vector, so that interference of surrounding environments on the zebra crossing can be removed through determining the interested image, the rest of vehicles are shielded through a clustering algorithm, the intermittent detection on a zebra crossing frame caused by the problems of zebra crossing breakage, paint dropping and the like is reduced, the influence of the rest of the vehicles on detection precision caused by the zebra crossing on the self breakage and the paint dropping and the like is reduced, the zebra crossing detection and the precision is improved.

The zebra stripes determining method provided by the embodiment of the application is described in detail below through specific embodiments and application scenes thereof with reference to the accompanying drawings.

Fig. 1 is a schematic flow chart of a zebra crossing determination method provided in an embodiment of the present application, and as shown in fig. 1, the zebra crossing determination method provided in the embodiment of the present application may include steps 110 to 160.

Step 110, acquiring a first image.

The first image may be an image acquired in real time based on a camera in the vehicle during running of the vehicle, or may be a pre-acquired image acquired from a cloud.

In some embodiments of the present application, zebra stripes may be included in the first image.

In some embodiments of the present application, since the first image is captured during the running process of the vehicle, the captured first image also has road lane line and road edge information, and tall trees, sky, etc. on both sides of the road.

And step 120, obtaining an interested image containing the zebra stripes based on the road lane lines and the road edge information in the first image.

In some embodiments of the present application, an adaptive region of interest method may be employed to delineate an image of interest including zebra crossings based on road lane line and road edge information in the first image.

In some embodiments of the present application, since the zebra stripes are located on the ground, and the road lane lines and the road edge information are also located on the ground, when the region of interest including the zebra stripes is defined, the region located on the ground may be defined as the region of interest according to the road lane lines and the road edge information, and redundant backgrounds (such as the background of the sky, tall trees, etc. that is higher than the ground) in the first image are removed, so as to obtain the image of interest including the zebra stripes.

In some embodiments in the present application, when the region of interest including the zebra stripes is defined, the region of the zebra stripes may be defined by the user's needs, so long as the region of the zebra stripes is defined, and redundant background is removed, which is not limited herein. In this way, the problem of ambient interference is solved by adapting the region of interest.

And 130, performing contour detection on zebra crossings in the interested image to obtain contour areas of a plurality of contours.

In some embodiments of the present application, the contour detection may be performed on the zebra stripes in the image of interest based on a contour detection algorithm, so as to obtain a plurality of contours, and then the contour area of each contour is calculated respectively.

In some embodiments of the present application, when calculating the contour area of each contour, specifically, each contour in the image of interest may be traversed, and the contour area of each contour is calculated according to the length and the width of each contour.

Step 140, clustering the first contour and the second contour to obtain at least one cluster-like structure under the condition that the contour area is larger than a preset area threshold value according to each contour area.

The preset area threshold may be a preset threshold of the contour area, which may be set by a user according to a user requirement, and is not limited herein.

The first profile and the second profile may each be any one of a plurality of profiles. I.e. the plurality of contours may comprise a first contour and a second contour.

In some embodiments of the present application, in the case where it is determined that the area of the contour is greater than the preset area threshold, the reason why the clustering is performed again is that the area of the contour is too small, which may be the area of the contour formed by the road scratch or the road painting, which do not belong to the zebra stripes, may not be preserved.

In some embodiments of the present application, in order to accurately cluster the first contour and the second contour, before step 140, the above-mentioned zebra crossing determination method may further include: acquiring a first center point coordinate of a minimum circumscribed rectangle of the first contour and a second center point coordinate of the minimum circumscribed rectangle of the second contour;

Determining a distance between the first center point coordinates and the second center point coordinates;

the clustering the first contour and the second contour may specifically include:

and clustering the first contour and the second contour under the condition that the distance is smaller than a preset distance threshold.

The first center point coordinate may be a center point coordinate of a minimum bounding rectangle of the first contour.

The second center point coordinate may be a center point coordinate of a smallest bounding rectangle of the second contour.

In the embodiment of the application, through obtaining the first central point coordinate of the minimum circumscribed rectangle of the first profile and the second central point coordinate of the minimum circumscribed rectangle of the second profile, calculate the distance between first central point coordinate and the second central point coordinate, under the condition that the distance is less than the preset distance threshold value, can cluster first profile and second profile, so can link up the intermittent zebra stripes that cause because zebra stripes fall off paint or self damage, solve the zebra stripes intermittent problem that causes because zebra stripes fall off paint or self damage, further promoted the detection precision of zebra stripes.

In some embodiments of the present application, after detecting a plurality of contours, the number of contours may be calculated, if the number of contours is small (for example, may be less than 4, and specific numerical values may be set according to user requirements), the coordinates of the upper left corner and the lower right corner of the minimum circumscribed rectangle of the contours may be directly output, that is, if the number of contours is small, the contours may be directly reserved, and no clustering operation is performed. This is because since the zebra stripes are determined in advance and the detected contour is small, the positions of the zebra stripes are generally detected, and no additional operation is required, thus saving the computing resources.

In some embodiments of the present application, if the number of contours is large, a clustering operation is performed. Before the clustering operation, the number of clustered clusters can be determined, specifically, the number k of the clustered clusters can be determined by adopting an elbow method, namely, the error square sum defined by the loss function under different k values is calculated, and the k value corresponding to the value with the minimum error square sum is determined as the final k value.

In some embodiments of the present application, after clustering is completed to obtain at least one cluster, each cluster, a minimum circumscribed rectangular frame of each cluster, and a gray image corresponding to the cluster may be output.

And 150, extracting features of the gray level images corresponding to the class clusters aiming at each class cluster to obtain a first feature vector corresponding to the class cluster.

The first feature vector may be a feature vector obtained by extracting features of the gray-scale image corresponding to each class cluster.

In some embodiments of the present application, when feature extraction is performed on a gray image corresponding to a class cluster, HOG feature extraction may be performed on a gray image corresponding to a class cluster. Specifically, firstly, an HOG descriptor object is defined, parameters suitable for detecting zebra crossing characteristics are set, including window size, block (block) size, block moving step length, cell (unit) size and the like, then gamma correction and normalization are carried out on a gray image, contrast is adjusted, noise influence is reduced, and the correction and transformation formulas are as follows:

G(x,y)＝F(x,y) ²

Wherein F (x, y) is the original image pixel point, and G (x, y) is the corrected image pixel point.

The corrected image size is adjusted to (128, 64), and the gradient value and direction of each pixel of the image are calculated so as to capture contour information, wherein the calculation formula is as follows:

where gx and gy represent gradient values in the horizontal and vertical directions, respectively.

Counting the number of different gradient values in the gradient histogram in each cell to obtain a feature descriptor of each cell, combining the feature descriptors of all cells in a block to obtain the feature descriptor of the block, and finally combining the feature descriptors of all blocks in an image to obtain the HOG feature vector (namely a first feature vector) of the gray image corresponding to the cluster and the minimum circumscribed rectangular frame thereof.

It should be noted that, the above-mentioned adjustment of the corrected image size to (128, 64) is to meet the requirement of HOG feature extraction on the one hand, and to reduce the image size on the other hand, so as to facilitate subsequent calculation and save calculation resources.

Step 160, determining the zebra stripes based on the first feature vector.

In some embodiments of the present application, the first feature vector may be input into a pre-trained detection model, so as to obtain a zebra crossing.

In some embodiments of the present application, the detection model may be pre-trained based on machine learning, specifically, may be a support vector machine, a random forest, a multi-layer perceptron, and the like, which is not limited herein.

It should be noted that the training process of the detection model is consistent with the existing model training process, and will not be described herein.

In the embodiment of the application, the zebra stripes are determined through the detection model, so that the detection efficiency and the detection precision of the zebra stripes are improved, the cost is reduced, and the vehicle end deployment application is facilitated.

In the prior art, when detecting the zebra stripes, the characteristics of the zebra stripes such as parallelism, black-white alternation and the like can be utilized, the image is preprocessed by extracting the region of interest, each row of the image is traversed, the two-dimensional gray information of the image is converted into one-dimensional signals based on Fourier transformation, and the zebra stripes are detected according to the change of signal quantity.

In order to solve the above-described problem, the first image in the embodiment of the present application may include a first sub-image and a second sub-image. The first sub-image may be taken by a forward looking wide angle camera and the second sub-image may be taken by a forward looking tele camera.

In some embodiments of the present application, where the first image includes a first sub-image, step 110 may include:

acquiring a first sub-image acquired by a forward-looking wide-angle camera;

correspondingly, the step 120 may specifically include:

obtaining an interested image containing zebra crossings based on the road lane lines and the road edge information in the first sub-image;

after obtaining the image of interest including the zebra stripes based on the road lane lines and the road edge information in the first sub-image, the above-mentioned zebra stripes determining method may further include:

performing image processing on the interested image to obtain a second image;

correspondingly, the step 130 may specifically include:

and performing contour detection on the zebra stripes in the second image to obtain the contour area of at least one contour.

The second image may be an image obtained by performing image processing on the image of interest.

In the embodiment of the application, after the first sub-image acquired by the forward-looking wide-angle camera is acquired, an interested image containing the zebra stripes is obtained according to the road lane line and the road edge information in the first sub-image, then the interested image is subjected to image processing to obtain a second image, and then the zebra stripes in the second image are subjected to contour detection to obtain the contour area of at least one contour, so that whether the zebra stripes exist in the first sub-image acquired by the forward-looking wide-angle camera can be accurately determined.

In some embodiments of the present application, in order to further improve accuracy of zebra stripes detection, the image processing on the image of interest to obtain the second image may specifically include:

performing gray level transformation on the interested image to obtain a third image;

denoising the third image to obtain a fourth image;

thresholding the fourth image to obtain a binarized image;

and carrying out morphological processing on the binarized image to obtain a second image.

The third image may be an image obtained by performing gray-scale transformation on the image of interest.

The fourth image may be an image obtained by denoising the third image.

The binarized image may be an image obtained by thresholding the fourth image.

In some embodiments of the present application, after the image of interest is obtained, it may be subjected to gray-scale conversion to obtain a third image.

And firstly, suppressing noise which is subjected to normal distribution by using a Gaussian filter for the third image, performing image smoothing and denoising, and then further eliminating isolated noise points around the image by using a median filter, and keeping the edge characteristics of the image, so that the image is not blurred obviously, and a fourth image is obtained.

Thresholding the fourth image, the specific operation being: firstly, setting a threshold value according to the distribution condition of RGB intensity of each pixel point in the fourth image (specifically, the threshold value can be set according to the requirement), and then dividing a foreground object (such as pedestrians and/or vehicles beside a zebra crossing) of the fourth image from the background according to the threshold value to obtain a binarized image.

Morphological processing is carried out on the binarized image, and the method specifically comprises the following steps: firstly, performing expansion operation on the binarized image for multiple times to fill up tiny holes in the foreground object, then performing corrosion operation on the expanded image to eliminate small particle noise in the image, and finally obtaining a binarized image (namely a second image) with clear outline.

In some embodiments of the present application, by performing image processing on the image of interest, a second image with a clear outline is obtained, so that the outline in the second image is conveniently detected subsequently, and the accuracy of zebra crossing detection is improved.

In some embodiments of the present application, where the first image includes a second sub-image, step 110 may include:

acquiring a second sub-image acquired by a forward-looking remote camera;

after the second sub-image acquired by the forward-looking remote camera is acquired, the zebra stripes determining method can further include:

According to the internal and external parameters of the front-view remote camera, converting the second sub-image into a bird's-eye view angle image;

and replacing the aerial view angle image with the first sub-image to obtain a second characteristic vector corresponding to the aerial view angle image.

Wherein the second feature vector may be a feature vector derived from the second sub-image.

In some embodiments of the present application, after the second sub-image acquired by the front-view remote camera is acquired, the second sub-image may be converted into the bird's-eye view image according to the internal and external parameters of the front-view remote camera, and then the bird's-eye view image is replaced with the first sub-image, and the image processing process, the clustering process and the feature extraction process of the first sub-image are performed to obtain the second feature vector corresponding to the bird's-eye view image.

In the embodiment of the application, the second sub-image acquired by the front-view remote camera is acquired, and is converted into the aerial view angle image according to the internal and external parameters of the front-view remote camera, so that the second feature vector corresponding to the aerial view angle image is obtained.

In some embodiments of the present application, after obtaining the second feature vector, the second feature vector may be input into another pre-trained detection model, and the zebra stripes are detected.

In some embodiments of the present application, the detection model may be trained in advance based on machine learning, and specifically may be a support vector machine, a random forest, a multi-layer perceptron, and the like, which is not limited herein.

In some embodiments of the present application, the detection model corresponding to the first feature vector and the detection model corresponding to the second feature vector may be integrated into one model, that is, step 160 may specifically include:

and inputting the first characteristic vector and the second characteristic vector into a pre-trained zebra crossing detection model to obtain the zebra crossing.

In the embodiment of the application, the first characteristic vector and the second characteristic vector are input into the zebra crossing detection model, so that the zebra crossing can be detected rapidly and accurately, the distance limitation of the zebra crossing detection can be effectively solved by combining the detection results of the two paths of images, the detection efficiency and the detection precision of the zebra crossing are improved, and the generalization performance is high.

In some embodiments of the present application, the zebra crossing detection model may include a first sub-model and a second sub-model.

Wherein the first sub-model may be one of the zebra crossing detection models. Specifically, the detection model corresponding to the first feature vector may be provided.

The second sub-model may be another sub-model of the zebra crossing detection model. Specifically, the detection model corresponding to the second feature vector may be provided.

Inputting the first feature vector and the second feature vector into a pre-trained zebra crossing detection model to obtain a zebra crossing, which specifically may include:

inputting the first characteristic vector into a first sub-model to obtain a first probability value with zebra stripes in a first sub-image;

inputting the second characteristic vector into a second sub-model to obtain a second probability value with a zebra stripes in a second sub-image;

and obtaining the zebra stripes based on the first probability value and the second probability value.

The first probability value may be a probability value of a zebra stripes in the first sub-image obtained by inputting the first feature vector into the first sub-model.

The second probability value may be a probability value of a zebra stripes in the second sub-image obtained by inputting the second feature vector into the second sub-model.

In the embodiment of the application, the first probability value of the zebra stripes in the first sub-image is obtained by inputting the first feature vector into the first sub-model, the second probability value of the zebra stripes in the second sub-image is obtained by inputting the second feature vector into the second sub-model, the zebra stripes are obtained based on the first probability value and the second probability value, the distance limitation of the zebra stripes detection can be effectively solved by combining the detection results of the two paths of images, the generalization performance is high, and the detection efficiency and the detection precision of the zebra stripes are improved.

In some embodiments of the present application, in order to more clearly understand the technical solutions of the present application, another implementation manner of the zebra stripes determination method is provided in the embodiments of the present application, and the zebra stripes determination method provided in the embodiments of the present application may include steps 210-260.

Step 210, reading two paths of camera images of the forward looking wide-angle camera and the forward looking long-distance camera.

And 220, removing redundant background of the first sub-image acquired by the forward-looking wide-angle camera by adopting a self-adaptive region-of-interest method, and only keeping the region of interest.

Compared with the method of directly designating the potential area of the zebra stripes, the method can effectively reduce the influence of environmental noise and furthest reserve the area where the zebra stripes are positioned.

Step 230, a series of image preprocessing operations including graying, filtering, thresholding, morphological transformation (expansion, corrosion) and the like are adopted on the region image of interest, and area screening is performed on the preprocessed image in a contour detection mode, and an area threshold is set to obtain a series of contours larger than the threshold.

And 240, clustering all rectangle center points by using a k-means++ method on the minimum circumscribed rectangle of the obtained series of contours to obtain a series of cluster clusters containing the rectangle center points, merging adjacent cluster clusters, and screening out all clusters to be detected and corresponding minimum circumscribed rectangle images.

And 250, extracting features of the minimum circumscribed rectangular image corresponding to the screened cluster, training by adopting an SVM (support vector machine) based on the extracted features, identifying whether a zebra stripes exist in the image according to a weight file obtained after training, and outputting the position of the minimum circumscribed rectangular frame of the zebra stripes if the zebra stripes exist.

Step 260, for the zebra stripes in the close-range scene, the first sub-image acquired by the forward-looking wide-angle camera is adopted, and the steps 230-250 are executed for detection.

And for the zebra stripes in the long-distance scene, adopting a second sub-image acquired by the forward-looking long-distance camera, generating a corresponding aerial view image based on an inverse perspective transformation IPM method and internal and external parameters of the forward-looking long-distance camera, and then executing the steps 230-250 for detection.

Specifically, the IPM method performs view conversion for a front-view telephoto camera image, and converts a front view into a top view. The specific operation is that firstly, vanishing points of an image plane are calculated through camera internal parameters (focal length and optical center) to obtain a designated image area, then, a mapping relation between a top view and a front view is obtained by combining camera external parameters (pitching angle and height from the ground), and finally, a bird's-eye view image is obtained.

In the embodiment of the application, the detection of the zebra stripes under different distances is realized based on the original image of the forward-looking wide-angle camera and the aerial view image of the forward-looking remote camera, the original image of the forward-looking wide-angle camera is mainly aimed at a close-range scene, the aerial view image of the forward-looking remote camera is mainly aimed at a remote-range scene, the detection distance of the system can be effectively improved by combining the detection results of the two paths of images, the accurate detection of the zebra stripes under the remote-range scene is ensured, and the method is more generalized.

In the embodiment of the application, the traditional image processing and the machine learning technology are combined, the low-cost and high-efficiency zebra crossing detection is realized based on the HOG feature extraction and the SVM method, and the vehicle end deployment application is facilitated.

In the embodiment of the application, the interference degree of surrounding environment on the detection of the zebra stripes is reduced by adopting the pretreatment means such as the self-adaptive interested area and the like, the problem of false detection caused by shielding of other vehicles, self damage of the zebra stripes and paint dropping is solved by adopting the clustering method, and the detection precision of the zebra stripes is effectively improved.

It should be noted that, in the zebra stripes determination method provided in the embodiments of the present application, the execution body may be a zebra stripes determination device, or a control module in the zebra stripes determination device for executing the zebra stripes determination method.

Based on the same inventive concept as the zebra crossing determination method, the application also provides a zebra crossing determination device. The zebra stripes determining device provided in the embodiment of the present application will be described in detail below with reference to fig. 3.

Fig. 3 is a schematic diagram showing a configuration of a zebra stripes determining device according to an exemplary embodiment.

As shown in fig. 3, the zebra stripes determining device 300 may include:

A first acquiring module 310, configured to acquire a first image; wherein the first image comprises zebra crossings;

a first determining module 320, configured to obtain an image of interest including the zebra stripes based on the road lane line and the road edge information in the first image;

a second determining module 330, configured to perform contour detection on zebra crossings in the image of interest, so as to obtain contour areas of a plurality of contours;

a third determining module 340, configured to cluster, for each contour area, the first contour and the second contour to obtain at least one cluster-like group when determining that the contour area is greater than a preset area threshold; wherein the plurality of contours includes a first contour and the second contour;

a fourth determining module 350, configured to perform feature extraction on a gray image corresponding to each class cluster, to obtain a first feature vector corresponding to the class cluster;

a fifth determining module 360 is configured to determine the zebra stripes based on the first feature vector.

In some embodiments of the present application, an image of interest including a zebra stripes is obtained based on road lane line and road edge information in an obtained first image, then contour detection is performed on the image of interest to obtain contour areas of a plurality of contours, and for each contour area, under the condition that the determined contour area is larger than a preset area threshold, the first contour and the second contour are clustered to obtain at least one class cluster, and for each class cluster, feature extraction is performed on gray images corresponding to the class cluster to obtain a first feature vector corresponding to the class cluster, and the zebra stripes are determined based on the first feature vector, so that interference of surrounding environments on the zebra stripes can be removed by determining the image of interest, and the zebra stripes caused by the problems of zebra stripes breakage, paint dropping and the like are intermittently detected by a clustering algorithm, so that influence of the detection precision of the zebra stripes on the detection precision is reduced, and the zebra stripes are broken and the paint dropping problem is solved, and the precision of the zebra stripes is improved.

In some embodiments of the present application, the first image comprises a first sub-image; to further improve the zebra stripes detection accuracy, the first acquisition module 310 may specifically be configured to:

acquiring a first sub-image acquired by a forward-looking wide-angle camera;

the first determining module 320 may specifically be configured to:

obtaining an interested image containing the zebra stripes based on the road lane lines and the road edge information in the first sub-image;

the zebra stripes determination device related to the above may further include:

the image processing module is used for carrying out image processing on the interested image to obtain a second image;

the second determining module 330 may specifically be configured to:

and performing contour detection on the zebra crossings in the second image to obtain the contour area of at least one contour.

In some embodiments of the present application, in order to further improve the zebra stripes detection accuracy, the image processing module may specifically be configured to:

denoising the third image to obtain a fourth image;

thresholding is carried out on the fourth image to obtain a binarized image;

and carrying out morphological processing on the binarized image to obtain the second image.

In some embodiments of the present application, the first image further comprises a second sub-image; to further improve the zebra stripes detection accuracy, the first acquisition module 310 may specifically be configured to:

acquiring a second sub-image acquired by a forward-looking remote camera;

the zebra stripes determination method related to the above may further include:

the image conversion module is used for converting the second sub-image into a bird's-eye view angle image according to the internal and external parameters of the forward-looking remote camera;

and a sixth determining module, configured to replace the aerial view image with the first sub-image, to obtain a second feature vector corresponding to the aerial view image.

In some embodiments of the present application, to improve the efficiency of zebra stripes detection, the fifth determination module 360 may specifically be configured to:

In some embodiments of the present application, the zebra crossing detection model includes a first sub-model and a second sub-model;

the fifth determination module 360 may specifically be configured to:

inputting the first characteristic vector into the first sub-model to obtain a first probability value with zebra stripes in the first sub-image;

Inputting the second eigenvector into the second sub-model to obtain a second probability value with zebra stripes in the second sub-image;

and obtaining the zebra crossing based on the first probability value and the second probability value.

In some embodiments of the present application, in order to further improve the zebra stripes detection accuracy, the zebra stripes determining device may further include:

the second acquisition module is used for acquiring a first center point coordinate of the minimum circumscribed rectangle of the first contour and a second center point coordinate of the minimum circumscribed rectangle of the second contour;

a seventh determining module, configured to determine a distance between the first center point coordinate and the second center point coordinate;

the third determining module 340 may specifically be configured to:

and clustering the first contour and the second contour under the condition that the distance is smaller than a preset distance threshold value.

The zebra stripes determining device provided in the embodiments of the present application may be used to execute the zebra stripes determining method provided in the embodiments of the methods described above, and its implementation principle and technical effects are similar, and for the sake of brevity, it is not repeated here.

Based on the same inventive concept, the embodiment of the application also provides electronic equipment.

Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 4, the electronic device may include a processor 401 and a memory 402 in which computer programs or instructions are stored.

In particular, the processor 401 described above may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured as one or more integrated circuits implementing embodiments of the present invention.

Memory 402 may include mass storage for data or instructions. By way of example, and not limitation, memory 402 may comprise a Hard Disk Drive (HDD), floppy Disk Drive, flash memory, optical Disk, magneto-optical Disk, magnetic tape, or universal serial bus (Universal Serial Bus, USB) Drive, or a combination of two or more of the foregoing. Memory 402 may include removable or non-removable (or fixed) media, where appropriate. Memory 402 may be internal or external to the integrated gateway disaster recovery device, where appropriate. In a particular embodiment, the memory 402 is a non-volatile solid state memory. The Memory may include read-only Memory (Read Only Memory image, ROM), random-Access Memory (RAM), magnetic disk storage media devices, optical storage media devices, flash Memory devices, electrical, optical, or other physical/tangible Memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software comprising computer-executable instructions and when the software is executed (e.g., by one or more processors) it is operable to perform the operations described in the zebra crossing determination methods provided by the above embodiments.

The processor 401 implements any of the zebra stripes determination methods of the above-described embodiments by reading and executing computer program instructions stored in the memory 402.

In one example, the electronic device may also include a communication interface 403 and a bus 410. As shown in fig. 4, the processor 401, the memory 402, and the communication interface 403 are connected by a bus 410 and perform communication with each other.

The communication interface 403 is mainly used to implement communication between each module, device, unit and/or device in the embodiment of the present invention.

Bus 410 includes hardware, software, or both, coupling components of the electronic device to one another. By way of example, and not limitation, the buses may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a micro channel architecture (MCa) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus, or a combination of two or more of the above. Bus 410 may include one or more buses, where appropriate. Although embodiments of the invention have been described and illustrated with respect to a particular bus, the invention contemplates any suitable bus or interconnect.

The electronic device may execute the zebra crossing determination method in the embodiment of the present invention, thereby implementing the zebra crossing determination methods described in fig. 1 and fig. 2.

In addition, in combination with the zebra stripes determining method of the above embodiment, the embodiment of the present invention may be implemented by providing a readable storage medium. The readable storage medium has program instructions stored thereon; the program instructions, when executed by a processor, implement any of the zebra crossing determination methods of the above embodiments.

In addition, in combination with the zebra stripes determination method of the above embodiment, the embodiments of the present invention may provide a computer program product, which when executed by a processor of an electronic device, causes the electronic device to perform any one of the zebra stripes determination methods of the above embodiment.

It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.

The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.

It should also be noted that the exemplary embodiments mentioned in this disclosure describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, or may be performed in a different order from the order in the embodiments, or several steps may be performed simultaneously.

Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to being, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware which performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In the foregoing, only the specific embodiments of the present invention are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present invention is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and they should be included in the scope of the present invention.

Claims

1. A zebra stripes determination method, the method comprising:

acquiring a first image; wherein the first image comprises zebra crossings;

the zebra stripes are determined based on the first feature vector.

2. The method of claim 1, wherein the first image comprises a first sub-image;

the acquiring the first image includes:

acquiring a first sub-image acquired by a forward-looking wide-angle camera;

the obtaining the interested image containing the zebra stripes based on the road lane lines and the road edge information in the first image comprises the following steps:

after the obtaining the image of interest including the zebra stripes based on the road lane lines and the road edge information in the first sub-image, the method further includes:

performing image processing on the interested image to obtain a second image;

the step of performing contour detection on the zebra crossings in the interested image to obtain a contour area of at least one contour comprises the following steps:

3. The method of claim 2, wherein the image processing the image of interest to obtain a second image comprises:

denoising the third image to obtain a fourth image;

thresholding is carried out on the fourth image to obtain a binarized image;

4. A method according to claim 3, wherein the first image further comprises a second sub-image;

The acquiring a first image includes:

acquiring a second sub-image acquired by a forward-looking remote camera;

after the acquiring the second sub-image acquired by the forward looking remote camera, the method further comprises:

converting the second sub-image into an aerial view angle image according to the internal and external parameters of the forward-looking remote camera;

5. The method of claim 4, wherein the determining the zebra stripes based on the first feature vector comprises:

6. The method of claim 5, wherein the zebra crossing detection model comprises a first sub-model and a second sub-model;

the step of inputting the first feature vector and the second feature vector into a pre-trained zebra crossing detection model to obtain the zebra crossing comprises the following steps:

7. The method of claim 1, wherein prior to said clustering the first contour and the second contour to obtain at least one cluster-like, the method further comprises:

acquiring a first center point coordinate of a minimum circumscribed rectangle of the first contour and a second center point coordinate of the minimum circumscribed rectangle of the second contour;

the clustering of the first contour and the second contour includes:

8. A zebra stripes determination method, the method comprising:

9. An electronic device comprising a processor, a memory, and a program or instruction stored on the memory and executable on the processor, which when executed by the processor, performs the steps of the zebra crossing determination method of any of claims 1-7.

10. A readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the zebra crossing determination method of any one of claims 1-7.