CN116279592A - Method for dividing travelable area of unmanned logistics vehicle - Google Patents
Method for dividing travelable area of unmanned logistics vehicle Download PDFInfo
- Publication number
- CN116279592A CN116279592A CN202310460686.1A CN202310460686A CN116279592A CN 116279592 A CN116279592 A CN 116279592A CN 202310460686 A CN202310460686 A CN 202310460686A CN 116279592 A CN116279592 A CN 116279592A
- Authority
- CN
- China
- Prior art keywords
- image
- obstacle
- line
- point
- road
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000001514 detection method Methods 0.000 claims abstract description 57
- 238000004364 calculation method Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims description 37
- 230000008569 process Effects 0.000 claims description 34
- 238000004422 calculation algorithm Methods 0.000 claims description 19
- 230000009466 transformation Effects 0.000 claims description 17
- 238000001914 filtration Methods 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000012937 correction Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000005286 illumination Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 6
- 230000002146 bilateral effect Effects 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000003014 reinforcing effect Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims 1
- 238000012360 testing method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 101100295091 Arabidopsis thaliana NUDT14 gene Proteins 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000004888 barrier function Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000011049 filling Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
- B60W60/0015—Planning or execution of driving tasks specially adapted for safety
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0019—Control system elements or transfer functions
- B60W2050/0028—Mathematical models, e.g. for simulation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0043—Signal treatments, identification of variables or parameters, parameter estimation or state estimation
- B60W2050/0044—In digital systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for dividing a drivable area of an unmanned logistics vehicle, which comprises the steps of establishing a lane line identification model and an obstacle detection model; identifying whether a central line or a side line exists by using a lane line identification model, and if so, entering an obstacle detection model for detection; if the road width is not existed, calculating the road width, judging whether the road width is enough for two vehicles to be parallel, if not, giving an alarm; if so, taking the middle point of the road surface, and generating a virtual center line or a side road line with the obstacle as an aid; the obstacle detection model detects whether an obstacle protruding out of the ground exists or not, and if the obstacle exists, the road width calculation is returned; if the vehicle does not exist, calculating a lane line fitting point, fitting a lane line, and then carrying out a lane line equation and polynomial curvature calculation to obtain a drivable area. The method can realize lane planning of the unmanned logistics vehicle, can detect the obstacle higher than the ground, avoid the problem of collision, and improve the intelligent level of the unmanned logistics vehicle.
Description
Technical Field
The invention relates to the technical field of logistics transportation, in particular to a method for dividing a drivable area of an unmanned logistics vehicle.
Background
Along with the gradual maturity of unmanned technique, unmanned technique is first in logistics park, and this kind of closed unmanned scene of container port unloading district falls to the ground. Under this closed scene of commodity circulation operation storehouse district, owing to there is the condition that the lane line is incomplete and some driving areas do not have the lane line, traditional automatic driving solution that relies on lane line discernment and lane line to keep can't accomplish the task, and owing to there is a large amount of goods shelves in the warehouse, traditional unmanned carries out the lane route planning to the barrier on ground only, has ignored the barrier in mid air or above ground to make unmanned commodity circulation car can be obstructed by the aerial barrier in going on.
Disclosure of Invention
The invention aims to provide a method for dividing a travelable area of an unmanned logistics vehicle. The method can realize lane planning of the unmanned logistics vehicle, can detect the obstacle higher than the ground, avoid the problem of collision, and improve the intelligent level of the unmanned logistics vehicle.
The technical scheme of the invention is as follows: the method for dividing the drivable area of the unmanned logistics vehicle is characterized by comprising the following steps of: the method comprises the following steps:
step 2, recognizing whether a central line or a side line exists by using a lane line recognition model, and if so, entering a step 3; if the road width is not existed, calculating the road width, judging whether the road width is enough for two vehicles to be parallel, if not, giving an alarm; if so, taking the middle point of the road surface, and generating a virtual center line or a side road line with the obstacle as an aid;
step 4: and obtaining a lane center line equation through polynomial curvature, calculating to obtain a road curvature radius, writing a calculation result into an image, and then dividing a drivable area.
According to the method for dividing the drivable area of the unmanned logistics vehicle, the lane line identification model is established as follows:
Step 1.1, image data acquisition is carried out by using a binocular camera, and image data is preprocessed; the preprocessing of the image data adopts a weighted average gray level conversion method to carry out gray level processing on the image, and the formula is as follows:
G ray (i，j)＝0.299×R(i，j)+0.578×G(i，j)+0.114×B(i，j)
wherein R, G, B represents the values of the 3 red, green and blue channels of the point in the original image respectively; (i, j) represents a pixel;
and Gamma correction is carried out on the image with darker light and stronger light, and nonlinear operation is carried out on the gray value of the input image by the Gamma correction, so that the gray value of the output image has an exponential relation:
G out ＝AG in γ ；
wherein: g out Representing the output image, G in Representing the input image, A being a constant of 0-1; gamma represents the gray average;
when gamma is less than 1, in the low gray value area, the dynamic range is enlarged, and the relatively concentrated low illumination area range in the image is stretched, so that the contrast of the image is enhanced; when gamma is larger than 1, in a high gray value region, the dynamic range is enlarged, and the region of high illumination concentrated in the image is stretched to enhance the contrast of the image, wherein the mathematical expression is as follows:
step 1.2, making and marking a data set, and then constructing a YOLOV7 model for pre-training; the pre-training is to extract the lane characteristic image by using a YOLOV7 model, and the process is as follows: assuming the coordinates of the pixel points of the lane line image as (x, y), the gradient direction of the detection area is the direction of the maximum change rate of the coordinates, and the gradient at the pixel points is as follows:
Wherein: g x The partial derivative of the pixel point in the horizontal direction; g y The partial derivative of the pixel point in the vertical direction; t is a gradient threshold;
when the gradient G (x, y) of a certain point is larger than the threshold T and the gradient value of the pixel point G (x+L, y) at a distance L from the gradient G (x, y) is also larger than the threshold T, the gradient is the boundary point of the lane line, the pixel values between the two points are all set to 255, and the pixel value of the pixel point which does not meet the condition is set to 0, so that the lane characteristic image is obtained.
Step 1.3, decoupling a head of the YOLOV7 model, and then using a FReLU function as an activation function to obtain a lane line recognition model;
step 1.4, using the data set in the step 1.2 for training a lane line recognition model;
step 1.5, reinforcing the lane line recognition model by using a channel attention mechanism and a space attention mechanism;
and step 1.6, calculating a loss function, returning to the step 1.4 to continue training if the confidence coefficient of the model is smaller than 0.5, and ending training if the confidence coefficient of the model is larger than 0.5 to obtain the trained lane line recognition model.
In the aforementioned method for dividing the drivable area of the unmanned logistics vehicle, in step 1.2, bilateral filtering denoising is performed on the data set image before the data set is manufactured, and the formula is as follows:
Wherein: g (x, y) is the filtered image, ω s (i, j) is a spatial domain weight, ω r (i, j) is the gray domain weight, ω p For normalization parameters, I (I, j) is the noise image, Ω is the domain range at pixel (I, j); wherein:
ω p ＝∑ i，j∈Ω ω s (i，j)ω r (i，j)。
in the foregoing method for dividing a drivable area of an unmanned logistics vehicle, in step 1.3, the FReLU function is expressed as follows:
y＝Max((x s，i，j ，T(x s，i，j ))；
wherein: x is x s，i，j For the input value of the activation function at the S-th channel (i, j), T (x) s，i，j ) To be a spatial context feature extractor, the calculation is as follows:
wherein:
expressed in x s，i，j A parameter pooling window for the center, +.>
Representing the window parameter.
In the aforementioned method for dividing the drivable area of the unmanned logistics vehicle, in step 1.6, the loss function formula of the lane line recognition model is as follows:
wherein: alpha is the weight coefficient of the weight coefficient,
IoU the intersection ratio of the calculated predicted detection frame and the actual detection frame;
wherein: w (w) gt Is the width of a real frame, h gt And w is the width of the prediction frame, and h is the height of the prediction frame.
In the aforementioned method for dividing the drivable area of the unmanned logistics vehicle, in step 2, the process of generating the virtual center line or the side line with the obstacle as an aid is as follows: obtaining point cloud data, layering and dividing the point cloud data, filtering obstacles, constructing a three-dimensional grid, judging whether the cosine value of an angle between the plane method adjacent and the z-axis is 0-0.1, if not, determining that the point cloud data is not road boundary point cloud data, if so, determining that the point cloud data is road boundary point cloud data, denoising the road boundary point cloud data to obtain complete road boundary point cloud data, and finally sequencing the complete road boundary point cloud data to draw a virtual center line or a side line;
The method comprises the steps of establishing a relationship between point cloud data and a space grid and between the space grid and the space grid by utilizing a three-dimensional grid:
firstly, reading point cloud data, and searching the maximum X of three directions of three-dimensional coordinates X, Y and Z of the point cloud max ，Y max ，Z max And a minimum value X min ，Y min ，Z min ，
Secondly, a small amount delta is given,
the most value of the anti-dead point cloud falls on the grid line and does not participate in operation;
by (X) max +δ，Y max +δ，Z max +δ) is the maximum value;
(X min -δ，Y min -δ，Z min delta) is a minimum value;
drawing a three-dimensional grid in the maximum range, and setting the length, width and height of the three-dimensional grid as X respectively grid ，Y grid ，Z grid The number of the grids in the X, Y and Z directions is n respectively x ，n y And n Z Obtaining the maximum value X of each space grid according to the length, width and height of the grid g_max ,Y g_max ,Z g_max And a minimum value X g_min ，Y g_min ，Z g_min ；
Has the following components
The denoising of the point cloud data is performed by adopting an improved K-means clustering algorithm, and the process is as follows:
1) Observing the point cloud, and selecting a proper initial cluster number N according to the target cluster number N;
2) Randomly selecting a point from the point cloud as a clustering center, and setting a proper threshold d according to the density of the point cloud and the distance between the noise point and the target clustering center;
3) Taking the point as a clustering center, circulating all the points, setting the coordinates of the clustering center point as (x, y, z), and setting the coordinates of a certain point as (x i ，y i ，z i ) The distance equation from a certain point to the clustering center point is applied as follows:
calculating the distance from all points to the clustering center point, setting a threshold value, clustering smaller than the threshold value to the center, and removing the points from the point cloud;
4) Removing the points selected as the clustering centers in the step 2), and respectively selecting the data clustered in the step 3) as the clustering centers; repeating the process until all the points meeting the threshold are clustered together, and then performing the next clustering after the clustering is finished;
5) Repeating the step 4) process, and obtaining N classes according to the initial clustering;
6) In the projection XOY plane of the road edge point cloud data after initial clustering, noise points are sheet-shaped, the road edge point cloud is strip-shaped, a bounding box is built for each type of the initial clustering, the height of the bounding box is in the range of 0.05-0.15 m according to the height threshold value and the length threshold value of the bounding box, the length of the bounding box is 0.9-1 times of the length of a grid, the noise points are removed, and finally n types of clustering results are obtained.
According to the method for dividing the drivable area of the unmanned logistics vehicle, the obstacle detection model detects whether an obstacle protruding out of the ground exists or not, and the process is as follows: taking a picture by a binocular camera, removing the influence of background information and road surface information, detecting the edge line segments of the U-V parallax map by carrying out Canny algorithm on the parallax map of the obstacle area, setting a threshold value for line segment detection by Hough transformation, and extracting the width information of the line segments of the obstacle area in the U parallax map and the height information of the line segments in the V parallax map after calculating the base points of the intersection of the obstacle and the road surface area, so as to accurately position the target obstacle; then, according to the obtained positioning information of the obstacle region, parallax of the region where the obstacle is located is further intercepted by the parallax map, and contour extraction is carried out on the obstacle in the selected region by adopting a region growing method; finally, overlapping the obstacle with the outline with an image edge line segment detected by a Canny algorithm, then establishing a detection frame of the obstacle according to a line segment fitting result of the U-V parallax image, and selecting the corresponding obstacle to realize the detection of the obstacle;
Wherein the effect of removing the background information and the road surface information is to calculate a texture main direction of the road surface area based on a 4-direction Gabor filter, the 4-direction Gabor filter being defined as:
wherein,,
ω 0 =2pi/λ is radial frequency, frequency multiplication constant c=pi/2, spatial frequency +.>
The Canny algorithm realizes the process of detecting the edge line segments of the U-V disparity map, which comprises the following steps:
1) The gaussian filter removes noise:
let G (x, y) be the smoothed image, the smoothing of image f (x, y) with a two-dimensional gaussian function G (x, y, σ) can be expressed as: g (x, y) =g (x, y, σ) ×f (x, y); 2) Obtaining gradient values and directions;
and calculating gradient amplitude and direction of the image with the noise removed by a two-dimensional Gaussian function first-order partial differential equation:
wherein k is a constant, and
the amplitude and the reverse direction of the gradient are obtained as follows:
wherein M x, y is the intensity information of the image edge, and theta x, y is the direction information of the image edge;
3) Non-maximum suppression;
performing non-maximum suppression processing on the calculated gradient amplitude values, and quantifying all possible directions into four edge directions of 0 degree, 135 degrees, 90 degrees and 45 degrees, wherein the four edge directions correspond to gradient directions perpendicular to the edge directions;
4) Determining edges by a double-threshold algorithm;
In the determination process of the edge pixel points, setting threshold coefficients TH and TL, wherein the ratio of the threshold coefficients TH to TL is 2:1 or 3:1; marking the pixel points larger than TH as edge points, and not marking the pixel points smaller than TL; and determining the pixel points between TH and TL by using an 8-connected region, marking the pixel points which are the same as the TL threshold value in the 8-connected region as edge connection points, and forming edge line segments by the edge connection points.
The lane line equation and the polynomial curvature calculation are performed by the given data point p (x i ，y i ) Where i=1, 2,3, …, m, an approximate curve is obtained
The fitting polynomial is set to:
wherein a is 0 ，a 1 ，a 2 ，…，a k Using the sum of squares of the deviations as a loss function for polynomial coefficients, to make
The deviation from the relation f (x) representing the input x and the output y is minimal:
to obtain the optimal polynomial coefficient a 0 ，a 1 ，a 2 ，…，a k ；
Then by
Coefficients representing the order terms of the lane line fitting polynomial, the lane line equation is represented by a K-th order polynomial:
wherein: a, a k,j A kth order term coefficient representing a fitted polynomial of the deduced jth lane line; k=5.
According to the method for dividing the drivable area of the unmanned logistics vehicle, the lane center line equation is calculated by obtaining an inverse transformation matrix through perspective transformation, the drawn left lane line and the drawn right lane line are transformed into an original image space through the inverse transformation matrix, the result is combined with the original image, the lane line equation formula and the inverse perspective image and the actual road transverse and longitudinal proportional relation coefficient formula are combined, and the lane center line equation on the front actual road of the vehicle is obtained by the following steps:
k x : ordinate scaling factor, k y : scale factor of abscissa, a 0 ，a 1 ，a 2 Representing curve equation parameters;
the calculation formula of the curvature radius is as follows:
wherein: k (K) 1 Is a curvature;
substituting the actual roadway center line equation into a curvature radius calculation formula:
thereby obtaining the radius of curvature of the road ahead of the vehicle.
In the above-mentioned method for dividing the drivable area of the unmanned logistics vehicle, the dividing process of the drivable area is to input the center line into the deep labv3+ model, output the feature head by using the deep labv3+ model, and then output the drivable area by using the detection head.
Compared with the prior art, the lane line recognition model can accurately recognize lanes with center lines and side lines, and for lanes with abnormal structures, the lane line recognition model is combined with the obstacle detection model to perform unstructured lane detection, so that lane planning of the unmanned logistics vehicle can be realized, obstacles higher than the ground can be detected, the problem of collision is avoided, and finally the drivable area can be divided, and the intelligent level of the unmanned logistics vehicle is improved.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
FIG. 2 is a schematic diagram of a lane line recognition model establishment flow;
FIG. 3 is a binocular stereoscopic imaging principle;
FIG. 4 is a schematic diagram of a decoupling head;
FIG. 5 is a flow chart of drawing a side road line with the assistance of an obstacle;
FIG. 6 is a schematic diagram of a spatial grid and a point cloud storage grid;
FIG. 7 is a schematic diagram of a detection flow of an obstacle detection model;
FIG. 8 is a Deeplabv3+ block diagram based on a ResNet50 feature extraction module;
FIG. 9 is a block diagram of ASPP;
fig. 10 is a structural view of the detection head.
Reference numerals
Detailed Description
The invention is further illustrated by the following figures and examples, which are not intended to be limiting.
Examples: a method for dividing a travelable area of an unmanned logistics vehicle, as shown in fig. 1, comprises the following steps:
step 1.1, image data acquisition is carried out by using a binocular camera, and image data is preprocessed; as shown in fig. 3, the binocular camera simulates both eyes of a person and obtains three-dimensional information in a real environment according to a parallax principle. A binocular camera generally captures environmental information at the same time from different angles in left and right directions by 2 cameras to form a digital image. Transforming a target point in three-dimensional space to obtain the corresponding coordinate P (X) w ，Y w ，Z w ) However, when the pixel coordinates of a point image are known to calculate the three-dimensional coordinates in the world coordinate system, depth information in the three-dimensional coordinates cannot be accurately determined according to the principle of transmission transformation. Therefore, the binocular vision model can accurately determine the target, and the accuracy of target object detection is improved.
In the step, the image data is preprocessed by considering the influence of the lamplight environment in the warehouse on the detection result.
Firstly, carrying out image graying treatment by adopting a weighted average gray level conversion method, wherein the formula is as follows:
G ray (i, j) =0.299×r (i, j) +0.578×g (i, j) +0.114×b (i, j) wherein R, G, B represents the values of the 3 red, green and blue channels of the dot in the original image, respectively; (i, j) represents a pixel;
when the local gray value is less than or equal to 0 f (x, y) < a and b is less than or equal to 0 f (x, y) < 255, the image in the area with a=110 and b=160 is the image with darker light and stronger light, gamma correction is needed, and the processed image is subjected to local contrast stretching to achieve a better effect; and similarly, when the local gray value is a less than or equal to f (x, y) < b, the image of the region is a normal illumination image, and the image is directly preprocessed.
The Gamma correction has the effect of improving the image enhancement effect of the image in dark light and exposure environments. In Gamma correction, nonlinear operation is performed on the gray values of the input image, so that the gray values of the output image are in an exponential relationship:
G out ＝AG in γ；
wherein: g out Representing the output image, G in Representing an input image, A representing a constant of 0-1; gamma represents the gray average;
when gamma is less than 1, in the low gray value area, the dynamic range is enlarged, and the relatively concentrated low illumination area range in the image is stretched, so that the contrast of the image is enhanced; when gamma is larger than 1, in a high gray value region, the dynamic range is enlarged, and the region of high illumination concentrated in the image is stretched to enhance the contrast of the image, wherein the mathematical expression is as follows:
the method can enhance the contrast characteristic of the image and has better detection effect.
Step 1.2, making and marking a data set, and then constructing a YOLOV7 model for pre-training; the method is characterized in that bilateral filtering denoising is carried out on the data set image before the data set is manufactured, the bilateral filtering is a nonlinear filtering method, spatial domain information is considered in the filtering process, gray similarity is considered, and compared with fuzzy edges with obvious mean filtering and median filtering, the method can eliminate image noise and simultaneously save image edge characteristics. In bilateral filtering, the filtered image pixels depend on weighting their neighborhood pixel values as follows:
Wherein: g (x, y) is the filtered image, ω s (i, j) is a spatial domain weight, ω r (i, j) is the gray domain weight, ω p For normalization parameters, I (I, j) is the noise image, Ω is the domain range at pixel (I, j); wherein:
ω p ＝∑ i，j∈Ω ω s (i，j)ω r (i，j)。
the data set making and marking steps are as follows:
(1) The text of the picture name lane line is modified. The marking of the lane lines is to modify the structured/unstructured roads in a warehouse in the actual pictures into names of the pictures, the suffixes of the pictures are kept unchanged, the data sets are modified according to the format required by the actual projects after the marking of the data sets is completed, the pictures are divided into training sets and test sets according to a certain proportion by using a Python script program, a data catalog is newly built in the projects, and two paths of a track/text and a test/text are respectively built in a data folder, wherein the track/text is used for storing the processed training set pictures, the processed track.pkl binary files are stored in the track, the test set pictures are stored in the test/text, and the test.pkl binary files of the test sets are stored in the test.
(2) Generating a pkl file. The pkl file is a file format for storing binary contents, and in the training process, the network reads text information and corresponding pictures from the pkl file for training. And respectively storing the picture names in the training set and the verification set into the newly built pkl file in sequence, naming the picture names as train. Pkl and test. Pkl, and storing the corresponding picture names as serial numbers.
The data set is then thresholded:
(1) A lane line dataset is prepared. Several pictures of structured and unstructured lanes in each preparation warehouse, wherein the types of lanes should also include data of lanes of various road conditions. The structured road lanes are named structured lanes and serial numbers, and the unstructured road lanes are named unstructured lanes and serial numbers.
(2) Code is written. The accuracy of the processed data under the condition of different thresholds is calculated through a program, the accuracy calculated after each time of threshold modification is saved, and finally a line graph is generated. In the implementation process, whether the confidence of the lane line is larger than a set threshold value is judged, the lane line which is higher than the threshold value is judged to be named as a structured lane of the structured road, the lane line which is lower than the threshold value is judged to be named as the same as the correct recognition of the unstructured road, and the accuracy of the number plate is calculated under the two conditions.
(3) A threshold is selected. And writing a script to count the accuracy of the verification data set under the condition of different thresholds, wherein a lane line lower than the set threshold is used as an unstructured road lane line, otherwise, the lane line is a structured road lane line, and the most suitable threshold under the condition of the highest accuracy is tested.
The threshold operation method is fixed threshold operation, the method converts a gray image into a binary image based on a gray threshold, and the key point of the method is to select the size of the threshold, and the pixels in the image are processed as follows:
finally, generating a pre-training weight:
(1) And analyzing the weight structure. The storage mode of the pre-training weight can be divided into CPU training, single GPU training and multi-GPU training according to the training mode, wherein the model structures stored under the conditions of the CPU and the single GPU are the same; according to the storage mode, the method can be divided into a network structure for storing the model and a weight file for storing training parameters, and only two types of training parameters to weight files are stored; according to the storage mode, the method can be divided into two modes of saving the batch trained in the training process, training parameters, weight files of the intermediate structure of the network structure and weight files only saving the final training parameters; according to the network structure, the network structure of the weight file can be printed out through the Python script, and according to the requirement, the network parameters needing to be modified in the weight can be modified.
(2) The dimensions of the pre-training weights are modified. Modifying parameters in the network may cause the network to change, so there are two ways to modify the pre-training weight file to adapt to the current network: eliminating node training parameters of an unsuitable network; and modifying the network nodes unsuitable for training, filling the nodes, and using the node names and the dimension information in the Python read weights to modify the parameter dimension of the first layer and the network node parameters of the last corresponding category number in the weight file into the modified category number. And saving the modified weight file as a new weight file.
Processing a data set, and reconstructing a YOLOV7 model for pre-training, wherein the pre-training is to extract a lane characteristic image by adopting the YOLOV7 model, and the process is as follows:
assuming the coordinates of the pixel points of the lane line image as (x, y), the gradient direction of the detection area is the direction of the maximum change rate of the coordinates, and the gradient at the pixel points is as follows:
wherein: g x The partial derivative of the pixel point in the horizontal direction; g y The partial derivative of the pixel point in the vertical direction; t is a gradient threshold;
when the gradient G (x, y) of a certain point is larger than the threshold T and the gradient value of the pixel point G (x+L, y) at a distance L from the gradient G (x, y) is also larger than the threshold T, the gradient is the boundary point of the lane line, the pixel values between the two points are all set to 255, and the pixel value of the pixel point which does not meet the condition is set to 0, so that the lane characteristic image is obtained.
Step 1.3, decoupling a head of the YOLOV7 model, and then using a FReLU function as an activation function to obtain a lane line recognition model; in this step, as shown in fig. 4, the convergence rate of the decoupling head is increased after the decoupling head replaces the YOLO head. The step of introducing a decoupling head: 1. modifying common file 2, adding the decoded_detect code 3 at yolo.py, modifying Build structures, anchors 4 in the model function, modifying the structure of the last layer of detected header in the yaml file of the model.
In this step, the FReLU function is expressed as follows:
y＝Max((x s，i，j ，T(x s，i，j ))；
wherein: x is x s，i，j For the input value of the activation function at the S-th channel (i, j), T (x) s，i，j ) To be a spatial context feature extractor, the calculation is as follows:
wherein:
expressed in x s，i，j A parameter pooling window for the center, +.>
Representing the window parameter.
Step 1.4, using the data set of step 1.2 for training of a lane line recognition model to obtain a prediction frame and a category including a center line first and a boundary line (also referred to as a side line).
Step 1.5, reinforcing the lane line recognition model by using a channel attention mechanism and a space attention mechanism;
in the step, the channel attention mechanism enhancement is that firstly, the characteristics are input into a channel attention mechanism module; then carrying out maximum pooling and average pooling on the whole characteristics to squeeze the characteristics and obtain global information among channels; obtaining the weight of each channel through a multi-layer sensor; and finally, carrying out weighted operation to output the channel attention information.
The spatial attention mechanism strengthens the maximum pooling and average pooling of each channel of the input features to generate two features representing different information, then the features are fused through a large convolution kernel of the receptive field, then weights are generated through a Sigmoid activation function, and finally the features are overlapped with the original features to generate the spatial attention information. The Sigmoid activation function formula is as follows:
And step 1.6, calculating a loss function, returning to the step 1.4 to continue training if the confidence coefficient of the model is smaller than 0.5, and ending training if the confidence coefficient of the model is larger than 0.5 to obtain the trained lane line recognition model. In this step, the loss function formula of the lane line recognition model is as follows:
wherein: alpha is the weight coefficient of the weight coefficient,
IoU the intersection ratio of the calculated predicted detection frame and the actual detection frame;
wherein: w (w) gt Is the width of a real frame, h gt And w is the width of the prediction frame, and h is the height of the prediction frame.
Step 2, recognizing whether a central line or a side line exists by using a lane line recognition model, and if so, entering a step 3; if the road width is not existed, calculating the road width, judging whether the road width is enough for two vehicles to be parallel, if not, giving an alarm; if so, taking the middle point of the road surface, and generating a virtual center line or a side road line with the obstacle as an aid;
in this step, with the obstacle as an assist, the process of generating the virtual center line or the side road line is as shown in fig. 5, firstly, the point cloud data is obtained, the point cloud data is layered and segmented, the obstacle is filtered, then a three-dimensional grid is constructed, whether the cosine value of the angle between the plane method adjacent and the z-axis is 0-0.1 is judged, if not, the point cloud data is not the road boundary point cloud data, if yes, the road boundary point cloud data is obtained, then the noise is removed by blending, the complete road boundary point cloud data is obtained, and finally the complete road boundary point cloud data is ordered, so that the virtual center line or the side road line is drawn.
Drawing a three-dimensional grid by using the maximum value of the space grid, and drawing all the three-dimensional grids through a certain cycle; finally, according to whether the coordinate value of the point cloud data falls within a certain space grid maximum value range, the point cloud data falling within the grid maximum value range is stored in the grid, and then all the point cloud data are respectively stored in the corresponding cube grids, so that the establishment of the relation between the point cloud data and the space grid and between the space grid and the space grid is completed, and the space grid and the point cloud are stored in the space grid schematic diagram as shown in fig. 6. The specific process is as follows:
firstly, reading point cloud data, and searching the maximum X of three directions of three-dimensional coordinates X, Y and Z of the point cloud max ，Y max ，Z max And a minimum value X min ，Y min ，Z min ，
Secondly, a small amount delta is given,
the most value of the anti-dead point cloud falls on the grid line and does not participate in operation;
by (X) max +δ，Y max +δ，Z max +δ) is the maximum value;
(X min -δ，Y min -δ，Z min delta) is a minimum value;
drawing a three-dimensional grid in the maximum range, and setting the length, width and height of the three-dimensional grid as X respectively grid ，Y grid ，Z grid The number of the grids in the X, Y and Z directions is n respectively x ，n y And n Z Obtaining the maximum value X of each space grid according to the length, width and height of the grid g_max ,Y g_max ,Z g_max And a minimum value X g_min ，Y g_min ，Z g_min ；
Has the following components
Further, in the step, denoising of the point cloud data is performed by adopting an improved K-means clustering algorithm, and the process is that 1) the point cloud is observed, and a proper initial clustering number N (N is that the point cloud data of a part has a plurality of discrete subclasses including road edges and other noise points) is selected according to the target clustering number N (the point cloud data of the part has a plurality of subclasses which are the point cloud data of the road edges);
2) Randomly selecting a point from the point cloud as a clustering center, and setting a proper threshold d according to the density of the point cloud and the distance between the noise point and the target clustering center;
3) Taking the point as a clustering center, circulating all the points, setting the coordinates of the clustering center point as (x, y, z), and setting the coordinates of a certain point as (x i ，y i ，z i ) The distance equation from a certain point to the clustering center point is applied as follows:
calculating the distance from all points to the clustering center point, setting a threshold value, clustering smaller than the threshold value to the center, and removing the points from the point cloud;
4) Removing the points selected as the clustering centers in the step 2), and respectively selecting the data clustered in the step 3) as the clustering centers; repeating the process until all the points meeting the threshold are clustered together, and then performing the next clustering after the clustering is finished;
5) Repeating the step 4) process, and obtaining N classes according to the initial clustering;
6) In the projection XOY plane of the road edge point cloud data after initial clustering, noise points are sheet-shaped, the road edge point cloud is strip-shaped, a bounding box is built for each type of the initial clustering, the height of the bounding box is in the range of 0.05-0.15 m according to the height threshold value and the length threshold value of the bounding box, the length of the bounding box is 0.9-1 times of the length of a grid, the noise points are removed, and finally n types of clustering results are obtained. The improved algorithm can effectively filter noise points in the point cloud, reduce complexity of the algorithm, and improve operation efficiency of the algorithm in a dynamic clustering center mode.
Still further, in this step, the sorting of the road boundary points is performed using an improved eight-domain method. The traditional eight-neighborhood searching method needs edge grid identification, and if point clouds exist in the edge grid, different neighborhood searching needs to be carried out according to different positions of the point clouds. The improved eight-neighborhood searching method is that a row of grids are respectively added at the leftmost side and the rightmost side of the X direction, and a row of grids are respectively added at the uppermost side and the lowermost side of the Y direction, so that eight neighborhoods of the grids with point clouds are all grids, and the method does not need to identify the edge grids.
In this step, as shown in fig. 7, the process of detecting whether there is an obstacle protruding from the ground by the obstacle detection model is: taking a picture by a binocular camera, removing the influence of background information and road surface information, detecting the edge line segments of the U-V parallax map by carrying out Canny algorithm on the parallax map of the obstacle area, setting a threshold value for line segment detection by Hough transformation, and extracting the width information of the line segments of the obstacle area in the U parallax map and the height information of the line segments in the V parallax map after calculating the base points of the intersection of the obstacle and the road surface area, so as to accurately position the target obstacle; then, according to the obtained positioning information of the obstacle region, parallax of the region where the obstacle is located is further intercepted by the parallax map, and contour extraction is carried out on the obstacle in the selected region by adopting a region growing method; and finally, overlapping the obstacle with the outline with an image edge line segment detected by a Canny algorithm, and then establishing a detection frame of the obstacle according to a line segment fitting result of the U-V parallax image, and selecting the corresponding obstacle to realize the detection of the obstacle.
The background information is removed by using a road vanishing point detection algorithm, and the process is based on a 4-direction Gabor filter to calculate the main direction of texture of the pavement area. The basic principle of the Gabor filter is that the image is filtered through a Gaussian window function with adjustable direction, and the image is subjected to local feature analysis more accurately, so that the extraction of the texture features of the image is realized. The 4-direction Gabor filter is defined as
Wherein,,
ω 0 =2pi/λ is radial frequency, frequency multiplication constant c=pi/2, spatial frequency +.>
The Canny can be divided into the following four steps:
1) Removing noise points by a Gaussian filter;
2)
let G (x, y) be the smoothed image, the smoothing of image f (x, y) with a two-dimensional gaussian function G (x, y, σ) can be expressed as:
h(x，y)＝G(x，y，σ)*f(x，y)；
3) Obtaining gradient values and directions;
calculating gradient amplitude and direction of the image with noise removed by using a two-dimensional Gaussian function first-order partial differential equation;
wherein k is a constant, and
the magnitude and the inverse of the gradient can be found as: />
Wherein M x, y is the intensity information of the image edge, and θ x, y is the direction information of the image edge.
3) Non-maximum suppression;
and performing non-maximum suppression processing on the calculated gradient amplitude values, and quantifying all possible directions into four edge directions of 0 degree, 135 degree, 90 degree and 45 degree, wherein the four edge directions correspond to gradient directions perpendicular to the edge directions. The non-maximum value inhibition is to compare the magnitudes of corresponding neighborhood values in the 3*3 neighborhood along the four gradient directions.
4) Determining edges by a double-threshold algorithm;
after the above steps are completed, the strong edges in the image are already in the currently acquired edge image. However, some virtual edges may also be within the edge image. These false edges may be generated by the real image or by noise.
In the determination process of the edge pixel points, setting threshold coefficients TH and TL, wherein the ratio of the threshold coefficients TH to TL is 2:1 or 3:1; marking the pixel points larger than TH as edge points, and not marking the pixel points smaller than TL; and determining the pixel points between TH and TL by using an 8-connected region, and marking the pixel points which are in the 8-connected region and have the same threshold value as the TL as edge connection points.
The Hough transform can convert line detection in the image into point estimation in the parameter space where a threshold is set to extract lines in the image. In the representation of straight lines
The standard expression of (2) replaces the general expression, the parameter space is changed into (theta, p), and the theta is more than or equal to 0 and less than or equal to 2 pi, so that the pixel point corresponds to a trigonometric function curve in the parameter space. And detecting the edges of the U-V parallax images by adopting a Canny algorithm, and calculating and extracting the base points of the obstacle region by utilizing Hough transformation.
Identifying the target obstacle by an obstacle structure detection frame with the outline extracted according to the U-V parallax map with the base points of the obstacle region calculated, determining height information of the obstacle through the V parallax map, determining the height distance by Hough transformation to extract straight line segments of the obstacle, constructing the height of the detection frame, determining width information of the obstacle through the result of straight line segment fitting of the U parallax map through Hough transformation, and constructing the width of the detection frame to realize detection of the target obstacle.
In this step, the lane-line equation and polynomial curvature calculation is performed by a given data point p (x i ，y i ) Where i=1, 2,3, …, m, an approximate curve is obtained
The fitting polynomial is set to:
wherein a is 0 ，a 1 ，a 2 ，…，a k Using the sum of squares of the deviations as a loss function for polynomial coefficients, to make
The deviation from the relation f (x) representing the input x and the output y is minimal:
to obtain the optimal polynomial coefficient a 0 ，a 1 ，a 2 ，…，a k ；
Then by
Coefficients representing the order terms of the lane line fitting polynomial, the lane line equation is represented by a K-th order polynomial:
wherein: a, a k,j A kth order term coefficient representing a fitted polynomial of the deduced jth lane line; k=5.
Step 4: and obtaining a lane center line equation through polynomial curvature, calculating to obtain a road curvature radius, writing a calculation result into an image, and then dividing a drivable area.
In the step, the calculation of the lane center line equation is to obtain an inverse transformation matrix through perspective transformation, transform the drawn left and right lane lines into an original image space through the inverse transformation matrix, combine the result with the original image, combine the lane line equation formula with the inverse perspective image and the actual road transverse and longitudinal proportional relation coefficient formula, and obtain the lane center line equation on the front actual road of the vehicle as follows:
k x : ordinate scaling factor, k y : scale factor of abscissa, a 0 ，a 1 ，a 2 Representing curve equation parameters;
the calculation formula of the curvature radius is as follows:
wherein: k (K) 1 Is a curvature;
substituting the actual roadway center line equation into a curvature radius calculation formula:
therefore, the radius of curvature of the road in front of the vehicle is obtained, and the acquisition of the radius of curvature has positive significance for judging the road condition in front of the vehicle. Meanwhile, a lane center line equation can be calculated, the lane center line equation is a road equation, and the acquisition of the lane center line equation is an important basis for calculating the lane curvature by the method.
The dividing process of the drivable area is to input the center line into the deep labv3+ model, output the characteristic head by using the deep labv3+ model, and then output the drivable area by using the detection head. In this embodiment, deeplabv3+ based on the res net50 feature extraction module is used. As shown in fig. 8, the size of the input picture of deep labv3+ is 416×416×3, after the network feature extraction of ResNet50 based on the deformable convolution design is used in the encoding stage, the last layer of output is parallelly calculated through ASPP modules (the structure of the ASPP modules is shown in fig. 9) to obtain a plurality of feature maps with different scales, and the channel merging operation is performed on the 5 feature maps output by the ASPP network along the channel direction, where the feature map size is 13×13×256. The combined feature map does not adopt a layer-by-layer up-sampling mode, but performs eight times up-sampling and low-layer feature channel combination operation to restore the boundary information of the target part, and finally four times up-sampling is performed after convolution kernel convolution calculation of 3*3 to obtain a final segmentation result.
Specifically, as shown in fig. 10, the structure of the detection head is that the CSPLayer module divides the output of deep labv3+ in the upper layer into two branches, one branch obtains the output through one 1*1 convolution block and one 3*3 convolution block, and the output obtained by performing Concat channel splicing with the other branch with only one 3*3 convolution block is not 0, so that the gradient vanishing or the gradient explosion is effectively prevented. Different layers will output different characteristic patterns to the detection head, and the 3*3 convolution block in the CSPLayer module can enlarge the receptive field for outputting the detected characteristic patterns, so that the detection performance of the detection head can be effectively enhanced.
The convolution block in the CSPLlayer module consists of DConv convolution (deformable convolution) and SiLU activation function layer (f (x) = -x-sigmoid (x)), the parameter quantity brought by DConv is very small, the calculated quantity of the detection head is greatly reduced, and the light weight of the detection head is realized. The DConv deformable convolution is actually that an offset is added to the sampling position in the standard convolution operation, so that the convolution kernel can be expanded to a large range in the training process. The deformable convolution generalizes various transformations of scale, aspect ratio and rotation.
In summary, the lane line recognition model can accurately recognize lanes with center lines and side lines, and for lanes with abnormal structures, the lane line recognition model is combined with the obstacle detection model to perform unstructured lane detection, so that lane planning of the unmanned logistics vehicle can be realized, obstacles higher than the ground can be detected, the problem of collision is avoided, and the intelligent level of the unmanned logistics vehicle is improved.
Claims (10)
1. The method for dividing the drivable area of the unmanned logistics vehicle is characterized by comprising the following steps of: the method comprises the following steps:
step 1, establishing a lane line identification model and an obstacle detection model;
step 2, recognizing whether a central line or a side line exists by using a lane line recognition model, and if so, entering a step 3; if the road width is not existed, calculating the road width, judging whether the road width is enough for two vehicles to be parallel, if not, giving an alarm; if so, taking the middle point of the road surface, and generating a virtual center line or a side road line with the obstacle as an aid;
step 3, detecting whether an obstacle protruding out of the ground exists or not by using the obstacle detection model, and if so, returning to the road surface width calculation in the step 2; if the curve fitting point does not exist, calculating a curve fitting point, fitting a curve, and then calculating a curve equation and polynomial curvature;
step 4: and obtaining a lane center line equation through polynomial curvature, calculating to obtain a road curvature radius, writing a calculation result into an image, and then dividing a drivable area.
2. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 1, wherein: the establishment process of the lane line identification model is as follows:
Step 1.1, image data acquisition is carried out by using a binocular camera, and image data is preprocessed; the preprocessing of the image data adopts a weighted average gray level conversion method to carry out gray level processing on the image, and the formula is as follows:
G ray (i，j)＝0.299×R(i，j)+0.587×G(i，j)+0.114×B(i，j)
wherein R, G, B represents the values of the 3 red, green and blue channels of the point in the original image respectively; (i, j) represents a pixel;
and Gamma correction is carried out on the image with darker light and stronger light, and nonlinear operation is carried out on the gray value of the input image by the Gamma correction, so that the gray value of the output image has an exponential relation:
G out ＝AG in γ ；
wherein: g out Representing the output image, G in Representing the input image, A being a constant of 0-1; gamma represents the gray average;
when gamma is less than 1, in the low gray value area, the dynamic range is enlarged, and the relatively concentrated low illumination area range in the image is stretched, so that the contrast of the image is enhanced; when gamma is larger than 1, in a high gray value region, the dynamic range is enlarged, and the region of high illumination concentrated in the image is stretched to enhance the contrast of the image, wherein the mathematical expression is as follows:
step 1.2, making and marking a data set, and then constructing a YOLOV7 model for pre-training; the pre-training is to extract the lane characteristic image by using a YOLOV7 model, and the process is as follows: assuming the coordinates of the pixel points of the lane line image as (x, y), the gradient direction of the detection area is the direction of the maximum change rate of the coordinates, and the gradient at the pixel points is as follows:
Wherein: g x The partial derivative of the pixel point in the horizontal direction; g y The partial derivative of the pixel point in the vertical direction; t is a gradient threshold;
when the gradient G (x, y) of a certain point is larger than the threshold T and the gradient value of the pixel point G (x+L, y) at a distance L from the gradient G (x, y) is also larger than the threshold T, the gradient is the boundary point of the lane line, the pixel values between the two points are all set to 255, and the pixel value of the pixel point which does not meet the condition is set to 0, so that the lane characteristic image is obtained.
Step 1.3, decoupling a head of the YOLOV7 model, and then using a FReLU function as an activation function to obtain a lane line recognition model;
step 1.4, using the data set in the step 1.2 for training a lane line recognition model;
step 1.5, reinforcing the lane line recognition model by using a channel attention mechanism and a space attention mechanism;
and step 1.6, calculating a loss function, returning to the step 1.4 to continue training if the confidence coefficient of the model is smaller than 0.5, and ending training if the confidence coefficient of the model is larger than 0.5 to obtain the trained lane line recognition model.
3. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 2, wherein: in step 1.2, the bilateral filtering denoising is performed on the data set image before the data set is manufactured, and the formula is as follows:
Wherein: g (x, y) is the filtered image, ω s (i, j) is a spatial domain weight, ω r (i, j) is the gray domain weight, ω p For normalization parameters, I (I, j) is the noise image, Ω is the domain range at pixel (I, j); wherein:
ω p ＝∑ i，j∈Ω ω s (i，j)ω r (i，j)。
4. the drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 2, wherein: in step 1.3, the FReLU function is expressed as follows:
y＝Max((x s，i，j ，T(x s，i，j ))；
wherein: x is x s，i，j For the input value of the activation function at the s-th channel (i, j), T (x) s，i，j ) To be a spatial context feature extractor, the calculation is as follows:
wherein:
expressed in x s，i，j A parameter pooling window for the center, +.>
Representing the window parameter.
5. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 2, wherein: in step 1.6, the loss function formula of the lane line recognition model is as follows:
wherein: alpha is the weight coefficient of the weight coefficient,
IoU the intersection ratio of the calculated predicted detection frame and the actual detection frame;
wherein: w (w) gt Is the width of a real frame, h gt Is the height of the real frame, w is the pre-determinedThe width of the measurement frame, h is the height of the prediction frame.
6. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 1, wherein: in the step 2, with the obstacle as an aid, the process of generating the virtual center line or the side line is as follows: obtaining point cloud data, layering and dividing the point cloud data, filtering obstacles, constructing a three-dimensional grid, judging whether the cosine value of an angle between the plane method adjacent and the z-axis is 0-0.1, if not, determining that the point cloud data is not road boundary point cloud data, if so, determining that the point cloud data is road boundary point cloud data, denoising the road boundary point cloud data to obtain complete road boundary point cloud data, and finally sequencing the complete road boundary point cloud data to draw a virtual center line or a side line;
The method comprises the steps of establishing a relationship between point cloud data and a space grid and between the space grid and the space grid by utilizing a three-dimensional grid:
firstly, reading point cloud data, and searching the maximum X of three directions of three-dimensional coordinates X, Y and Z of the point cloud max ，Y max ，Z max And a minimum value X min ，Y min ，Z min ；
Secondly, a tiny quantity delta is given, and the maximum value of the anti-dead point cloud falls on a grid line and does not participate in operation;
by (X) max +δ，Y max +δ，Z max +δ) is the maximum value;
(X min -δ，Y min -δ，Z min delta) is a minimum value;
drawing a three-dimensional grid in the maximum range, and setting the length, width and height of the three-dimensional grid as X respectively grid ，Y grid ，Z grid The number of the grids in the X, Y and Z directions is n respectively x ，n y And n Z Obtaining the maximum value X of each space grid according to the length, width and height of the grid g_max ，Y g_max ，Z g_max And a minimum value X g_min ，Y g_min ，Z g_min ；
Has the following components
The denoising of the point cloud data is performed by adopting an improved K-means clustering algorithm, and the process is as follows:
1) Observing the point cloud, and selecting a proper initial cluster number N according to the target cluster number N;
2) Randomly selecting a point from the point cloud as a clustering center, and setting a proper threshold d according to the density of the point cloud and the distance between the noise point and the target clustering center;
3) Taking the point as a clustering center, circulating all the points, setting the coordinates of the clustering center point as (x, y, z), and setting the coordinates of a certain point as (x i ，y i ，z i ) The distance equation from a certain point to the clustering center point is applied as follows:
calculating the distance from all points to the clustering center point, setting a threshold value, clustering smaller than the threshold value to the center, and removing the points from the point cloud;
4) Removing the points selected as the clustering centers in the step 2), and respectively selecting the data clustered in the step 3) as the clustering centers; repeating the process until all the points meeting the threshold are clustered together, and then performing the next clustering after the clustering is finished;
5) Repeating the step 4) process, and obtaining N classes according to the initial clustering;
6) In the projection XOY plane of the road edge point cloud data after initial clustering, noise points are sheet-shaped, the road edge point cloud is strip-shaped, a bounding box is built for each type of the initial clustering, the height of the bounding box is in the range of 0.05-0.15 m according to the height threshold value and the length threshold value of the bounding box, the length of the bounding box is 0.9-1 times of the length of a grid, the noise points are removed, and finally n types of clustering results are obtained.
7. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 1, wherein: the obstacle detection model detects whether an obstacle protruding out of the ground exists or not, and the obstacle detection model comprises the following steps: taking a picture by a binocular camera, removing the influence of background information and road surface information, detecting the edge line segments of the U-V parallax map by carrying out Canny algorithm on the parallax map of the obstacle area, setting a threshold value for line segment detection by Hough transformation, and extracting the width information of the line segments of the obstacle area in the U parallax map and the height information of the line segments in the V parallax map after calculating the base points of the intersection of the obstacle and the road surface area, so as to accurately position the target obstacle; then, according to the obtained positioning information of the obstacle region, parallax of the region where the obstacle is located is further intercepted by the parallax map, and contour extraction is carried out on the obstacle in the selected region by adopting a region growing method; finally, overlapping the obstacle with the outline with an image edge line segment detected by a Canny algorithm, then establishing a detection frame of the obstacle according to a line segment fitting result of the U-V parallax image, and selecting the corresponding obstacle to realize the detection of the obstacle;
Wherein the effect of removing the background information and the road surface information is to calculate a texture main direction of the road surface area based on a 4-direction Gabor filter, the 4-direction Gabor filter being defined as:
wherein,,
ω 0 =2pi/λ is radial frequency, frequency multiplication constant c=pi/2, spatial frequency +.>
The Canny algorithm realizes the process of detecting the edge line segments of the U-V disparity map, which comprises the following steps:
1) The gaussian filter removes noise:
let G (x, y) be the smoothed image, the smoothing of image f (x, y) with a two-dimensional gaussian function G (x, y, σ) can be expressed as: g (x, y) =g (x, y, σ) ×f (x, y);
2) Obtaining gradient values and directions;
and calculating gradient amplitude and direction of the image with the noise removed by a two-dimensional Gaussian function first-order partial differential equation:
wherein k is a constant, and
the amplitude and the reverse direction of the gradient are obtained as follows:
wherein M x, y is the intensity information of the image edge, and theta x, y is the direction information of the image edge;
3) Non-maximum suppression;
performing non-maximum suppression processing on the calculated gradient amplitude values, and quantifying all possible directions into four edge directions of 0 degree, 135 degrees, 90 degrees and 45 degrees, wherein the four edge directions correspond to gradient directions perpendicular to the edge directions;
4) Determining edges by a double-threshold algorithm;
in the determination process of the edge pixel point, setting threshold coefficients TH and TL, wherein the ratio of the threshold coefficients TH to TL is 2:1 or 3:1; marking the pixel points larger than TH as edge points, and not marking the pixel points smaller than TL; and determining the pixel points between TH and TL by using an 8-connected region, marking the pixel points which are the same as the TL threshold value in the 8-connected region as edge connection points, and forming edge line segments by the edge connection points.
8. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 1, wherein: the lane line equation and polynomial curvature calculation is performed by a given data point p (x i ，y i ) Where i=1, 2,3, …, m, an approximate curve is obtained
The fitting polynomial is set to:
wherein a is 0 ，a 1 ，a 2 ，…，a k Using the sum of squares of the deviations as a loss function for polynomial coefficients, to make
The deviation from the relation f (x) representing the input x and the output y is minimal:
to obtain the optimal polynomial coefficient a 0 ，a 1 ，a 2 ，…，a k ；
Then by
Coefficients representing the order terms of the lane line fitting polynomial, the lane line equation is represented by a K-th order polynomial:
wherein: a, a k，j A kth order term coefficient representing a fitted polynomial of the deduced jth lane line; k=5.
9. The drivable zone dividing method for an unmanned logistics vehicle of claim 8, wherein: the calculation of the lane center line equation is to obtain an inverse transformation matrix through perspective transformation, transform the drawn left lane line and the drawn right lane line into an original image space through the inverse transformation matrix, combine the result with the original image, combine the lane line equation formula with the coefficient formula of the transverse and longitudinal proportional relation between the inverse perspective image and the actual road, and obtain the equation of the lane center line on the actual road in front of the vehicle, wherein the equation is as follows:
k x : ordinate scaling factor, k y : scale factor of abscissa, a 0 ，a 1 ，a 2 Representing curve equation parameters;
the calculation formula of the curvature radius is as follows:
wherein: k (K) 1 Is a curvature;
substituting the actual roadway center line equation into a curvature radius calculation formula:
thereby obtaining the radius of curvature of the road ahead of the vehicle.
10. The drivable zone dividing method for an unmanned logistics vehicle as set forth in claim 1, wherein: the dividing process of the drivable area is to input the center line into the deep labv3+ model, output the characteristic head by using the deep labv3+ model, and then output the drivable area by using the detection head.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310460686.1A CN116279592A (en) | 2023-04-26 | 2023-04-26 | Method for dividing travelable area of unmanned logistics vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310460686.1A CN116279592A (en) | 2023-04-26 | 2023-04-26 | Method for dividing travelable area of unmanned logistics vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116279592A true CN116279592A (en) | 2023-06-23 |
Family
ID=86794330
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310460686.1A Pending CN116279592A (en) | 2023-04-26 | 2023-04-26 | Method for dividing travelable area of unmanned logistics vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116279592A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116612451A (en) * | 2023-07-20 | 2023-08-18 | 城市之光(深圳)无人驾驶有限公司 | Road edge identification method, device and equipment for unmanned sweeper and storage medium |
| CN116883945A (en) * | 2023-07-21 | 2023-10-13 | 江苏省特种设备安全监督检验研究院 | Personnel identification positioning method integrating target edge detection and scale invariant feature transformation |
| CN117274932A (en) * | 2023-09-06 | 2023-12-22 | 广州城建职业学院 | Lane line self-adaptive detection method, system, device and storage medium |
| CN118015369A (en) * | 2024-02-20 | 2024-05-10 | 常州市双爱家私股份有限公司 | Obstacle detection system and method and electric lifting table |
| CN118469676A (en) * | 2024-07-11 | 2024-08-09 | 上海大华电子秤厂 | Intelligent city big data management platform based on intelligent bar code scale |
| CN118644974A (en) * | 2024-08-19 | 2024-09-13 | 中铁水利水电规划设计集团有限公司 | Automatic monitoring and early warning system and method for dyke seepage |
-
2023
- 2023-04-26 CN CN202310460686.1A patent/CN116279592A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116612451A (en) * | 2023-07-20 | 2023-08-18 | 城市之光(深圳)无人驾驶有限公司 | Road edge identification method, device and equipment for unmanned sweeper and storage medium |
| CN116612451B (en) * | 2023-07-20 | 2023-09-29 | 城市之光(深圳)无人驾驶有限公司 | Road edge identification method, device and equipment for unmanned sweeper and storage medium |
| CN116883945A (en) * | 2023-07-21 | 2023-10-13 | 江苏省特种设备安全监督检验研究院 | Personnel identification positioning method integrating target edge detection and scale invariant feature transformation |
| CN116883945B (en) * | 2023-07-21 | 2024-02-06 | 江苏省特种设备安全监督检验研究院 | Personnel identification positioning method integrating target edge detection and scale invariant feature transformation |
| CN117274932A (en) * | 2023-09-06 | 2023-12-22 | 广州城建职业学院 | Lane line self-adaptive detection method, system, device and storage medium |
| CN117274932B (en) * | 2023-09-06 | 2024-05-07 | 广州城建职业学院 | Lane line self-adaptive detection method, system, device and storage medium |
| CN118015369A (en) * | 2024-02-20 | 2024-05-10 | 常州市双爱家私股份有限公司 | Obstacle detection system and method and electric lifting table |
| CN118469676A (en) * | 2024-07-11 | 2024-08-09 | 上海大华电子秤厂 | Intelligent city big data management platform based on intelligent bar code scale |
| CN118644974A (en) * | 2024-08-19 | 2024-09-13 | 中铁水利水电规划设计集团有限公司 | Automatic monitoring and early warning system and method for dyke seepage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112101175B (en) | Expressway vehicle detection and multi-attribute feature extraction method based on local image | |
| CN116279592A (en) | Method for dividing travelable area of unmanned logistics vehicle | |
| CN111242041B (en) | Laser radar three-dimensional target rapid detection method based on pseudo-image technology | |
| CN108694386B (en) | Lane line detection method based on parallel convolution neural network | |
| CN104778721B (en) | The distance measurement method of conspicuousness target in a kind of binocular image | |
| CN111429514A (en) | Laser radar 3D real-time target detection method fusing multi-frame time sequence point clouds | |
| CN110728658A (en) | High-resolution remote sensing image weak target detection method based on deep learning | |
| CN111723721A (en) | Three-dimensional target detection method, system and device based on RGB-D | |
| CN105261017A (en) | Method for extracting regions of interest of pedestrian by using image segmentation method on the basis of road restriction | |
| CN112488046B (en) | Lane line extraction method based on high-resolution images of unmanned aerial vehicle | |
| CN109840483B (en) | Landslide crack detection and identification method and device | |
| CN116188999B (en) | Small target detection method based on visible light and infrared image data fusion | |
| CN113095152A (en) | Lane line detection method and system based on regression | |
| CN114463736A (en) | Multi-target detection method and device based on multi-mode information fusion | |
| CN113592894A (en) | Image segmentation method based on bounding box and co-occurrence feature prediction | |
| CN112446292B (en) | 2D image salient object detection method and system | |
| CN117475428A (en) | Three-dimensional target detection method, system and equipment | |
| CN112613392A (en) | Lane line detection method, device and system based on semantic segmentation and storage medium | |
| CN117037142A (en) | 3D target detection method based on deep learning | |
| CN114898321A (en) | Method, device, equipment, medium and system for detecting road travelable area | |
| CN113219472B (en) | Ranging system and method | |
| CN114332796A (en) | Multi-sensor fusion voxel characteristic map generation method and system | |
| CN116778262B (en) | Three-dimensional target detection method and system based on virtual point cloud | |
| CN117726880A (en) | Traffic cone 3D real-time detection method, system, equipment and medium based on monocular camera | |
| Zhang et al. | Fusion of lidar and camera by scanning in lidar imagery and image-guided diffusion for urban road detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |