CN117606477A - Path planning method for multi-information fusion - Google Patents
Path planning method for multi-information fusion Download PDFInfo
- Publication number
- CN117606477A CN117606477A CN202311358940.3A CN202311358940A CN117606477A CN 117606477 A CN117606477 A CN 117606477A CN 202311358940 A CN202311358940 A CN 202311358940A CN 117606477 A CN117606477 A CN 117606477A
- Authority
- CN
- China
- Prior art keywords
- path
- deviation
- cotton
- coordinates
- path planning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000004927 fusion Effects 0.000 title claims abstract description 24
- 229920000742 Cotton Polymers 0.000 claims abstract description 109
- 230000000007 visual effect Effects 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 20
- 230000008569 process Effects 0.000 claims abstract description 12
- 230000009471 action Effects 0.000 claims abstract description 4
- 238000012937 correction Methods 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 9
- 230000014509 gene expression Effects 0.000 claims description 8
- 239000013598 vector Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000006073 displacement reaction Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000008447 perception Effects 0.000 abstract description 5
- 238000001514 detection method Methods 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 3
- 238000012876 topography Methods 0.000 description 3
- DTTPWCNKTMQMTE-TYNNPWLESA-N [(1R,2S,3S,4S,5R,6S,8R,12S,13S,16R,19S,20R,21S)-14-ethyl-2-hydroxy-4,6,19-trimethoxy-16-methyl-9,11-dioxa-14-azaheptacyclo[10.7.2.12,5.01,13.03,8.08,12.016,20]docosan-21-yl] acetate Chemical compound CCN1C[C@]2(C)CC[C@H](OC)[C@]34[C@@H]2[C@H](OC(C)=O)[C@@]2(OCO[C@@]22C[C@H](OC)[C@H]5C[C@]3(O)[C@@H]2[C@H]5OC)[C@@H]14 DTTPWCNKTMQMTE-TYNNPWLESA-N 0.000 description 2
- DTTPWCNKTMQMTE-UHFFFAOYSA-N delphelatine Natural products O1COC2(C3C4OC)CC(OC)C4CC3(O)C34C(OC)CCC5(C)CN(CC)C4C21C(OC(C)=O)C53 DTTPWCNKTMQMTE-UHFFFAOYSA-N 0.000 description 2
- WNIGHBYIOLQQSJ-UHFFFAOYSA-N deltaline Natural products CCN1CC2(COC)CCC(O)C34C2C(OC(=O)C)C5(OCOC56CC(OC)C7CC3(O)C6C7O)C14 WNIGHBYIOLQQSJ-UHFFFAOYSA-N 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000035807 sensation Effects 0.000 description 2
- 241000238631 Hexapoda Species 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 235000014443 Pyrus communis Nutrition 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010413 gardening Methods 0.000 description 1
- 238000003306 harvesting Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010899 nucleation Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- OWWYREKLGMILGW-UHFFFAOYSA-N δline Chemical compound COC1C2C3C4(C5C6OC(C)=O)C(OC)CCC5(C)CN(CC)C4C46OCOC42CC(OC)C1C3 OWWYREKLGMILGW-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a path planning method for multi-information fusion, which comprises the following steps: step 1: global path planning; step 2: establishing a cotton ridge line identification model and self-training; step 3: the ridge line information obtained through the recognition of the cotton ridge line recognition model is calculated, and deviation between the recognized ridge line and the global path is used as visual deviation; step 4: establishing and training a path deviation prediction model, and then predicting the path deviation by using signals of a touch sensor in the working process as the touch deviation; step 5: and solving a local path planning result under the combined action of the touch deviation and the visual deviation through a weight distribution model, so that the global path is corrected in real time, and the local path planning is completed. The path planning method based on the multi-information fusion realizes the framework of three-information fusion of global path priori knowledge, visual detection posterior perception and tactile sensing posterior perception, and improves the reliability of path planning.
Description
Technical Field
The invention relates to the technical field of unmanned cotton picker path planning, in particular to a path planning method with multi-information fusion.
Background
Unmanned cotton pickers still face many challenges in practical applications, wherein planning a real-time, accurate, modifiable trajectory is critical to ensure proper operation of the cotton picker. The path planning of the cotton picker needs to plan a harvesting path from a starting point to an end point, but the planned path cannot meet the path of actual operation because the shape and the size of cotton lands are not regular enough and the change of cotton topography and possible obstacles are not considered. If the cotton picking machine only relies on GPS to position and move along the linear track, the cotton picking machine is easy to roll cotton and the picking head is easy to stop rotating, and large-area damage and safety accidents of the cotton are caused. In the path planning algorithm of the unmanned cotton picker, the design of the global path planning algorithm which accords with the operation rule of the cotton picker and has local correction is a key problem.
The global planning algorithm based on cotton field shape considers the law of cotton planting, and needs to acquire the operation direction, the start and stop end points of the paths and the distance between adjacent paths in advance. The straight line path is a straight line expression generated in the working area according to a certain breadth, and the turning path is generated by turning modes of arc shape, semicircle shape, pear shape and the like (Bo Fan, gao Wenjie, yan Ting and the like; development study of agricultural machinery automatic driving path planning technology [ J ]. Agricultural equipment technology 2021,47 (06): 4-6+10.), (Chen Peng; gardening electric tractor autonomous working path planning algorithm study [ D ]. Zhenjiang: jiangsu university, 2019.). However, the full-coverage cotton field path cannot be established by means of the information, and the requirements on straightness of the farmland are strict, so that the method cannot be directly used for irregular scenes at the cotton field head.
The ridge line identification method based on visual identification provides priori path information for agricultural machinery by identifying cotton ridges. The traditional method is used for fitting ridge lines, data processing images are used, and the positions of the ridge lines are obtained based on least square and probability Hough transformation, but the accuracy and the instantaneity cannot meet the requirements. Zhu Yihang et al of Zhejiang university establish a quater-Unet model for identifying the paths among cotton ridges in real time, and improve the accuracy and the instantaneity of identifying the cotton ridges. (Zhu Yihang, zhang Yanning, zhang Xiaomin, etc.. Real-time road recognition between cotton ridges based on semantic segmentation), (Slaugmenter D.C., chen P., curley R.G. vision Guided Precision guidance.precision guidance reference line recognition based on Hough transform), (Ma Gongxia, ma Mingjian, ma Na, etc.. Based on Hough transform, the data of the cotton ridge is recorded by the data of the cotton ridge. However, in cotton fields, the environmental characteristics are too complex and are easily affected by strong illumination, and direct and accurate travel path information cannot be provided for cotton pickers.
In addition, foreign John diel corporation adopts a ridge line alignment scheme based on a touch sensor, inspiration is derived from insect feelers, and the feelers sense the position and state of cotton, so that a path deviation signal for correcting the cotton picker in the driving process can be provided, and the path is corrected and the like. There are also devices in China that adjust the steering wheel of a car by tactile sensation, and related functions of the car can be triggered by the magnitude and position of the grip strength of the hand. (Zhu Bing, marz, qian Zhihui, etc. an intelligent automotive steering wheel device with multi-modal haptic sensations) this way of correcting direction by haptic signals is limited by the sensitivity of the sensor and does not guarantee a dynamic adjustment of the steering wheel by the signal.
Disclosure of Invention
The invention aims at solving the technical defect that the prior cotton picker path planning cannot meet the requirement, and provides a multi-information fusion path planning method which comprehensively utilizes various information of cotton topography, visual identification and tactile perception, effectively adapts to the cotton topography and carries out dynamic path adjustment, and meets the requirement of unmanned cotton picker path planning in a complex unstructured road.
The technical scheme adopted for realizing the purpose of the invention is as follows:
a path planning method of multi-information fusion comprises the following steps:
step 1: respectively planning the working paths of a parallel equidistant part of the cotton picker and a turning part of the cotton picker to finish the overall path planning;
step 2: establishing a cotton ridge line identification model and self-training;
step 3: in the operation process, ridge line information obtained by the recognition of the cotton ridge line recognition model is converted into UTM coordinates from pixel coordinates, and deviation between the recognized ridge line and a global path is calculated and used as visual deviation;
step 4: establishing and training a path deviation prediction model, and then predicting the path deviation by using signals of a touch sensor in the working process as the touch deviation;
step 5: and establishing and training a weight distribution model, solving a local path planning result under the combined action of the tactile deviation and the visual deviation through the weight distribution model, and fusing the local path planning result with the global path, so that the global path is corrected in real time, and the local path planning is completed.
In the technical scheme, the operation path planning of the parallel equidistant parts of the cotton picker comprises the following steps:
firstly, acquiring longitude and latitude coordinates of a cotton field through man-machine interaction software, and converting the longitude and latitude coordinates of the cotton field into a UTM coordinate system;
then, calculating according to an intercept equation to obtain an expression of a first straight line, and performing discrete processing on the straight line;
finally, according to the interval between two ridges of cotton fields, obtaining expressions of a plurality of straight lines parallel to the first straight line, and performing discrete treatment on the plurality of straight lines; and (5) completing the working path planning of the parallel equidistant parts.
In the technical scheme, a Bezier curve is adopted to fit the path of the turning part, and the operation path of the turning part of the cotton picker is planned;
the parameter equation of the Bezier curve is:
wherein the method comprises the steps ofT is the parameter variation of the Bezier curve calculated from t in the above formula i The value range is 0-1, 0 represents the starting point, and 1 represents the end point;
num points representing the number of discrete points on a turn path;
p0 and P3 are the travel end of each parallel straight-line path and the travel start of the next straight-line path, respectively;
p1 and P2 are control point coordinates that direct the straight path to extend outward.
In the technical scheme, unit vectors U0 and U3 are calculated firstly to ensure that the direction of the control point is correct;
the calculation formula of the unit vectors U0 and U3 is as follows:
then, the coordinates of the control points P1 and P2 are found:
P1＝P0-α*U0*||V0||，P2＝P3+β*U3*||V3||
(5)；
wherein alpha and beta are scaling factors, and the adjustment range is 0-1;
the coordinates of a certain discrete number of points on the Bezier curve can be obtained by using different t values in the parameter equation to obtain a smoother path by adjusting num points The curve is discretized according to the corresponding proportion, so that the operation path of the turning part is obtained.
In the technical scheme, in the step 2, cotton field pictures are collected in operation, global paths in UTM coordinates planned in the step 1 are converted into pixel coordinates through coordinate transformation, then ridge lines are marked in the cotton field pictures, and the self-marked cotton field pictures are used for self-training of a cotton ridge line recognition model through manual screening, so that an optimized cotton ridge line recognition model is obtained.
In the above technical solution, the coordinate transformation method includes: firstly, extracting UTM coordinates of path points in each straight line path in a planned parallel equidistant part, and converting the UTM coordinates into a camera coordinate system; then, converting the camera coordinate system into an image coordinate system; finally, the image coordinates are converted into pixel coordinates.
In the above technical solution, UTM coordinates of the path points are converted into a camera coordinate system, that is:
r, T represents a rotation matrix and a translation vector respectively, and is obtained through external parameter calibration of a camera;
converting the camera coordinate system into the image coordinate system, namely:
where f is the focal length of the camera;
converting the image coordinates into pixel coordinates, namely:
simultaneous (6) (7) (8) is available:
the external reference matrix and the internal reference matrix of the camera can be obtained through calibration, so that the mutual conversion from the pixel coordinates to the UTM coordinates can be realized.
In the above technical solution, in step 3, the deviation between the identified ridge line and the global path is:
in the above technical solution, in step 4, the tactile sensor records the voltage signal change of the cotton picker during the running process, uses the data obtained by the signal as input, uses the path deviation degree of the actual running path of the cotton picker and the planned global path as output, performs iterative training, learns the relationship between the tactile signal and the path deviation degree, and optimizes the path deviation prediction model through continuous iterative training.
In the above technical solution, in step 5, the weight of the haptic deviation is:
ω tactile ＝1-w1*|Δtrctile|
(11)；
w1 is the gain factor of the path deviation predicted by the tactile sensor;
the weight of the visual deviation can be expressed as:
max_displacement is the maximum offset of the ridge line;
after normalization processing, the haptic deviation weight and the visual deviation weight are respectively as follows:
final path correction amount Φ final The following formula may be used:
the global path and the correction are fused, so that a more accurate cotton picker operation path can be obtained.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the multi-information fusion path planning method, aiming at the problem of unmanned cotton picker path planning of a large-area cotton field, the global path planning algorithm based on the minimum turning radius constraint and the Bezier curve of the vehicle is provided by utilizing the characteristic of overall regularity of the ridge distance of the cotton field, the priori knowledge of the characteristics of the cotton field of human beings is fused into the planning algorithm, and the path planning efficiency is improved.
2. The invention provides a multi-information fusion path planning method, which aims at solving the problems of difficult visual detection, time consumption of manual marking, inaccuracy and the like of cotton ridge lines, provides an intelligent marking method based on automatic mapping of an effective cotton picking path of a vehicle to cotton field picture ridge lines, and further establishes a closed loop self-growing visual correction planning algorithm which forms a cotton picker 'planning while driving, marking, training and re-planning'.
3. According to the multi-information fusion path planning method, aiming at the problems that the visual detection is inaccurate and the path planning precision is affected due to the fuzzy edge of the cotton ridge line, the local path correction algorithm assisted by the cotton picking head touch sensor is provided, the framework of three information fusion of global path priori knowledge, visual detection posterior perception and tactile sensing posterior perception is realized, and the reliability of path planning is improved.
Drawings
FIG. 1 is a flow chart of the operation of a path planning method for multiple information fusion;
FIG. 2 is a schematic diagram of image coordinate conversion;
FIG. 3 is a schematic diagram of a comparison of a middle identified ridge line with a global path;
FIG. 4 is a schematic signal diagram of an analog tactile sensor;
FIG. 5 is a graph of the deviation of the tactile sensor signal from the global path, the actual path;
FIG. 6 is a global path generated in example 2;
fig. 7 is a graph showing the comparison of the corrected paths.
Detailed Description
The present invention will be described in further detail with reference to specific examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Example 1
A path planning method for multi-information fusion, as shown in figure 1, comprises the following steps:
step 1: planning the working paths of the parallel equidistant part of the cotton picker and the turning part of the cotton picker, and primarily completing global path planning.
Firstly, planning the working paths of parallel equidistant parts of the cotton picker, and then planning the working paths of a U-turn part of the cotton picker.
And acquiring cotton field longitude and latitude coordinates through man-machine interaction software, and converting the cotton field longitude and latitude coordinates into a UTM coordinate system. In the plane coordinate conversion process, two principles are followed, namely, the distance between two points on the map is unchanged, and the plane relation of two straight lines is unchanged.
Since inertial navigation positioning is used in cotton planting, seeding is performed according to rows, a plurality of pairs of parallel equidistant straight lines can be calculated to determine the working path. According to the intercept equation, the expression of the first straight line is:
y＝kx+b
(1)。
the equation of the straight line AB is calculated according to the substitution of the coordinates of A, B points converted to the UTM coordinate system into the expression, and then the straight line is discretely processed. The coordinates of the new point (x ', y') are calculated for each increment of Ω m step starting from the starting point, denoted as (x 1, y 1), (x 2, y 2), and the coordinates of A, B are denoted as (x ', y'):
in this way, after the working direction is determined, the corresponding linear expression is circularly calculated, and the points on the straight line are subjected to discrete processing, so that the path planning of the parallel equidistant parts can be completed. For equidistant requirements, the cotton fields need to be processed according to the actual spacing of the cotton fields, and the spacing of the cotton fields is relatively fixed due to the fact that the cotton fields are automatically sown by using a navigation tractor. The distance between two ridges of cotton fields can be obtained through field measurement, and parallel equidistant straight lines are planned according to the calculation method.
Because cotton picker is different from general agricultural machine, huge size makes it unable light to turn around between two groups of cotton (six ridges are a set), need to separate many ridges cotton just can turn around smoothly. As required, in two groups of cotton (n=2, 3,4, …) separated by 6n ridges, the invention adopts a Bezier curve to fit the path of the turning part, and the parameter equation of the Bezier curve is as follows:
wherein t is the parameter variation of the Bezier curve, and the calculation is carried out by t in the above formula i The value range is 0-1, 0 represents the starting point, and 1 represents the ending point. num (num) points Representing the number of discrete points on a u-turn path, P0 and P3 are the travel end point of each parallel straight-line path and the travel start point of the next straight-line path, respectively, and P1 and P2 are control point coordinates that guide the straight-line paths to extend outward.
In order to ensure the correct direction of the control point, the unit vectors U0 and U3 need to be calculated first:
finally, the coordinates of the control points P1 and P2 can be determined:
P1＝P0-α*U0*||V0||，P2＝P3+β*U3*||V3||
(5)；
the alpha and beta are scaling factors, the adjusting range of the scaling factors is 0-1, the scaling factors represent the outward extending degree of the turning paths, the larger the value is, the larger the turning radius is needed, and the scaling factors can be properly adjusted according to the size of cotton fields.
The coordinates of a certain discrete number of points on the Bezier curve can be obtained by using different t values in the parameter equation to obtain a smoother path by adjusting num points The size of the curve can be discretized according to the corresponding proportion, so that the operation path of the turning part is obtained.
Step 2: and (3) collecting cotton field pictures in the operation process, converting the global path in the UTM coordinates planned in the step (1) into pixel coordinates through coordinate conversion, and then self-labeling ridge lines in the cotton field pictures. And establishing a cotton ridge line identification model, and using the self-marked cotton field pictures for model self-training through manual screening to obtain the optimized cotton ridge line identification model more efficiently and accurately.
The coordinate transformation method is as shown in fig. 3: and extracting UTM coordinates of path points from each straight line path in the planned parallel equidistant parts, and converting the UTM coordinates into a camera coordinate system, namely:
r, T represents a rotation matrix and a translation vector respectively, and is obtained through calibration of an external parameter matrix of the camera; m1 is the extrinsic matrix of the camera.
The camera coordinate system is then converted into an image coordinate system, namely:
where f is the focal length of the camera.
Finally, converting the image coordinates into pixel coordinates, namely:
simultaneous (6) (7) (8) is available:
the camera external reference matrix M1 is calibrated and calculated by a monocular vision method, and the camera internal reference matrix M2 is calculated by a checkerboard calibration method, so that the mutual conversion from pixel coordinates to UTM coordinates can be realized.
Step 3: as shown in fig. 3, ridge line information obtained by the recognition of the cotton ridge line recognition model is converted from pixel coordinates to UTM coordinates, and deviation between the recognized ridge line and the global path is calculated as visual deviation.
When cotton picking operation is performed, the identified cotton ridge line is converted into a UTM coordinate system from a pixel coordinate system through the inverse process in the step 2, and the deviation value between the identified ridge line and two straight lines of the parallel equidistant part operation path planned in the step 1 is calculated:
and then taking the deviation value delta line as the visual deviation to carry out local path planning.
Step 4: a path deviation prediction model is built and trained, and then the path deviation is predicted as a haptic deviation using the signal of the haptic sensor.
The touch sensor records the voltage signal change of the cotton picker in the driving process, data obtained by the signal are used as input, the path deviation degree between the actual cotton picker driving path and the planned global path is used as output, iterative training is carried out, the relation between the touch signal and the path deviation degree is learned, and the path deviation prediction model is optimized through continuous iterative training.
The voltage signal is used for representing the path deviation degree and is used as the touch deviation to conduct local path planning.
Step 5: and establishing and training a weight distribution model, solving a local path planning result under the combined action of the tactile deviation and the visual deviation through the weight distribution model, and fusing the local path planning result with a global path, so that the global path is corrected in real time, and the local path correction is completed.
Wherein the weight of the haptic deviation can be expressed as:
ω tactile ＝1-w1*|Δtrctile|
(11)；
and w1 is a gain coefficient of the path deviation predicted by the tactile sensor, and the weight occupied by the tactile deviation can be adjusted by only adjusting w1 in practical application, so that the offset of the tactile signal to the path is adjusted. Δtrtive is the voltage signal change value measured by the tactile sensor.
The weight of the visual deviation can be expressed as:
and (3) calculating the offset delta line between the identified ridge line and the global path according to the ridge line identification result, and recording the maximum offset of the identified ridge line as max_displacement, wherein the maximum offset is given by the processing result in the step (3).
After normalization processing of the haptic deviation weight and the visual deviation weight:
setting the range of path correction as [ -1,1]Where 0 indicates no correction is required, -1 indicates maximum correction to the left and 1 indicates maximum correction to the right. Let the correction of the touch sensor be phi tactile Visual recognition ridge line correction is phi line . The final path correction amount Φ is calculated using the following formula final ：
And (3) fusing the global path planned in the step (1) with the local path correction, so as to obtain a more accurate cotton picker operation path.
Example 2
In this embodiment we will use a transformation based on latitude and longitude coordinates and UTM coordinates, in combination with deep learning and tactile sensor data, to plan and correct the path of the cotton picker in the cotton field. The following steps and data are specific:
table 1 example 1 specific parameter settings
And converting the coordinates of the starting point and the end point into a UTM coordinate system through longitude and latitude information provided by man-machine interaction software. Equation for calculating straight line AB from start point a (X1, Y1) end point B (X2, Y2): linear equation: y=kx+b where k= (Y2-Y1)/(X2-X1) and b=y1-kX 1, parallel equidistant straight-line paths are determined taking into account the cotton field actual spacing and job requirements. Coordinate points on each parallel equidistant straight line path are calculated. And then adopting a Bezier curve to fit and plan the path of the turning part. The final planned global path is shown in fig. 6.
And performing ridge line identification on the image of the cotton field by using a deep learning model. And extracting coordinates of the ridge lines under a pixel coordinate system, and converting the pixel coordinates into UTM coordinates by using an external reference matrix and an internal reference matrix of the camera. Recording UTM coordinates of the converted ridge lines asUTM coordinates of the line planned +.>And (3) calculating deviation:
the voltage signal changes recorded by the tactile sensor and the deviation of the actual travel path are used as inputs and the actual path deviation is used as an output. And (3) learning the relation between the voltage signal and the path deviation of the touch sense by using iterative training to obtain a path deviation prediction model, and obtaining the path deviation value corresponding to the voltage signal by using the model.
The actual spacing l=0.765m of cotton fields is measured, and the maximum deviation max of ridge lines obtained through visual identification is obtained deviation =0.1m, the voltage signal change Δtrtile=0.03y at this time is measured, the gain w1=0.5 of the path deviation value corresponding to the haptic voltage signal, and weights ω_tile and ω_line of the visual deviation and the haptic deviation are calculated:
ω tactile ＝1-w1*|Δtrctile|＝1-0.5*0.003＝0.9985
normalizing the processing weights to obtain eta_tactile and eta_line:
finally, the path correction amount is calculated according to the formula of the weight and the correction amount:
Φ final ＝η tactile *Δtrctile+η line *Δline＝0.554*0.003m+0.446*0.02m≈0.00942m。
the running environment of each algorithm is a domain controller carrying an Orin chip, the requirement on the calculation speed is met, the time consumption is short, and the real-time performance is high. As shown in fig. 7, compared with the global planned path, the deviation between the corrected local path and the actual path is reduced by 89%, and the path is more accurate and reliable.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (10)
1. A path planning method for multi-information fusion is characterized in that: the method comprises the following steps:
step 1: respectively planning the working paths of a parallel equidistant part of the cotton picker and a turning part of the cotton picker to finish the overall path planning;
step 2: establishing a cotton ridge line identification model and self-training;
step 3: in the operation process, ridge line information obtained by the recognition of the cotton ridge line recognition model is converted into UTM coordinates from pixel coordinates, and deviation between the recognized ridge line and a global path is calculated and used as visual deviation;
step 4: establishing and training a path deviation prediction model, and then predicting the path deviation by using signals of a touch sensor in the working process as the touch deviation;
step 5: and establishing and training a weight distribution model, solving a local path planning result under the combined action of the tactile deviation and the visual deviation through the weight distribution model, and fusing the local path planning result with the global path, so that the global path is corrected in real time, and the local path planning is completed.
2. The multiple information fusion path planning method of claim 1, wherein: the working path planning of the parallel equidistant parts of the cotton picker comprises the following steps:
firstly, acquiring longitude and latitude coordinates of a cotton field through man-machine interaction software, and converting the longitude and latitude coordinates of the cotton field into a UTM coordinate system;
then, calculating according to an intercept equation to obtain an expression of a first straight line, and performing discrete processing on the straight line;
finally, according to the interval between two ridges of cotton fields, obtaining expressions of a plurality of straight lines parallel to the first straight line, and performing discrete treatment on the plurality of straight lines; and (5) completing the working path planning of the parallel equidistant parts.
3. The multiple information fusion path planning method of claim 1, wherein: fitting the paths of the turning parts by using Bezier curves, and planning the operation paths of the turning parts of the cotton picker;
the parameter equation of the Bezier curve is:
wherein t is the parameter variation of the Bezier curve, and the calculation is carried out by t in the above formula i The value range is 0-1, 0 represents the starting point, and 1 represents the end point;
num points representing the number of discrete points on a turn path;
p0 and P3 are the travel end of each parallel straight-line path and the travel start of the next straight-line path, respectively;
p1 and P2 are control point coordinates that direct the straight path to extend outward.
4. A multiple information fusion path planning method according to claim 3 and characterized in that: firstly, calculating unit vectors U0 and U3 to ensure that the direction of a control point is correct;
the calculation formula of the unit vectors U0 and U3 is as follows:
then, the coordinates of the control points P1 and P2 are found:
P1＝P0-α*U0*||V0||，P2＝P3+β*U3*||V3||
(5)；
wherein alpha and beta are scaling factors, and the adjustment range is 0-1;
the coordinates of a certain discrete number of points on the Bezier curve can be obtained by using different t values in the parameter equation to obtain a smoother path by adjusting num points Is to curve according to the corresponding sizeThe proportion is discretized, so that a working path of the turning part is obtained.
5. The multiple information fusion path planning method of claim 1, wherein: and 2, collecting cotton field pictures in operation, converting the global path in the UTM coordinates planned in the step 1 into pixel coordinates through coordinate transformation, marking ridge lines in the cotton field pictures, and using the self-marked cotton field pictures for self-training of a cotton ridge line recognition model through manual screening to obtain an optimized cotton ridge line recognition model.
6. The multiple information fusion path planning method of claim 5, wherein: the coordinate transformation method comprises the following steps: firstly, extracting UTM coordinates of path points in each straight line path in a planned parallel equidistant part, and converting the UTM coordinates into a camera coordinate system; then, converting the camera coordinate system into an image coordinate system; finally, the image coordinates are converted into pixel coordinates.
7. The multiple information fusion path planning method of claim 6, wherein: converting UTM coordinates of the path points into a camera coordinate system, namely:
r, T represents a rotation matrix and a translation vector respectively, and is obtained through external parameter calibration of a camera;
converting the camera coordinate system into the image coordinate system, namely:
where f is the focal length of the camera;
converting the image coordinates into pixel coordinates, namely:
simultaneous (6) (7) (8) is available:
the external reference matrix and the internal reference matrix of the camera can be obtained through calibration, so that the mutual conversion from the pixel coordinates to the UTM coordinates can be realized.
8. The multiple information fusion path planning method of claim 1, wherein: in step 3, the deviation between the identified ridge line and the global path is:
9. the multiple information fusion path planning method of claim 1, wherein: in step 4, the touch sensor records the voltage signal change of the cotton picker in the running process, the data obtained by the signal is used as input, the path deviation degree of the actual cotton picker running path and the planned global path is used as output, iterative training is carried out, the relation between the touch signal and the path deviation degree is learned, and the path deviation prediction model is optimized through continuous iterative training.
10. The multiple information fusion path planning method of claim 1, wherein: in step 5, the weights of the haptic bias are:
ω tactile ＝1-w1*|Δtrctile|
(11)；
w1 is the gain factor of the path deviation predicted by the tactile sensor;
the weight of the visual deviation can be expressed as:
max_displacement is the maximum offset of the ridge line;
after normalization processing, the haptic deviation weight and the visual deviation weight are respectively as follows:
final path correction amount Φ final The following formula may be used:
the global path and the correction are fused, so that a more accurate cotton picker operation path can be obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311358940.3A CN117606477A (en) | 2023-10-19 | 2023-10-19 | Path planning method for multi-information fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311358940.3A CN117606477A (en) | 2023-10-19 | 2023-10-19 | Path planning method for multi-information fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117606477A true CN117606477A (en) | 2024-02-27 |
Family
ID=89944931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311358940.3A Pending CN117606477A (en) | 2023-10-19 | 2023-10-19 | Path planning method for multi-information fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117606477A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102013212153A1 (en) * | 2013-06-26 | 2014-12-31 | Robert Bosch Gmbh | Haptic feedback on the execution of an optimal trajectory in mobile machines |
| CN109240284A (en) * | 2018-08-10 | 2019-01-18 | 江苏大学 | A kind of autonomous paths planning method and device of unmanned agricultural machinery |
| CN109885063A (en) * | 2019-03-15 | 2019-06-14 | 江苏大学 | A kind of application robot farmland paths planning method merging vision and laser sensor |
| CN113467461A (en) * | 2021-07-13 | 2021-10-01 | 燕山大学 | Man-machine cooperation type path planning method under mobile robot unstructured environment |
| CN114911244A (en) * | 2021-06-17 | 2022-08-16 | 北京博创联动科技有限公司 | Ridge obstacle avoidance control method and device and agricultural automatic driving equipment |
| JP2022185941A (en) * | 2021-06-03 | 2022-12-15 | 日立チャネルソリューションズ株式会社 | Agricultural support system and movable body |
-
2023
- 2023-10-19 CN CN202311358940.3A patent/CN117606477A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102013212153A1 (en) * | 2013-06-26 | 2014-12-31 | Robert Bosch Gmbh | Haptic feedback on the execution of an optimal trajectory in mobile machines |
| CN109240284A (en) * | 2018-08-10 | 2019-01-18 | 江苏大学 | A kind of autonomous paths planning method and device of unmanned agricultural machinery |
| CN109885063A (en) * | 2019-03-15 | 2019-06-14 | 江苏大学 | A kind of application robot farmland paths planning method merging vision and laser sensor |
| JP2022185941A (en) * | 2021-06-03 | 2022-12-15 | 日立チャネルソリューションズ株式会社 | Agricultural support system and movable body |
| CN114911244A (en) * | 2021-06-17 | 2022-08-16 | 北京博创联动科技有限公司 | Ridge obstacle avoidance control method and device and agricultural automatic driving equipment |
| CN113467461A (en) * | 2021-07-13 | 2021-10-01 | 燕山大学 | Man-machine cooperation type path planning method under mobile robot unstructured environment |
Non-Patent Citations (2)
| Title |
|---|
| JINGBIN LI; BINGQI CHEN; YANG LIU; TAO ZHA: "Study on image detection method of navigation route for cotton harvester", APPLIED MECHANICS AND MATERIALS, vol. 246, 14 November 2013 (2013-11-14), pages 219 - 24 * |
| 罗晨晖: "多信息特征融合的作物行识别与路径规划研究", 中国优秀硕士学位论文全文数据库 农业科技辑, no. 02, 15 February 2021 (2021-02-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11892855B2 (en) | Robot with perception capability of livestock and poultry information and mapping approach based on autonomous navigation | |
| CN102914967B (en) | Autonomous navigation and man-machine coordination picking operating system of picking robot | |
| Zhang et al. | Tractor path tracking control based on binocular vision | |
| WO2022021525A1 (en) | Automatic navigation and steering system for unmanned tractor | |
| CN107065887A (en) | Backward air navigation aid in omni-directional mobile robots passage | |
| WO2018076725A1 (en) | Automatic calibration method of angle sensor for automatic drive control system of farm machine | |
| Reinstein et al. | Terrain adaptive odometry for mobile skid-steer robots | |
| CN106125740A (en) | Vision navigation control method and system based on fuzzy algorithmic approach | |
| CN113052369B (en) | Intelligent agricultural machinery operation management method and system | |
| CN115900726A (en) | Navigation path generation method based on crop geographic coordinate positioning | |
| Wei et al. | Design of robot automatic navigation under computer intelligent algorithm and machine vision | |
| CN116012438A (en) | Agricultural robot ridge searching method based on image recognition | |
| CN110134129B (en) | Automatic seedling avoiding system of paddy field weeding machine based on deep learning and ADRC control | |
| CN114063615B (en) | Reversing navigation control method and system of intelligent pesticide spraying vehicle among ridges in greenhouse | |
| CN117606477A (en) | Path planning method for multi-information fusion | |
| CN117115769A (en) | Plant detection and positioning method based on semantic segmentation network | |
| Juntao et al. | Research of AGV positioning based on the two-dimensional Code Recognition Method | |
| CN115993089B (en) | PL-ICP-based online four-steering-wheel AGV internal and external parameter calibration method | |
| CN116739739A (en) | Loan amount evaluation method and device, electronic equipment and storage medium | |
| Bakken et al. | Robot-supervised learning of crop row segmentation | |
| US20220377973A1 (en) | Method and apparatus for modeling an environment proximate an autonomous system | |
| CN109542099B (en) | Agricultural machinery control method | |
| CN113587946A (en) | Visual navigation system and method for field agricultural machine | |
| CN115280960A (en) | Combine harvester steering control method based on field vision SLAM | |
| CN110125945B (en) | Plant row following method of harvesting robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |