CN116882612A

CN116882612A - Intelligent agricultural machinery path planning method and device based on remote sensing image and deep learning

Info

Publication number: CN116882612A
Application number: CN202311156169.1A
Authority: CN
Inventors: 余涛; 孙兴冻; 郑玉凯; 李宁; 尹志强; 刘子曦
Original assignee: Anhui Agricultural University AHAU
Current assignee: Anhui Agricultural University AHAU
Priority date: 2023-09-08
Filing date: 2023-09-08
Publication date: 2023-10-13

Abstract

The invention provides an intelligent agricultural machinery path planning method and device based on remote sensing images and deep learning, wherein the method comprises the following steps: pre-training a deep learning target detection network by using an unmanned aerial vehicle aerial photograph; detecting crop positions using a transfer learning network and Yolov5 modifying the detection layer code; marking the row starting point coordinates of crops as initial point coordinates on the detected output image, finding out the point with the minimum Euclidean distance with the initial point in the attention range of the initial point according to the detected unordered crop position points, updating the point with the minimum distance as the initial point, fitting the initial point set by using a Bezier curve to form seedling zones, and repeating the steps until all the seedling zones are all found out; and carrying out global path planning based on the seedling zone. The method effectively solves the problems of unsatisfactory seedling zone identification effect and seedling pressing caused by crop row inclination and seedling missing, and further improves the farmland operation coverage rate and the unmanned operation efficiency.

Description

Intelligent agricultural machinery path planning method and device based on remote sensing image and deep learning

Technical Field

The invention relates to the field of deep learning and path planning, in particular to an intelligent agricultural machinery path planning method and device based on remote sensing images and deep learning.

Background

With the rapid development of automatic navigation technology, robots, unmanned agricultural machines and unmanned aerial vehicles are widely used in agricultural production. Agricultural robots have gradually replaced humans in engaging in agricultural production activities such as farming, spraying, fertilizing, harvesting, etc. The unmanned agricultural machinery brings remarkable advantages to the application, such as labor intensity reduction, operation efficiency and quality improvement and energy consumption reduction. Therefore, a reasonable agricultural machinery path planning algorithm is particularly important, and the reasonable path planning algorithm is an important step for solving the problems of ageing and shortage of rural labor force, meeting the requirements of agricultural production and realizing agricultural modernization.

The existing path planning algorithm is mostly based on a traditional experience or mathematical model, has low applicability to the distribution situation of real farmland crops, is easy to damage crops, and particularly has unsatisfactory performance under the conditions of crop row inclination or seedling missing and the like. Therefore, it is necessary to develop a more intelligent agricultural machinery path planning algorithm adapted to the distribution of real farmland crops so as to adapt to different working environments and improve the efficiency and quality of farmland operation.

Disclosure of Invention

In view of the above, the invention aims to provide an intelligent agricultural machinery path planning method and device based on remote sensing images and deep learning, which utilizes the remote sensing images to acquire real-time information of farmlands, and combines the deep learning technology to identify the positions and distribution conditions of crops so as to generate a reasonable agricultural machinery operation path. By introducing the deep learning target detection network, the algorithm can more accurately identify the positions of crops, and factors such as crop row inclination, seedling shortage and the like are considered in the path planning process, so that damage is reduced, and the coverage rate and efficiency of farmland operation are improved. When the real farmland path is planned, the situation that the seedling zone identification effect is not ideal due to the inclination of crop rows and the lack of seedlings can be well overcome, meanwhile, the seedling pressing is avoided, and the farmland operation coverage rate and the unmanned operation efficiency are further improved.

Based on the above object, in a first aspect, the present invention provides an intelligent agricultural machinery path planning method based on remote sensing images and deep learning, comprising the following steps:

s1: aerial photographing is carried out through the unmanned aerial vehicle, and a remote sensing image of a farmland area is obtained;

s2, identifying and positioning the crop position in the remote sensing image by using a target detection network based on deep learning;

s3, determining crop row starting point coordinates based on the detected crop position, and setting a predefined attention range around the crop row starting point coordinates to limit the selection range of candidate points;

s4, selecting a point with the smallest distance from the initial point as the next initial point in the attention range of each starting point based on Euclidean distance calculation, and repeating the step to generate a seedling zone;

s5, fitting the initial point set through a Bezier curve to generate a smooth agricultural machine operation path which accords with the farmland shape.

As a further scheme of the invention, the intelligent agricultural machinery path planning method based on remote sensing images and deep learning further comprises the following steps:

according to the turning radius of the agricultural machinery and the width of the mounting machine, taking the minimum energy consumption and the minimum operation time as optimization targets, and carrying out global path planning on the generated seedling belt;

Setting a spacing distance of an agricultural machine operation path, and selecting an optimal turning strategy based on operation breadth so as to ensure uniform coverage and efficient completion of the operation;

the control points of the Bezier curve are in one-to-one correspondence with the initial point sets in the generated seedling zone, so that the walking path of the agricultural machinery is determined, and the accuracy and the high efficiency of farmland operation are ensured.

As a further scheme of the invention, the target detection network is a pre-trained Yolov5 model, and detection layer codes are modified to adapt to farmland crop characteristics, so that the target detection network is used for accurately identifying and positioning crop positions in the remote sensing image.

In step S3, the coordinates of the starting points of the crop rows are marked as the coordinates of the initial points on the output image after the crop positions are detected in step S2, the point with the smallest euclidean distance with the initial points is found out in the attention range of the initial points according to the unordered crop positions detected in step S2, the point with the smallest distance is updated as the initial point, the bezier curve is used for fitting the initial point set to form the seedling zones, and the step is repeated until all the seedling zones are found out.

As a further scheme of the present invention, step S1 uses a path planning performed by deep learning in combination with aerial photographs, and the path planning method by deep learning in combination with aerial photographs includes the following steps:

Collecting and preparing a data set containing aerial photo corn seedling images and marking frames, and ensuring that the data set is based on a marking file (txt) in a YOLO format;

creating a deep learning network model for target detection based on the data set, wherein the deep learning network model is a Yolov5 model with an attention mechanism for enhancing extraction of main features of corn seedlings, and the Yolov5 model is a basic model for target detection constructed based on a Yolov5 frame;

the pre-trained weight is used for transfer learning, a channel attention mechanism (Channel Attention Module, CAM) and a space attention mechanism (Spatial Attention Module, SAM) are integrated in the structure of the Yolov5 model, so that the convergence speed of the model is accelerated, the performance is improved, the extraction capacity of the model to main features of corn seedlings is improved, and the response to other irrelevant features is reduced, so that the accuracy of a target detection model is improved;

through transfer learning, the pre-trained weight is used for fine adjustment of the Yolov5 model, and the trained model is operated on a farmland remote sensing image so as to accurately identify and position corn seedlings.

As a further scheme of the invention, a Yolov5 modified detection layer code is used in the step S2, and on the basis, when the resolution of the aerial image is overlarge, the intelligent agricultural machinery path planning method based on the remote sensing image and the deep learning comprises the following steps of:

Decomposing the farmland remote sensing image into a plurality of overlapped small image blocks, wherein each small image block covers a proper area;

sending each small image block into a modified Yolov5 network for target detection to obtain crop position information in each small image block;

recovering crop position information from detection results of each small image block, and calculating coordinate values relative to an original image;

performing non-maximum suppression on the positions of crops detected in all small image blocks, and removing repeated detection results, so that the detection effect is improved;

and obtaining accurate farmland crop position information by using the relative coordinate values and the detection result subjected to non-maximum value inhibition, and providing input for subsequent path planning.

The path planning method uses the steps of deep learning and attention mechanism and the processing method under the condition of larger image resolution so as to improve the accuracy and efficiency of target detection.

As a further aspect of the present invention, determining crop line start coordinates based on the detected crop position includes the steps of:

forming unordered crop position point sets based on the crop coordinates detected in the step S2Starting with the marked crop line +. >For the first initial point, a range of attention is set to define the abscissa of the attention rangeRest and find ++>Points within the attention range in the set of location points +.>And then (I) is->Middle through European distance calculation formula->Find out and->Points with minimum Euclidean distance ∈ ->And will->Updating the point to be a new initial point, repeating the above steps until the initial point set +.>。

As a further aspect of the present invention, when generating a smooth agricultural machine working path conforming to a farmland shape by fitting an initial point set by a bezier curve, the method comprises the steps of:

in farmland seedling zone generation, the control point of Bezier curve is combined with the initial point of each seedling zoneCorrespondingly, generating a smooth seedling zone which accords with the farmland shape;

is provided withControl points->Wherein->Is the order of the Bezier curve, bezier curve +.>The parametric equation for (a) is defined as follows:

，/>from 0 to->

Wherein, the liquid crystal display device comprises a liquid crystal display device,is in the value range->Parameter of->，/>Is->Control points->Is the order of the curve, +.>The calculation mode of the value is as follows: />。

As a further aspect of the present invention, the intelligent agricultural machinery path planning method for remote sensing image and deep learning further includes:

according to the turning radius R of the agricultural machine, the width W of the mounting machine at the tail end of the agricultural machine takes the minimum energy consumption and the minimum operation time as the optimization total target;

Setting the distance between working paths of agricultural machinery, and selecting according to the working breadthOr->An optimal turning strategy;

wherein, the global path planning is based on the generation of seedling belts, when the width of the mounting machine at the tail end of the agricultural machine is W, the walking path of the agricultural machine is determined by translating the seedling belts according to W/2, so that the agricultural machine achieves the minimum energy consumption and the minimum operation time, and then the agricultural machine passes through the step of R<W/2, preferably usingTurning mode, when R>W/2 is preferably +.>And selecting a turning strategy to link the walking path.

In a second aspect, the present invention provides an intelligent agricultural machinery path planning apparatus based on remote sensing images and deep learning, comprising:

the image acquisition module is used for acquiring remote sensing images of farmland areas through aerial photographing of the unmanned aerial vehicle;

the target detection module uses a pre-trained Yolov5 model, modifies the detection layer code to adapt to the characteristics of farmland crops, and accurately identifies and positions the crops in the remote sensing image;

an initial point determining module for determining crop line start coordinates based on the detected crop position and setting a predefined attention range around the crop line start coordinates to define a selection range of candidate points;

The path generation module is used for selecting a point with the smallest distance from the initial point as the next initial point based on Euclidean distance calculation in the attention range of each starting point, and repeating the step to generate a seedling zone;

the path optimization module performs global path planning on the generated seedling belt by taking minimum energy consumption and minimum operation time as optimization targets according to the turning radius of the agricultural machinery and the width of the mounting machine;

the turning strategy module is used for setting the spacing distance of the operation paths of the agricultural machinery, and selecting an optimal turning strategy based on the operation breadth so as to ensure uniform coverage and efficient completion of the operation;

and the Bezier curve fitting module is used for corresponding control points of the Bezier curve to the initial point sets in the generated seedling zone one by one, determining the walking path of the agricultural machinery and ensuring the accuracy and the high efficiency of farmland operation.

As a further aspect of the invention, the objective detection module uses a pre-trained Yolov5 model that incorporates an attention mechanism.

As a further scheme of the invention, when the target detection module processes the high-resolution aerial photo, the image is decomposed into a plurality of overlapped small image blocks, and target detection and non-maximum suppression are carried out on each small image block so as to improve the accuracy and efficiency of target detection.

As a further scheme of the invention, the path generation module generates a smooth agricultural machine operation path which accords with the farmland shape through Euclidean distance calculation and Bezier curve fitting.

As a further scheme of the invention, the path generation module determines the agricultural machinery walking path according to the initial point set and the turning radius, takes the minimum energy consumption and the minimum operation time as optimization targets, and realizes intelligent planning of the agricultural machinery path.

In yet another aspect of the present invention, there is also provided a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, performs any one of the above-described intelligent agricultural machinery path planning methods based on remote sensing images and deep learning according to the present invention.

In yet another aspect of the present invention, there is also provided a computer readable storage medium storing computer program instructions that when executed implement any of the above-described intelligent agricultural machinery path planning methods based on remote sensing images and deep learning according to the present invention.

Compared with the prior art, the intelligent agricultural machinery path planning method and device based on remote sensing images and deep learning provided by the invention have the following beneficial effects:

1. Efficient and accurate crop position identification: by using a pre-trained Yolov5 model and adding an attention mechanism, the crop position can be efficiently and accurately identified and positioned in the remote sensing image, and key input data of path planning is provided.

2. Adapting the high resolution image: for the aerial image with high resolution, the image is decomposed into small blocks, the modified Yolov5 network is utilized for target detection, and non-maximum suppression is carried out, so that the complexity and the calculation cost of the image can be effectively applied, and the precision and the speed of crop detection are improved.

3. Intelligent path planning: and (3) generating a smooth agricultural machine operation path which accords with the farmland shape by calculating the Euclidean distance and utilizing Bezier curve fitting. By combining global path planning and turning strategy optimization, energy consumption and minimum operation time can be minimized, intelligent path planning is realized, and the operation efficiency and quality of the agricultural machinery are improved.

4. The method is suitable for actual requirements of agricultural machinery: according to the turning radius of the agricultural machinery and the width of the mounting machine, the path planning is flexibly adjusted, the uniform coverage and the high-efficiency completion of the operation path are ensured, and the agricultural production benefit is improved.

5. The energy consumption and the loss are reduced: the intelligent path planning can effectively reduce the back and forth movement of the agricultural machinery in the operation process, reduce energy consumption, avoid unnecessary loss and damage, and further improve the sustainability of agricultural production.

In summary, the intelligent agricultural machinery path planning method and device based on remote sensing images and deep learning are used for recognizing the positions of crops by utilizing the target detection network of the attention mechanism by combining the remote sensing images and the deep learning technology, and further the agricultural machinery operation path is generated based on Euclidean distance calculation and Bezier curve fitting. Meanwhile, aiming at the high-resolution aerial image, the image is decomposed into small blocks, and is detected through a Yolov5 network, and the detection result is subjected to non-maximum suppression, so that the accuracy and the efficiency of target detection are improved. The combination of the steps enables the agricultural machinery to plan the path more accurately during farmland operation, improves the operation efficiency and quality, reduces the energy consumption, has important application value in agricultural production, can effectively improve the operation efficiency of the agricultural machinery and reduce the resource consumption, and provides a feasible solution for agricultural modernization.

These and other aspects of the application will be more readily apparent from the following description of the embodiments. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.

In the figure:

fig. 1 is a flowchart of an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the application.

Fig. 2 is a crop identification chart in an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the application.

Fig. 3 is a seedling zone identification chart in an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the application.

FIG. 4 is a schematic diagram of an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the applicationA model turning diagram.

FIG. 5 is a schematic diagram of an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the applicationA model turning diagram.

FIG. 6 is a schematic diagram of an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the applicationTurn strategy path planning diagram.

FIG. 7 is a schematic diagram of an intelligent agricultural machinery path planning method based on remote sensing images and deep learning according to an embodiment of the applicationTurn strategy path planning diagram.

Description of the embodiments

The present application will be further described with reference to the accompanying drawings and detailed description, wherein it is to be understood that, on the premise of no conflict, the following embodiments or technical features may be arbitrarily combined to form new embodiments.

In order to make the objects, technical solutions and advantages of the present application more apparent, the following embodiments of the present application will be described in further detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

It should be noted that, in the embodiments of the present application, all the expressions "first" and "second" are used to distinguish two non-identical entities with the same name or non-identical parameters, and it is noted that the "first" and "second" are only used for convenience of expression, and should not be construed as limiting the embodiments of the present application. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or device that comprises a list of steps or elements does not include other steps or elements inherent to the apparatus.

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations.

Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.

Aiming at the problems that the existing path planning method is not strong in applicability to the distribution situation of real farmland crops and is easy to damage the crops, the application provides an intelligent agricultural machine path planning method based on remote sensing images and deep learning.

Referring to fig. 1, the embodiment of the application provides an intelligent agricultural machinery path planning method based on remote sensing images and deep learning, which aims to effectively solve the path planning problem in farmland operation and improve the efficiency and quality of agricultural machinery operation. The method comprises the following steps:

S1, pre-training a deep learning target detection network by using an unmanned aerial vehicle aerial photograph;

s2, detecting the position of the crop by using a transfer learning network and Yolov5 for modifying the code of the detection layer;

s3, marking the coordinates of the starting points of the crop rows on the output image detected in the S2 as the coordinates of the initial points, finding out the point with the minimum Euclidean distance with the initial points in the attention range of the initial points according to the unordered crop position points detected in the S2, updating the point with the minimum distance as the initial point, fitting the initial point set by using a Bezier curve to form seedling zones, and repeating the steps until all the seedling zones are found out;

s4, global path planning is conducted based on the S3.

Therefore, when agricultural machinery path planning is performed, aerial photographing is performed through the unmanned aerial vehicle, high-resolution remote sensing images of the farmland area are acquired, and the images are used as input data for subsequent target detection and path planning processes. Next, in step S2, the crop position in the farm is identified and located using a deep learning based target detection network, i.e. a Yolov5 model that is pre-trained and incorporates an attention mechanism. In order to adapt to different farmland environments and crop distribution characteristics, under the condition of larger resolution of aerial images, a target image is decomposed into a plurality of overlapped small image blocks by modifying a detection layer code, the overlapped small image blocks are sent into a Yolov5 network for detection, and detection effects are improved by using methods such as non-maximum suppression and the like. Subsequently, in step S3, crop row start coordinates are determined based on the detected crop position, and a predefined attention range is set to define the selection range of candidate points. This step is repeated to generate a seedling belt by calculating the euclidean distance and selecting the point having the smallest distance from the initial point as the next initial point. Then, the initial point set is fitted by using a Bezier curve to form a smooth agricultural machine working path conforming to the farmland shape.

Finally, in step S4, global path planning is performed based on the generated seedling strips. And optimizing a path planning scheme by taking the minimum energy consumption and the minimum working time as optimization targets according to the turning radius of the agricultural machinery and the width of the mounting machine. Meanwhile, the distance between the working paths of the agricultural machinery is set, and an optimal turning strategy is selected so as to ensure uniform coverage and efficient completion of the operation.

The step S1 uses path planning performed by deep learning in combination with aerial photo, and is a target detection network added with an attention mechanism, and specifically includes the following steps:

preparing a data set: collecting and preparing a data set containing aerial photo corn seedling images and marking frames, and ensuring that the data set is based on a marking file (txt) in a YOLO format;

creating a Yolov5 model: creating a basic model for target detection using a Yolov5 framework;

adding CBAM attention mechanisms: adding a CBAM attention mechanism module into a model structure of Yolov5, enhancing the extraction of the main features of corn seedlings in the feature map by the model, weakening the attention to other irrelevant features, and improving the accuracy of the target detection model;

migration learning: and the pre-trained weight is used for transfer learning, so that the convergence speed of the model is increased and the performance is improved.

In the step S2, a network based on S1 transfer learning is used, the Yolov5 of a detection layer code is modified, under the condition that the resolution of an aerial image is too large, a target image is decomposed into a plurality of images and then sent into the Yolov5 network for detection, all the images are recovered, the relative values of coordinates are calculated, and non-maximum suppression is carried out collectively, so that a better detection effect is achieved.

Wherein, step S3 is specifically;

(1) Forming unordered crop position point sets based on the crop coordinates detected in the step S2Starting with the marked crop line +.>For the first initial point, an attention range is set to find +.>Points within the attention range in the set of location points +.>And then (I) is->Middle through European distance calculation formula->Find out and->Points with minimum Euclidean distance ∈ ->And will->Updating the point to be a new initial point, repeating the above steps until the initial point set +.>。

in farmland seedling zone generation, the control point of Bezier curve is combined with the initial point of each seedling zone Correspondingly, generating a smooth seedling zone which accords with the farmland shape;

，/>from 0 to->

setting the distance between working paths of agricultural machinery, and selecting according to the working breadthIs->An optimal turning strategy;

the overall path planning of the invention is based on the seedling zone found by S3, when the width of the mounting machine at the tail end of the agricultural machine is W, the invention determines the walking path of the agricultural machine by translating the seedling zone according to W/2, so as to achieve the minimum energy consumption and the minimum operation time, and then, when R is used<W/2, preferably usingTurning mode, when R>W/2 is preferably +.>Selecting a turning strategy to link itA walking path.

The intelligent agricultural machinery path planning method based on remote sensing images and deep learning fully utilizes advanced image processing and deep learning technology, can accurately identify the positions of crops, generates an efficient operation path, and has strong adaptability in different farmland environments. It is hopeful to provide an advanced intelligent solution for agricultural production and promote the development of agricultural modernization. Meanwhile, the method can reduce energy consumption and loss, reduce production cost, promote agricultural sustainable development and have wide application prospects.

When the intelligent agricultural machine path planning method based on remote sensing images and deep learning is used for agricultural machine path planning, referring to fig. 1 to 7, the method specifically comprises the following steps:

s10: aerial photographing is carried out through the unmanned aerial vehicle, and a remote sensing image of a farmland area is obtained.

In the step, aerial photographing is carried out by using an unmanned aerial vehicle, and a remote sensing image of a farmland area is obtained. The images can capture the actual conditions of farmlands, including crop distribution, land shape, obstacle positions and the like. The remote sensing image is used as input data, and basic data is provided for subsequent path planning.

S20: using a deep learning based target detection network, crop locations are identified and located in the remote sensing image.

In this step, a target detection network based on deep learning is adopted to identify and locate the crop position in the remote sensing image. The key to this step is to select a target detection model suitable for the characteristics of the farm crop and accurately identify and locate the crop position in the farm image using the pre-trained Yolov5 model as a basis.

S30: based on the detected crop position, crop line origin coordinates are determined and a predefined attention range is set around the crop line origin coordinates to define a selection range of candidate points.

In this step, crop line start coordinates are determined from the detected crop position and a predefined attention range is set around it. This range will limit the selection range of subsequent candidate points for path planning within a limited range, thereby reducing the amount of computation and improving the efficiency of path planning.

S40: in the attention range of each starting point, a point with the smallest distance from the initial point is selected as the next initial point based on Euclidean distance calculation, and the step is repeated to generate a seedling zone.

In this step, a point having the smallest distance from the initial point is selected as the next initial point based on the euclidean distance calculation for the attention range of each of the starting points. The purpose of this step is to create a seedling belt, ensuring continuity and efficiency of the agricultural work. By repeating this step, a seedling zone is gradually formed connecting the initial points, ready for subsequent path planning.

S50: and generating a smooth agricultural machine operation path which accords with the farmland shape through Bezier curve fitting of the initial point set.

In this step, a smooth agricultural work path conforming to the shape of the farm field is generated by fitting the initial set of points by Bezier curves. The smoothness of the path can be ensured by using the Bezier curve, so that the pesticide can smoothly operate, the influence of the farmland shape on path planning is reduced, and the operation quality is improved.

In some embodiments, the following steps are further added to optimize path planning:

(1) Global path planning and energy consumption optimization:

the path planning objectives, including minimizing energy consumption and minimizing working time, are optimized for the turning radius of the agricultural machine and the breadth of the mounting implement. And the generated seedling zone is optimized through global path planning, so that the operation process is more efficient, and the energy consumption is reduced.

(2) Turning strategies and jobs evenly cover:

and setting the spacing distance of the operation paths of the agricultural machinery, and selecting an optimal turning strategy to ensure uniform coverage and efficient completion of the operation. Through reasonable turning strategy, repeated operation and missing operation are avoided, and the operation efficiency is improved.

(3) The path control point corresponds to an agricultural machine walking path:

the control points of the Bezier curve are in one-to-one correspondence with the initial point sets in the generated seedling zone, so that the walking path of the agricultural machinery is ensured to be consistent with the planned path, and the accuracy and the high efficiency of farmland operation are ensured.

In this embodiment, the target detection network is a pre-trained Yolov5 model, and the detection layer codes are modified to adapt to the characteristics of farmland crops, so as to accurately identify and position the crop in the remote sensing image.

In this embodiment, in step S30, the coordinates of the starting points of the crop rows are marked as the coordinates of the initial points on the output image after the crop positions are detected in step S20, the point with the smallest euclidean distance to the initial points is found in the attention range of the initial points according to the unordered crop position points detected in step S20, the point with the smallest distance is updated as the initial point, the bezier curve is used to fit the initial point set to form the seedling strips, and the step is repeated until all the seedling strips are found.

In this embodiment, step S10 uses path planning performed by deep learning in combination with aerial photographs, where the path planning method by deep learning in combination with aerial photographs includes the following steps:

In this embodiment, the step S20 uses Yolov5 with modified detection layer codes, and on this basis, when the resolution of the aerial image is too large, the intelligent agricultural machine path planning method based on remote sensing images and deep learning includes the following steps:

In this embodiment, the intelligent agricultural machinery path planning method for remote sensing image and deep learning further includes:

In conclusion, the intelligent agricultural machinery path planning method based on remote sensing images and deep learning can realize the high efficiency, accuracy and sustainability of agricultural machinery operation through effective crop positioning, path planning and optimizing strategies, and provides powerful technical support for modern agricultural production.

It is noted that the above-described figures are only schematic illustrations of processes involved in a method according to an exemplary embodiment of the invention, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.

It should be understood that although described in a certain order, the steps are not necessarily performed sequentially in the order described. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, some steps of the present embodiment may include a plurality of steps or stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily sequential, but may be performed alternately or alternately with at least a part of the steps or stages in other steps or other steps.

In a second aspect of the embodiment of the present invention, the present invention further provides an intelligent agricultural machinery path planning apparatus based on remote sensing images and deep learning, including:

the image acquisition module is used for acquiring remote sensing images of farmland areas through aerial photographing of the unmanned aerial vehicle; the obtained remote sensing image is used as input data, and basic information of actual farmland conditions is provided for subsequent path planning.

In this embodiment, the objective detection module uses a pre-trained Yolov5 model that joins the attention mechanism.

2-7, it is assumed that the intelligent agricultural machine path planning apparatus based on remote sensing images and deep learning is applied to intelligent agricultural machine path planning of a wheat field; aerial photographing is carried out by the unmanned aerial vehicle, a remote sensing image of a wheat field is obtained, and the target detection module utilizes a pre-trained Yolov5 model to identify and position the wheat crop position; according to the detected wheat position, an initial point determining module determines the starting point of the wheat row and sets the attention range; the path generation module generates seedling zones connected with all starting points based on Euclidean distance calculation; then, the path optimization module performs global path planning, and optimizes the seedling zone path with the minimum energy consumption and the minimum operation time as targets; the turning strategy module sets the interval distance of the operation path and selects the optimal turning strategy; and finally, the Bezier curve fitting module corresponds the generated seedling zone to control points of the Bezier curve, and an agricultural machinery operation path which is smooth and accords with the farmland shape is generated.

In this embodiment, when the target detection module processes the high-resolution aerial image, the image is decomposed into a plurality of overlapped small image blocks, and target detection and non-maximum suppression are performed on each small image block, so as to improve the accuracy and efficiency of target detection.

In this embodiment, the path generation module generates a smooth agricultural machine working path conforming to the farmland shape through euclidean distance calculation and bezier curve fitting.

In this embodiment, the path generation module determines the agricultural machinery walking path according to the initial point set and the turning radius, and uses the minimum energy consumption and the minimum operation time as optimization targets to implement intelligent planning of the agricultural machinery path.

According to the intelligent agricultural machinery path planning device based on remote sensing images and deep learning, the remote sensing images and the deep learning technology are combined, and the intelligent agricultural machinery path planning is realized through steps of target detection, path generation, path optimization and the like. The specific working procedure is as follows:

(1) Image acquisition and target detection: firstly, aerial photographing is carried out by using an unmanned aerial vehicle, and a remote sensing image of a farmland area is obtained. Then, the position of the crop is identified and located in the remote sensing image by the pre-trained Yolov5 model and the modified detection layer code. The step utilizes the deep learning technology to realize accurate detection of the position of crops.

(2) Initial point determination and path generation: based on the detected crop position, crop line origin coordinates are determined and a predefined attention range is set around it to define a selection range of candidate points. And selecting a point with the smallest distance from the initial point as the next initial point according to Euclidean distance calculation in the attention range of each initial point, and repeating the steps to generate the seedling zone. The generation of the seedling zone is realized at this stage, and a foundation is provided for the optimization of the subsequent path.

(3) Path optimization and turning strategy: and carrying out global path planning on the generated seedling zone by taking the turning radius of the agricultural machine and the width of the mounting machine as parameters and taking the minimum energy consumption and the minimum operation time as optimization targets. And setting the spacing distance of the working paths of the agricultural machinery, and selecting an optimal turning strategy based on the working breadth so as to ensure uniform coverage and efficient completion of the working. In the stage, by comprehensively considering factors such as turning, interval and the like, the whole path planning is optimized, and the efficiency and quality of agricultural machinery operation are improved.

(4) Bezier curve fitting and path determination: the control points of the Bezier curve are in one-to-one correspondence with the initial point sets in the generated seedling zone, so that the walking path of the agricultural machinery is determined, and the accuracy and the high efficiency of farmland operation are ensured. This step realizes the smoothness of the agricultural machine operation path and conforms to the farmland shape.

Through the steps, the intelligent agricultural machinery path planning device based on remote sensing images and deep learning acquires the positions of farmland crops through a deep learning target detection technology, and combines the path generation, optimization and fitting technologies to realize intelligent agricultural machinery path planning. The device comprehensively considers factors such as agricultural machinery operation efficiency, energy consumption, operation quality and the like, and provides an intelligent path planning scheme for farmland operation.

In summary, the intelligent agricultural machinery path planning method and device based on remote sensing images and deep learning combine the modern remote sensing technology and the deep learning algorithm, and intelligent planning and optimization of the agricultural machinery operation path are realized. Through a series of steps such as image acquisition, target detection, path generation, path optimization, turning strategy, bezier curve fitting and the like, the agricultural machine can efficiently and accurately run in the farmland operation process, and therefore the efficiency and the operation quality of the agricultural machine operation are improved. The invention is not only beneficial to reducing the labor cost and the resource waste in the agricultural production, but also provides an innovative solution for the intellectualization of farmland operation.

In a third aspect of the embodiments of the present invention, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, implements the method of any of the embodiments described above.

A processor and a memory are included in the computer device, and may further include: input means and output means. The processor, memory, input device, and output device may be connected by a bus or other means, and the input device may receive input numerical or character information and generate signal inputs related to migration of intelligent agricultural machine path planning based on telemetry images and deep learning. The output means may comprise a display device such as a display screen.

The memory is used as a non-volatile computer readable storage medium and can be used for storing non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions/modules corresponding to the intelligent agricultural machinery path planning method based on remote sensing images and deep learning in the embodiment of the application. The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating device, an application program required for at least one function; the storage data area may store data created based on remote sensing images and the use of a deep learning intelligent agricultural machinery path planning method, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the local module through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to execute the program code stored in the memory or process the data. The processors of the multiple computer devices of the computer device of the embodiment execute various functional applications and data processing of the server by running nonvolatile software programs, instructions and modules stored in the memory, namely, the steps of the intelligent agricultural machinery path planning method based on remote sensing images and deep learning of the method embodiment are realized.

It should be appreciated that all of the embodiments, features and advantages set forth above for the remote sensing image and deep learning based intelligent agricultural machine path planning method according to the present invention apply equally, without conflict, to the remote sensing image and deep learning based intelligent agricultural machine path planning and storage medium according to the present invention.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall device. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Finally, it should be noted that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, RAM may be available in a variety of forms such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.

The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP and/or any other such configuration.

The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.

It should be understood that as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items. The foregoing embodiment of the present invention has been disclosed with reference to the number of embodiments for the purpose of description only, and does not represent the advantages or disadvantages of the embodiments.

Those of ordinary skill in the art will appreciate that: the above discussion of any embodiment is merely exemplary and is not intended to imply that the scope of the disclosure of embodiments of the invention, including the claims, is limited to such examples; combinations of features of the above embodiments or in different embodiments are also possible within the idea of an embodiment of the invention, and many other variations of the different aspects of the embodiments of the invention as described above exist, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. of the embodiments should be included in the protection scope of the embodiments of the present invention.

Claims

1. An intelligent agricultural machinery path planning method based on remote sensing images and deep learning is characterized by comprising the following steps:

Aerial photographing is carried out through the unmanned aerial vehicle, and a remote sensing image of a farmland area is obtained;

identifying and locating crop positions in the remote sensing image using a deep learning based target detection network;

determining crop line origin coordinates based on the detected crop position, and setting a predefined attention range around the crop line origin coordinates to define a selection range of candidate points;

selecting a point with the smallest distance from the initial point as the next initial point based on Euclidean distance calculation in the attention range of each starting point, and repeating the step to generate a seedling zone;

and generating a smooth agricultural machine operation path which accords with the farmland shape through Bezier curve fitting of the initial point set.

2. The intelligent agricultural machinery path planning method based on remote sensing image and deep learning of claim 1, further comprising the steps of:

setting a spacing distance of an agricultural machine operation path, and selecting an optimal turning strategy based on operation breadth;

And (3) corresponding the control points of the Bezier curve to the initial point sets in the generated seedling zone one by one, and determining the walking path of the agricultural machine.

3. The intelligent agricultural machinery path planning method based on remote sensing images and deep learning of claim 1, wherein the target detection network is a pre-trained Yolov5 model and the detection layer codes are modified to adapt to the characteristics of farm crops for accurately identifying and locating the crop positions in the remote sensing images.

4. The intelligent agricultural machinery path planning method based on remote sensing image and deep learning according to claim 3, wherein when generating seedling strips, based on the detected positions of the crops, marking the coordinates of the starting points of the rows of the crops on the output image as the coordinates of the initial points, finding out the point with the minimum Euclidean distance to the initial points in the attention range of the initial points according to the detected position points of unordered crops, updating the minimum distance point as the initial points, fitting the initial points to form the seedling strips by using Bezier curve, and repeating the steps until all the seedling strips are found out.

5. The intelligent agricultural machinery path planning method based on remote sensing images and deep learning as claimed in claim 4, wherein the method uses the path planning by combining the deep learning with the aerial image, and the path planning method by combining the deep learning with the aerial image comprises the following steps:

performing migration learning by using pre-trained weights, and integrating a channel attention mechanism and a spatial attention mechanism in the structure of the Yolov5 model;

and through transfer learning, the pre-trained weight is used for fine tuning the Yolov5 model, the trained model is operated on a farmland remote sensing image, and the positions of corn seedlings are identified and positioned.

6. The intelligent agricultural machinery path planning method based on remote sensing image and deep learning according to claim 5, wherein the use of the target detection network based on deep learning uses Yolov5 with modified detection layer codes, and on the basis, when the resolution of the aerial image is too large, the intelligent agricultural machinery path planning method based on remote sensing image and deep learning comprises the following steps:

Decomposing a farmland remote sensing image into a plurality of overlapped small image blocks, wherein each small image block covers an area;

and performing non-maximum suppression on the positions of crops detected in all the small image blocks, removing repeated detection results, and obtaining accurate farmland crop position information by using the relative coordinate values and the detection results subjected to non-maximum suppression.

7. The intelligent agricultural machinery path planning method based on remote sensing images and deep learning of claim 1, wherein determining crop row start coordinates based on the detected crop position comprises the steps of:

forming a set of unordered crop position points based on the detected crop coordinatesStarting with the marked crop line +.>For the first initial point, an attention range is set to find +.>Points within the attention range in the set of location points +.>Then from->Middle through European distance calculation formula- >Find out and->Points with minimum Euclidean distance ∈ ->And will->Updating the point to be a new initial point, repeating the above steps until the initial point set +.>。

8. The intelligent agricultural machine path planning method based on remote sensing images and deep learning of claim 7, wherein when generating a smooth agricultural machine working path conforming to a farmland shape by bezier curve fitting an initial point set, comprising the steps of:

，/>from 0 to->Wherein (1)>Is in the value range->Is used for the control of the temperature of the liquid crystal display device,，/>is->Control points->Is the order of the curve, +.>The calculation mode of the value is as follows: />。

9. The intelligent agricultural machinery path planning method based on remote sensing image and deep learning of claim 7, further comprising:

wherein, the global path planning is based on the generation of seedling belts, when the width of the mounting machine at the tail end of the agricultural machine is W, the walking path of the agricultural machine is determined by translating the seedling belts according to W/2, so that the agricultural machine achieves the minimum energy consumption and the minimum operation time, and then the agricultural machine passes through the step of R<W/2, preferably usingTurning mode, when R>W/2 timePreferably use->And selecting a turning strategy to link the walking path.

10. An intelligent agricultural machinery path planning device based on remote sensing image and deep learning, which is characterized in that the intelligent agricultural machinery path planning device based on remote sensing image and deep learning is used for executing the intelligent agricultural machinery path planning method based on remote sensing image and deep learning as set forth in any one of claims 1-9, and comprises:

the target detection module is used for using a pre-trained Yolov5 model, modifying detection layer codes to adapt to the characteristics of farmland crops, and accurately identifying and positioning the positions of the crops in the remote sensing image;

the path optimization module is used for carrying out global path planning on the generated seedling zone by taking the minimum energy consumption and the minimum operation time as optimization targets according to the turning radius of the agricultural machinery and the width of the mounting machine;

the turning strategy module is used for setting the spacing distance of the operation paths of the agricultural machinery and selecting an optimal turning strategy based on the operation breadth;

the Bezier curve fitting module is used for corresponding control points of the Bezier curve to the initial point sets in the generated seedling zone one by one to determine the walking path of the agricultural machine.