CN115167506A

CN115167506A - Method, device, equipment and storage medium for updating and planning flight line of unmanned aerial vehicle

Info

Publication number: CN115167506A
Application number: CN202210735942.9A
Authority: CN
Inventors: 崔海霞; 张楠; 刘鹏; 陆江南; 刘圣锋
Original assignee: South China Normal University
Current assignee: South China Normal University
Priority date: 2022-06-27
Filing date: 2022-06-27
Publication date: 2022-10-11
Also published as: WO2024001267A1

Abstract

The invention relates to the technical field of unmanned aerial vehicle wireless communication, in particular to a method, a device, equipment and a storage medium for updating and planning a flight path of an unmanned aerial vehicle, wherein the unmanned aerial vehicle can detect the current state of the unmanned aerial vehicle in real time when executing a flight task, update the flight path of the unmanned aerial vehicle through a multi-step deep learning network model, control the unmanned aerial vehicle to move along the updated flight path, and avoid the unmanned aerial vehicle from approaching an interference source or entering a weak signal coverage area, so that the quality of data transmission of the unmanned aerial vehicle is ensured, the multi-step deep learning network model can be continuously updated, the accuracy of updating and planning the flight path of the unmanned aerial vehicle is improved, and the efficiency of the unmanned aerial vehicle executing the task is improved.

Description

Method, device, equipment and storage medium for updating and planning flight line of unmanned aerial vehicle

Technical Field

The invention relates to the technical field of unmanned aerial vehicle wireless communication, in particular to a method, a device, equipment and a storage medium for updating and planning flight routes of an unmanned aerial vehicle.

Background

In the field of drone wireless communication technology, a drone has two typical application frameworks, namely, drone-assisted wireless communication, in which a drone provides communication services for ground users as air mobile Base Stations (BSs) or mobile relays (relays): and the second is a cellular connection unmanned aerial vehicle, wherein the unmanned aerial vehicle is used as an air communication user and provides communication service for the air communication user through a ground base station, so that the purpose of sending or receiving related data to complete the expected task of the air communication user is achieved.

Currently, in a communication system using drones as air Base Stations (BSs) to implement air-ground communication, the drones mainly establish communication connection with ground base stations and ground users by means of line-of-sight links. However, a large amount of interference and weak signal coverage areas exist in the real-world communication environment, when the unmanned aerial vehicle executes a flight task, the unmanned aerial vehicle is close to an interference source or enters the weak signal coverage areas, and transmission of data by the unmanned aerial vehicle is blocked, so that the unmanned aerial vehicle cannot well complete an expected task target, and the task efficiency is low.

Disclosure of Invention

Based on this, the present invention aims to provide a method, an apparatus, a device and a storage medium for updating and planning a flight path of an unmanned aerial vehicle, which specifically include the following steps:

in a first aspect, an embodiment of the present application provides a method for updating and planning a flight path of an unmanned aerial vehicle, including the following steps:

s1: acquiring a flight route of the unmanned aerial vehicle in a target area, and controlling the unmanned aerial vehicle to fly along the flight route, wherein the flight route comprises a termination point;

s2: acquiring real-time signal-to-noise ratio data of the unmanned aerial vehicle in the flight process; according to the real-time signal-to-noise ratio data, performing interference detection on the unmanned aerial vehicle to obtain an interference detection result; acquiring position data of the unmanned aerial vehicle at the current moment as position data of a current interference state according to the interference detection result, and acquiring signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state and real-time signal-to-noise ratio data;

s3: inputting the position data of the interference state of the unmanned aerial vehicle into an action module of a preset multistep deep learning network model, and acquiring moving direction data of the interference state of the unmanned aerial vehicle; updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route;

s4: inputting the position data of the unmanned aerial vehicle interference state and the signal interruption probability data into an evaluation module of the multi-step deep learning network model to obtain mobile income data of the unmanned aerial vehicle interference state, and combining the mobile income data with the position data of the unmanned aerial vehicle interference state to be used as a mobile income association of the unmanned aerial vehicle;

s5: the method comprises the steps of obtaining a plurality of mobile income association sets of the unmanned aerial vehicles corresponding to interference state positions of the unmanned aerial vehicles in a preset number in the flight process, updating action modules and evaluation modules of a multi-step deep learning network model according to the mobile income association sets of the unmanned aerial vehicles, and obtaining the updated multi-step deep learning network model;

s6: when the unmanned aerial vehicle detects the position data of the next interference state, inputting the position data of the next interference state of the unmanned aerial vehicle into the updated multi-step deep learning network model to obtain moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle; updating the flight route according to moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight route;

s7: and repeating the steps S4-S6, updating the flight route, and controlling the unmanned aerial vehicle to move along the updated flight route until the unmanned aerial vehicle moves to the termination point.

In a second aspect, an embodiment of the present application provides an apparatus for updating and planning flight routes of an unmanned aerial vehicle, including:

the route setting module is used for acquiring a flight route of the unmanned aerial vehicle in a target area and controlling the unmanned aerial vehicle to fly along the flight route, wherein the flight route comprises a termination point;

the interference detection module is used for acquiring real-time signal-to-noise ratio data in the flight process of the unmanned aerial vehicle; according to the real-time signal-to-noise ratio data, performing interference detection on the unmanned aerial vehicle to obtain an interference detection result; according to the interference detection result, acquiring position data of the unmanned aerial vehicle at the current moment as position data of a current interference state, and acquiring signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state and real-time signal to noise ratio data;

the first route updating module is used for inputting the position data of the unmanned aerial vehicle interference state into an action module of a preset multi-step deep learning network model and acquiring the moving direction data of the unmanned aerial vehicle interference state; updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route;

the mobile profit calculation module is used for inputting the position data of the unmanned aerial vehicle interference state and the signal interruption probability data into the evaluation module of the multistep deep learning network model to obtain the mobile profit data of the unmanned aerial vehicle interference state, and combining the mobile profit data with the position data of the unmanned aerial vehicle interference state to be used as a mobile profit association of the unmanned aerial vehicle;

the model updating module is used for acquiring a plurality of mobile income association sets of the unmanned aerial vehicles corresponding to the interference state positions of the unmanned aerial vehicles in a preset number in the flight process, updating the action module and the evaluation module of the multi-step deep learning network model according to the mobile income association sets of the unmanned aerial vehicles, and acquiring the updated multi-step deep learning network model;

the second air route updating module is used for inputting the position data of the next interference state of the unmanned aerial vehicle into the updated multi-step deep learning network model when the unmanned aerial vehicle detects the position data of the next interference state, and obtaining moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle; updating the flight path according to moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight path;

and the mobile execution module is used for updating the flight route and controlling the unmanned aerial vehicle to move along the updated flight route until the unmanned aerial vehicle moves to the termination point.

In a third aspect, an embodiment of the present application provides an apparatus, including: a processor, a memory, and a computer program stored on the memory and executable on the processor; the computer program when executed by the processor implements the steps of the method for updating a plan of flight paths for a drone according to the first aspect.

In a fourth aspect, an embodiment of the present application provides a storage medium storing a computer program, which when executed by a processor, implements the steps of the method for updating a plan of flight path of a drone according to the first aspect.

In the embodiment of the application, a method, a device, equipment and a storage medium for updating and planning a flight path of an unmanned aerial vehicle are provided, when the unmanned aerial vehicle executes a flight task, the current state of the unmanned aerial vehicle can be detected in real time, the flight path of the unmanned aerial vehicle is updated through a multi-step deep learning network model, the unmanned aerial vehicle is controlled to move along the updated flight path, the unmanned aerial vehicle is prevented from approaching an interference source or entering a weak signal coverage area, the quality of data transmission of the unmanned aerial vehicle is guaranteed, the multi-step deep learning network model can be continuously updated, the accuracy of updating and planning the flight path of the unmanned aerial vehicle is improved, and the efficiency of the unmanned aerial vehicle executing the task is improved.

For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.

Drawings

Fig. 1 is a schematic flow chart of a method for updating a planning of a flight path of an unmanned aerial vehicle according to an embodiment of the present application;

fig. 2 is a schematic flowchart of S2 in a method for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application;

fig. 3 is a schematic flowchart of S3 in a method for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application;

fig. 4 is a schematic flowchart of S4 in a method for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application;

fig. 5 is a schematic flowchart of S5 in a method for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an apparatus for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if as used herein may be interpreted as" at "8230; \8230when" or "when 8230; \823030, when" or "in response to a determination", depending on the context.

Referring to fig. 1, fig. 1 is a schematic flow chart of a method for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application, where the method includes the following steps:

s1: and acquiring a flight route of the unmanned aerial vehicle in the target area, and controlling the unmanned aerial vehicle to fly along the flight route.

The execution main body of the method for updating and planning the flight path of the unmanned aerial vehicle is planning equipment (hereinafter referred to as planning equipment for short) of the method for updating and planning the flight path of the unmanned aerial vehicle.

Unmanned aerial vehicle can be connected with planning equipment through communication methods such as self radio station or 4G, and in this embodiment, planning equipment acquires unmanned aerial vehicle's flight route in the target area, control unmanned aerial vehicle along flight route flies to the termination point from the initial point that predetermines, carries out the flight task, provides communication service for the user in target area.

S2: the method comprises the steps of obtaining real-time signal to noise ratio data of an unmanned aerial vehicle in the flight process, carrying out interference detection on the unmanned aerial vehicle according to the real-time signal to noise ratio data, obtaining an interference detection result, obtaining position data of the unmanned aerial vehicle at the current moment according to the interference detection result, using the position data as the position data of an interference state, and obtaining signal interruption probability data of the interference state of the unmanned aerial vehicle according to the position data and the real-time signal to noise ratio data of the interference state of the unmanned aerial vehicle.

Because the unmanned aerial vehicle enters a weak coverage area or an interference area of a cellular network in the flight process, the unmanned aerial vehicle is blocked in data transmission, in this embodiment, the planning device acquires signal-to-noise ratio data sent by the unmanned aerial vehicle in real time in the flight process, and performs interference detection on the unmanned aerial vehicle according to the real-time signal-to-noise ratio data to acquire an interference detection result, wherein the interference detection result includes an interference generation result and an interference-free result.

Specifically, the planning equipment compares a value corresponding to the acquired real-time signal-to-noise ratio data with a signal-to-noise ratio threshold value through presetting the signal-to-noise ratio threshold value, acquires an interference generation result when the value of the acquired real-time signal-to-noise ratio data is greater than or equal to the signal-to-noise ratio threshold value, and acquires position data of the unmanned aerial vehicle at the current moment according to the interference generation result as position data of an interference state; and when the value of the acquired real-time signal-to-noise ratio data is smaller than the signal-to-noise ratio threshold value, acquiring an interference-free result, and not performing the acquisition operation of the position data of the unmanned aerial vehicle at the current moment by the planning equipment.

In order to further confirm the current moment of the unmanned aerial vehicle, namely the interference condition received in the interference state, the planning equipment acquires the signal interruption probability data of the interference state of the unmanned aerial vehicle according to the position data and the real-time signal-to-noise ratio data of the interference state of the unmanned aerial vehicle.

Referring to fig. 2, fig. 2 is a schematic flow chart of a method for updating and planning a flight path of an unmanned aerial vehicle, according to an embodiment of the present application, where the method includes steps S201 to S202, and specifically includes the following steps:

s201: dividing the target area into a plurality of communication areas, determining the communication area related to the interference state of the unmanned aerial vehicle according to the position data of the interference state of the unmanned aerial vehicle, and acquiring the position data of the communication area related to the interference state of the unmanned aerial vehicle.

In this embodiment, the planning device obtains, through a preset urban macro (UMa) model, position data of all base stations in a target area where the unmanned aerial vehicle executes a flight mission, divides the target area into a plurality of communication areas according to the position data of the base stations, and determines, according to the position data of the interference state of the unmanned aerial vehicle, a communication area where the unmanned aerial vehicle is located at the current moment, thereby determining the position data of the communication area associated with the interference state of the unmanned aerial vehicle, and obtaining the position data of the communication area associated with the interference state of the unmanned aerial vehicle.

S202: and acquiring the signal interruption probability data of the interference state of the unmanned aerial vehicle according to the position data of the interference state of the unmanned aerial vehicle, the position data of the associated communication area, the real-time signal to noise ratio data and a preset signal interruption probability algorithm.

The signal interruption probability algorithm is as follows:

P _out (t)＝P(s _t ,b _t ,SIR _t )

in the formula, P _out (t) signal interruption probability data, s, for the tth interference state of the drone _t Position data of the tth interference state of the unmanned aerial vehicle, b _t Position data, SIR, of a communication area associated with the tth interference state of the UAV _t Real-time signal-to-noise ratio data of the t interference state of the unmanned aerial vehicle; p () is a signal interruption probability data calculation function regarding the interference state of the drone.

In this embodiment, the planning device obtains the signal interruption probability data of the interference state of the unmanned aerial vehicle according to the position data of the interference state of the unmanned aerial vehicle, the position data of the associated communication area, the real-time signal to noise ratio data, and a preset signal interruption probability algorithm.

S3: inputting the position data of the interference state of the unmanned aerial vehicle into an action module of a preset multi-step deep learning network model, and acquiring the moving direction data of the interference state of the unmanned aerial vehicle; and updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route.

The planning equipment adopts a DDPG (Deep Deterministic Policy Gradient) depth Deterministic strategy Gradient algorithm and multi step learning, namely a multi-step learning algorithm, to be combined to construct a multi-step Deep learning network model, wherein the multi-step Deep learning network model comprises an action module and an evaluation module, the action module is used for moving direction data of an interference state of the unmanned aerial vehicle, and the evaluation module is used for evaluating the behavior of the unmanned aerial vehicle in moving operation according to the moving direction data, so that the multi-step Deep learning network model is updated, and the efficiency of the unmanned aerial vehicle in executing flight tasks is improved.

The evaluation module and the action module are both of a double-network structure, the evaluation module comprises a first sub-evaluation module and a second sub-evaluation module which are identical and connected in sequence, and the action module comprises a first sub-action module and a second sub-action module which are identical and connected in sequence. Before the multi-step deep learning network model is operated, the planning equipment initializes the evaluation weight parameters of a first sub-evaluation module and a second sub-evaluation module of the multi-step deep learning network model and the action weight parameters of the first sub-action module and the second sub-action module, and replaces the evaluation weight parameters of the second sub-evaluation module with the evaluation weight parameters of the first sub-evaluation module according to preset updating duration; and replacing the action weight parameter of the second sub-action module with the action weight parameter of the first sub-action module according to the preset updating time length.

In this embodiment, the planning device inputs the position data of the interference state of the unmanned aerial vehicle into an action module of a preset multi-step deep learning network model, and obtains moving direction data of the interference state of the unmanned aerial vehicle output by the action module; and updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route.

Referring to fig. 3, fig. 3 is a schematic flow chart of S3 in the method for updating and planning the flight path of the unmanned aerial vehicle according to an embodiment of the present application, including steps S301 to S302, which are specifically as follows:

s301: and inputting the position data of the interference state of the unmanned aerial vehicle into a first sub-action module of the action module, and acquiring the moving direction data of the interference state of the unmanned aerial vehicle output by the first sub-action module according to a preset moving direction data calculation algorithm.

The moving direction data calculation algorithm is as follows:

a _t ＝μ(s _t |θ _μ )+N _t

in the formula, a _t For the moving direction data of the t interference state of the unmanned aerial vehicle, mu () is the action calculation function in the first sub-action module, s _t Position data of the tth interference state of the unmanned aerial vehicle, theta _μ Is the action weight parameter, N, of the first sub-action module _t Noise data for a predetermined t-th interference state.

In this embodiment, the planning device inputs the position data of the interference state of the unmanned aerial vehicle into a first sub-action module of the action module, and obtains the moving direction data of the interference state of the unmanned aerial vehicle output by the first sub-action module according to a preset moving direction data calculation algorithm.

S302: and according to the moving direction data and a preset moving direction comparison table, obtaining a moving direction corresponding to the moving direction data, updating the flight route according to the moving direction, and controlling the unmanned aerial vehicle to move along the updated flight route.

The moving direction comparison table includes data corresponding to a plurality of moving direction types, the data may be a determined numerical value or a numerical value within a preset interval range, and the moving direction includes front, back, left, right, and the like.

In this embodiment, the planning device acquires the moving direction corresponding to the moving direction data according to the moving direction data and a preset moving direction comparison table, updates the flight route according to the moving direction, specifically, the planning device matches the moving direction data with data corresponding to the moving direction type in the moving direction comparison table, acquires the matched moving direction, uses the moving direction data as the moving direction corresponding to the moving direction data, connects the position point corresponding to the position data of the unmanned aerial vehicle interference state with the end point according to the moving direction, completes updating the flight route, and controls the unmanned aerial vehicle to move along the updated flight route.

S4: and inputting the position data and the signal interruption probability data of the unmanned aerial vehicle interference state into an evaluation module of the multistep deep learning network model to obtain the mobile income data of the unmanned aerial vehicle interference state, and combining the mobile income data with the position data of the unmanned aerial vehicle interference state to serve as the mobile income association of the unmanned aerial vehicle.

In this embodiment, the planning device inputs the position data of the interference state of the unmanned aerial vehicle and the signal interruption probability data into the evaluation module of the multi-step deep learning network model, acquires the mobile profit data of the interference state of the unmanned aerial vehicle output by the evaluation module, and combines the mobile profit data with the position data of the interference state of the unmanned aerial vehicle to serve as the mobile profit association set of the unmanned aerial vehicle.

Referring to fig. 4, fig. 4 is a schematic flow chart of S4 in the method for updating and planning the flight path of the unmanned aerial vehicle according to an embodiment of the present application, which includes step S401, specifically as follows:

s401: inputting the position data and the signal interruption probability data of the interference state of the unmanned aerial vehicle into a first sub-evaluation module of the evaluation module, and obtaining the mobile profit data of the interference state of the unmanned aerial vehicle output by the first sub-evaluation module according to a preset mobile profit data calculation algorithm.

The mobile income data calculation algorithm comprises the following steps:

r _t ＝-1-δ*P _out (t+1)

in the formula, r _t Moving profit data of the t-th interference state of the unmanned aerial vehicle, wherein delta is a punishment weight parameter P _out (t + 1) is the signal interruption probability data of the t +1 th interference state of the unmanned aerial vehicle.

In order to reasonably judge the moving direction data of the current interference state of the unmanned aerial vehicle, in the embodiment, the planning device acquires the signal interruption probability data of the next interference state of the unmanned aerial vehicle, inputs the position data of the current interference state of the unmanned aerial vehicle and the signal interruption probability data of the next interference state into the first sub-evaluation module of the evaluation module, and acquires the moving profit data of the current interference state of the unmanned aerial vehicle output by the first sub-evaluation module according to a preset moving profit data calculation algorithm.

S5: the method comprises the steps of obtaining a plurality of interference state positions of unmanned aerial vehicles in the flying process, wherein the interference state positions of the unmanned aerial vehicles correspond to the interference state positions of the unmanned aerial vehicles, and updating an action module and an evaluation module of a multistep deep learning network model according to the interference state positions of the unmanned aerial vehicles to obtain the updated multistep deep learning network model.

In this embodiment, the planning device obtains a plurality of interference state positions of the unmanned aerial vehicle corresponding to the preset number in the flight process, and updates the action module and the evaluation module of the multi-step deep learning network model according to the mobile profit association of the unmanned aerial vehicle to obtain the updated multi-step deep learning network model.

Because the mobile revenue data of adjacent interference states are similar, but obvious, in order to improve the accuracy and efficiency of updating the multi-step deep learning network model, in an optional embodiment, the planning device may obtain the mobile revenue associated sets of the plurality of non-adjacent interference states from the plurality of mobile revenue associated sets of the unmanned aerial vehicle corresponding to the acquired interference state positions of the preset number by a sample acquisition method, input the mobile revenue associated sets of the plurality of non-adjacent interference states into the multi-step deep learning network model, update the action module and the evaluation module of the multi-step deep learning network model, and obtain the updated multi-step deep learning network model, so as to reduce loss, accelerate the training learning speed of the algorithm, and improve the accuracy and efficiency of updating the multi-step deep learning network model.

Referring to fig. 5, fig. 5 is a schematic flow chart of S5 in the method for updating and planning the flight path of the unmanned aerial vehicle according to an embodiment of the present application, which includes steps S501 to S505, and specifically includes the following steps:

s501: and judging the position of the unmanned aerial vehicle in the interference state according to the position data of the interference state of the unmanned aerial vehicle in the mobile profit association set of the unmanned aerial vehicle, and acquiring a position judgment result.

The position determination result includes an end point position result, a position offset result and a position coincidence result, and the flight route is updated, and a situation that the flight route deviates from a target area may be generated, so that the unmanned aerial vehicle cannot provide a communication service for a user in the target area well.

S502: and obtaining expected values corresponding to the mobile income association groups of the unmanned aerial vehicles according to the position judgment result of the interference state of the unmanned aerial vehicles, the mobile income data of the interference state of the unmanned aerial vehicles and a preset expected value calculation algorithm.

The expected value calculation algorithm is as follows:

in the formula, y _t Is the expected value, r, of the tth interference state of the unmanned aerial vehicle _t Is the mobile profit data of the t-th interference state of the unmanned aerial vehicle, s _t+1 Position data of t +1 th interference state of the unmanned aerial vehicle, c _t Determining a position judgment result of the t interference state of the unmanned aerial vehicle, wherein A is an end position result, B is a position deviation result, C is a position coincidence result, gamma is a preset expectation coefficient, Q '() is an evaluation calculation function in the second sub-evaluation module, mu' () is an action calculation function in the second sub-action module, and theta is ^μ' The action weight parameter of the second sub-action module; theta.theta. ^Q' An evaluation weight parameter for the second sub-evaluation module; r _des For a predetermined prize value, P _ob Is a preset penalty value;

in this embodiment, the planning device obtains the expected value corresponding to the mobile revenue association group of each unmanned aerial vehicle according to the position judgment result of the interference state of the unmanned aerial vehicle, the mobile revenue data of the interference state of the unmanned aerial vehicle and a preset expected value calculation algorithm, embodies the rationality of the mobile direction data of the t-th interference state of the unmanned aerial vehicle, and is used for updating the subsequent multi-step deep learning network.

S503: and according to the moving direction data and the position data corresponding to the mobile income association of each unmanned aerial vehicle, according to the expected value, the moving direction data and the preset error calculation algorithm corresponding to the mobile income association of each unmanned aerial vehicle, obtaining an error value of the first sub-evaluation module, according to the error value, updating the evaluation weight parameter of the first sub-evaluation module, and obtaining the updated evaluation weight parameter of the first sub-evaluation module.

The error calculation algorithm is as follows:

wherein L is the error value, N is the total number of the mobile revenue association set of the UAV, Q () is an evaluation calculation function in the first sub-evaluation module, s _t Position data of the tth interference state of the unmanned aerial vehicle, a _t Moving direction data of the t-th interference state of the unmanned aerial vehicle, theta ^Q An evaluation weight parameter for the first sub-evaluation module;

in this embodiment, the planning device obtains an error value of the first sub-estimation module according to the movement direction data and the position data corresponding to the mobile revenue association of each unmanned aerial vehicle, according to the expected value, the movement direction data and the preset error calculation algorithm corresponding to the mobile revenue association of each unmanned aerial vehicle, and continuously modifies the estimation weight parameter of the first sub-estimation module according to the error value, so that the value output by the estimation calculation function Q () according to the mobile revenue association of the unmanned aerial vehicle is close to the expected value corresponding to the mobile revenue association of the unmanned aerial vehicle, so that the error value of the first sub-estimation module is reduced until the error value of the first sub-estimation module remains unchanged, and obtains the estimation weight parameter of the current first sub-estimation module as the estimation weight parameter of the updated first sub-estimation module.

S504: and according to the expected value corresponding to the mobile revenue association group of each unmanned aerial vehicle and a preset gradient updating calculation algorithm, acquiring a gradient updating value of the first sub-action module, updating the action weight parameter of the first sub-action module according to the gradient updating value, and acquiring the updated action weight parameter of the first sub-action module.

The gradient updating calculation algorithm is as follows:

in the formula (I), the compound is shown in the specification,

updating the gradient of the t-th interference state, μ () being a function of the motion computation in said first sub-motion block,

for Q () in the t interference state, for the moving direction data, μ(s) _t ) The value of the gradient update to be solved,

an action weight parameter theta for mu () on the first sub-action module in the tth interference state _μ The solved gradient update value, alpha is a preset gradient decreasing step length, theta ^μ An action weight parameter of the first sub-action module;

in this embodiment, the planning device obtains a gradient update value of the first sub-action module according to an expected value corresponding to the mobile profit association of each unmanned aerial vehicle and a preset gradient update calculation algorithm, updates the action weight parameter of the first sub-action module according to the gradient update value and a preset gradient decreasing step length, obtains the updated action weight parameter of the first sub-action module, and implements update iteration of the first sub-action module.

S505: and updating the weight parameter of the second sub-evaluation module and the weight parameter of the second sub-action module respectively according to the updated evaluation weight parameter of the first sub-evaluation module, the updated action weight parameter of the first sub-action module and a preset weight parameter updating algorithm to obtain the updated evaluation weight parameter of the second sub-evaluation module and the updated action weight parameter of the second sub-action module.

The weight parameter updating algorithm is as follows:

θ ^Q '←τθ ^Q +(1-τ)θ ^Q' ,θ ^μ' ←τθ ^μ +(1-τ)θ ^μ'

in the formula, theta ^Q' An evaluation weight parameter for the second sub-evaluation module; theta ^μ' Is the action weight parameter, theta, of the second sub-action module ^μ And the motion weight parameter tau of the first sub-motion module is a preset optimization parameter.

In this embodiment, the planning device is based on: according to the updated evaluation weight parameter of the first sub-evaluation module, the updated action weight parameter of the first sub-action module and a preset weight parameter updating algorithm, the weight parameter of the second sub-evaluation module and the weight parameter of the second sub-action module are updated respectively to obtain the updated evaluation weight parameter of the second sub-evaluation module and the updated action weight parameter of the second sub-action module, and the weight parameters of the second sub-evaluation module and the second sub-action module are updated in a soft updating mode to solve the problem of difficult convergence of a multi-step deep learning network model and improve the accuracy of updating and planning of the flight route of the unmanned aerial vehicle.

S6: when the unmanned aerial vehicle detects the position data of the next interference state, inputting the position data of the next interference state of the unmanned aerial vehicle into the updated multi-step deep learning network model to obtain moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle; and updating the flight path according to the moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight path.

After the multi-step deep learning network model is updated, in this embodiment, when the unmanned aerial vehicle detects position data of a next interference state, the planning device inputs the position data of the next interference state of the unmanned aerial vehicle to a second sub-action module of an action module in the updated multi-step deep learning network model, and obtains moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle output by the second sub-action module; and updating the flight route according to the moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight route.

In this embodiment, the planning device updates the flight path according to a preset updated number of position data of the interference states, for example, the planning device sets the updated number to 4 in advance, the planning device takes the position data of the 4 interference states each time when acquiring the position data of the 4 interference states, inputs the position data into a second sub-evaluation module of an evaluation module of the multi-step deep learning network module, acquires moving direction profit data of the 4 interference states, and acquires expected values of the 4 interference states, updates the multi-step deep learning network model, that is, sets an update cycle according to the set updated number, a first cycle is to update the model by using the position data of the interference states 1, 2, 3, and 4, and then a second cycle is to update the model by using the position data of the interference states 5, 6, 7, and 8, and so on, continuously updates the flight path, and controls the unmanned aerial vehicle to move along the updated flight path until the unmanned vehicle moves to the termination point.

Referring to fig. 6, fig. 6 is a schematic structural diagram of an apparatus for updating and planning a flight path of an unmanned aerial vehicle according to an embodiment of the present application, where the apparatus may implement all or a part of the apparatus for updating and planning a flight path of an unmanned aerial vehicle through software, hardware, or a combination of the two, and the apparatus 6 includes:

the route setting module 61 is used for acquiring a flight route of the unmanned aerial vehicle in a target area and controlling the unmanned aerial vehicle to fly along the flight route, wherein the flight route comprises a termination point;

the interference detection module 62 is configured to obtain real-time signal-to-noise ratio data of the unmanned aerial vehicle in the flight process; according to the real-time signal-to-noise ratio data, performing interference detection on the unmanned aerial vehicle to obtain an interference detection result; acquiring position data of the unmanned aerial vehicle at the current moment as position data of a current interference state according to the interference detection result, and acquiring signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state and real-time signal-to-noise ratio data;

the first route updating module 63 is configured to input the position data of the interference state of the unmanned aerial vehicle into an action module of a preset multi-step deep learning network model, and acquire moving direction data of the interference state of the unmanned aerial vehicle; updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route;

a mobile benefit calculation module 64, configured to input the position data of the unmanned aerial vehicle interference state and the signal interruption probability data into an evaluation module of the multi-step deep learning network model, obtain mobile benefit data of the unmanned aerial vehicle interference state, and combine the mobile benefit data with the position data of the unmanned aerial vehicle interference state to serve as a mobile benefit association of the unmanned aerial vehicle;

the model updating module 65 is configured to obtain a plurality of mobile benefit association sets of the unmanned aerial vehicles corresponding to interference state positions of the unmanned aerial vehicles in a preset number during a flight process, update the action module and the evaluation module of the multi-step deep learning network model according to the mobile benefit association sets of the unmanned aerial vehicles, and obtain an updated multi-step deep learning network model;

the second route updating module 66 is configured to, when the unmanned aerial vehicle detects position data of a next interference state, input the position data of the next interference state of the unmanned aerial vehicle into the updated multi-step deep learning network model, and obtain moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle; updating the flight path according to moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight path;

and the mobile execution module 67 is used for updating the flight route and controlling the unmanned aerial vehicle to move along the updated flight route until the unmanned aerial vehicle moves to the termination point.

In the embodiment of the application, a flight path of the unmanned aerial vehicle in a target area is obtained through a path setting module, and the unmanned aerial vehicle is controlled to fly along the flight path, wherein the flight path comprises a termination point; acquiring real-time signal-to-noise ratio data of the unmanned aerial vehicle in the flight process through an interference detection module; according to the real-time signal-to-noise ratio data, performing interference detection on the unmanned aerial vehicle to obtain an interference detection result; acquiring position data of the unmanned aerial vehicle at the current moment as position data of a current interference state according to the interference detection result, and acquiring signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state and real-time signal-to-noise ratio data; inputting the position data of the interference state of the unmanned aerial vehicle into an action module of a preset multi-step deep learning network model through a first route updating module, and acquiring moving direction data of the interference state of the unmanned aerial vehicle; updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route; inputting the position data of the unmanned aerial vehicle interference state and the signal interruption probability data into an evaluation module of the multi-step deep learning network model through a mobile profit calculation module to obtain mobile profit data of the unmanned aerial vehicle interference state, and combining the mobile profit data with the position data of the unmanned aerial vehicle interference state to be used as a mobile profit association of the unmanned aerial vehicle; the method comprises the steps that a model updating module is used for obtaining a plurality of mobile profit association groups of the unmanned aerial vehicles corresponding to interference state positions of the unmanned aerial vehicles in a preset number in the flight process, and according to the mobile profit association groups of the unmanned aerial vehicles, action modules and evaluation modules of a multi-step deep learning network model are updated to obtain an updated multi-step deep learning network model; through a second route updating module, when the unmanned aerial vehicle detects the position data of the next interference state, inputting the position data of the next interference state of the unmanned aerial vehicle into an updated multi-step deep learning network model, and obtaining moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle; updating the flight route according to moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight route; through removing the execution module, it is right the flight route updates, control unmanned aerial vehicle and remove along the flight route after the update, until unmanned aerial vehicle removes to the termination point. When the unmanned aerial vehicle executes a flight task, the current state of the unmanned aerial vehicle can be detected in real time, the flight line of the unmanned aerial vehicle is updated through the multi-step deep learning network model, the unmanned aerial vehicle is controlled to move along the updated flight line, the unmanned aerial vehicle is prevented from being close to an interference source or entering a weak signal coverage area, the quality of data transmission of the unmanned aerial vehicle is guaranteed, the multi-step deep learning network model can be continuously updated, the accuracy of updating and planning the flight line of the unmanned aerial vehicle is improved, and the efficiency of the unmanned aerial vehicle executing the task is improved.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application, where the computer device 7 includes: a processor 71, a memory 72, and a computer program 73 stored on the memory 72 and operable on the processor 71; the computer device may store a plurality of instructions, where the instructions are suitable for being loaded by the processor 71 and executing the method steps in the embodiments shown in fig. 1 to fig. 5, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to fig. 5, which is not described herein again.

Processor 71 may include one or more processing cores, among others. The processor 71 is connected to various parts in the server by various interfaces and lines, and executes various functions and processes data of the apparatus 6 for performing unmanned aerial vehicle flight path update planning by operating or executing instructions, programs, code sets or instruction sets stored in the memory 72 and calling data in the memory 72, and optionally, the processor 71 may be implemented in at least one hardware form of Digital Signal Processing (DSP), field-Programmable Gate Array (FPGA), programmable Logic Array (PLA). The processor 71 may integrate one or a combination of a Central Processing Unit (CPU) 71, a Graphics Processing Unit (GPU) 71, a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the touch display screen; the modem is used to handle wireless communications. It is understood that the modem may be implemented by a single chip without being integrated into the processor 71.

The Memory 72 may include a Random Access Memory (RAM) 72 or a Read-Only Memory (Read-Only Memory) 72. Optionally, the memory 72 includes a non-transitory computer-readable medium. The memory 72 may be used to store instructions, programs, code sets, or instruction sets. The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 72 may alternatively be at least one memory device located remotely from the processor 71.

An embodiment of the present application further provides a storage medium, where the storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executing the method steps in the embodiments shown in fig. 1 to 5, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to 5, which is not described herein again.

It should be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional units and modules is only used for illustration, and in practical applications, the above function distribution may be performed by different functional units and modules as needed, that is, the internal structure of the apparatus may be divided into different functional units or modules to perform all or part of the above described functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. For the specific working processes of the units and modules in the system, reference may be made to the corresponding processes in the foregoing method embodiments, which are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc.

The present invention is not limited to the above-described embodiments, and various modifications and variations of the present invention are intended to be included within the scope of the claims and the equivalent technology of the present invention if they do not depart from the spirit and scope of the present invention.

Claims

1. A method for updating and planning flight routes of unmanned aerial vehicles is characterized by comprising the following steps:

s2: acquiring real-time signal-to-noise ratio data of the unmanned aerial vehicle in the flight process; according to the real-time signal-to-noise ratio data, performing interference detection on the unmanned aerial vehicle to obtain an interference detection result; according to the interference detection result, acquiring position data of the unmanned aerial vehicle at the current moment as position data of a current interference state, and acquiring signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state and real-time signal to noise ratio data;

s5: the method comprises the steps of obtaining a plurality of mobile income association groups of the unmanned aerial vehicles corresponding to a preset number of interference state positions of the unmanned aerial vehicles in the flight process, updating action modules and evaluation modules of a multi-step deep learning network model according to the mobile income association groups of the unmanned aerial vehicles, and obtaining the updated multi-step deep learning network model;

s6: when the unmanned aerial vehicle detects the position data of the next interference state, inputting the position data of the next interference state of the unmanned aerial vehicle into the updated multi-step deep learning network model to obtain moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle; updating the flight path according to moving direction data corresponding to the position data of the next interference state of the unmanned aerial vehicle, and controlling the unmanned aerial vehicle to move along the updated flight path;

2. The method for updating the planning of the flight path of the unmanned aerial vehicle as claimed in claim 1, wherein the step of obtaining the signal interruption probability data of the interference state of the unmanned aerial vehicle according to the position data and the real-time signal-to-noise ratio data of the interference state of the unmanned aerial vehicle comprises the steps of:

dividing the target area into a plurality of communication areas, determining the communication area related to the interference state of the unmanned aerial vehicle according to the position data of the interference state of the unmanned aerial vehicle, and acquiring the position data of the communication area related to the interference state of the unmanned aerial vehicle;

acquiring the signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state, the position data of the associated communication area, the real-time signal to noise ratio data and a preset signal interruption probability algorithm, wherein the signal interruption probability algorithm is as follows:

P _out (t)＝P(s _t ,b _t ,SIR _t )

in the formula, P _out (t) signal interruption probability data, s, for the tth interference state of the drone _t Position data of the tth interference state of the unmanned aerial vehicle, b _t Position data, SIR, of a communication area associated with the tth interference state of the UAV _t Real-time signal-to-noise ratio data of the t interference state of the unmanned aerial vehicle; p () is a signal outage probability data calculation function with respect to the interference state of the drone.

3. The method for updating a plan for a flight path of a drone of claim 2, wherein: the evaluation module and the action module are both of a double-network structure, the evaluation module comprises a first sub-evaluation module and a second sub-evaluation module which are identical and connected in sequence, and the action module comprises a first sub-action module and a second sub-action module which are identical and connected in sequence.

4. The method for updating and planning flight path of unmanned aerial vehicle according to claim 3, wherein the step of inputting the position data of the interference state of unmanned aerial vehicle into an action module of a preset multi-step deep learning network model, and acquiring the moving direction data of the interference state of unmanned aerial vehicle output by the action module comprises the steps of:

inputting the position data of the interference state of the unmanned aerial vehicle into a first sub-action module of the action module, and acquiring the moving direction data of the interference state of the unmanned aerial vehicle output by the first sub-action module according to a preset moving direction data calculation algorithm, wherein the moving direction data calculation algorithm is as follows:

a _t ＝μ(s _t |θ _μ )+N _t

5. The method for updating the planning of the flight path of the unmanned aerial vehicle according to claim 3, wherein the flight path is updated according to the moving direction data, and the unmanned aerial vehicle is controlled to move along the updated flight path, comprising the following steps:

and according to the moving direction data and a preset moving direction comparison table, obtaining a moving direction corresponding to the moving direction data, updating the flight route according to the moving direction, and controlling the unmanned aerial vehicle to move along the updated flight route.

6. The method for updating and planning flight path of unmanned aerial vehicle according to claim 3, wherein the step of inputting the position data and the signal interruption probability data of the interference state of unmanned aerial vehicle into the evaluation module of the multi-step deep learning network model, and obtaining the mobile revenue data of the interference state of unmanned aerial vehicle output by the evaluation module comprises the steps of:

inputting the position data and the signal interruption probability data of the interference state of the unmanned aerial vehicle into a first sub-evaluation module of the evaluation module, and acquiring the mobile revenue data of the interference state of the unmanned aerial vehicle output by the first sub-evaluation module according to a preset mobile revenue data calculation algorithm, wherein the mobile revenue data calculation algorithm is as follows:

r _t ＝-1-δ*P _out (t+1)

in the formula, r _t For the mobile profit data of the t-th interference state of the unmanned aerial vehicle, delta is a penalty weight parameter, P _out (t + 1) is the signal interruption probability data of the t +1 th interference state of the unmanned aerial vehicle.

7. The method for updating the planning of the flight path of the unmanned aerial vehicle as claimed in claim 6, wherein the step of updating the action module and the evaluation module of the multi-step deep learning network model according to the mobile profit association of the unmanned aerial vehicle to obtain the updated multi-step deep learning network model comprises the steps of:

judging the position of the unmanned aerial vehicle in the interference state according to the position data of the interference state of the unmanned aerial vehicle in the mobile revenue association set of the unmanned aerial vehicle, and acquiring a position judgment result, wherein the position judgment result comprises an end point position result, a position offset result and a position coincidence result;

obtaining expected values corresponding to the mobile revenue association groups of the unmanned aerial vehicles according to the position judgment result of the interference state of the unmanned aerial vehicles, the mobile revenue data of the interference state of the unmanned aerial vehicles and a preset expected value calculation algorithm, wherein the expected value calculation algorithm is as follows:

in the formula, y _t Is the expected value, r, of the tth interference state of the unmanned aerial vehicle _t Is the mobile profit data, s, of the t-th interference state of the unmanned aerial vehicle _t+1 Position data of t +1 th interference state of the unmanned aerial vehicle, c _t Determining a position judgment result of the t interference state of the unmanned aerial vehicle, wherein A is an end position result, B is a position deviation result, C is a position coincidence result, gamma is a preset expectation coefficient, Q '() is an evaluation calculation function in the second sub-evaluation module, mu' () is an action calculation function in the second sub-action module, and theta ^μ' The action weight parameter of the second sub-action module; theta ^Q ' is an evaluation weight parameter of the second sub-evaluation module; r _des For a predetermined prize value, P _ob Is a preset penalty value;

obtaining an error value of the first sub-evaluation module according to the movement direction data and the position data corresponding to the mobile profit association of each unmanned aerial vehicle, an expected value, movement direction data and a preset error calculation algorithm corresponding to the mobile profit association of each unmanned aerial vehicle, updating the evaluation weight parameter of the first sub-evaluation module according to the error value, and obtaining an updated evaluation weight parameter of the first sub-evaluation module, wherein the error calculation algorithm is as follows:

wherein L is the error value, N is the total number of the mobile revenue association set of the UAV, Q () is an evaluation calculation function in the first sub-evaluation module, s _t Position data of the tth interference state of the unmanned aerial vehicle, a _t Is the movement of the t interference state of the unmanned aerial vehicleDirection data, theta ^Q An evaluation weight parameter for the first sub-evaluation module;

obtaining a gradient update value of the first sub-action module according to an expected value corresponding to the mobile revenue association group of each unmanned aerial vehicle and a preset gradient update calculation algorithm, updating the action weight parameter of the first sub-action module according to the gradient update value, and obtaining an updated action weight parameter of the first sub-action module, wherein the gradient update calculation algorithm is as follows:

in the formula (I), the compound is shown in the specification,

for Q () in the t interference state, μ(s) for the moving direction data _t ) The value of the gradient update to be solved,

an action weight parameter theta for mu () on said first sub-action module in the tth interference state _μ The solved gradient update value, alpha is a preset gradient decreasing step length, theta ^μ An action weight parameter of the first sub-action module;

according to the updated evaluation weight parameter of the first sub-evaluation module, the updated action weight parameter of the first sub-action module and a preset weight parameter updating algorithm, updating the weight parameter of the second sub-evaluation module and the weight parameter of the second sub-action module respectively, and acquiring the updated evaluation weight parameter of the second sub-evaluation module and the updated action weight parameter of the second sub-action module, wherein the weight parameter updating algorithm is as follows:

θ ^Q' ←τθ ^Q +(1-τ)θ ^Q' ,θ ^μ' ←τθ ^μ +(1-τ)θ ^μ'

8. An apparatus for updating a plan of a flight path of an unmanned aerial vehicle, comprising:

the interference detection module is used for acquiring real-time signal-to-noise ratio data in the flight process of the unmanned aerial vehicle; according to the real-time signal-to-noise ratio data, performing interference detection on the unmanned aerial vehicle to obtain an interference detection result; acquiring position data of the unmanned aerial vehicle at the current moment as position data of a current interference state according to the interference detection result, and acquiring signal interruption probability data of the unmanned aerial vehicle interference state according to the position data of the unmanned aerial vehicle interference state and real-time signal-to-noise ratio data;

the first air route updating module is used for inputting the position data of the unmanned aerial vehicle interference state into an action module of a preset multi-step deep learning network model and acquiring the moving direction data of the unmanned aerial vehicle interference state; updating the flight route according to the moving direction data, and controlling the unmanned aerial vehicle to move along the updated flight route;

the mobile profit calculation module is used for inputting the position data of the unmanned aerial vehicle interference state and the signal interruption probability data into the evaluation module of the multistep deep learning network model, obtaining the mobile profit data of the unmanned aerial vehicle interference state, and combining the mobile profit data with the position data of the unmanned aerial vehicle interference state to be used as a mobile profit association group of the unmanned aerial vehicle;

9. A computer device, comprising: a processor, a memory, and a computer program stored on the memory and executable on the processor; the computer program when executed by the processor implements the steps of the method of unmanned aerial vehicle flight path update planning of any of claims 1 to 7.

10. A storage medium, characterized by: the storage medium stores a computer program which, when executed by a processor, implements the steps of the method of unmanned aerial vehicle flight path update planning of any of claims 1 to 7.