CN110287945B - Unmanned aerial vehicle target detection method in 5G environment - Google Patents
Unmanned aerial vehicle target detection method in 5G environment Download PDFInfo
- Publication number
- CN110287945B CN110287945B CN201910599417.7A CN201910599417A CN110287945B CN 110287945 B CN110287945 B CN 110287945B CN 201910599417 A CN201910599417 A CN 201910599417A CN 110287945 B CN110287945 B CN 110287945B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- base station
- target detection
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 90
- 238000005457 optimization Methods 0.000 claims abstract description 18
- 238000009826 distribution Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 18
- 238000005520 cutting process Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 9
- 238000000034 method Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000013144 data compression Methods 0.000 claims description 6
- 230000006854 communication Effects 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 230000009471 action Effects 0.000 claims description 3
- 230000006855 networking Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000013441 quality evaluation Methods 0.000 claims description 3
- 230000007175 bidirectional communication Effects 0.000 claims description 2
- 230000002457 bidirectional effect Effects 0.000 claims description 2
- 238000007906 compression Methods 0.000 claims 1
- 230000006835 compression Effects 0.000 claims 1
- 238000004364 calculation method Methods 0.000 abstract description 11
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 238000013210 evaluation model Methods 0.000 description 2
- 238000013439 planning Methods 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 238000010223 real-time analysis Methods 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003467 diminishing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/52—Network services specially adapted for the location of the user terminal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Astronomy & Astrophysics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Remote Sensing (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention provides an unmanned aerial vehicle target detection method in a 5G environment, and belongs to the technical field of 5G technology, unmanned aerial vehicles, target detection and edge calculation. In addition, edge intelligent computing nodes are managed in a unified mode through a cloud center, the edge nodes are subjected to personalized distribution of intelligent models and services according to unmanned aerial vehicle target detection tasks, and meanwhile images collected by the unmanned aerial vehicles are collected and used for continuous optimization of the cloud target detection models.
Description
Technical Field
The invention relates to a 5G technology, an unmanned aerial vehicle, a target detection and edge calculation technology, in particular to a target detection method of the unmanned aerial vehicle in a 5G environment.
Background
An unmanned aircraft, abbreviated as "drone", and abbreviated in english as "UAV", is an unmanned aircraft that is operated by a radio remote control device and a self-contained program control device, or is operated autonomously, either completely or intermittently, by an onboard computer. Along with the development of unmanned aerial vehicle technique and the continuous decline of key spare part cost, greatly reduced the threshold of developing unmanned aerial vehicle system, this also makes unmanned aerial vehicle obtain extensive application, and unmanned aerial vehicle's volume and weight also are constantly diminishing in order to adapt to wider application scene, and unmanned aerial vehicle also becomes more intelligent simultaneously, and intelligent functions such as automatic obstacle avoidance, automatic tracking target all begin to popularize.
With the coming of a new information revolution represented by technologies and businesses such as cloud computing and mobile internet, revolutionary transition from product provision to service provision is brought, and the information industry is a key element for promoting the upgrading of the information industry and promoting the integration of the vertical industry and the internet. The cloud center aggregates a large amount of physical hardware resources, realizes unified allocation, scheduling and management of heterogeneous network computing resources by adopting a virtualization technology, and greatly reduces computing and storage costs by intensively building a data center. Along with the increasing huge cloud data volume, the transmission rate is reduced, and even a large network delay sometimes exists, and the network delay becomes a large factor restricting the application. The current situation of 5G is technically changed, 5G has the characteristics of ultrahigh bandwidth, low delay and large capacity, and the end-to-end network slicing capability can flexibly and dynamically allocate required network resources to different requirements, so that a customized network is provided for users aiming at the vertical industry. On the other hand, with the development of artificial intelligence technology in recent years, especially the algorithm evolution taking deep learning as a core, the large-scale convolutional neural network similar to the human brain structure is obtained through training and construction with the support of big data and the super-strong evolutionary capability, and landing is realized in different fields. Under the 5G environment, along with the continuous surge of the demand of the edge side, the capability of artificial intelligence gradually sinks to the edge side from the cloud, the artificial intelligence is closer to the demand side of the equipment, and the edge computing node represented by the 5G base station brings new possibility to the whole edge side application industry.
Utilize unmanned aerial vehicle to carry out target detection is unmanned aerial vehicle's important application, nevertheless receive unmanned aerial vehicle size, load, computational resource's restriction, unmanned aerial vehicle is more as an acquisition device, and the data of collection also are mostly to shoot with the angle of overlooking perpendicularly, and target detection's rate of accuracy has certain restriction. Under the circumstances, how to effectively utilize the 5G network and the edge computing capability and improve the target detection accuracy of the unmanned aerial vehicle become problems which need to be solved urgently.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method for detecting the target of the unmanned aerial vehicle in the 5G environment, which effectively utilizes a 5G high-bandwidth high-speed network and a 5G base station edge intelligent service, combines the computing capability of the terminal side of the unmanned aerial vehicle, detects the image obtained by aerial shooting of the unmanned aerial vehicle in real time, transmits the undetermined target back to the 5G base station side through 5G for analysis, feeds the result back to the unmanned aerial vehicle in real time, changes the advancing mode of the unmanned aerial vehicle, and changes the angle or adjusts sensing equipment to complete the target detection task. In addition, edge intelligent computing nodes are managed in a unified mode through a cloud center, the edge nodes are subjected to personalized distribution of intelligent models and services according to unmanned aerial vehicle target detection tasks, and meanwhile images collected by the unmanned aerial vehicles are collected and used for continuous optimization of the cloud target detection models.
According to the invention, a large amount of computing resources are gathered by the cloud center, model training is carried out on a target detection task executed by the unmanned aerial vehicle, multi-dimensional cutting optimization is carried out, and services and models are respectively pushed to the 5G edge base station terminal and the unmanned aerial vehicle terminal by combining business logic. The unmanned aerial vehicle interacts with the edge computing node in real time through a 5G network, and real-time target detection of images collected by the unmanned aerial vehicle is achieved jointly. In addition, the edge base station provides the storage function, can have the edge base station with the image that unmanned aerial vehicle gathered temporarily to unified timesharing is uploaded to cloud center and is used for future model optimization, further promotes the recognition accuracy, has also promoted unmanned aerial vehicle's duration to a certain extent.
Wherein,
the unmanned aerial vehicle is provided with an image acquisition device and has certain calculation and storage capacities, a target detection model specially aiming at the unmanned aerial vehicle can be loaded after the cloud center training, cutting and optimizing, real-time target detection reasoning is carried out at the unmanned aerial vehicle end, interaction with a 5G base station and the cloud center is completed through a 5G communication module, and a cooperative task is completed.
In addition, unmanned aerial vehicle possesses the data compression function, further reduces storage and network bandwidth.
The 5G base station is a node with edge computing capability, comprises the capabilities of computing, storing and networking, provides 5G network service externally, is upwards connected with a core network in a butt joint mode and is connected with a cloud center, can realize the services of target detection, data temporary storage, safety certification, unmanned aerial vehicle interaction control, data compression, time-sharing uploading and the like, realizes two-way message transmission and action interaction with the unmanned aerial vehicle, carries out two-way communication with the cloud center, and timely updates and manages the node assembly service.
The cloud center realize the distribution of target detection task and the multidimension degree of target detection model and tailor the optimization to with service and model, carry out the propelling movement with and the business demand according to 5G basic station and unmanned aerial vehicle's actual specification, realize the management to 5G basic station and unmanned aerial vehicle, the high in the clouds carries out the model training through the image that comes from marginal basic station and unmanned aerial vehicle collection that continues to collect and optimizes, continues to promote the discernment precision.
The invention is used for an unmanned aerial vehicle to execute image acquisition and target detection tasks, and comprises the following steps:
101, the cloud center gathers a large amount of hardware resources, and performs model training by using actually collected images to form corresponding detection models for target detection tasks of pictures collected by the unmanned aerial vehicle;
102, establishing an image acquisition task for the unmanned aerial vehicle through the cloud center, and determining the target detection requirement and the general driving route of the unmanned aerial vehicle;
103, cutting and optimizing a corresponding model according to the actual specification of the unmanned aerial vehicle and the requirement of a target detection task to form a detection model which is most suitable for the unmanned aerial vehicle, and downloading the detection model into the unmanned aerial vehicle through a safety channel;
104, selecting a model which is most suitable for the 5G base station to operate according to the target detection task requirement of the unmanned aerial vehicle, and directionally pushing a detection model aiming at the task, a picture quality evaluation model and related services interacted with the unmanned aerial vehicle to the 5G base station through a safety channel according to a driving route, so that the 5G base station can be conveniently and quickly loaded in the future;
105, the unmanned aerial vehicle starts to execute an image acquisition task, flies above a relevant area, acquires images for ground targets, performs real-time target detection and analysis by using a local model of the unmanned aerial vehicle, determines target objects and takes pictures for multiple times;
106, the unmanned aerial vehicle quickly transmits the acquired image to the 5G base station through a 5G network, and simultaneously, the image with low certainty factor is specially marked, so that the 5G base station is requested to provide service, and further image processing is completed;
step 107, the 5G base station receives the image sent by the unmanned aerial vehicle, detects the image by adopting a target detection model of the base station local to the unmanned aerial vehicle image acquisition task, and determines a detection result;
step 108, the 5G base station analyzes and detects the image sequence from the unmanned aerial vehicle, synthesizes multiple factors such as confidence coefficient, picture quality and the like, plans an optimal shooting place and feeds back the optimal shooting place to the unmanned aerial vehicle;
step 109, the unmanned aerial vehicle adjusts the flight line of the unmanned aerial vehicle in real time according to the feedback from the 5G base station, changes the position and the angle of the unmanned aerial vehicle and shoots;
and step 110, repeating the steps 105 to 109, and continuously improving the quality of the acquired image to meet the acquisition task.
111, temporarily storing the images acquired by the unmanned aerial vehicle in a local place by the 5G base station, classifying and compressing the images according to actual acquisition task requirements, and uploading the images to the cloud center;
step 112, (optional) when the timeliness of the acquisition task is not strong, the 5G base station can select the relative idle time of the network, and the network is compressed and uploaded to the cloud center;
step 113, the unmanned aerial vehicle executes an image acquisition task;
and step 114, the cloud center further processes the image according to the task execution condition, and uses the image collected this time for training the task type target detection model, so as to continuously improve the detection precision.
The invention has the advantages that
The unmanned aerial vehicle tasks are managed in a unified mode through the cloud center, targeted model training is performed according to target detection requirements generated by different tasks executed by the unmanned aerial vehicle and by combining business logic, multi-dimensional model cutting optimization is performed according to the specification of the unmanned aerial vehicle and the specification of the tasks, the unmanned aerial vehicle is enabled to have the capability of real-time analysis and processing, and the problem of high energy consumption calculation caused by the power consumption and volume limitation of the unmanned aerial vehicle is solved; simultaneously according to the task that unmanned aerial vehicle executed, the optimization back relevant detection model that will relate to, evaluation model and relevant service, push to 5G edge base station end, make full use of the high-speed transmission ability of 5G network and the marginal intelligent ability of 5G basic station, through the interdynamic between edge calculation node and the unmanned aerial vehicle, alleviate the not enough of unmanned aerial vehicle resource, promote unmanned aerial vehicle's real-time target detection rate of accuracy simultaneously, and according to the feedback of 5G edge calculation node, carry out image acquisition route planning, promote the execution efficiency who gathers the image task, and then promote the image quality who finally gathers the image. In addition, images acquired by the unmanned aerial vehicle temporarily have edge base stations, the cruising ability of the unmanned aerial vehicle is improved to a certain extent, the images are uploaded to the cloud center in a unified time-sharing mode for future model optimization, the network bandwidth and the computing resources are used more efficiently, and meanwhile the target detection and identification accuracy is also continuously improved.
Drawings
Fig. 1 is a schematic diagram of a node where an unmanned aerial vehicle performs a task to participate.
Fig. 2 is a flowchart of the unmanned aerial vehicle performing image acquisition and target detection tasks.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
According to the invention, the tasks of the unmanned aerial vehicle are uniformly managed through the cloud center, targeted model training is carried out according to target detection requirements generated by different tasks executed by the unmanned aerial vehicle and by combining business logic, multi-dimensional model cutting optimization is carried out according to the specification and the task specification of the unmanned aerial vehicle, the unmanned aerial vehicle is enabled to have the capability of real-time analysis and processing, and the problem of high energy consumption calculation caused by the power consumption and volume limitation of the unmanned aerial vehicle is solved; simultaneously according to the task that unmanned aerial vehicle executed, the optimization back relevant detection model that will relate to, evaluation model and relevant service, push to 5G edge base station end, make full use of the high-speed transmission ability of 5G network and the marginal intelligent ability of 5G basic station, through the interdynamic between edge calculation node and the unmanned aerial vehicle, alleviate the not enough of unmanned aerial vehicle resource, promote unmanned aerial vehicle's real-time target detection rate of accuracy simultaneously, and according to the feedback of 5G edge calculation node, carry out image acquisition route planning, promote the execution efficiency who gathers the image task, and then promote the image quality who finally gathers the image. In addition, images acquired by the unmanned aerial vehicle temporarily have edge base stations, the cruising ability of the unmanned aerial vehicle is improved to a certain extent, the images are uploaded to the cloud center in a unified time-sharing mode for future model optimization, the network bandwidth and the computing resources are used more efficiently, and meanwhile the target detection and identification accuracy is also continuously improved.
As shown in fig. 1, the cloud center aggregates a large amount of computing resources, performs model training on a target detection task executed by the unmanned aerial vehicle, performs multidimensional cutting optimization, and pushes services and models to a 5G edge base station end and an unmanned aerial vehicle terminal respectively in combination with business logic. The unmanned aerial vehicle interacts with the edge computing node in real time through a 5G network, and real-time target detection of images collected by the unmanned aerial vehicle is achieved jointly. In addition, the edge base station provides the storage function, can have the edge base station with the image that unmanned aerial vehicle gathered temporarily to unified timesharing is uploaded to cloud center and is used for future model optimization, further promotes the recognition accuracy, has also promoted unmanned aerial vehicle's duration to a certain extent.
Wherein,
the unmanned aerial vehicle is provided with an image acquisition device, has certain calculation and storage capacities, can load a target detection model specially aiming at the unmanned aerial vehicle after training, cutting and optimizing of a cloud center, carries out real-time target detection reasoning at an unmanned aerial vehicle end, completes interaction with a 5G base station and the cloud center through a 5G communication module, and completes a cooperative task, and in addition, the unmanned aerial vehicle has a data compression function, so that the storage and network bandwidth are further reduced;
the 5G base station is a node with edge computing capability, comprises the capabilities of computing, storing and networking, provides 5G network service to the outside, is upwards butted with a core network and is connected with a cloud center, can realize the services of target detection, data temporary storage, security authentication, unmanned aerial vehicle interaction control, data compression, time-sharing uploading and the like, realizes bidirectional message transmission and action interaction with the unmanned aerial vehicle, performs bidirectional communication with the cloud center, and timely updates and manages the node assembly service;
the cloud center realize the distribution of target detection task and the multidimension degree of target detection model and tailor the optimization to with service and model, carry out the propelling movement with and the business demand according to 5G basic station and unmanned aerial vehicle's actual specification, realize the management to 5G basic station and unmanned aerial vehicle, the high in the clouds carries out the model training through the image that comes from marginal basic station and unmanned aerial vehicle collection that continues to collect and optimizes, continues to promote the discernment precision.
For clarity of description, the unmanned aerial vehicle mentioned below is provided with an image acquisition module, a data storage module, a calculation analysis module, a position positioning module, a security authentication module and the like in addition to a 5G communication module, wherein a handshake protocol can be adopted between the unmanned aerial vehicle and an edge node of the 5G base station for mutual authentication to form a security channel, and data is transmitted through encryption to ensure the security of the unmanned aerial vehicle; the target detection algorithm adopted by the unmanned aerial vehicle can be an SSD algorithm, a YOLO algorithm and the like. Those skilled in the art will appreciate that the configurations according to embodiments of the present invention can be applied on other scenarios in addition to using the above terminals and algorithms.
The method provided by the invention will be described in detail with reference to specific examples.
Unmanned aerial vehicle executes image acquisition and target detection tasks
Referring to fig. 2, the unmanned aerial vehicle performing image acquisition and target detection tasks includes the following steps:
101, the cloud center gathers a large amount of hardware resources, and performs model training by using actually collected images to form corresponding detection models for target detection tasks of pictures collected by the unmanned aerial vehicle;
102, establishing an image acquisition task for the unmanned aerial vehicle through the cloud center, and determining the target detection requirement and the general driving route of the unmanned aerial vehicle;
103, cutting and optimizing a corresponding model according to the actual specification of the unmanned aerial vehicle and the requirement of a target detection task to form a detection model which is most suitable for the unmanned aerial vehicle, and downloading the detection model into the unmanned aerial vehicle through a safety channel;
104, selecting a model which is most suitable for the 5G base station to operate according to the target detection task requirement of the unmanned aerial vehicle, and directionally pushing a detection model aiming at the task, a picture quality evaluation model and related services interacted with the unmanned aerial vehicle to the 5G base station through a safety channel according to a driving route, so that the 5G base station can be conveniently and quickly loaded in the future;
105, the unmanned aerial vehicle starts to execute an image acquisition task, flies above a relevant area, acquires images for ground targets, performs real-time target detection and analysis by using a local model of the unmanned aerial vehicle, determines target objects and takes pictures for multiple times;
106, the unmanned aerial vehicle quickly transmits the acquired image to the 5G base station through a 5G network, and simultaneously, the image with low certainty factor is specially marked, so that the 5G base station is requested to provide service, and further image processing is completed;
step 107, the 5G base station receives the image sent by the unmanned aerial vehicle, detects the image by adopting a target detection model of the base station local to the unmanned aerial vehicle image acquisition task, and determines a detection result;
step 108, the 5G base station analyzes and detects the image sequence from the unmanned aerial vehicle, synthesizes multiple factors such as confidence coefficient, picture quality and the like, plans an optimal shooting place and feeds back the optimal shooting place to the unmanned aerial vehicle;
step 109, the unmanned aerial vehicle adjusts the flight line of the unmanned aerial vehicle in real time according to the feedback from the 5G base station, changes the position and the angle of the unmanned aerial vehicle and shoots;
and step 110, repeating the steps 105 to 109, and continuously improving the quality of the acquired image to meet the acquisition task.
111, temporarily storing the images acquired by the unmanned aerial vehicle in a local place by the 5G base station, classifying and compressing the images according to actual acquisition task requirements, and uploading the images to the cloud center;
step 112, (optional) when the timeliness of the acquisition task is not strong, the 5G base station can select the relative idle time of the network, and the network is compressed and uploaded to the cloud center;
step 113, the unmanned aerial vehicle executes an image acquisition task;
and step 114, the cloud center further processes the image according to the task execution condition, and uses the image collected this time for training the task type target detection model, so as to continuously improve the detection precision.
The above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (2)
1. A method for detecting the target of an unmanned aerial vehicle in a 5G environment is characterized in that,
the cloud center is used for gathering computing resources, model training is carried out on a target detection task executed by the unmanned aerial vehicle, multi-dimensional cutting optimization is carried out, and services and models are respectively pushed to a 5G edge base station end and an unmanned aerial vehicle terminal in combination with business logic; the unmanned aerial vehicle interacts with the edge computing node in real time through a 5G network, and real-time target detection of images acquired by the unmanned aerial vehicle is realized together; in addition, the edge base station provides a storage function, images acquired by the unmanned aerial vehicle are temporarily stored in the edge base station and are uploaded to the cloud center in a unified time-sharing mode for future model optimization;
by utilizing a 5G high-bandwidth high-speed network and 5G base station edge intelligent service and combining the computing capacity of a terminal side of the unmanned aerial vehicle, detecting images obtained by aerial photography of the unmanned aerial vehicle in real time, transmitting an undetermined target back to the 5G base station side through 5G for analysis, feeding back the result to the unmanned aerial vehicle in real time, changing the advancing mode of the unmanned aerial vehicle, changing the angle or adjusting sensing equipment to complete a target detection task;
edge intelligent computing nodes are managed through a cloud center in a unified mode, individualized distribution of intelligent models and services is conducted on the edge nodes according to target detection tasks of the unmanned aerial vehicles, and meanwhile images collected by the unmanned aerial vehicles are collected and used for continuous optimization of a cloud target detection model;
the unmanned aerial vehicle has computing and storing capabilities, can load a target detection model specially aiming at the unmanned aerial vehicle after training, cutting and optimizing of a cloud center, carries out real-time target detection inference at the unmanned aerial vehicle end, and finishes interaction with a 5G base station and the cloud center through a 5G communication module to finish a cooperative task;
the unmanned aerial vehicle has a data compression function, so that the storage and network bandwidth can be further reduced;
the 5G base station is a node with edge computing capability, comprises the capabilities of computing, storing and networking, provides 5G network service to the outside, is upwards butted with a core network and is connected with a cloud center, can realize target detection, data temporary storage, safety authentication, unmanned aerial vehicle interaction control, data compression and time-sharing uploading service, realizes bidirectional message transmission and action interaction with an unmanned aerial vehicle, performs bidirectional communication with the cloud center, and timely updates and manages the node assembly service;
the cloud center distributes a target detection task and optimizes a target detection model in a multi-dimensional cutting mode, services and the model are pushed according to the actual specifications of the 5G base station and the unmanned aerial vehicle and the service requirements of the 5G base station and the unmanned aerial vehicle, the 5G base station and the unmanned aerial vehicle are managed, model training optimization is carried out on the cloud end through images collected continuously from the edge base station and the unmanned aerial vehicle, and the identification precision is continuously improved;
the operation steps are as follows:
the method comprises the following steps:
step 101, gathering hardware resources by a cloud center, and performing model training by using actually collected images to form corresponding detection models for target detection tasks of pictures collected by an unmanned aerial vehicle;
102, creating an image acquisition task for the unmanned aerial vehicle through a cloud center, and determining the target detection requirement and the driving route of the unmanned aerial vehicle;
103, cutting and optimizing a corresponding model according to the actual specification of the unmanned aerial vehicle and the requirement of a target detection task to form a detection model which is most suitable for the unmanned aerial vehicle, and downloading the detection model into the unmanned aerial vehicle through a safety channel;
104, selecting a model which is most suitable for the 5G base station to operate according to the target detection task requirement of the unmanned aerial vehicle, and directionally pushing a detection model aiming at the task, a picture quality evaluation model and related services interacted with the unmanned aerial vehicle to the 5G base station through a safety channel according to a driving route, so that the 5G base station can be conveniently and quickly loaded in the future;
105, the unmanned aerial vehicle starts to execute an image acquisition task, flies above a relevant area, acquires images for ground targets, performs real-time target detection and analysis by using a local model of the unmanned aerial vehicle, determines target objects and takes pictures for multiple times;
106, the unmanned aerial vehicle quickly transmits the acquired image to the 5G base station through a 5G network, and simultaneously, the image with low certainty factor is specially marked, so that the 5G base station is requested to provide service, and further image processing is completed;
step 107, the 5G base station receives the image sent by the unmanned aerial vehicle, detects the image by adopting a target detection model of the base station local to the unmanned aerial vehicle image acquisition task, and determines a detection result;
step 108, the 5G base station analyzes and detects the image sequence from the unmanned aerial vehicle, integrates confidence coefficient and picture quality factors, plans an optimal shooting place and feeds back the optimal shooting place to the unmanned aerial vehicle;
step 109, the unmanned aerial vehicle adjusts the flight line of the unmanned aerial vehicle in real time according to the feedback from the 5G base station, changes the position and the angle of the unmanned aerial vehicle and shoots;
step 110, repeating the steps 105 to 109, continuously improving the quality of the collected image and meeting the collection task;
step 111, the 5G base station temporarily stores the images acquired by the unmanned aerial vehicle in the local, classifies and compresses the images according to the actual acquisition task requirements, and uploads the images to the cloud center;
step 112, the unmanned aerial vehicle executes an image acquisition task;
and step 113, the cloud center further processes the image according to the task execution condition, and uses the image collected this time for training the task type target detection model, so as to continuously improve the detection precision.
2. The method of claim 1,
when the timeliness of the acquisition task is not strong, the 5G base station selects the relative idle time of the network, performs image compression and uploads the image to the cloud center.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910599417.7A CN110287945B (en) | 2019-07-04 | 2019-07-04 | Unmanned aerial vehicle target detection method in 5G environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910599417.7A CN110287945B (en) | 2019-07-04 | 2019-07-04 | Unmanned aerial vehicle target detection method in 5G environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110287945A CN110287945A (en) | 2019-09-27 |
| CN110287945B true CN110287945B (en) | 2021-03-16 |
Family
ID=68021866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910599417.7A Active CN110287945B (en) | 2019-07-04 | 2019-07-04 | Unmanned aerial vehicle target detection method in 5G environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110287945B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111431986B (en) * | 2020-03-18 | 2023-02-28 | 宁波智诚祥科技发展有限公司 | Industrial intelligent quality inspection system based on 5G and AI cloud edge cooperation |
| CN111582016A (en) * | 2020-03-18 | 2020-08-25 | 宁波送变电建设有限公司永耀科技分公司 | Intelligent maintenance-free power grid monitoring method and system based on cloud edge collaborative deep learning |
| CN111486869B (en) * | 2020-04-22 | 2022-03-18 | 电子科技大学 | Distributed flight verification system and method based on 5G |
| CN113746524A (en) * | 2020-05-29 | 2021-12-03 | 青岛云世纪信息科技有限公司 | Unmanned aerial vehicle interconnection system based on 5G |
| CN112113150A (en) * | 2020-08-11 | 2020-12-22 | 苏州辰航致远智能科技有限公司 | Pipeline detection system based on 5G communication and remote control method thereof |
| CN112130588B (en) * | 2020-08-27 | 2022-03-01 | 东风汽车集团有限公司 | Method for searching target person, vehicle-mounted terminal and unmanned aerial vehicle |
| CN112055178A (en) * | 2020-09-07 | 2020-12-08 | 河南浩宇空间数据科技有限责任公司 | Be applied to unmanned aerial vehicle low latency 5G communication system |
| CN112289032B (en) * | 2020-11-05 | 2022-01-14 | 浙江点辰航空科技有限公司 | Automatic inspection method for unmanned aerial vehicle expressway |
| CN112367400B (en) * | 2020-11-12 | 2022-04-29 | 广东电网有限责任公司 | Intelligent inspection method and system for power internet of things with edge cloud coordination |
| CN112859907A (en) * | 2020-12-25 | 2021-05-28 | 湖北航天飞行器研究所 | Rocket debris high-altitude detection method based on three-dimensional special effect simulation under condition of few samples |
| CN112950396A (en) * | 2021-03-08 | 2021-06-11 | 中国联合网络通信集团有限公司 | Automatic claim settlement method, automatic claim settlement platform and automatic claim settlement system based on unmanned aerial vehicle |
| CN113067872B (en) * | 2021-03-19 | 2023-01-06 | 航天时代飞鸿技术有限公司 | Private network cloud control system for unmanned system |
| CN113534829B (en) * | 2021-06-11 | 2024-04-05 | 南京邮电大学 | Unmanned aerial vehicle daily patrol detecting system based on edge calculation |
| CN113534832B (en) * | 2021-08-03 | 2024-03-15 | 国网江苏省电力有限公司泰州供电分公司 | Unmanned aerial vehicle routing inspection tracking distribution network line flight method based on edge calculation |
| CN114020006B (en) * | 2021-09-26 | 2023-04-07 | 佛山中科云图智能科技有限公司 | Unmanned aerial vehicle auxiliary landing method and device, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107679484A (en) * | 2017-09-28 | 2018-02-09 | 辽宁工程技术大学 | A kind of Remote Sensing Target automatic detection and recognition methods based on cloud computing storage |
| CN108549384A (en) * | 2018-05-21 | 2018-09-18 | 济南浪潮高新科技投资发展有限公司 | A kind of remote control automatic Pilot method under 5G environment |
| CN108684047A (en) * | 2018-07-11 | 2018-10-19 | 北京邮电大学 | A kind of unmanned plane carries small base station communication system and method |
| CN108845885A (en) * | 2018-07-04 | 2018-11-20 | 济南浪潮高新科技投资发展有限公司 | A kind of edge calculations method for managing resource towards automatic Pilot |
| CN109828845A (en) * | 2019-03-01 | 2019-05-31 | 长沙理工大学 | A kind of insulator thermal imaging real-time diagnosis system based on edge calculations |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9111143B2 (en) * | 2013-09-27 | 2015-08-18 | At&T Mobility Ii Llc | Method and apparatus for image collection and analysis |
| CN108198439B (en) * | 2018-01-24 | 2021-05-14 | 浪潮集团有限公司 | Urban intelligent traffic control method based on fog calculation |
| CN109765939A (en) * | 2018-12-21 | 2019-05-17 | 中国科学院自动化研究所南京人工智能芯片创新研究院 | Cloud platform control method, device and the storage medium of unmanned plane |
-
2019
- 2019-07-04 CN CN201910599417.7A patent/CN110287945B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107679484A (en) * | 2017-09-28 | 2018-02-09 | 辽宁工程技术大学 | A kind of Remote Sensing Target automatic detection and recognition methods based on cloud computing storage |
| CN108549384A (en) * | 2018-05-21 | 2018-09-18 | 济南浪潮高新科技投资发展有限公司 | A kind of remote control automatic Pilot method under 5G environment |
| CN108845885A (en) * | 2018-07-04 | 2018-11-20 | 济南浪潮高新科技投资发展有限公司 | A kind of edge calculations method for managing resource towards automatic Pilot |
| CN108684047A (en) * | 2018-07-11 | 2018-10-19 | 北京邮电大学 | A kind of unmanned plane carries small base station communication system and method |
| CN109828845A (en) * | 2019-03-01 | 2019-05-31 | 长沙理工大学 | A kind of insulator thermal imaging real-time diagnosis system based on edge calculations |
Non-Patent Citations (3)
| Title |
|---|
| 5G时代测绘地理信息的新发展;徐小夏等;《中国测绘》;20190510;第10-13页 * |
| A Novel Protocol for Securing Network Slice Component Association and Slice Isolation in 5G Networks;Vipin N Sathi等;《the 21st ACM International Conference》;20181231;第249-253页 * |
| 基于无人机平台的动态目标检测系统开发;张宁;《中国优秀硕士学位论文全文数据库工程科技II辑》;20180615(第6期);第C031-183页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110287945A (en) | 2019-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110287945B (en) | Unmanned aerial vehicle target detection method in 5G environment | |
| US11683831B2 (en) | Systems and methods for providing resource analysis for autonomous mobility on demand | |
| CN111552313B (en) | Multi-unmanned aerial vehicle path planning method based on edge calculation dynamic task arrival | |
| CN114690799A (en) | Air-space-ground integrated unmanned aerial vehicle Internet of things data acquisition method based on information age | |
| CN113156992B (en) | Three-layer architecture collaborative optimization method for unmanned aerial vehicle in edge environment | |
| CN110650487B (en) | Internet of things edge computing configuration method based on data privacy protection | |
| CN112381265A (en) | Unmanned aerial vehicle-based charging and task unloading system and task time consumption optimization method thereof | |
| CN112929849B (en) | Reliable vehicle-mounted edge calculation unloading method based on reinforcement learning | |
| Munir et al. | Intelligent service fulfillment for software defined networks in smart city | |
| Jung et al. | Adaptive and stabilized real-time super-resolution control for UAV-assisted smart harbor surveillance platforms | |
| Giyenko et al. | Intelligent unmanned aerial vehicle platform for smart cities | |
| Wang et al. | Edge intelligence for mission cognitive wireless emergency networks | |
| CN113422803A (en) | Seamless migration method for intelligent unmanned aerial vehicle inspection task based on end edge cloud cooperation | |
| Zeng et al. | MEC-assisted real-time data acquisition and processing for UAV with general missions | |
| Yao et al. | Power control in Internet of Drones by deep reinforcement learning | |
| Gad et al. | Joint self-organizing maps and knowledge distillation-based communication-efficient federated learning for resource-constrained UAV-IoT systems | |
| Vishnevsky et al. | SDN-assisted unmanned aerial system for monitoring sensor data | |
| CN117519944A (en) | Unmanned vehicle based on cooperation of computing power perception and edge cloud computing and cooperation method thereof | |
| CN114143893A (en) | Unmanned aerial vehicle resource allocation and track optimization method based on mobile edge calculation and microwave energy transmission | |
| US11975835B2 (en) | Method and system for managing unmanned aerial vehicle data transmission in smart city based on the internet of things | |
| CN117809205A (en) | Unmanned aerial vehicle airborne AI discernment and integration of multiple networks communication, controlling means | |
| CN110471988A (en) | One kind being based on modular three section of five laminar artificial intelligence system | |
| Iacovelli et al. | An iterative stochastic approach to constrained drones' communications | |
| Lakew et al. | A review on AI-driven aerial access networks: Challenges and open research issues | |
| CN117177306B (en) | Unmanned aerial vehicle MEC network system based on NFV and SDN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210223 Address after: No. 1036, Shandong high tech Zone wave road, Ji'nan, Shandong Applicant after: INSPUR GROUP Co.,Ltd. Address before: North 6th floor, S05 building, Langchao Science Park, 1036 Langchao Road, hi tech Zone, Jinan City, Shandong Province, 250100 Applicant before: SHANDONG INSPUR ARTIFICIAL INTELLIGENCE RESEARCH INSTITUTE Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230423 Address after: S02 Building, 1036 Langchao Road, Jinan Area, China (Shandong) Pilot Free Trade Zone, Jinan City, Shandong Province, 250000 Patentee after: Shandong Inspur innovation and entrepreneurship Technology Co.,Ltd. Address before: No. 1036, Shandong high tech Zone wave road, Ji'nan, Shandong Patentee before: INSPUR GROUP Co.,Ltd. |