CN112257434B - Unmanned aerial vehicle control method, unmanned aerial vehicle control system, mobile terminal and storage medium - Google Patents
Unmanned aerial vehicle control method, unmanned aerial vehicle control system, mobile terminal and storage medium Download PDFInfo
- Publication number
- CN112257434B CN112257434B CN201910589408.XA CN201910589408A CN112257434B CN 112257434 B CN112257434 B CN 112257434B CN 201910589408 A CN201910589408 A CN 201910589408A CN 112257434 B CN112257434 B CN 112257434B
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- unmanned aerial
- natural language
- information
- language text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The application discloses an unmanned aerial vehicle control method, a system, a mobile terminal and a storage medium, wherein the method comprises the following steps: after receiving voice information input by a user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information; converting natural language text information classified as positive samples into unmanned aerial vehicle operation instructions; and sending the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle so as to control the unmanned aerial vehicle to execute corresponding operation according to the unmanned aerial vehicle operation instruction. According to the application, the mobile terminal receives the voice input of the user, converts the voice instruction of the user into the instruction executed by the unmanned aerial vehicle, and then controls the unmanned aerial vehicle to execute corresponding operation, and meanwhile, the unmanned aerial vehicle is controlled without manually operating the mobile terminal by the user, so that the unmanned aerial vehicle is accurately, timely, flexible and convenient to operate.
Description
Technical Field
The application relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle control method, an unmanned aerial vehicle control system, a mobile terminal and a storage medium.
Background
In the air detection field, unmanned aerial vehicle detects harmful gas content in the air and has replaced artifical detection to become mainstream. The unmanned aerial vehicle can reach some places that human beings can not reach or comparatively dangerous and carry out real-time detection, but current unmanned aerial vehicle controls mainly manual operation, for example controls unmanned aerial vehicle through mobile terminal and also needs manual operation, and the user still looks over unmanned aerial vehicle's flight condition in one side operation unmanned aerial vehicle, and the unmanned aerial vehicle has perhaps deviated the appointed direction in the time of being in operation in some cases, and the accuracy of operation is not high like this, and it is inconvenient to operate, not intelligent enough.
Accordingly, the prior art is still in need of improvement and development.
Disclosure of Invention
The application mainly aims to provide an unmanned aerial vehicle control method, a unmanned aerial vehicle control system, a mobile terminal and a storage medium, and aims to solve the problems that unmanned aerial vehicle control needs to be carried out manually, operation is inconvenient and control is inaccurate in the prior art.
In order to achieve the above object, the present application provides an unmanned aerial vehicle control method, comprising the steps of:
after receiving voice information input by a user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information;
converting natural language text information classified as positive samples into unmanned aerial vehicle operation instructions;
and sending the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle so as to control the unmanned aerial vehicle to execute corresponding operation according to the unmanned aerial vehicle operation instruction.
Optionally, in the unmanned aerial vehicle control method, after receiving the voice information input by the user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information includes:
positive samples and negative samples of natural language text information are defined in advance;
receiving voice information input by a user, and converting the voice information into natural language text information through a voice-to-text function of the mobile terminal;
and the natural language text information is subjected to a trained deep neural network classification model to complete positive and negative sample classification.
Optionally, the unmanned aerial vehicle control method, wherein the defining the positive and negative samples of the natural language text information in advance includes:
the positive sample is defined as information related to the implementation of unmanned aerial vehicle flight attitude control or gas detection functions;
the negative samples are defined as related information unrelated to unmanned aerial vehicle attitude control or gas monitoring function implementation.
Optionally, in the unmanned aerial vehicle control method, the performing positive and negative sample classification on the natural language text information through a trained deep neural network classification model includes:
the deep neural network classification model extracts text characteristics of the input natural language text information, and calculates probability values P of positive samples and negative samples through a multi-layer network;
when the probability value P of the negative sample is greater than a preset threshold value, classifying natural language text information into the negative sample, and popping up a prompt box to prompt that the message of the negative sample is illegal;
when the probability value P of the positive sample is greater than a preset threshold value, the natural language text information is classified into the positive sample, and the conversion processing stage is entered.
Optionally, the method for controlling a drone, wherein the converting the natural language text information classified as the positive sample into the operation instruction of the drone includes:
inputting natural language text information classified as positive samples into a trained deep learning model;
carrying out word segmentation processing on the natural language text information of the positive sample, extracting the characteristics of each word by an encoder, extracting at least one of the position information, the syntactic characteristics and the semantic characteristics of text input through calculation of a multi-layer network, and outputting an intermediate semantic information vector;
and the decoder decodes the semantic information vector and outputs an unmanned aerial vehicle operation instruction corresponding to the natural language text information.
Optionally, the unmanned aerial vehicle control method comprises the steps that the deep learning model comprises an encoder and a decoder, the classified natural language text information is directly input into the encoder of the deep learning model to be subjected to semantic coding, and the decoder of the deep learning model outputs corresponding unmanned aerial vehicle operation instructions.
Optionally, in the unmanned aerial vehicle control method, the outputting the intermediate semantic information vector specifically includes:
performing word segmentation on the natural language text information of the positive sample, adding the word2Vector processed by word segmentation to the word position coding Vector to generate a new Vector, and sending the new Vector into a network;
and the encoder of the deep learning model extracts the characteristics of each word, extracts at least one of the position information, the syntactic characteristics and the semantic characteristics of the text input through the calculation of a multi-layer network, and outputs an intermediate semantic information vector.
In addition, to achieve the above object, the present application also provides a mobile terminal, wherein the mobile terminal includes:
the system comprises a memory, a processor and a unmanned aerial vehicle control program stored on the memory and capable of running on the processor, wherein the unmanned aerial vehicle control program realizes the steps of the unmanned aerial vehicle control method when being executed by the processor.
In addition, in order to achieve the above purpose, the application also provides the unmanned aerial vehicle control system, wherein the unmanned aerial vehicle control system comprises the mobile terminal and the unmanned aerial vehicle which establishes communication connection with the mobile terminal, and the unmanned aerial vehicle receives the instruction transmitted by the mobile terminal in real time through a wireless network and executes corresponding operation according to the unmanned aerial vehicle operation instruction.
In addition, in order to achieve the above object, the present application also provides a storage medium storing a drone control program which, when executed by a processor, implements the steps of the drone control method described above.
According to the method, the mobile terminal receives voice input of the user, the voice instruction of the user is converted into the voice instruction which is executed by the unmanned aerial vehicle, then the unmanned aerial vehicle is controlled to fly to the appointed area, the unmanned aerial vehicle detects real-time items (such as gas) according to the fact that the flight instruction arrives at the area where human is inconvenient to arrive or the danger coefficient is high, convenience in air detection is greatly improved, a convenient and reliable detection method is improved for collecting environment gas detection data in the future, meanwhile, the unmanned aerial vehicle is controlled without manual operation of the mobile terminal by the user, the unmanned aerial vehicle can be controlled to fly accurately only by sending the voice instruction, and operation of the unmanned aerial vehicle is accurate, timely, flexible and convenient.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the unmanned aerial vehicle control method of the present application;
FIG. 2 is a flowchart of step S10 in a preferred embodiment of the unmanned aerial vehicle control method of the present application;
FIG. 3 is a schematic flow chart of the deep neural network classification model for completing classification of positive and negative samples in the preferred embodiment of the unmanned aerial vehicle control method of the present application;
FIG. 4 is a flowchart of step S20 in a preferred embodiment of the unmanned aerial vehicle control method of the present application;
FIG. 5 is a schematic flow chart of the deep learning model processing the natural language text information of the positive sample in the preferred embodiment of the unmanned aerial vehicle control method of the present application;
FIG. 6 is a schematic diagram of the power supply of a preferred embodiment of the unmanned control system of the present application;
fig. 7 is a flowchart of the overall system operation of a preferred embodiment of the drone control method of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear and clear, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
According to the unmanned aerial vehicle control method disclosed by the preferred embodiment of the application, as shown in fig. 1, the unmanned aerial vehicle control method comprises the following steps:
and S10, after receiving voice information input by a user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information.
Fig. 2 is a flowchart of step S10 in the unmanned aerial vehicle control method provided by the present application.
As shown in fig. 2, the step S10 includes:
s11, defining positive samples and negative samples of natural language text information in advance;
s12, receiving voice information input by a user, and converting the voice information into natural language text information through a voice-to-text function of the mobile terminal;
s13, the natural language text information is subjected to a trained deep neural network classification model to complete positive and negative sample classification.
Specifically, when positive and negative samples of natural language text information are defined and marked, the following processing is performed: the positive sample is defined as information related to unmanned aerial vehicle flight attitude control or gas detection function implementation, and can be marked as 1 for representation; the unmanned aerial vehicle flight attitude control instruction includes: take-off, landing, forward, backward, leftward, rightward, acceleration, deceleration, stopping, etc.; the gas detection function instruction includes: start to detect sulfur dioxide concentration, end to detect sulfur dioxide concentration, start to detect PM2.5 concentration, end to detect PM2.5 concentration, start to detect dust content, end to detect dust content, etc.
The negative sample is defined as related information irrelevant to the attitude control or the gas monitoring function implementation of the unmanned aerial vehicle, for example, the negative sample is marked with 0 for representation; such as how much weather is today independent of the words of flight and gas detection.
The user directly sends a voice instruction (voice information) to the mobile terminal, the voice instruction is processed by STT (speech to text) of the mobile terminal, voice input is converted into natural language text information, and then the natural language text information is sent into a deep neural network classification model trained on a large amount of text data to complete positive and negative sample classification.
Further, as shown in fig. 3, the deep neural network classification model (classification model or classifier) adopts a Pre-trained GPT network (GPT, generator Pre-Training model), which is a new neural network Pre-Training model in the natural language processing field, followed by a layer of fully-connected network+softmax network (the classification model is obtained by adding a layer of fully-connected network to the basis of the existing trained GPT network structure parameters, and then receiving the softmax function for classification output, the classification model is a typical Pre-training+fine-tuning model) structure, as the main structure of the classifier, loading the trained GPT network parameters, and then performing fine-tuning on the following forward network.
The GPT network consists of several blocks, each block has 2 sublayers, the 1 st sublayer calculates the self-attention value of the token between each other (self-attention is one of the attention mechanisms, the self-attention value is calculated as the self-attention calculation of each token, a value is output, the calculation result is added into a residual connection (the residual connection is a connection mode between the neural networks), the purpose is mainly to solve the degradation problem and gradient problem of the model with the increase of the network depth, the formula H (x) =x+f (x), x is understood as the input of the model, F (x) is the output of the model, the residual connection is H (x)) is then transmitted to the 2 nd sublayer after being made into a layer of laye_rnormal (a method for normalizing the data in the neural network), the sublayer is made into a forward network, and the output of the network is added into the residual connection, and the laye_m is the output of the network.
The GPT network adopts a block layer number L of 6, a value of hidde_size (vector dimension size of network hidden state) of 256, a number of multi-head attention of 4, and feedback/size=4×h=1024. Each token of text information input sent into the network is mapped into a multidimensional word vector after word-embedding and position information coding, the word vector is sent into the GPT network for information extraction, a layer of full-connection network is added after the GPT network, and a softmax function is added after the full-connection layer for sample classification.
The deep neural network classification model firstly extracts text characteristics of input text information (natural language text information), then carries out calculation of a multi-layer network, classifies texts related to unmanned aerial vehicle attitude control and gas detection functions contained in the text information into positive samples according to the input semantic information, and classifies the text into negative samples according to the related information, wherein the method comprises the following specific steps of:
and the deep neural network classification model extracts text characteristics of the input natural language text information, and calculates probability values P of positive samples and negative samples through a multi-layer network.
When the probability value P of the negative sample is greater than a preset threshold value (for example, the preset threshold value is 0.5), the text information is classified into the negative sample, the text is not sent to a subsequent module for processing, and a prompt box is popped up to prompt that the message is illegal.
When the probability value P of the positive sample is greater than a preset threshold value, the text information is classified into the positive sample, and the conversion processing stage is performed, wherein the conversion processing stage is used for converting the natural language text information classified into the positive sample into an unmanned aerial vehicle operation instruction, so that the unmanned aerial vehicle can conveniently execute corresponding operation according to the operation instruction.
And step S20, converting the natural language text information classified as the positive sample into an unmanned aerial vehicle operation instruction.
Fig. 4 is a flowchart of step S20 in the unmanned aerial vehicle control method provided by the present application.
As shown in fig. 4, the step S20 includes:
s21, natural language text information classified into positive samples is input into a trained deep learning model, wherein the deep learning model comprises an encoder and a decoder;
s22, word segmentation is carried out on the natural language text information of the positive sample, the encoder extracts the characteristic of each word, extracts at least one of the position information, the syntactic characteristic and the semantic characteristic of the text input through multi-layer network calculation, and outputs an intermediate semantic information vector;
s23, the decoder decodes the semantic information vector and outputs an unmanned aerial vehicle operation instruction corresponding to the natural language text information.
Specifically, natural language text information classified as positive samples is subjected to conversion treatment and finally translated into unmanned aerial vehicle operation instructions; and sending the natural language text information into a trained deep learning model, and converting the natural language text information into unmanned aerial vehicle operation instructions.
For example, natural text information: the unmanned aerial vehicle takes off, and the unmanned aerial vehicle takes off operation is converted into a machine instruction capable of executing unmanned aerial vehicle take-off operation.
As shown in fig. 5, the deep learning model mainly uses a multi-layer multi-head attention transducer model, the model is an end-to-end translation model, text information sent by classification is directly sent to an Encoder of the model for semantic coding, and the output of the Encoder is a machine operation instruction for operating a flight and function switch of the unmanned aerial vehicle; that is to say the model comprises two parts, enconner and Encoder.
For the Encoder, there are several Encoder blocks, the number L of blocks used by the deep learning model in the present application is 4, the value of hidde_size is 256, the number of multi-headed attention a=4, and feedback/size=4×h=1024. The transducer is used as a powerful feature extractor, and can accurately acquire the input intrinsic features and send the encoded information to the Decoder.
For a Decoder, a plurality of Decoder blocks are formed quickly, and an Encoder-Decoder-attribute sublayer is added in the middle of the Decoder blocks compared with the Encoder blocks; the number L of blocks is 4, the number of hidden_size is 256, the number A of multi-head attention is 4, a full connection layer is added to the outermost layer of the blocks, and the final output structure is the operation instruction of the machine.
The method comprises the steps of firstly carrying out word segmentation on input text information, adding a word2Vector (a word segmentation tool for efficiently realizing word segmentation, which is a word conversion Vector tool) Vector and a word segmentation position coding Vector to generate a new Vector, sending the new Vector into a network, firstly extracting the characteristic of each word by an encoding machine of a transform model according to a self-attribute mechanism, and fully extracting the position information, the syntax characteristic and the semantic characteristic of the text input by calculation of a multi-layer network, and then outputting an intermediate semantic information Vector Z.
The semantic information vector Z is decoded through a Decoder, the output of the Decoder is a corresponding machine instruction, for example, the input of an unmanned aerial vehicle takes off, the machine instruction which can operate the unmanned aerial vehicle to take off is generated through a model, for example, the function of starting sulfur dioxide content detection can be realized through the machine instruction output by the model, and the unmanned aerial vehicle can be opened for sulfur dioxide content detection.
And step S30, sending the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle so as to control the unmanned aerial vehicle to execute corresponding operation according to the unmanned aerial vehicle operation instruction.
Specifically, the unmanned aerial vehicle operation instruction is sent to the unmanned aerial vehicle through a wireless network; and controlling the unmanned aerial vehicle to execute the flying action according to the unmanned aerial vehicle operation instruction and then performing detection operation after the flying action reaches the designated area.
Specifically, the unmanned aerial vehicle operation instruction is mainly sent to the unmanned aerial vehicle from the mobile terminal through a network (such as a 4G network), and the unmanned aerial vehicle receives the operation instruction sent by the mobile terminal and then completes the flight action control and gas detection functions (the unmanned aerial vehicle in the application can execute various conventional operations according to the operation instruction, and besides, the unmanned aerial vehicle mainly completes the gas detection functions). For example, a GRPS module is pre-installed on the gas detection unmanned aerial vehicle, and the real-time transmission of the mobile terminal command is received through the mobile 4G network.
The machine instruction converted at the mobile terminal is sent to the unmanned aerial vehicle through the network, the unmanned aerial vehicle end receives the machine instruction sent by the mobile terminal, the control of the flight gestures such as take-off, landing, forward, backward, acceleration and stopping is completed, after the unmanned aerial vehicle is driven to reach a specified detection area (such as high-altitude, high-risk areas around a smoke emission tower and the like), the mobile terminal sends an unmanned aerial vehicle operation instruction, and the unmanned aerial vehicle executes the related instruction to fly to the related area to detect harmful gas, so that operations such as sulfur dioxide detection, sulfur dioxide detection stopping, PM2.5 concentration detection stopping and PM2.5 concentration detection are realized.
Further, the unmanned aerial vehicle collects detected data information, the detected data information is sent to a data control center through the unmanned aerial vehicle, real-time video information of a detection area returned through a camera carried by the unmanned aerial vehicle is finely adjusted through real-time attitude control of the unmanned aerial vehicle according to detection conditions, and the unmanned aerial vehicle reaches a specific area (a designated area) more accurately to detect required data.
The unmanned aerial vehicle based on the artificial intelligence gas detection can reach the area where human is inconvenient to reach or the danger coefficient is high to detect the gas in real time, so that convenience in the aspect of air detection is greatly improved, and a convenient and reliable method is added for collecting the later environmental gas detection data.
Further, as shown in fig. 6, based on the above unmanned aerial vehicle control method, the present application further provides a mobile terminal correspondingly, where the mobile terminal includes: the mobile terminal includes a processor 10, a memory 20 and a display 30. Fig. 6 shows only some of the components of the mobile terminal, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may alternatively be implemented.
The memory 20 may in some embodiments be an internal storage unit of the mobile terminal, such as a hard disk or a memory of the mobile terminal. The memory 20 may in other embodiments also be an external storage device of the mobile terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the mobile terminal. Further, the memory 20 may also include both an internal storage unit and an external storage device of the mobile terminal. The memory 20 is used for storing application software installed in the mobile terminal and various data, such as program codes for installing the mobile terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores a drone control program 40, and the drone control program 40 is executable by the processor 10 to implement the drone control method of the present application.
The processor 10 may in some embodiments be a central processing unit (Central Processing Unit, CPU), microprocessor or other data processing chip for executing program code or processing data stored in the memory 20, for example for performing the drone control method or the like.
The display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like in some embodiments. The display 30 is used for displaying information on the mobile terminal and for displaying a visual user interface. The components 10-30 of the mobile terminal communicate with each other via a system bus.
In one embodiment, the following steps are implemented when the processor 10 executes the drone control program 40 in the memory 20:
after receiving voice information input by a user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information;
converting natural language text information classified as positive samples into unmanned aerial vehicle operation instructions;
and sending the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle so as to control the unmanned aerial vehicle to execute corresponding operation according to the unmanned aerial vehicle operation instruction.
After receiving the voice information input by the user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information comprises the following steps:
positive samples and negative samples of natural language text information are defined in advance;
receiving voice information input by a user, and converting the voice information into natural language text information through a voice-to-text function of the mobile terminal;
and the natural language text information is subjected to a trained deep neural network classification model to complete positive and negative sample classification.
The pre-defining positive and negative examples of the natural language text information includes:
the positive sample is defined as information related to the implementation of unmanned aerial vehicle flight attitude control or gas detection functions;
the negative samples are defined as related information unrelated to unmanned aerial vehicle attitude control or gas monitoring function implementation.
The step of completing positive and negative sample classification of the natural language text information through a trained deep neural network classification model comprises the following steps:
the deep neural network classification model extracts text characteristics of the input natural language text information, and calculates probability values P of positive samples and negative samples through a multi-layer network;
when the probability value P of the negative sample is greater than a preset threshold value, classifying natural language text information into the negative sample, and popping up a prompt box to prompt that the message of the negative sample is illegal;
when the probability value P of the positive sample is greater than a preset threshold value, the natural language text information is classified into the positive sample, and the conversion processing stage is entered.
The converting the natural language text information classified as positive samples into unmanned aerial vehicle operation instructions comprises:
inputting natural language text information classified as positive samples into a trained deep learning model;
carrying out word segmentation processing on the natural language text information of the positive sample, extracting the characteristics of each word by an encoder, extracting at least one of the position information, the syntactic characteristics and the semantic characteristics of text input through calculation of a multi-layer network, and outputting an intermediate semantic information vector;
and the decoder decodes the semantic information vector and outputs an unmanned aerial vehicle operation instruction corresponding to the natural language text information.
The deep learning model comprises an encoder and a decoder, wherein the classified natural language text information is directly input into the encoder of the deep learning model for semantic coding, and the decoder of the deep learning model outputs corresponding unmanned aerial vehicle operation instructions.
The output intermediate semantic information vector specifically comprises:
performing word segmentation on the natural language text information of the positive sample, adding the word2Vector processed by word segmentation to the word position coding Vector to generate a new Vector, and sending the new Vector into a network;
and the encoder of the deep learning model extracts the characteristics of each word, extracts at least one of the position information, the syntactic characteristics and the semantic characteristics of the text input through the calculation of a multi-layer network, and outputs an intermediate semantic information vector.
The application also provides an unmanned aerial vehicle control system, which comprises the mobile terminal and an unmanned aerial vehicle (aircraft) which is in communication connection with the mobile terminal, wherein the unmanned aerial vehicle receives the instruction transmitted by the mobile terminal in real time through a wireless network, and executes corresponding operation according to the unmanned aerial vehicle operation instruction.
Further, as shown in fig. 7, the implementation process of the unmanned aerial vehicle control system of the present application is as follows:
step S101, starting;
step S102, the mobile terminal receives voice information input by a user;
step S103, converting the voice information into natural language text information through a voice-to-text function of the mobile terminal;
step S104, inputting natural language text information into an information classification module of the mobile terminal;
step S105, the natural language text information is subjected to a trained deep neural network classification model through an information classification module to complete positive and negative sample classification;
executing step S106 when the natural language text information is judged to be a positive sample, and returning to executing step S102 when the natural language text information is judged to be a negative sample;
wherein the positive sample is defined as information related to unmanned aerial vehicle flight attitude control or gas detection function implementation; the negative sample is defined as related information irrelevant to the implementation of the unmanned aerial vehicle attitude control or gas monitoring function;
step S106, inputting the natural language text information classified as the positive sample into an instruction generation module of the mobile terminal;
step S107, converting natural language text information classified as a positive sample into an unmanned aerial vehicle operation instruction through an instruction generation module;
step S108, the mobile terminal sends an unmanned aerial vehicle operation instruction to an unmanned aerial vehicle for gas detection through a 4G network;
step S109, the unmanned aerial vehicle executes flight actions (attitude flight control) according to the unmanned aerial vehicle operation instruction;
the flying actions comprise ascending, descending, starting and stopping, advancing, retreating, leftwards and rightwards, hovering, pitching, accelerating, decelerating and other flying postures;
step S110, the unmanned aerial vehicle starts gas detection after reaching a designated area according to a flight instruction, so that intelligent control and function realization of the gas monitoring unmanned aerial vehicle are realized;
wherein, the gas detection comprises the detection of sulfur dioxide concentration, PM2.5 concentration, dust content and the like;
in addition, after the unmanned aerial vehicle detects a certain detection project, the data of the gas detection result can be sent to the data control center for the data control center to collect and process.
The present application also provides a storage medium storing a drone control program which, when executed by a processor, implements the steps of the drone control method described above.
In summary, the present application provides a method, a system, a mobile terminal and a storage medium for controlling an unmanned aerial vehicle, where the method includes: after receiving voice information input by a user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information; converting natural language text information classified as positive samples into unmanned aerial vehicle operation instructions; and sending the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle so as to control the unmanned aerial vehicle to execute corresponding operation according to the unmanned aerial vehicle operation instruction. According to the application, the mobile terminal receives the voice input of the user, the voice instruction of the user is converted into the instruction executed by the unmanned aerial vehicle, and then the unmanned aerial vehicle is controlled to fly to the appointed area, so that the unmanned aerial vehicle can reach the area where the human is inconvenient to reach or the danger coefficient is high to detect the gas in real time, the convenience in air detection is greatly improved, and meanwhile, the unmanned aerial vehicle is controlled without the need of manually operating the mobile terminal by the user, so that the unmanned aerial vehicle is accurately, timely, flexible and convenient to operate.
Of course, those skilled in the art will appreciate that implementing all or part of the above-described methods may be implemented by a computer program for instructing relevant hardware (such as a processor, a controller, etc.), where the program may be stored in a computer-readable storage medium, and where the program may include the steps of the above-described method embodiments when executed. The storage medium may be a memory, a magnetic disk, an optical disk, or the like.
It is to be understood that the application is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (8)
1. The unmanned aerial vehicle control method is characterized by comprising the following steps of:
after receiving voice information input by a user, converting the voice information into natural language text information, and classifying positive and negative samples of the natural language text information; positive and negative samples of the positive and negative samples are predefined; the positive and negative samples are information related to the implementation of unmanned aerial vehicle flight attitude control or gas detection functions; the negative sample is related information irrelevant to the implementation of the unmanned aerial vehicle attitude control or gas monitoring function;
converting natural language text information classified as positive samples into unmanned aerial vehicle operation instructions;
the converting the natural language text information classified as positive samples into unmanned aerial vehicle operation instructions comprises:
inputting natural language text information classified as positive samples into a trained deep learning model;
carrying out word segmentation processing on the natural language text information of the positive sample, extracting the characteristics of each word by an encoder, extracting at least one of the position information, the syntactic characteristics and the semantic characteristics of text input through calculation of a multi-layer network, and outputting an intermediate semantic information vector;
the decoder decodes the semantic information vector and outputs an unmanned aerial vehicle operation instruction corresponding to the natural language text information;
the unmanned aerial vehicle collects detected data information, the detected data information is sent to a data control center through the unmanned aerial vehicle, real-time video information of a detection area returned by a camera carried by the unmanned aerial vehicle is subjected to real-time attitude control fine adjustment through the mobile terminal according to detection conditions, and data required by detection of a specified area are more accurately reached;
and sending the unmanned aerial vehicle operation instruction to the unmanned aerial vehicle so as to control the unmanned aerial vehicle to execute corresponding operation according to the unmanned aerial vehicle operation instruction.
2. The unmanned aerial vehicle control method of claim 1, wherein after receiving the voice information input by the user, converting the voice information into natural language text information, and classifying the natural language text information into positive and negative samples comprises:
receiving voice information input by a user, and converting the voice information into natural language text information through a voice-to-text function of the mobile terminal;
and the natural language text information is subjected to a trained deep neural network classification model to complete positive and negative sample classification.
3. The unmanned aerial vehicle control method of claim 2, wherein the performing the positive and negative sample classification on the natural language text information via the trained deep neural network classification model comprises:
the deep neural network classification model extracts text characteristics of the input natural language text information, and calculates probability values P of positive samples and negative samples through a multi-layer network;
when the probability value P of the negative sample is greater than a preset threshold value, classifying natural language text information into the negative sample, and popping up a prompt box to prompt that the message of the negative sample is illegal;
when the probability value P of the positive sample is greater than a preset threshold value, the natural language text information is classified into the positive sample, and the conversion processing stage is entered.
4. The unmanned aerial vehicle control method of claim 1, wherein the deep learning model comprises an encoder and a decoder; the classified natural language text information is directly input into an encoder of the deep learning model for semantic coding, and a decoder of the deep learning model outputs a corresponding unmanned aerial vehicle operation instruction.
5. The unmanned aerial vehicle control method of claim 4, wherein the outputting the intermediate semantic information vector specifically comprises:
performing word segmentation on the natural language text information of the positive sample, adding the word2Vector processed by word segmentation to the word position coding Vector to generate a new Vector, and sending the new Vector into a network;
and the encoder of the deep learning model extracts the characteristics of each word, extracts at least one of the position information, the syntactic characteristics and the semantic characteristics of the text input through the calculation of a multi-layer network, and outputs an intermediate semantic information vector.
6. A mobile terminal, the mobile terminal comprising: memory, a processor and a drone control program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the drone control method of any one of claims 1-5.
7. An unmanned aerial vehicle control system, comprising the mobile terminal according to claim 6, and an unmanned aerial vehicle in communication connection with the mobile terminal, wherein the unmanned aerial vehicle receives an instruction transmitted in real time by the mobile terminal through a wireless network, and performs a corresponding operation according to the unmanned aerial vehicle operation instruction.
8. A storage medium storing a drone control program which when executed by a processor performs the steps of the drone control method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910589408.XA CN112257434B (en) | 2019-07-02 | 2019-07-02 | Unmanned aerial vehicle control method, unmanned aerial vehicle control system, mobile terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910589408.XA CN112257434B (en) | 2019-07-02 | 2019-07-02 | Unmanned aerial vehicle control method, unmanned aerial vehicle control system, mobile terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112257434A CN112257434A (en) | 2021-01-22 |
| CN112257434B true CN112257434B (en) | 2023-09-08 |
Family
ID=74223809
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910589408.XA Active CN112257434B (en) | 2019-07-02 | 2019-07-02 | Unmanned aerial vehicle control method, unmanned aerial vehicle control system, mobile terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112257434B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114815904B (en) * | 2022-06-29 | 2022-09-27 | 中国科学院自动化研究所 | Attention network-based unmanned cluster countermeasure method and device and unmanned equipment |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104914859A (en) * | 2014-10-13 | 2015-09-16 | 江苏华音信息科技有限公司 | Full-automatic foreign language voice field control driving automobile system |
| CN106647803A (en) * | 2016-11-29 | 2017-05-10 | 湖北大秀天域科技发展有限公司 | UAV based smart home system |
| CN106682091A (en) * | 2016-11-29 | 2017-05-17 | 深圳市元征科技股份有限公司 | Method and device for controlling unmanned aerial vehicle |
| CN106992001A (en) * | 2017-03-29 | 2017-07-28 | 百度在线网络技术（北京）有限公司 | Processing method, the device and system of phonetic order |
| CN107588804A (en) * | 2017-09-16 | 2018-01-16 | 北京神鹫智能科技有限公司 | A kind of monitoring system for gases based on unmanned plane |
| CN107773982A (en) * | 2017-10-20 | 2018-03-09 | 科大讯飞股份有限公司 | Game voice interactive method and device |
| CN108375986A (en) * | 2018-03-30 | 2018-08-07 | 深圳市道通智能航空技术有限公司 | Control method, device and the terminal of unmanned plane |
| CN108447477A (en) * | 2018-01-30 | 2018-08-24 | 华南理工大学 | A kind of robot control method based on natural language understanding |
| KR101896973B1 (en) * | 2018-01-26 | 2018-09-10 | 가천대학교 산학협력단 | Natural Laguage Generating System Using Machine Learning Moodel, Method and Computer-readable Medium Thereof |
| CN108986801A (en) * | 2017-06-02 | 2018-12-11 | 腾讯科技（深圳）有限公司 | A kind of man-machine interaction method, device and human-computer interaction terminal |
| CN109271623A (en) * | 2018-08-16 | 2019-01-25 | 龙马智芯（珠海横琴）科技有限公司 | Text emotion denoising method and system |
| CN109360568A (en) * | 2018-12-20 | 2019-02-19 | 西安Tcl软件开发有限公司 | Unmanned plane sound control method, system and computer readable storage medium |
| CN109933652A (en) * | 2019-01-17 | 2019-06-25 | 深圳壹账通智能科技有限公司 | Intelligent answer method, apparatus, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102608469B1 (en) * | 2017-12-22 | 2023-12-01 | 삼성전자주식회사 | Method and apparatus for generating natural language |
-
2019
- 2019-07-02 CN CN201910589408.XA patent/CN112257434B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104914859A (en) * | 2014-10-13 | 2015-09-16 | 江苏华音信息科技有限公司 | Full-automatic foreign language voice field control driving automobile system |
| CN106647803A (en) * | 2016-11-29 | 2017-05-10 | 湖北大秀天域科技发展有限公司 | UAV based smart home system |
| CN106682091A (en) * | 2016-11-29 | 2017-05-17 | 深圳市元征科技股份有限公司 | Method and device for controlling unmanned aerial vehicle |
| CN106992001A (en) * | 2017-03-29 | 2017-07-28 | 百度在线网络技术（北京）有限公司 | Processing method, the device and system of phonetic order |
| CN108986801A (en) * | 2017-06-02 | 2018-12-11 | 腾讯科技（深圳）有限公司 | A kind of man-machine interaction method, device and human-computer interaction terminal |
| CN107588804A (en) * | 2017-09-16 | 2018-01-16 | 北京神鹫智能科技有限公司 | A kind of monitoring system for gases based on unmanned plane |
| CN107773982A (en) * | 2017-10-20 | 2018-03-09 | 科大讯飞股份有限公司 | Game voice interactive method and device |
| KR101896973B1 (en) * | 2018-01-26 | 2018-09-10 | 가천대학교 산학협력단 | Natural Laguage Generating System Using Machine Learning Moodel, Method and Computer-readable Medium Thereof |
| CN108447477A (en) * | 2018-01-30 | 2018-08-24 | 华南理工大学 | A kind of robot control method based on natural language understanding |
| CN108375986A (en) * | 2018-03-30 | 2018-08-07 | 深圳市道通智能航空技术有限公司 | Control method, device and the terminal of unmanned plane |
| CN109271623A (en) * | 2018-08-16 | 2019-01-25 | 龙马智芯（珠海横琴）科技有限公司 | Text emotion denoising method and system |
| CN109360568A (en) * | 2018-12-20 | 2019-02-19 | 西安Tcl软件开发有限公司 | Unmanned plane sound control method, system and computer readable storage medium |
| CN109933652A (en) * | 2019-01-17 | 2019-06-25 | 深圳壹账通智能科技有限公司 | Intelligent answer method, apparatus, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112257434A (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108304846B (en) | Image recognition method, device and storage medium | |
| CN110705267B (en) | Semantic parsing method, semantic parsing device and storage medium | |
| CN107134279A (en) | A kind of voice awakening method, device, terminal and storage medium | |
| US11393465B2 (en) | Artificial intelligence apparatus for speech interaction and method for the same | |
| US11605379B2 (en) | Artificial intelligence server | |
| CN112599124A (en) | Voice scheduling method and system for power grid scheduling | |
| CN111241245B (en) | Human-computer interaction processing method and device and electronic equipment | |
| CN112923523B (en) | Intelligent fresh air system regulation and control method based on data link of Internet of things | |
| CN114127847A (en) | Synchronized sound generation from video | |
| US11568853B2 (en) | Voice recognition method using artificial intelligence and apparatus thereof | |
| CN105788596A (en) | Speech recognition television control method and system | |
| CN112257434B (en) | Unmanned aerial vehicle control method, unmanned aerial vehicle control system, mobile terminal and storage medium | |
| US11810575B2 (en) | Artificial intelligence robot for providing voice recognition function and method of operating the same | |
| US20200202844A1 (en) | Mobile terminal and method of operating the same | |
| KR20210033809A (en) | Control server and method for controlling robot using artificial neural network, and the robot implementing the same | |
| CN104898831A (en) | Human action collection and action identification system and control method therefor | |
| CN114420102B (en) | Method and device for speech sentence-breaking, electronic equipment and storage medium | |
| CN111210824B (en) | Voice information processing method and device, electronic equipment and storage medium | |
| CN108320740A (en) | A kind of audio recognition method, device, electronic equipment and storage medium | |
| US11366984B1 (en) | Verifying a target object based on confidence coefficients generated by trained models | |
| CN113469023A (en) | Method, device, equipment and storage medium for determining alertness | |
| CN111292731A (en) | Voice information processing method and device, electronic equipment and storage medium | |
| CN112232089A (en) | Pre-training method, device and storage medium of semantic representation model | |
| Majewski et al. | Building innovative speech interfaces using patterns and antipatterns of commands for controlling loader cranes | |
| CN115994211A (en) | Text processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 516006 TCL science and technology building, No. 17, Huifeng Third Road, Zhongkai high tech Zone, Huizhou City, Guangdong Province Applicant after: TCL Technology Group Co.,Ltd. Address before: 516006 Guangdong province Huizhou Zhongkai hi tech Development Zone No. nineteen District Applicant before: TCL Corp. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |