CN109376660A - A kind of target monitoring method, apparatus and system - Google Patents
A kind of target monitoring method, apparatus and system Download PDFInfo
- Publication number
- CN109376660A CN109376660A CN201811261900.6A CN201811261900A CN109376660A CN 109376660 A CN109376660 A CN 109376660A CN 201811261900 A CN201811261900 A CN 201811261900A CN 109376660 A CN109376660 A CN 109376660A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- sent
- ground
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
Abstract
The invention discloses a kind of target monitoring methods, apparatus and system, are related to flying vehicles control technical field, and the target monitoring method applied to ground-based server end includes: the image data for receiving unmanned aerial vehicle onboard end and sending in real time；Classified to image data, filtration treatment, filters out the classification image data for needing to carry out complex data analysis beyond the clouds；The image data that will classify is sent to cloud；Receive the target data that cloud is sent；Target data is identified, target identification data are obtained；Target identification data are sent to client；The director data that client is sent is received, and director data is forwarded to unmanned aerial vehicle onboard end, for making unmanned aerial vehicle onboard end realize object real-time tracking shooting or target focus operation according to described instruction data control load.The present invention has the advantages that real-time is high, the real-time control to target except hundreds of kilometer even thousands of kilometers can be completed from client end interface, and crisis treatment effeciency greatly improved.
Description
Technical field
The present invention relates to flying vehicles control technical fields, and in particular to a kind of target monitoring method, apparatus and system.
Background technique
With China's economy, the lasting propulsion of the rapid development of society and airspace management reform, low altitude airspace is gradually opened
It puts, unmanned aerial vehicle onboard end will obtain development energetically and more be widely applied, for example, unmanned aerial vehicle onboard end can be applied
To fields such as electric power, communication, meteorology, agricultural, ocean, exploration, insurances, it specifically such as can be used for earth observation, forest fire protection
With fire extinguishing, disaster detection, communication relay, maritime surveillance, oil-gas pipeline inspection, pesticide spraying, land resources survey, wild animal
Monitoring, flood-control and drought relief monitoring, the locating fish, video display are taken photo by plane, drug law enforcement is seized smugglers or smuggled goods, border patrol, public security anti-terrorism etc..Based on nobody
The target monitoring system at the airborne end of machine will also obtain development energetically and more be widely applied.
Target monitoring system is mostly by the way that the HD video of camera or cameras record or photo to be transmitted in real time
Monitoring center carries out the long-range monitoring and management of centralization.However, video and photo transmission can continue to generate on a large scale in real time
Data flow, there is statistical data to show, monitoring 8 channel highways vehicle flowrate situation need the data of up to 40Gb/s to transmit
Speed.The mass data that unmanned aerial vehicle onboard end and wireless service generate all the time brings arduous choose to target monitoring system
War, if will the data that generate of so more equipment, sensor be completely transferred to server end and handled, network communication band can be given
Carry out huge burden, while the computing capability of server end is similarly difficult to meet the needs of mass data sustainable growth.And it takes
Device end be engaged in farther out from user distance, data transmission is quite limited by factors such as bandwidth.Thus, it can usually cause so past over long distances
The problems such as returning delay, network congestion, service quality decline.In addition, the unmanned plane real-time target monitoring side provided in the prior art
Method is essentially all the process of a later period observation, artificial interpretation processing.
Defect existing for above-mentioned two aspect in the prior art, results in the real-time of current unmanned plane target monitoring system
Difference, user can not be low to target monitoring progress real-time control, crisis treatment effeciency.
Summary of the invention
Therefore, technical problems to be solved of the embodiment of the present invention are the real-time of target monitoring system in the prior art
Difference, user can not be low to target monitoring progress real-time control, crisis treatment effeciency.
For this purpose, a kind of target monitoring method of the embodiment of the present invention, is applied to ground-based server end, comprising the following steps:
Receive the image data that unmanned aerial vehicle onboard end is sent in real time；
Classified to the image data, filtration treatment, filters out the classification for needing to carry out complex data analysis beyond the clouds
Image data；
The image data of having classified is sent to cloud, for carrying out complex data to the image data of having classified beyond the clouds
Analysis obtains target data；
Receive the target data that cloud is sent；
The target data is identified, target identification data are obtained；
The target identification data are sent to client, include mesh for being based on the target identification data acquisition in client
The director data that mark tracking or target identify；
The described instruction data that client is sent are received, and by described instruction data forwarding to unmanned aerial vehicle onboard end, for making nothing
Man-machine airborne end object real-time tracking shooting or target focus operation are realized according to described instruction data control load.
Preferably, described that the target data is identified, obtain target identification data the step of include:
Training set data and test set data are constructed, target data, training deep learning network are obtained；
The target data is predicted, target data prediction result is obtained；
The target data prediction result is sent to client, is obtained for being based on the target data prediction result in client
Take target object judging result and target identification data；
Receive the target identification data that client is sent；
The target identification data are added to training set data, re -training deep learning network obtains updated depth
Learning network.
A kind of target monitoring method of the embodiment of the present invention is applied to cloud, comprising the following steps:
Receive the classification image data that ground-based server end is sent；
The picture frame of adjacent n frame is obtained based on the image data of having classified；
Described image frame is registrated, registration image is obtained；
Difference is carried out to the registration image, obtains difference diagram；
Process of refinement is carried out to the difference diagram, is extracted by static nature and obtains suspected target data；
The described the step of picture frame of adjacent n frame is obtained based on the image data of having classified is repeated to described to the difference diagram
Process of refinement is carried out, is extracted step K times of suspected target data by static nature, K suspected target data is obtained, to institute
It states K suspected target data and carries out target association processing, screened by behavioral characteristics and obtain target data, and to the track of target
It is filtered, obtains target trajectory data；
The target data and target trajectory data are sent to ground-based server end.
A kind of target monitoring method of the embodiment of the present invention is applied to unmanned aerial vehicle onboard end, comprising the following steps:
Image data is sent to ground-based server end in real time；
Receive the director data that ground-based server end is sent；
Judge that the content that described instruction data include is that target following or target identify；
When the content that described instruction data include is target following, generates the first state modulator value and export to load, be used for
Control load realizes object real-time tracking shooting operation；
When the content that described instruction data include is that target identifies, generates the second state modulator value and export to load, be used for
Control load realizes target focus operation.
A kind of object monitoring device for ground-based server end of the embodiment of the present invention, comprising:
Image data receiving unit, the image data sent in real time for receiving unmanned aerial vehicle onboard end；
Categorical filtering unit is filtered out and needs to carry out complexity beyond the clouds for being classified to the image data, filtration treatment
The classification image data of data analysis；
Image data transmission unit of having classified is used for for the image data of having classified to be sent to cloud beyond the clouds to institute
It states image data of having classified and carries out complex data analysis acquisition target data；
Target data receiving unit, for receiving the target data of cloud transmission；
Unit is identified, for being identified to the target data, obtains target identification data；
Target identification data transmission unit, for the target identification data to be sent to client, for being based in client
The target identification data acquisition includes target following or the director data that target identifies；
Director data Transmit-Receive Unit, for receiving the described instruction data of client transmission, and extremely by described instruction data forwarding
Unmanned aerial vehicle onboard end, for make unmanned aerial vehicle onboard end according to described instruction data control load realize object real-time tracking shooting or
Target focus operation.
Preferably, the mark unit includes:
Deep learning network training unit obtains target data, training depth for constructing training set data and test set data
Learning network；
Target data predicting unit obtains target data prediction result for predicting the target data；
Target data prediction result transmission unit, for the target data prediction result to be sent to client, in visitor
Family end group obtains target object judging result and target identification data in the target data prediction result；
Target identification data receipt unit, for receiving the target identification data of client transmission；
Deep learning network updating unit, for the target identification data to be added to training set data, re -training depth
Learning network obtains updated deep learning network.
A kind of object monitoring device for cloud of the embodiment of the present invention, comprising:
Classify image data receiving unit, for receiving the classification image data of ground-based server end transmission；
Adjacent image frame acquiring unit, for obtaining the picture frame of adjacent n frame based on the image data of having classified；
Registration unit obtains registration image for being registrated to described image frame；
Difference unit obtains difference diagram for carrying out difference to the registration image；
Suspected target data extracting unit is extracted by static nature and is obtained for carrying out process of refinement to the difference diagram
Suspected target data；
Target data screening unit, for repeating the picture frame for obtaining adjacent n frame based on the image data of having classified
Step carries out process of refinement to the difference diagram to described, is extracted step K times of suspected target data, is obtained by static nature
K suspected target data are obtained, target association processing is carried out to the K suspected target data, passes through behavioral characteristics and screens acquisition
Target data, and the track of target is filtered, obtain target trajectory data；
Target data transmission unit, for the target data and target trajectory data to be sent to ground-based server end.
A kind of object monitoring device for unmanned aerial vehicle onboard end of the embodiment of the present invention, comprising:
Image data transmission unit, for image data to be sent to ground-based server end in real time；
Director data receiving unit, for receiving the director data of ground-based server end transmission；
Judging unit, for judging that the content that described instruction data include is that target following or target identify；
First state modulator value generates output unit, when the content for including when described instruction data is target following, generates
First state modulator value is simultaneously exported to load, realizes object real-time tracking shooting operation for control load；
Second state modulator value generates output unit and generates when the content for including when described instruction data is that target identifies
Second state modulator value is simultaneously exported to load, realizes target focus operation for control load.
A kind of target monitoring system of the embodiment of the present invention, comprising: unmanned aerial vehicle onboard end, ground-based server end, cloud and
Client, ground-based server end are connect with unmanned aerial vehicle onboard end, cloud and client by UAV TT & C's network in real time respectively；
The ground-based server end includes the object monitoring device for ground-based server end, for executing above-mentioned target monitoring
Method；
The cloud includes the object monitoring device for cloud, for executing above-mentioned target monitoring method；
The unmanned aerial vehicle onboard end includes the object monitoring device for unmanned aerial vehicle onboard end, for executing above-mentioned target monitoring
Method；
The client carries out the target identification data for receiving the target identification data of ground-based server end transmission
Interface display obtains the instruction identified comprising target following or target that user shows the processing mode of selection according to target interface
Described instruction data are sent to ground-based server end by data.
Preferably, the client is also used to receive the target data prediction result of ground-based server end transmission, will be described
Target data prediction result carries out interface display, and the target object judgement and mark that acquisition user carries out according to interface display obtain
Target object judging result and target identification data, the target identification data are sent to ground-based server end.
The technical solution of the embodiment of the present invention, has the advantages that
Target monitoring method provided in an embodiment of the present invention, apparatus and system, unmanned aerial vehicle onboard end image data are passed by network
To ground-based server end, ground-based server termination receives preliminary classification, parsing and filtering of image data and complete paired data etc.
Process, then the data-pushing for needing AI system to handle to cloud is subjected to further data processing, realize the shunting of data
Processing avoids the network communication to cloud from causing undue burden and provides the calculating pressure of magnanimity to cloud, reduces network and gather around
Plug and calculation delay, effectively increase real-time.Ground received server-side AI system resolves the data completed and completion simultaneously
To the mark and superposition processing of target, finally by the data distribution handled well to client end interface, user passes through showing interface
Content gives the process instruction of next step, carries out real-time control to target monitoring to realize user, complete from client end interface
The real-time control of target, saves resource except pairs of hundreds of kilometer even thousands of kilometers, and crisis processing effect greatly improved
Rate.
Detailed description of the invention
In order to illustrate more clearly of the technical solution in the specific embodiment of the invention, specific embodiment will be retouched below
Attached drawing needed in stating is briefly described, it should be apparent that, the accompanying drawings in the following description is some realities of the invention
Mode is applied, it for those of ordinary skill in the art, without creative efforts, can also be attached according to these
Figure obtains other attached drawings.
Fig. 1 is the flow chart of a specific example of target monitoring method in the embodiment of the present invention 1；
Fig. 2 is the flow chart of a specific example of target identification in the embodiment of the present invention 1；
Fig. 3 is the flow chart of a specific example of target monitoring method in the embodiment of the present invention 2；
Fig. 4 is the flow chart of a specific example of target monitoring method in the embodiment of the present invention 3；
Fig. 5 is the functional block diagram of a specific example of object monitoring device in the embodiment of the present invention 4；
Fig. 6 is the functional block diagram of a specific example of target monitoring system in the embodiment of the present invention 7.
Specific embodiment
Technical solution of the present invention is clearly and completely described below in conjunction with attached drawing, it is clear that described implementation
Example is a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill
Personnel's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the term as used herein is only used for the mesh of description specific embodiment
, and it is not intended to limit the present invention.Unless clearly indicated by the context, otherwise singular " one " as used herein,
The intentions such as "one" and "the" also include plural form.When using terms such as " include " and or " include ", it is intended to illustrate exist
This feature, integer, step, operation, element and/or component, and it is not excluded for one or more other features, integer, step, behaviour
Work, element, component, and/or other presence or increase for combining.Term "and/or" includes that one or more correlations list project
Any and all combinations.Term " first ", " second " etc. are used for description purposes only, and are not understood to indicate or imply opposite
Importance.Term " connected ", " connection " shall be understood in a broad sense, for example, it may be connected directly, it can also be by between intermediary
It connects connected, can also be the connection inside two elements；It can be wireless connection, be also possible to wired connection.For this field
Those of ordinary skill for, the concrete meaning of above-mentioned term in the present invention can be understood with concrete condition.
Although exemplary embodiment is described as executing example process using multiple units, however it will be appreciated that
It is that the example process can also be executed by one or more modules.Further it will be understood that term controller/control
Unit refer to include memory and processor hardware device.Memory is configured to store module, and processor is specially matched
It is set to the process for executing and storing in above-mentioned memory module, thereby executing one or more processes.
As long as in addition, the non-structure each other of technical characteristic involved in invention described below different embodiments
It can be combined with each other at conflict.
Embodiment 1
The present embodiment provides a kind of target monitoring methods, are applied to ground-based server end, the system background of the target monitoring method
It mainly include unmanned aerial vehicle onboard end, ground-based server end, cloud, client, ground-based server end and unmanned aerial vehicle onboard end, cloud
It is connected in real time between end, client by UAV TT & C's network implementations.Biography and the instruction down of measurement and control network realization image data
The upload of data.The server at ground-based server end is the data processing terminal of local computer room deployment, is mainly completed better simply
Data handling procedure, and complicated machine-learning process and Objective extraction process requires high-performance server cluster beyond the clouds
Upper realization.The target monitoring method is as shown in Figure 1, comprising the following steps:
S11, the image data that unmanned aerial vehicle onboard end is sent in real time is received；Image data is enrolled by unmanned plane real-time flight, machine
It carries end mission computer and obtains the image data enrolled in real time and transmission；Image data is adopted from the acquisition of airborne end Loading End
The data of collection need to meet basic requirement, i.e. data need to be high-definition image, it is preferable that H.264 the data of acquisition are passed through
Standard agreement completes compression, completes vacant lot real-time Transmission by measurement and control network；Preferably, image data information may include one
Timestamp corresponds to the time that every frame image data is passed to measurement and control network, correspondingly can establish the time of an addition network
Order, ground-based server end carry out data processing and distribution to image data according to temporal information；
S12, classified to image data, filtration treatment, filter out the classification for needing to carry out complex data analysis beyond the clouds
Image data；Preferably, the image data transmitted in above-mentioned steps S11 is by H.264 compressed data content, at this
Firstly the need of the processing unziped it to data content after the arrival ground-based server end of step；
S13, the image data that will classify are sent to cloud, for carrying out complex data analysis to image data of having classified beyond the clouds
Obtain target data；Cloud AI system is the key that realize target detection to the analysis of data, if be can be realized to object
Accurately identify with the later period identify, the process that a large amount of training machines of related objective object characteristic point are learnt dependent on AI system；Cloud
It holds AI system to carry out crucial point identification and characteristic matching to data, the intelligent interpretation to object content is realized, finally by target pair
As being back to local server；AI system data to be treated refer mainly to image information, pass through to great amount of images data early period
Training study, AI system has been can to execute the mature system of task in real time.
S14, the target data that cloud is sent is received；
S15, target data is identified, obtains target identification data；After target object is identified in cloud, to object
What mark, the superposition of related data information and the work of data distribution were mainly realized on ground-based server end.Ground clothes
Mark and superposition processing to target are completed in business device end, finally by the data distribution handled well to client end interface, it is preferable that place
The final data managed, which can be, to be automatically updated and is distributed in each client node according to network configuration；
S16, target identification data are sent to client, for client be based on target identification data acquisition include target with
The director data that track or target identify；Client gives the process instruction of next step by the content of showing interface；Client is shown
The object identified in video after user sees real time data from observation interface, carries out artificial secondary interpretation to target object,
The Treatment Options to object can be selected at client operation interface, complete to operate the remote-controllable of remote object object.
S17, the director data that client is sent is received, and director data is forwarded to unmanned aerial vehicle onboard end, for making nothing
Man-machine airborne end object real-time tracking shooting or target focus operation are realized according to director data control load.Preferably, airborne
The mission computer at end completes parsing and forwarding work to surface instruction, if command content is directed to target object after parsing
, then transmit commands to target identification and tracking module, target identification and tracking module by the adjustment to load parameter, from
And realize the real-time monitoring process to targets of interest.
Above-mentioned target monitoring method, unmanned aerial vehicle onboard end image data reach ground-based server end, ground clothes by network
Business device termination receives the processes such as preliminary classification, parsing and the filtering of image data and complete paired data, then will need at AI system
The data-pushing of reason to cloud carries out further data processing, realizes the shunting processing of data, avoids to the network in cloud
Communication causes undue burden and provides the calculating pressure of magnanimity to cloud, reduces network congestion and calculation delay, effectively improves
Real-time.Ground received server-side AI system resolves the data completed and completes to the mark of target and superposition simultaneously
Reason, finally by the data distribution handled well to client end interface, user gives the processing of next step by the content of showing interface
Instruction carries out real-time control to target monitoring to realize user, completes from client end interface even thousands of to hundreds of kilometer
The real-time control of target, saves resource, and crisis treatment effeciency greatly improved except kilometer.
Preferably, as shown in Fig. 2, above-mentioned steps S15's is identified target data, the step of target identification data is obtained
Suddenly include:
S15-1, building training set data；
S15-2, building test set data；
S15-3, target data, training deep learning network are obtained；
S15-4, target data is predicted, obtains target data prediction result；
S15-5, target data prediction result is sent to client, for obtaining in client based on target data prediction result
Target object judging result and target identification data；It by the way that prediction result is fed back to client, observe that user can intuitively
To Motion parameters as a result, judged by user and identified, human-computer interaction is improved, target mark is further improved
The confidence level of knowledge.
S15-6, the target identification data that client is sent are received, target identification data are added to by return step S15-1
Training set data, re -training deep learning network obtain updated deep learning network.
Ground-based server end may include geography information and attribute-bit to the real-time identification information of target object.For example,
Geographical indication can be the high identification information of longitude and latitude, be used to be limited in the status information in job area, can permit mark tool
Body geographic name is to improve identification.Attribute-bit is mainly the classification declaration to object, is determined with will pass through type attribute
Handle rank.
Embodiment 2
The present embodiment provides a kind of target monitoring methods, are applied to cloud, as shown in Figure 3, comprising the following steps:
S21, the classification image data that ground-based server end is sent is received；
S22, the picture frame that adjacent n frame is obtained based on image data of having classified；Data reach cloud AI system after, AI system according to
The sample frequency of setting obtains image information to decoding and samples, and obtains the image data that will be handled；
S23, picture frame is registrated, obtains registration image；Since unmanned plane is moving, the image data of different moments is shot
The different but adjacent image data background of background overlap, therefore need after obtaining image data to neighbor map
As carrying out image registration, the corresponding relationship between image background is found；
S24, difference is carried out to registration image, obtains difference diagram；After the corresponding relationship for determining image background, two images are carried out
Difference obtains difference diagram, and difference diagram can detecte the variation between two images, is the base of prospect (namely target) detection
Plinth；
S25, process of refinement is carried out to difference diagram, is extracted by static nature and obtains suspected target data；Image registration by
The influence of interference and noise, can not accomplish no error, therefore there is also error, simple difference diagrams for the corresponding relationship between image
The variation of prospect is not can determine that, it is also necessary to carry out process of refinement, remove by the methods of Morphological scale-space, gray feature dry
It disturbs, determines suspected target；Step K times of S22-S25 is repeated, K suspected target data are obtained；
S26, target association processing is carried out to K suspected target data, is screened by behavioral characteristics and obtain target data, and to mesh
Target track is filtered, and obtains target trajectory data；Multiple adjacent images are to available multiple suspected targets, some of them
Suspected target is the same object, therefore they can be associated, and relationship between them (such as the mobile speed of object is passed through
Degree, color change etc.) further determine whether it is required target, and the track of target is filtered, obtain compared with
For accurate actual path；
S27, target data and target trajectory data are sent to ground-based server end.
Embodiment 3
The present embodiment provides a kind of target monitoring methods, are applied to unmanned aerial vehicle onboard end, as shown in Figure 4, comprising the following steps:
S31, image data is sent to ground-based server end in real time；
S32, the director data that ground-based server end is sent is received；
The content that S33, decision instruction data include is that target following or target identify, i.e., to the state modulator content of load；
When the content that director data includes is target following, S34 is entered step；When the content that director data includes is that target identifies
When, enter step S35；
S34, it generates the first state modulator value and exports to load, realize object real-time tracking shooting operation for control load；
S35, it generates the second state modulator value and exports to load, realize target focus operation for control load.Pass through parsing
The state modulator content to load is obtained, to send corresponding state modulator value to load, load responds relevant parameter
The instruction to target is completed to handle.
Embodiment 4
Corresponding to embodiment 1, the present embodiment provides a kind of object monitoring devices for ground-based server end, as shown in figure 5, packet
It includes:
Image data receiving unit 11, the image data sent in real time for receiving unmanned aerial vehicle onboard end；
Categorical filtering unit 12 is filtered out and needs to carry out complicated number beyond the clouds for being classified to image data, filtration treatment
According to the classification image data of analysis；
Classify image data transmission unit 13, for will classify, image data is sent to cloud, for beyond the clouds to having divided
Class image data carries out complex data analysis and obtains target data；
Target data receiving unit 14, for receiving the target data of cloud transmission；
Unit 15 is identified, for being identified to target data, obtains target identification data；
Target identification data transmission unit 16, for target identification data to be sent to client, for being based on mesh in client
It marks mark data and obtains the director data identified comprising target following or target；
Director data for receiving the director data of client transmission, and is forwarded to unmanned plane by director data Transmit-Receive Unit 17
Airborne end, for making unmanned aerial vehicle onboard end realize object real-time tracking shooting or target focusing behaviour according to director data control load
Make.
Above-mentioned object monitoring device, unmanned aerial vehicle onboard end image data reach ground-based server end, ground clothes by network
Business device termination receives the processes such as preliminary classification, parsing and the filtering of image data and complete paired data, then will need at AI system
The data-pushing of reason to cloud carries out further data processing, realizes the shunting processing of data, avoids to the network in cloud
Communication causes undue burden and provides the calculating pressure of magnanimity to cloud, reduces network congestion and calculation delay, effectively improves
Real-time.Ground received server-side AI system resolves the data completed and completes to the mark of target and superposition simultaneously
Reason, finally by the data distribution handled well to client end interface, user gives the processing of next step by the content of showing interface
Instruction carries out real-time control to target monitoring to realize user, completes from client end interface even thousands of to hundreds of kilometer
The real-time control of target, saves resource, and crisis treatment effeciency greatly improved except kilometer.
Preferably, mark unit includes:
Deep learning network training unit, for constructing training set data and test set data, training deep learning network；
Target data predicting unit obtains target data prediction result for predicting target data；
Target data prediction result transmission unit, for target data prediction result to be sent to client, in client
Target object judging result and target identification data are obtained based on target data prediction result；
Target identification data receipt unit, for receiving the target identification data of client transmission；
Deep learning network updating unit, for target identification data to be added to training set data, re -training deep learning
Network obtains updated deep learning network.
Embodiment 5
Corresponding to embodiment 2, the present embodiment provides a kind of object monitoring devices for cloud, comprising:
Classify image data receiving unit, for receiving the classification image data of ground-based server end transmission；
Adjacent image frame acquiring unit, for obtaining the picture frame of adjacent n frame based on image data of having classified；
Registration unit obtains registration image for being registrated to picture frame；
Difference unit obtains difference diagram for carrying out difference to registration image；
It is doubtful to extract acquisition by static nature for carrying out process of refinement to difference diagram for suspected target data extracting unit
Target data；
Target data screening unit, for repeating the step of obtaining the picture frame of adjacent n frame based on image data of having classified to right
Difference diagram carries out process of refinement, is extracted step K times of suspected target data by static nature, obtains K suspected target number
According to K suspected target data progress target association processing, by behavioral characteristics screening acquisition target data, and to target
Track is filtered, and obtains target trajectory data；
Target data transmission unit, for target data and target trajectory data to be sent to ground-based server end.
Embodiment 6
Corresponding to embodiment 3, the present embodiment provides a kind of object monitoring devices for unmanned aerial vehicle onboard end, comprising:
Image data transmission unit, for image data to be sent to ground-based server end in real time；
Director data receiving unit, for receiving the director data of ground-based server end transmission；
Judging unit, the content for including for decision instruction data is that target following or target identify；
First state modulator value generates output unit, when the content for including when director data is target following, generates first
State modulator value is simultaneously exported to load, realizes object real-time tracking shooting operation for control load；
Second state modulator value generates output unit and generates second when the content for including when director data is that target identifies
State modulator value is simultaneously exported to load, realizes target focus operation for control load.
Embodiment 7
The present embodiment provides a kind of target monitoring systems, as shown in Figure 6, comprising: unmanned aerial vehicle onboard end 3, ground-based server end 1,
Cloud 2 and client 4, ground-based server end pass through UAV TT & C's network with unmanned aerial vehicle onboard end, cloud and client respectively
Connection in real time；
Ground-based server end 1 includes the object monitoring device 10 for ground-based server end, and the target for executing embodiment 1 is supervised
Survey method；
Cloud 2 includes the object monitoring device 20 for cloud, for executing the target monitoring method of embodiment 2；
Unmanned aerial vehicle onboard end 3 includes the object monitoring device 30 for unmanned aerial vehicle onboard end, and the target for executing embodiment 3 is supervised
Survey method；
Client 4 shows target identification data progress interface for receiving the target identification data of ground-based server end transmission
Show, obtain the director data identified comprising target following or target that user shows the processing mode of selection according to target interface,
Director data is sent to ground-based server end.
Preferably, client 4 is also used to receive the target data prediction result of ground-based server end transmission, by target data
Prediction result carries out interface display, obtains the target object judgement that user carries out according to interface display and the target pair that mark obtains
As judging result and target identification data, target identification data are sent to ground-based server end.
Above-mentioned target monitoring system, unmanned aerial vehicle onboard end image data reach ground-based server end, ground clothes by network
Business device termination receives the processes such as preliminary classification, parsing and the filtering of image data and complete paired data, then will need at AI system
The data-pushing of reason to cloud carries out further data processing, realizes the shunting processing of data, avoids to the network in cloud
Communication causes undue burden and provides the calculating pressure of magnanimity to cloud, reduces network congestion and calculation delay, effectively improves
Real-time.Ground received server-side AI system resolves the data completed and completes to the mark of target and superposition simultaneously
Reason, finally by the data distribution handled well to client end interface, user gives the processing of next step by the content of showing interface
Instruction carries out real-time control to target monitoring to realize user, completes from client end interface even thousands of to hundreds of kilometer
The real-time control of target, saves resource, and crisis treatment effeciency greatly improved except kilometer.
Obviously, the above embodiments are merely examples for clarifying the description, and does not limit the embodiments.It is right
For those of ordinary skill in the art, can also make on the basis of the above description it is other it is various forms of variation or
It changes.There is no necessity and possibility to exhaust all the enbodiments.And it is extended from this it is obvious variation or
It changes still within the protection scope of the invention.
Claims (10)
1. a kind of target monitoring method is applied to ground-based server end, which comprises the following steps:
Receive the image data that unmanned aerial vehicle onboard end is sent in real time；
Classified to the image data, filtration treatment, filters out the classification for needing to carry out complex data analysis beyond the clouds
Image data；
The image data of having classified is sent to cloud, for carrying out complex data to the image data of having classified beyond the clouds
Analysis obtains target data；
Receive the target data that cloud is sent；
The target data is identified, target identification data are obtained；
The target identification data are sent to client, include mesh for being based on the target identification data acquisition in client
The director data that mark tracking or target identify；
The described instruction data that client is sent are received, and by described instruction data forwarding to unmanned aerial vehicle onboard end, for making nothing
Man-machine airborne end object real-time tracking shooting or target focus operation are realized according to described instruction data control load.
2. target monitoring method according to claim 1, which is characterized in that it is described that the target data is identified,
Obtain target identification data the step of include:
Training set data and test set data are constructed, target data, training deep learning network are obtained；
The target data is predicted, target data prediction result is obtained；
The target data prediction result is sent to client, is obtained for being based on the target data prediction result in client
Take target object judging result and target identification data；
Receive the target identification data that client is sent；
The target identification data are added to training set data, re -training deep learning network obtains updated depth
Learning network.
3. a kind of target monitoring method is applied to cloud, which comprises the following steps:
Receive the classification image data that ground-based server end is sent；
The picture frame of adjacent n frame is obtained based on the image data of having classified；
Described image frame is registrated, registration image is obtained；
Difference is carried out to the registration image, obtains difference diagram；
Process of refinement is carried out to the difference diagram, is extracted by static nature and obtains suspected target data；
The described the step of picture frame of adjacent n frame is obtained based on the image data of having classified is repeated to described to the difference diagram
Process of refinement is carried out, is extracted step K times of suspected target data by static nature, K suspected target data is obtained, to institute
It states K suspected target data and carries out target association processing, screened by behavioral characteristics and obtain target data, and to the track of target
It is filtered, obtains target trajectory data；
The target data and target trajectory data are sent to ground-based server end.
4. a kind of target monitoring method is applied to unmanned aerial vehicle onboard end, which comprises the following steps:
Image data is sent to ground-based server end in real time；
Receive the director data that ground-based server end is sent；
Judge that the content that described instruction data include is that target following or target identify；
When the content that described instruction data include is target following, generates the first state modulator value and export to load, be used for
Control load realizes object real-time tracking shooting operation；
When the content that described instruction data include is that target identifies, generates the second state modulator value and export to load, be used for
Control load realizes target focus operation.
5. a kind of object monitoring device for ground-based server end characterized by comprising
Image data receiving unit, the image data sent in real time for receiving unmanned aerial vehicle onboard end；
Categorical filtering unit is filtered out and needs to carry out complexity beyond the clouds for being classified to the image data, filtration treatment
The classification image data of data analysis；
Image data transmission unit of having classified is used for for the image data of having classified to be sent to cloud beyond the clouds to institute
It states image data of having classified and carries out complex data analysis acquisition target data；
Target data receiving unit, for receiving the target data of cloud transmission；
Unit is identified, for being identified to the target data, obtains target identification data；
Target identification data transmission unit, for the target identification data to be sent to client, for being based in client
The target identification data acquisition includes target following or the director data that target identifies；
Director data Transmit-Receive Unit, for receiving the described instruction data of client transmission, and extremely by described instruction data forwarding
Unmanned aerial vehicle onboard end, for make unmanned aerial vehicle onboard end according to described instruction data control load realize object real-time tracking shooting or
Target focus operation.
6. object monitoring device according to claim 5, which is characterized in that the mark unit includes:
Deep learning network training unit obtains target data, training depth for constructing training set data and test set data
Learning network；
Target data predicting unit obtains target data prediction result for predicting the target data；
Target data prediction result transmission unit, for the target data prediction result to be sent to client, in visitor
Family end group obtains target object judging result and target identification data in the target data prediction result；
Target identification data receipt unit, for receiving the target identification data of client transmission；
Deep learning network updating unit, for the target identification data to be added to training set data, re -training depth
Learning network obtains updated deep learning network.
7. a kind of object monitoring device for cloud characterized by comprising
Classify image data receiving unit, for receiving the classification image data of ground-based server end transmission；
Adjacent image frame acquiring unit, for obtaining the picture frame of adjacent n frame based on the image data of having classified；
Registration unit obtains registration image for being registrated to described image frame；
Difference unit obtains difference diagram for carrying out difference to the registration image；
Suspected target data extracting unit is extracted by static nature and is obtained for carrying out process of refinement to the difference diagram
Suspected target data；
Target data screening unit, for repeating the picture frame for obtaining adjacent n frame based on the image data of having classified
Step carries out process of refinement to the difference diagram to described, is extracted step K times of suspected target data, is obtained by static nature
K suspected target data are obtained, target association processing is carried out to the K suspected target data, passes through behavioral characteristics and screens acquisition
Target data, and the track of target is filtered, obtain target trajectory data；
Target data transmission unit, for the target data and target trajectory data to be sent to ground-based server end.
8. a kind of object monitoring device for unmanned aerial vehicle onboard end characterized by comprising
Image data transmission unit, for image data to be sent to ground-based server end in real time；
Director data receiving unit, for receiving the director data of ground-based server end transmission；
Judging unit, for judging that the content that described instruction data include is that target following or target identify；
First state modulator value generates output unit, when the content for including when described instruction data is target following, generates
First state modulator value is simultaneously exported to load, realizes object real-time tracking shooting operation for control load；
Second state modulator value generates output unit and generates when the content for including when described instruction data is that target identifies
Second state modulator value is simultaneously exported to load, realizes target focus operation for control load.
9. a kind of target monitoring system characterized by comprising unmanned aerial vehicle onboard end, ground-based server end, cloud and client
End, ground-based server end are connect with unmanned aerial vehicle onboard end, cloud and client by UAV TT & C's network in real time respectively；
The ground-based server end includes the object monitoring device for ground-based server end, for executing such as claims 1 or 2
The target monitoring method；
The cloud includes the object monitoring device for cloud, for executing target monitoring method as claimed in claim 3；
The unmanned aerial vehicle onboard end includes the object monitoring device for unmanned aerial vehicle onboard end, for executing such as claim 4 institute
The target monitoring method stated；
The client carries out the target identification data for receiving the target identification data of ground-based server end transmission
Interface display obtains the instruction identified comprising target following or target that user shows the processing mode of selection according to target interface
Described instruction data are sent to ground-based server end by data.
10. target monitoring system according to claim 9, which is characterized in that the client is also used to receive ground clothes
Be engaged in device end send target data prediction result, by the target data prediction result carry out interface display, obtain user according to
The target object judgement that interface display carries out and target object judging result and target identification data that mark obtains, by the mesh
Mark mark data is sent to ground-based server end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811261900.6A CN109376660B (en) | 2018-10-26 | 2018-10-26 | Target monitoring method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811261900.6A CN109376660B (en) | 2018-10-26 | 2018-10-26 | Target monitoring method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109376660A true CN109376660A (en) | 2019-02-22 |
| CN109376660B CN109376660B (en) | 2022-04-08 |
Family
ID=65390071
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811261900.6A Active CN109376660B (en) | 2018-10-26 | 2018-10-26 | Target monitoring method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109376660B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298866A (en) * | 2019-06-06 | 2019-10-01 | 武汉易科空间信息技术股份有限公司 | Ground object tracking and system based on unmanned plane image technology |
| CN110958294A (en) * | 2019-10-22 | 2020-04-03 | 国网浙江省电力有限公司宁波供电公司 | Power transmission and transformation inspection video processing method based on fog calculation |
| CN111766579A (en) * | 2020-06-28 | 2020-10-13 | 中国科学院空天信息创新研究院 | SAR air-ground combined processing method, airborne SAR and ground controller |
| CN112327935A (en) * | 2019-08-05 | 2021-02-05 | 旭日蓝天(武汉)科技有限公司 | AI technology-based unmanned aerial vehicle cloud object identification and tracking system and method |
| CN112365468A (en) * | 2020-11-11 | 2021-02-12 | 南通大学 | AA-gate-Unet-based offshore wind power tower coating defect detection method |
| WO2021025747A3 (en) * | 2019-05-06 | 2021-06-03 | Sap National Security Services, Inc. | Systems and methods for aerostat management including identifying, classifying and determining predictive trends of an entity of interest |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104932529A (en) * | 2015-06-05 | 2015-09-23 | 北京中科遥数信息技术有限公司 | Unmanned plane autonomous flight cloud control system |
| CN105518555A (en) * | 2014-07-30 | 2016-04-20 | 深圳市大疆创新科技有限公司 | Systems and methods for target tracking |
| CN106407984A (en) * | 2015-07-31 | 2017-02-15 | 腾讯科技（深圳）有限公司 | Target object recognition method and device |
| CN106981073A (en) * | 2017-03-31 | 2017-07-25 | 中南大学 | A kind of ground moving object method for real time tracking and system based on unmanned plane |
| CN109447056A (en) * | 2018-12-07 | 2019-03-08 | 苏州米特希赛尔人工智能有限公司 | The feature-extraction images sensor that DSP is realized |
-
2018
- 2018-10-26 CN CN201811261900.6A patent/CN109376660B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105518555A (en) * | 2014-07-30 | 2016-04-20 | 深圳市大疆创新科技有限公司 | Systems and methods for target tracking |
| CN104932529A (en) * | 2015-06-05 | 2015-09-23 | 北京中科遥数信息技术有限公司 | Unmanned plane autonomous flight cloud control system |
| CN106407984A (en) * | 2015-07-31 | 2017-02-15 | 腾讯科技（深圳）有限公司 | Target object recognition method and device |
| CN106981073A (en) * | 2017-03-31 | 2017-07-25 | 中南大学 | A kind of ground moving object method for real time tracking and system based on unmanned plane |
| CN109447056A (en) * | 2018-12-07 | 2019-03-08 | 苏州米特希赛尔人工智能有限公司 | The feature-extraction images sensor that DSP is realized |
Non-Patent Citations (3)
| Title |
|---|
| HE S F ET AL: "Robust object tracking via locality sensitive histograms", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
| 张军阳等: "深度学习相关研究综述", 《计算机应用研究》 * |
| 葛宝义等: "基于特征融合的长时目标跟踪算法", 《光学学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021025747A3 (en) * | 2019-05-06 | 2021-06-03 | Sap National Security Services, Inc. | Systems and methods for aerostat management including identifying, classifying and determining predictive trends of an entity of interest |
| CN110298866A (en) * | 2019-06-06 | 2019-10-01 | 武汉易科空间信息技术股份有限公司 | Ground object tracking and system based on unmanned plane image technology |
| CN112327935A (en) * | 2019-08-05 | 2021-02-05 | 旭日蓝天(武汉)科技有限公司 | AI technology-based unmanned aerial vehicle cloud object identification and tracking system and method |
| CN110958294A (en) * | 2019-10-22 | 2020-04-03 | 国网浙江省电力有限公司宁波供电公司 | Power transmission and transformation inspection video processing method based on fog calculation |
| CN110958294B (en) * | 2019-10-22 | 2021-12-17 | 国网浙江省电力有限公司宁波供电公司 | Power transmission and transformation inspection video processing method based on fog calculation |
| CN111766579A (en) * | 2020-06-28 | 2020-10-13 | 中国科学院空天信息创新研究院 | SAR air-ground combined processing method, airborne SAR and ground controller |
| CN112365468A (en) * | 2020-11-11 | 2021-02-12 | 南通大学 | AA-gate-Unet-based offshore wind power tower coating defect detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109376660B (en) | 2022-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109376660A (en) | A kind of target monitoring method, apparatus and system | |
| CN108037770B (en) | Unmanned aerial vehicle power transmission line inspection system and method based on artificial intelligence | |
| CN108388927B (en) | Small sample polarization SAR terrain classification method based on deep convolution twin network | |
| CN107862705B (en) | Unmanned aerial vehicle small target detection method based on motion characteristics and deep learning characteristics | |
| US9251425B2 (en) | Object retrieval in video data using complementary detectors | |
| CN110084094B (en) | Unmanned aerial vehicle target identification and classification method based on deep learning | |
| CN104200671A (en) | Method and system for managing virtual gate based on big data platform | |
| CN112990262B (en) | Integrated solution system for monitoring and intelligent decision of grassland ecological data | |
| CN106682592A (en) | Automatic image recognition system and method based on neural network method | |
| CN106791696B (en) | Wireless video monitoring system and its image transfer method and device | |
| CN109905696A (en) | A kind of recognition methods of the Video service Quality of experience based on encryption data on flows | |
| CN110659391A (en) | Video detection method and device | |
| CN110968941A (en) | Unmanned aerial vehicle control platform and control method based on airspace safety assessment | |
| CN113284144B (en) | Tunnel detection method and device based on unmanned aerial vehicle | |
| CN113037783B (en) | Abnormal behavior detection method and system | |
| Hossain et al. | A UAV-based traffic monitoring system for smart cities | |
| CN113159166A (en) | Embedded image identification detection method, system, medium and equipment based on edge calculation | |
| Culman et al. | A novel application for identification of nutrient deficiencies in oil palm using the internet of things | |
| CN111126411B (en) | Abnormal behavior identification method and device | |
| CN114494916A (en) | Black-neck crane monitoring and tracking method based on YOLO and DeepsORT | |
| Hussain et al. | Low latency and non-intrusive accurate object detection in forests | |
| CN114898238A (en) | Wild animal remote sensing identification method and device | |
| CN208335208U (en) | Image co-registration acquisition system comprising meteorologic parameter | |
| CN110211159A (en) | A kind of aircraft position detection system and method based on image/video processing technique | |
| CN113536564A (en) | Unmanned swarm autonomous collaborative evaluation method and system based on virtual simulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210309 Address after: 100079 6357, 1-13, 6th floor, building 2, yard 1, Haiying Road, Fengtai District, Beijing Applicant after: TIANYU JINGWEI (BEIJING) TECHNOLOGY Co.,Ltd. Applicant after: East China branch of National Petroleum Pipeline Network Group Co.,Ltd. Address before: 100079 room 617, 6th floor, building 2, yard 1, Hangfeng Road, Fengtai District, Beijing (room 717, elevator floor) Applicant before: TIANYU JINGWEI (BEIJING) TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |