CN104899831B - A kind of unmanned plane image data real-time processing method and system - Google Patents
A kind of unmanned plane image data real-time processing method and system Download PDFInfo
- Publication number
- CN104899831B CN104899831B CN201510311975.0A CN201510311975A CN104899831B CN 104899831 B CN104899831 B CN 104899831B CN 201510311975 A CN201510311975 A CN 201510311975A CN 104899831 B CN104899831 B CN 104899831B
- Authority
- CN
- China
- Prior art keywords
- image data
- feature
- information
- frame image
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
Abstract
The invention discloses a kind of unmanned plane image data real-time processing method and systems, this method is matched by the union feature information to continuous image data, the feature extraction and matching process being applied in combination using FREAK and SIFT feature, has responded the requirement to unmanned plane image data real-time.Then in the case where no POS information assists, using characteristic matching as a result, being estimated based on attitude data of the SFM to continuous image data, the flight track of multiple segments of unmanned plane is restored, and mark the frame image data in each segment.It is last to carry out image mosaic using characteristic matching result according to the frame image data for being marked with segment label, generate corresponding spliced map.By the above method, realize and solve that telemetry can not obtain or time-space registration is poor, i.e., without POS system auxiliary in the case where, the image data of unmanned plane is handled in real time purpose.
Description
Technical field
The present invention relates to air vehicle technique field, more specifically to a kind of unmanned plane image data side of processing in real time
Method and system.
Background technique
For the application increasingly extension of the current unmanned plane industry in current China and deeply development, unmanned plane is as a kind of remote sensing
The extendable platform of application plays increasingly important role in all trades and professions.User is integrated from current unmanned plane
Formula is analyzed, and the research and development of unmanned plane and the research and development of unmanned plane load are individually separated and have between the two great only
Vertical property.Unmanned plane itself and unmanned plane payload are caused as a result, each other in terms of in addition to other in mechanical structure
The not open and payload data information Space of the reduction of coupling, especially certain aircraft telemetry intelligence (TELINT)s is asynchronous
Property.
What the load of unmanned plane carry mainly integrated at present is optical imaging sensor, and data type mainly includes continuous
Take photo by plane photograph and video data.Since the processing of unmanned plane load data is related to unmanned plane load type, in unmanned plane reality
In the application process of border, for photograph of continuously taking photo by plane, the mode of data processing are as follows: match necessary aircraft POS (Position
And Orientation System, low stable accuracy platform and high-precision attitude measuring system, by inertial measuring unit IMU and
Global position system GPS is composed) information or under precision and the higher situation of reliability requirement guarantee aircraft telemetering letter
Breath and the space-time synchronous of load data and the consistency of use pattern are executed in real time to the processing of single width image data and information
It extracts；For continuous video data, the mode of data processing are as follows: under post-processing mode, can to image and telemetering into
Row saves respectively, is spliced after later period matching, only needs to guarantee spatial synchrony at this time, reduce to POS system
It is required that.
Current unmanned plane is in the field for needing to take photo by plane on a large scale, according to above-mentioned for the boat obtained after a wide range of flight
Beat of data can not meet at data to a certain extent in such a way that the mode of post-processing carries out batch processing to data of taking photo by plane
Manage the requirement of result timeliness；If dependent on requiring under telemetry intelligence (TELINT) and load data space-time synchronous in real time to continuously taking photo by plane
The processing of photograph then needs to meet higher professional requirement for data link and unmanned plane load are integrated, this with work as
The preceding more and more rapid unmanned plane of development, can not especially open the application collision of the multi-rotor unmanned aerial vehicle of telemetry interface.
Therefore, how to solve not obtaining in telemetry or time-space registration is poor, i.e., the feelings required without POS system
Under condition, the image data of unmanned plane is carried out being treated as current urgent problem to be solved in real time.
Summary of the invention
The object of the present invention is to provide a kind of unmanned plane image data real-time processing method and systems, to solve in telemetering number
According to can not obtain or in the case that time-space registration is poor, the problem of processing in real time the image data of unmanned plane.
To achieve the above object, the present invention provides the following technical scheme that
A kind of unmanned plane image data real-time processing method, comprising:
Continuous image data are obtained, the union feature information of the continuous image data is extracted, the union feature is believed
Breath is matched with threshold value, obtains the characteristic matching result of successful match, wherein include FREAK in the union feature information
Feature and SIFT feature；
POS information auxiliary is judged whether there is, if assisting without POS information, using the characteristic matching as a result, based on movement
Estimate that structure SFM estimates the attitude data of the continuous image data, and obtains multiframe estimation attitude data；
Based on multiframe estimation attitude data to the affiliated segment of each frame image data in the continuous image data
It is analyzed, since a certain frame image data, the corresponding frame of frame image data of continuous default frame number estimates attitude data
In translation variation, the new segment since a certain frame image data is determined, and carry out segment label；
Segment belonging to SFM progress is based on to subsequent frame image data to analyze, and will belong to the new segment
Frame image carries out segment label；
According to the frame image data for being marked with segment label, image mosaic is carried out using the characteristic matching result, is generated
Corresponding spliced map.
Preferably, after being analyzed based on segment belonging to SFM progress subsequent frame image data, further includes:
Deviate the new segment when continuously presetting the corresponding frame estimation attitude data of frame number in subsequent frame image data, stops
Only to the image mosaic of the new segment, and start the control information of the new segment to the corresponding spliced map in the new segment into
The anti-of row segment image penetrates transformation correction, obtains the geographical map for meeting GIS requirement；
The control information includes track information, or the geographic factor information being manually entered.
Preferably, further includes: if judgement has POS information auxiliary, obtain the image data for being accompanied with POS attribute, be based on
The characteristic matching result carries out image mosaic, generates corresponding spliced map.
Preferably, the union feature information for obtaining and extracting continuous image data by the union feature information and is preset
Characteristic information is matched, and the characteristic matching result of successful match is obtained, comprising:
The continuous image data that the unmanned plane shooting is obtained by real-time link or forwarded, alternatively, directly acquiring
It is pre-stored within the continuous image data of earth station；
The continuous image data are imported according to image frame format, the joint for extracting the frame image data that each frame imports is special
Reference is ceased and is matched with threshold value；
The matching result for obtaining each frame image data based on successful match generates the characteristic matching result of successful match；
It extracts the union feature information for the frame image data that each frame imports and carries out matched process with threshold value and include:
It extracts the FREAK feature in the frame image data currently imported and carries out signature, and be based on characteristic matching letter
Number matches the FREAK feature after the label with first threshold；
If match point number is no less than the first threshold, it is determined that the frame image data successful match currently imported；
If match point number is less than the first threshold, switch to in the frame image data currently imported
SIFT feature is extracted and is marked；
It is matched to by the SIFT feature after the label with second threshold based on the characteristic matching function；
If match point number is less than the second threshold, the feature is constructed according to previous characteristic matching result
Matching relationship parameter with function；
If match point number is no less than the second threshold, it is determined that the frame image data successful match currently imported；
Wherein, while importing current frame image data, previous frame image data is cached.
Preferably, the composition of the Feature Descriptor in the FREAK feature is by thick information to smart information, by the mark
FREAK feature after note is matched with first threshold, comprising:
The thick information for representing description is matched with first threshold, if distance is less than first threshold in matching
Value then continues to match the first threshold with the smart information of description.
A kind of unmanned plane image data real time processing system, comprising:
Coalignment is extracted, for obtaining continuous image data, extracts the union feature information of the continuous image data,
The union feature information is matched with threshold value, obtains the characteristic matching result of successful match, wherein the union feature
It include FREAK feature and SIFT feature in information；
Judge estimation device, for judging whether there is POS information auxiliary, if assisting without POS information, utilizes the feature
Matching result is estimated based on attitude data of the motion estimation architecture SFM to the continuous image data, and obtains multiframe estimation appearance
State data；
Segment analytical equipment, for estimating attitude data to each frame in the continuous image data based on the multiframe
The affiliated segment of image data is analyzed, and since a certain frame image data, the frame image data of continuous default frame number is corresponding
Frame estimation attitude data be in translation variation, determine the new segment since a certain frame image data, and carry out segment
Label；
Segment judgment means are analyzed for being based on segment belonging to SFM progress to subsequent frame image data, will
The frame image for belonging to the new segment carries out segment label；
Splice map device, for according to be marked with segment label frame image data, using the characteristic matching result into
Row image mosaic generates corresponding spliced map.
Preferably, further includes:
Means for correcting, for when continuously the corresponding frame estimation attitude data of default frame number deviates in subsequent frame image data
The new segment stops the image mosaic to the new segment, and starts the control information of the new segment to the new segment
Corresponding spliced map carries out the anti-of segment image and penetrates transformation correction, obtains the geographical map for meeting GIS requirement；
The control information includes track information, or the geographic factor information being manually entered.
Preferably, the judgement estimation device, further includes:
Jump-transfer unit, if obtaining the image data for being accompanied with POS attribute and described for judging there is a POS information auxiliary
Characteristic matching result jumps to the splicing map device and carries out image mosaic, generates corresponding spliced map.
Preferably, the extraction coalignment, comprising:
Feature extraction unit, for obtaining the continuous image number of the unmanned plane shooting by real-time link or forwarded
According to alternatively, directly acquiring the continuous image data for being pre-stored within earth station；
Characteristic matching unit extracts what each frame imported for importing the continuous image data according to image frame format
The union feature information of frame image data is simultaneously matched with threshold value；
Acquiring unit, the matching result for obtaining each frame image data based on successful match generate successful match
Characteristic matching result；
The characteristic matching unit, comprising:
First matching module carries out signature for extracting the FREAK feature in the frame image data currently imported,
And the FREAK feature after the label is matched with first threshold based on characteristic matching function；
First determining module, if being no less than the threshold value for match point number, it is determined that the frame image currently imported
Data successful match；
Switching module switches to if being less than the threshold value for match point number to the frame figure currently imported
As the SIFT feature in data extracts and is marked；
Second matching module, for based on the characteristic matching function to by after the label SIFT feature and the second threshold
Value is matched；
Constructing module constructs if being less than the threshold value for match point number according to previous characteristic matching result
The matching relationship parameter of the characteristic matching function；
Second determining module, if being no less than the threshold value for match point number, it is determined that the frame image currently imported
Data successful match；
Wherein, while importing current frame image data, previous frame image data is cached.
Preferably, comprising: when the composition of the Feature Descriptor in the FREAK feature be by thick information to smart information,
By after the label FREAK feature and first threshold carry out matched first matching module, for retouching described in representing
The thick information for stating son is matched with first threshold, if distance is less than the first threshold in matching, is continued described first
Threshold value is matched with the smart information of description.
By above scheme it is found that a kind of unmanned plane image data real-time processing method provided in an embodiment of the present invention and being
System, this method are matched union feature information with threshold value by the union feature information of extraction continuous image data, are obtained
The characteristic matching of successful match is as a result, the feature extraction and matching process being applied in combination using FREAK feature and SIFT feature, is rung
The requirement to unmanned plane image data real-time is answered.Then in the case where no POS information assists, characteristic matching knot is utilized
Fruit is estimated based on attitude data of the SFM to continuous image data, and obtains multiframe estimation attitude data；And estimated based on multiframe
Meter attitude data analyzes the affiliated segment of each frame image data in continuous image data, when from a certain frame image data
Start, the corresponding frame estimation attitude data of the frame image data of continuous default frame number is in translation variation, is determined from a certain frame
Image data starts new segment, and carries out segment label；To subsequent frame image data be based on the SFM carry out belonging to segment into
The frame image for belonging to new segment is carried out segment label by row analysis；Here in the case where no POS system, the method for SFM is utilized
The recovery for carrying out unmanned plane running track restores the flight track of multiple segments, each by obtaining opposite attitude data
Splicing data in real time are constructed in segment.Last foundation is marked with the frame image data of segment label, utilizes the characteristic matching knot
Fruit carries out image mosaic, generates corresponding spliced map.By the above method, realization solution telemetry can not obtain or time-space registration
Property it is poor, i.e., without POS system auxiliary in the case where, the image data of unmanned plane is handled in real time purpose.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of flow chart of unmanned plane image data real-time processing method provided by the embodiments of the present application；
Fig. 2 is the flow chart of another unmanned plane image data real-time processing method provided by the embodiments of the present application；
Fig. 3 is the part process in another unmanned plane image data real-time processing method provided by the embodiments of the present application
Figure；
Fig. 4 is a kind of structural schematic diagram of unmanned plane image data real time processing system provided by the embodiments of the present application；
Fig. 5 is the structural schematic diagram of another unmanned plane image data real time processing system provided by the embodiments of the present application.
Specific embodiment
Specification and claims and term " first " in above-mentioned attached drawing, " second ", " third " " the 4th " etc. (if
In the presence of) it is part for distinguishing similar, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so that embodiments herein described herein can be in addition to illustrating herein
Sequence in addition is implemented.
The following are the explanations of english abbreviation involved in description of the invention and Figure of description:
POS:Position and Orientation System, low stable accuracy platform and high-precision attitude measurement system
System, is composed of inertial measuring unit IMU and global position system GPS；
SFM:structure from motion, motion estimation architecture；
FREAK feature and SIFT feature: being image local feature.
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Embodiment one
The embodiment of the present invention one provides a kind of unmanned plane image data real-time processing method, is applied to unmanned plane, such as Fig. 1
It is shown, it mainly comprises the steps that
Step S101 obtains continuous image data, extracts the union feature information of the continuous image data, will be described
It closes characteristic information to be matched with threshold value, obtains the characteristic matching result of successful match；
It in step s101, include FREAK feature and SIFT feature in the union feature information；Implement in the present invention
In example, the feature extraction and matching process being applied in combination using FREAK feature and SIFT feature is met to unmanned plane image data
The requirement handled in real time.
Step S102 judges whether there is POS information auxiliary if assisting without POS information and thens follow the steps S103；
In step s 102, POS information auxiliary whether sentences during handling current unmanned plane image data
It is disconnected, if judgement has POS information auxiliary, the image data for being accompanied with POS attribute can be directly acquired, is obtained based on step S101 is executed
The characteristic matching result taken carries out image mosaic, generates corresponding spliced map.If judging no POS information auxiliary, after
Mode continuous to execute, that unmanned plane image data is handled when calling without POS information, i.e. execution step S103.
Step S103, using the characteristic matching as a result, based on motion estimation architecture SFM to the continuous image data
Attitude data estimation, and obtain multiframe estimation attitude data；
During executing step S103, in the embodiment of the present invention using motion estimation architecture SFM by the way of to even
The attitude data of continuous image data is estimated, the estimation attitude data of multiframe is obtained.The SFM is two views of one kind by moving to
The method of structure sets up basic epipolar-line constraint relationship between continuous two frames image data by matched characteristic point,
Using the fundamental relation, can restore to encode the unmanned plane camera motion in the epipolar-line constraint relationship in essential matrix E, i.e.,
By N number of respective value formed it is N number of about 9 elements homogeneous equation.Under ideal noise-free case, essential matrix E is unusual
, i.e. SVD has 0 singular value after decomposing.
Step S104, based on multiframe estimation attitude data to each frame image data in the continuous image data
Affiliated segment is analyzed, since a certain frame image data, the corresponding frame estimation of the frame image data of continuous default frame number
Attitude data is in translation variation, determines the new segment since a certain frame image data, and carry out segment label；
In step S104, the corresponding frame estimation attitude data of the frame image data of continuous default frame number is in translation variation
In continuous default frame number, be under normal conditions ten frames, when the embodiment of the present invention is not limited to that, which can be controlled
Float 2~3 frames.
Step S105 is based on segment belonging to SFM progress to subsequent frame image data and analyzes, will belong to described
The frame image of new segment carries out segment label；
During executing step S103~step S105, unmanned plane running track can be carried out using the method for SFM
Recovery, and resolved by the movement of successive frame and obtain relative attitude, restore the flight track of multiple segments, in each segment
The processing result of data is spliced in building in real time, that is, determines the frame image data in each segment and the affiliated segment, and carry out
Corresponding operation.
Step S106 carries out image using the characteristic matching result according to the frame image data for being marked with segment label
Splicing, generates corresponding spliced map.
During executing step S106, based on the frame figure in the above-mentioned each segment got and in affiliated segment
As data, acquired corresponding characteristic matching result carries out image spelling when executing step S101 in conjunction with each frame image data
It connects, generates the corresponding spliced map of continuous image data that current unmanned plane obtains.
Disclose a kind of unmanned plane image data real-time processing method by the embodiments of the present invention, this method by pair
The union feature information of continuous image data is matched, the feature extraction that is applied in combination using FREAK and SIFT feature and
With process, the requirement to unmanned plane image data real-time has been responded.Then in the case where no POS information assists, spy is utilized
Matching result is levied, is estimated based on attitude data of the SFM to continuous image data, it is extensive to the flight track of multiple segments of unmanned plane
It is multiple, and mark the frame image data in each segment.Last foundation is marked with the frame image data of segment label, utilizes feature
Image mosaic is carried out with result, generates corresponding spliced map.By the above method, realization solution telemetry can not obtain or space-time
Matching is poor, i.e., without POS system auxiliary in the case where, the image data of unmanned plane is handled in real time purpose.
Embodiment two
Based on a kind of unmanned plane image data real-time processing method disclosed in the embodiments of the present invention, as illustrated in FIG. 1
Step S105, execute subsequent frame image data is based on the SFM carry out belonging to after segment analyzes, further include
It is as shown in Figure 2:
Step S107 estimates described in attitude data deviation when continuously presetting the corresponding frame of frame number in subsequent frame image data
New segment stops the image mosaic to the new segment, and the control information for starting the new segment is corresponding to the new segment
Spliced map carry out segment image it is anti-penetrate transformation correction, obtain meet GIS requirement geographical map；
In step s 107, the control information includes track information, or the geographic factor information being manually entered.
Based on a kind of unmanned plane image data real-time processing method disclosed in the embodiments of the present invention, as illustrated in FIG. 1
Step S101, specific implementation procedure, as shown in figure 3, mainly comprising the steps that
Step S1011 obtains the continuous image data of the unmanned plane shooting by real-time link or forwarded, or
Person directly acquires the continuous image data for being pre-stored within earth station；
Step S1012 imports the continuous image data according to image frame format, extracts the frame picture number that each frame imports
According to union feature information and matched with threshold value；
Step S1013, the matching result for obtaining each frame image data based on successful match generate the spy of successful match
Levy matching result.
Wherein, during executing step S1012, for the union feature information for the frame image data that each frame imports
And matched process is carried out with threshold value and includes:
It extracts the FREAK feature in the frame image data currently imported and carries out signature, and be based on characteristic matching letter
Number matches the FREAK feature after the label with first threshold；
If match point number is no less than the first threshold, it is determined that the frame image data successful match currently imported；
If match point number is less than the first threshold, switch to in the frame image data currently imported
SIFT feature is extracted and is marked；
It is matched to by the SIFT feature after the label with second threshold based on the characteristic function；
If match point number is less than the second threshold, the feature is constructed according to previous characteristic matching result
Matching relationship parameter with function；
If match point number is no less than the second threshold, it is determined that the frame image data successful match currently imported；
Wherein, while importing current frame image data, previous frame image data is cached；Here
First threshold and the value size of second threshold can be identical, can also be identical, are usually set by technical staff or adjusted.
Based on above-mentioned matching process, it should be noted that the FREAK feature disclosed in the embodiment of the present invention is using such as
Under:
A, critical point detection:
Here using multiple dimensioned AGAST detection proposed in BRISK algorithm.
B, the building of binary descriptor F:
Binary descriptor F such as formula (1) is obtained by the corresponding Gaussian kernel in thresholding receiving area:
Wherein, PaIt is a pair of acceptable field, N is the length of goal description；
HereFor first to acceptable field to PaSmoothness.
By tens acceptable area, it is sub to a higher-dimension description can be constructed to obtain thousands of pairs of regions.Pass through
The space length strategy introduced in description that BRISK algorithm obtains removes some regions pair useless.By this method
To region to may high correlation lack discrimination.
SIFT feature disclosed in the embodiment of the present invention uses as follows:
A, key point monitoring:
SIFT feature has comprehensively considered position, scale and the rotational invariants of characteristic point.The detection of extreme point can pass through
Scale space or gaussian pyramid, the embodiment of the present invention realize that extreme point is detected by building gaussian pyramid.
Scale space, Gaussian function formula (2) are established using Gaussian function are as follows:
G (x, y, e)=[1/2*pi*e^2] * exp [- (x^2+y^2)/2e^2] (2)
Above-mentioned formula G (x, y, e), as changeable scale Gaussian function.
Detect key point on this basis, in order to find the extreme point of scale space, firstly, each sampled point will and it
All consecutive points compare, and see whether it is bigger than the consecutive points of its image area and scale domain or small.Then, key is clicked through
Line direction distribution is to distribute a direction to each key point with the local feature of image, and description is made to have rotation not property, benefit
With the gradient of key point neighborhood territory pixel and the characteristic of directional spreding, available gradient modulus value and direction such as following formula (3) and (4):
θ (x, y)=tan-1((L(x,y+1)-L(x,y-1))/L(x+1,y)-L(x-1,y)))(4)
B, description building:
Each key point is gathered around there are three information: position, scale and direction.A description is established for each key point
Symbol, changes it with various change, such as illumination variation, visual angle change etc..Reference axis is rotated to be into key point first
Direction, to ensure rotational invariance.The window that 8 × 8 are taken centered on key point calculates 8 sides on every 4 × 4 fritter
To gradient orientation histogram, draw the accumulated value of each gradient direction, a key point by 2 × 2 totally 4 seed points form,
Each seed point has 8 direction vector information.It stores in Feature Descriptor, it, then can be with by the length normalization method of feature vector
Further remove the influence of illumination variation.
Based on the above-mentioned explanation to FREAK feature and SIFT feature, due to the Feature Descriptor in the FREAK feature
Composition is by thick information to smart information, therefore matching process is also in this way, represents description by preceding 16bytes first
Thick information.If distance is less than threshold value in matching, continue to be compared the fine-feature of description.This manner of comparison
As a result it can accelerate matched speed.
And SIFT feature description uses the Euclidean distance of key point feature vector as key point in two images
Similarity determination measurement.
By unmanned plane image data real-time processing method disclosed in the embodiments of the present invention, this method passes through the company of extraction
The union feature information of continuous image data, union feature information is matched with threshold value, obtains the characteristic matching of successful match
As a result, the feature extraction and matching process being applied in combination using FREAK feature and SIFT feature, has been responded to unmanned plane image number
According to the requirement of real-time.Then in the case where judging no POS information auxiliary, using characteristic matching as a result, based on SFM to continuous
The attitude data of image data is estimated, and obtains multiframe estimation attitude data；And based on multiframe estimation attitude data to continuous
The affiliated segment of each frame image data in image data is analyzed, and since a certain frame image data, continuously presets frame
The corresponding frame estimation attitude data of several frame image datas is in translation variation, and determination is newly navigated since a certain frame image data
Section, and carry out segment label；Segment belonging to SFM progress is based on to subsequent frame image data to analyze, and will be belonged to new
The frame image of segment carries out segment label；Here in the case where no POS system, unmanned plane operation is carried out using the method for SFM
The recovery of track restores the flight track of multiple segments, constructs in each segment real-time by obtaining opposite attitude data
Splice data.It is last to carry out image mosaic using the characteristic matching result according to the frame image data for being marked with segment label,
Generate corresponding spliced map.By the above method, realize that solution telemetry can not obtain or time-space registration is poor, i.e., without POS
In the case that system assists, the image data of unmanned plane is handled in real time purpose.
Embodiment three
Based on a kind of image data real-time processing method of unmanned plane disclosed in the embodiments of the present invention, corresponding hair
Bright embodiment also discloses a kind of unmanned plane image data real time processing system, applies equally to unmanned plane, structural schematic diagram
As shown in figure 4, specifically including that
Coalignment 101 is extracted, for obtaining continuous image data, extracts the union feature letter of the continuous image data
Breath, the union feature information is matched with threshold value, obtains the characteristic matching result of successful match, wherein the joint
It include FREAK feature and SIFT feature in characteristic information；
Judge estimation device 102, for judging whether there is POS information auxiliary, if judging no POS information auxiliary, utilizes
The characteristic matching based on attitude data of the motion estimation architecture SFM to the continuous image data as a result, estimated, and obtain more
Frame estimates attitude data；
Segment analytical equipment 103, for estimating attitude data to every in the continuous image data based on the multiframe
The affiliated segment of one frame image data is analyzed, since a certain frame image data, the frame image data of continuous default frame number
Corresponding frame estimation attitude data is in translation variation, determines the new segment since a certain frame image data, and carry out
Segment label；
Segment judgment means 104 are divided for being based on segment belonging to SFM progress to subsequent frame image data
The frame image for belonging to the new segment is carried out segment label by analysis；
Splice map device 105, for utilizing the characteristic matching result according to the frame image data for being marked with segment label
Image mosaic is carried out, corresponding spliced map is generated.
As shown in figure 5, unmanned plane image data disclosed by the embodiments of the present invention is handled in real time on the basis of above-mentioned Fig. 4
System, further includes:
Means for correcting 106, for when continuously the corresponding frame of default frame number estimates attitude data in subsequent frame image data
Deviate the new segment, stop the image mosaic to the new segment, and starts the control information of the new segment to described new
The corresponding spliced map in segment carries out the anti-of segment image and penetrates transformation correction, obtains the geographical map for meeting GIS requirement；
Wherein, the control information includes track information, or the geographic factor information being manually entered.
Based on judgement estimation device 102 disclosed in above-mentioned Fig. 4, wherein further include:
Jump-transfer unit, if obtaining the image data for being accompanied with POS attribute and described for judging there is a POS information auxiliary
Characteristic matching result jumps to the splicing map device and carries out image mosaic, generates corresponding spliced map.
Based on extraction coalignment 101 disclosed in above-mentioned Fig. 4, specifically include that
Feature extraction unit, for obtaining the continuous image number of the unmanned plane shooting by real-time link or forwarded
According to alternatively, directly acquiring the continuous image data for being pre-stored within earth station；
Characteristic matching unit extracts what each frame imported for importing the continuous image data according to image frame format
The union feature information of frame image data is simultaneously matched with threshold value；
Acquiring unit, the matching result for obtaining each frame image data based on successful match generate successful match
Characteristic matching result；
The characteristic matching unit, comprising:
First matching module carries out signature for extracting the FREAK feature in the frame image data currently imported,
And the FREAK feature after the label is matched with first threshold based on characteristic matching function；
It should be noted that the composition of the Feature Descriptor in the FREAK feature be by thick information to smart information,
By after the label FREAK feature and first threshold carry out matched first matching module, for retouching described in representing
The thick information for stating son is matched with first threshold, if distance is less than the first threshold in matching, is continued described first
Threshold value is matched with the smart information of description.
First determining module, if being no less than the threshold value for match point number, it is determined that the frame image currently imported
Data successful match；
Switching module switches to if being less than the threshold value for match point number to the frame figure currently imported
As the SIFT feature in data extracts and is marked；
Second matching module, for based on the characteristic matching function to by after the label SIFT feature and the second threshold
Value is matched；
Constructing module constructs if being less than the threshold value for match point number according to previous characteristic matching result
The matching relationship parameter of the characteristic matching function；
Second determining module, if being no less than the threshold value for match point number, it is determined that the frame image currently imported
Data successful match；
Wherein, while importing current frame image data, previous frame image data is cached.
Disclosed in unmanned plane image data real time processing system disclosed by the embodiments of the present invention and the embodiments of the present invention
Unmanned plane image data real-time processing method is corresponding, above-mentioned each device, and the specific implementation procedure of unit and module section can join
Square method part, is not discussed here.
Each embodiment is described in a progressive manner in description of the invention, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts in each embodiment may refer to each other.For disclosed in embodiment
For device, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method portion
It defends oneself bright.The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly use hardware, processing
The combination of software module or the two that device executes is implemented.Software module can be placed in random access memory (RAM), memory, only
Read memory (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or
In any other form of storage medium well known in technical field.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (10)
1. a kind of unmanned plane image data real-time processing method characterized by comprising
Obtain continuous image data, extract the union feature information of the continuous image data, by the union feature information with
Threshold value is matched, and the characteristic matching result of successful match is obtained, wherein includes FREAK feature in the union feature information
And SIFT feature；
POS information auxiliary is judged whether there is, if assisting without POS information, using the characteristic matching as a result, being based on estimation
Structure SFM estimates the attitude data of the continuous image data, and obtains multiframe estimation attitude data；
The affiliated segment of each frame image data in the continuous image data is carried out based on multiframe estimation attitude data
Analysis, since a certain frame image data, the corresponding frame estimation attitude data of the frame image data of continuous default frame number is in
In translation variation, the new segment since a certain frame image data is determined, and carry out segment label；
Segment belonging to SFM progress is based on to subsequent frame image data to analyze, and will belong to the frame figure of the new segment
As carrying out segment label；
According to the frame image data for being marked with segment label, image mosaic is carried out using the characteristic matching result, generates and corresponds to
Spliced map.
2. the method according to claim 1, wherein being based on the SFM to subsequent frame image data carries out institute
After category segment is analyzed, further includes:
Deviate the new segment, stopping pair when continuously presetting the corresponding frame estimation attitude data of frame number in subsequent frame image data
The image mosaic of the new segment, and start the control information of the new segment and navigate to the corresponding spliced map in the new segment
The affine transformation correction of section image, obtains the geographical map for meeting GIS requirement；
The control information includes track information, or the geographic factor information being manually entered.
3. the method as described in claim 1, which is characterized in that further include: if judgement has POS information auxiliary, obtain subsidiary
There is the image data of POS attribute, image mosaic is carried out based on the characteristic matching result, generates corresponding spliced map.
4. the method as described in any one of claims 1 to 3, which is characterized in that obtain and extract continuous image data
The union feature information is matched with default characteristic information, obtains the characteristic matching of successful match by union feature information
As a result, comprising:
The continuous image data that the unmanned plane shooting is obtained by real-time link or forwarded, alternatively, directly acquiring in advance
It is stored in the continuous image data of earth station；
The continuous image data are imported according to image frame format, extract the union feature letter for the frame image data that each frame imports
It ceases and is matched with threshold value；
The matching result for obtaining each frame image data based on successful match generates the characteristic matching result of successful match；
It extracts the union feature information for the frame image data that each frame imports and carries out matched process with threshold value and include:
It extracts the FREAK feature in the frame image data currently imported and carries out signature, and will based on characteristic matching function
FREAK feature after the label is matched with first threshold；
If match point number is no less than the first threshold, it is determined that the frame image data successful match currently imported；
If match point number is less than the first threshold, switch to the SIFT in the frame image data currently imported
Feature is extracted and is marked；
It is matched to by the SIFT feature after the label with second threshold based on the characteristic matching function；
If match point number is less than the second threshold, the characteristic matching letter is constructed according to previous characteristic matching result
Several matching relationship parameters；
If match point number is no less than the second threshold, it is determined that the frame image data successful match currently imported；
Wherein, while importing current frame image data, previous frame image data is cached.
5. method as claimed in claim 4, which is characterized in that the composition of the Feature Descriptor in the FREAK feature be by
Thick information is matched by the FREAK feature after the label with first threshold to smart information, comprising:
The thick information for representing description is matched with first threshold, if distance is less than the first threshold in matching,
Then continue to match the first threshold with the smart information of description.
6. a kind of unmanned plane image data real time processing system characterized by comprising
It extracts coalignment and the union feature information of the continuous image data is extracted, by institute for obtaining continuous image data
It states union feature information to be matched with threshold value, obtains the characteristic matching result of successful match, wherein the union feature information
In include FREAK feature and SIFT feature；
Judge estimation device, for judging whether there is POS information auxiliary, if assisting without POS information, utilizes the characteristic matching
As a result, estimating based on attitude data of the motion estimation architecture SFM to the continuous image data, and obtain multiframe estimation posture number
According to；
Segment analytical equipment, for estimating attitude data to each frame image in the continuous image data based on the multiframe
The affiliated segment of data is analyzed, since a certain frame image data, the corresponding frame of frame image data of continuous default frame number
Estimate that attitude data is in translation variation, determines the new segment since a certain frame image data, and carry out segment label；
Segment judgment means are analyzed for being based on segment belonging to SFM progress to subsequent frame image data, will be belonged to
The frame image of the new segment carries out segment label；
Splice map device, for carrying out figure using the characteristic matching result according to the frame image data for being marked with segment label
As splicing, corresponding spliced map is generated.
7. system as claimed in claim 6, which is characterized in that further include:
Means for correcting, for when continuously the corresponding frame of default frame number is estimated described in attitude data deviation in subsequent frame image data
New segment stops the image mosaic to the new segment, and the control information for starting the new segment is corresponding to the new segment
Spliced map carry out segment image affine transformation correction, obtain meet GIS requirement geographical map；
The control information includes track information, or the geographic factor information being manually entered.
8. system as claimed in claim 6, which is characterized in that the judgement estimation device, further includes:
Jump-transfer unit, if obtaining the image data and the feature for being accompanied with POS attribute for judging there is POS information auxiliary
Matching result jumps to the splicing map device and carries out image mosaic, generates corresponding spliced map.
9. the system as described in any one of claim 6~8, which is characterized in that the extraction coalignment, comprising:
Feature extraction unit, for obtaining the continuous image data of the unmanned plane shooting by real-time link or forwarded,
Alternatively, directly acquiring the continuous image data for being pre-stored within earth station；
Characteristic matching unit extracts the frame figure that each frame imports for importing the continuous image data according to image frame format
As data union feature information and matched with threshold value；
Acquiring unit, the matching result for obtaining each frame image data based on successful match generate the feature of successful match
Matching result；
The characteristic matching unit, comprising:
First matching module carries out signature, and base for extracting the FREAK feature in the frame image data currently imported
The FREAK feature after the label is matched with first threshold in characteristic matching function；
First determining module, if being no less than the threshold value for match point number, it is determined that the frame image data currently imported
Successful match；
Switching module switches to if being less than the threshold value for match point number to the frame picture number currently imported
SIFT feature in is extracted and is marked；
Second matching module, for based on the characteristic matching function to by after the label SIFT feature and second threshold into
Row matching；
Constructing module, if being less than the threshold value for match point number, according to described in previous characteristic matching result construction
The matching relationship parameter of characteristic matching function；
Second determining module, if being no less than the threshold value for match point number, it is determined that the frame image data currently imported
Successful match；
Wherein, while importing current frame image data, previous frame image data is cached.
10. system according to claim 9 characterized by comprising the Feature Descriptor in the FREAK feature
Composition be by thick information to smart information, by after the label FREAK feature and first threshold carry out matched described the
One matching module, for matching the thick information for representing description with first threshold, if distance is less than institute in matching
First threshold is stated, then continues to match the first threshold with the smart information of description.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510311975.0A CN104899831B (en) | 2015-06-09 | 2015-06-09 | A kind of unmanned plane image data real-time processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510311975.0A CN104899831B (en) | 2015-06-09 | 2015-06-09 | A kind of unmanned plane image data real-time processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104899831A CN104899831A (en) | 2015-09-09 |
| CN104899831B true CN104899831B (en) | 2019-05-24 |
Family
ID=54032479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510311975.0A Active CN104899831B (en) | 2015-06-09 | 2015-06-09 | A kind of unmanned plane image data real-time processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104899831B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105513119B (en) * | 2015-12-10 | 2018-08-28 | 北京恒华伟业科技股份有限公司 | A kind of road and bridge three-dimensional rebuilding method and device based on unmanned plane |
| CN105551043B (en) * | 2015-12-17 | 2018-07-13 | 北京猎鹰无人机科技有限公司 | Unmanned plane image data real-time processing method |
| CN106227732B (en) * | 2016-07-08 | 2019-09-03 | 广州市增城区城乡规划与测绘地理信息研究院 | A kind of real-time method for obtaining mobile video photographed scene position |
| CN106326920B (en) * | 2016-08-16 | 2020-02-07 | 天津航天中为数据系统科技有限公司 | Off-line synchronization method and device for telemetering data and video image data |
| CN108088488B (en) * | 2016-11-22 | 2020-04-14 | 江西洪都航空工业集团有限责任公司 | Target range telemetry data splicing method |
| CN108492256B (en) * | 2018-04-18 | 2022-03-04 | 中国人民解放军陆军工程大学 | Unmanned aerial vehicle video fast splicing method |
| CN111060075B (en) * | 2019-12-10 | 2021-01-12 | 中国人民解放军军事科学院国防科技创新研究院 | Local area terrain ortho-image rapid construction method and system based on unmanned aerial vehicle |
| CN112965507B (en) * | 2021-02-03 | 2022-10-04 | 南京航空航天大学 | Cluster unmanned aerial vehicle cooperative work system and method based on intelligent optimization |
| WO2023097494A1 (en) * | 2021-11-30 | 2023-06-08 | 深圳市大疆创新科技有限公司 | Panoramic image photographing method and apparatus, unmanned aerial vehicle, system, and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009176164A (en) * | 2008-01-25 | 2009-08-06 | Sharp Corp | Image processing device and image processing method |
| CN102799665A (en) * | 2012-07-13 | 2012-11-28 | 中国航天空气动力技术研究院 | Unmanned aerial vehicle video data processing method |
| CN102982515A (en) * | 2012-10-23 | 2013-03-20 | 中国电子科技集团公司第二十七研究所 | Method of unmanned plane image real-time splicing |
| TW201345263A (en) * | 2012-01-25 | 2013-11-01 | Samplify Systems Inc | Raw format image data processing |
| CN104282005A (en) * | 2014-09-19 | 2015-01-14 | 天津航天中为数据系统科技有限公司 | Video image stitching method and device |
| CN104301630A (en) * | 2014-09-10 | 2015-01-21 | 天津航天中为数据系统科技有限公司 | Video image splicing method and device |
-
2015
- 2015-06-09 CN CN201510311975.0A patent/CN104899831B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009176164A (en) * | 2008-01-25 | 2009-08-06 | Sharp Corp | Image processing device and image processing method |
| TW201345263A (en) * | 2012-01-25 | 2013-11-01 | Samplify Systems Inc | Raw format image data processing |
| CN102799665A (en) * | 2012-07-13 | 2012-11-28 | 中国航天空气动力技术研究院 | Unmanned aerial vehicle video data processing method |
| CN102982515A (en) * | 2012-10-23 | 2013-03-20 | 中国电子科技集团公司第二十七研究所 | Method of unmanned plane image real-time splicing |
| CN104301630A (en) * | 2014-09-10 | 2015-01-21 | 天津航天中为数据系统科技有限公司 | Video image splicing method and device |
| CN104282005A (en) * | 2014-09-19 | 2015-01-14 | 天津航天中为数据系统科技有限公司 | Video image stitching method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104899831A (en) | 2015-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104899831B (en) | A kind of unmanned plane image data real-time processing method and system | |
| JP7221089B2 (en) | Stable simultaneous execution of location estimation and map generation by removing dynamic traffic participants | |
| US10546387B2 (en) | Pose determination with semantic segmentation | |
| CN108489482B (en) | The realization method and system of vision inertia odometer | |
| Neubert et al. | Superpixel-based appearance change prediction for long-term navigation across seasons | |
| CN104520732B (en) | The method and relevant device of alignment sensor | |
| CN103679674B (en) | Method and system for splicing images of unmanned aircrafts in real time | |
| US8855369B2 (en) | Self learning face recognition using depth based tracking for database generation and update | |
| Ragot et al. | Benchmark of visual slam algorithms: Orb-slam2 vs rtab-map | |
| CN106910217A (en) | Vision map method for building up, computing device, computer-readable storage medium and intelligent vehicle | |
| CN110675453B (en) | Self-positioning method for moving target in known scene | |
| Armagan et al. | Learning to align semantic segmentation and 2.5 d maps for geolocalization | |
| Porzi et al. | Learning contours for automatic annotations of mountains pictures on a smartphone | |
| CN109583290A (en) | The system and method for improving visual signature detection using motion-dependent data | |
| EP3624060A1 (en) | Information processing device, information processing method, and program | |
| CN110349212A (en) | Immediately optimization method and device, medium and the electronic equipment of positioning and map structuring | |
| CN110533661A (en) | Adaptive real-time closed-loop detection method based on characteristics of image cascade | |
| JP6982242B2 (en) | Learning data generator, change area detection method and computer program | |
| CN112815958A (en) | Navigation object display method, device, equipment and storage medium | |
| CN110956115B (en) | Scene recognition method and device | |
| CN106558065A (en) | The real-time vision tracking to target is realized based on color of image and texture analysiss | |
| Sujiwo et al. | Robust and accurate monocular vision-based localization in outdoor environments of real-world robot challenge | |
| CN112683305B (en) | Visual-inertial odometer state estimation method based on point-line characteristics | |
| CN111461141B (en) | Equipment pose calculating method and device | |
| Ayadi et al. | A parametric algorithm for skyline extraction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |