CN110617802A - Satellite-borne moving target detection and speed estimation method - Google Patents
Satellite-borne moving target detection and speed estimation method Download PDFInfo
- Publication number
- CN110617802A CN110617802A CN201910684818.2A CN201910684818A CN110617802A CN 110617802 A CN110617802 A CN 110617802A CN 201910684818 A CN201910684818 A CN 201910684818A CN 110617802 A CN110617802 A CN 110617802A
- Authority
- CN
- China
- Prior art keywords
- connected domain
- satellite
- domains
- coordinate system
- moving target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000001514 detection method Methods 0.000 title claims abstract description 30
- 230000033001 locomotion Effects 0.000 claims abstract description 55
- 238000006243 chemical reaction Methods 0.000 claims abstract description 16
- 238000000605 extraction Methods 0.000 claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims abstract description 9
- 239000013598 vector Substances 0.000 claims description 39
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000004891 communication Methods 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000006073 displacement reaction Methods 0.000 claims description 8
- 230000001629 suppression Effects 0.000 claims description 8
- 230000003287 optical effect Effects 0.000 claims description 7
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 230000010339 dilation Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P3/00—Measuring linear or angular speed; Measuring differences of linear or angular speeds
- G01P3/36—Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light
- G01P3/38—Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light using photographic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Power Engineering (AREA)
- Image Analysis (AREA)
Abstract
A satellite-borne moving target detection and speed estimation method comprises the following steps: first, for each of a plurality of frame images continuously captured, respectively: (1) processing the original star map to obtain a background-removed image; (2) performing threshold segmentation on the background-removed image by using a self-adaptive threshold method, and storing all pixel coordinates and gray levels which are higher than the threshold; (3) carrying out four-connected domain target extraction on the stored pixels to extract potential targets; (4) removing false targets by using the area of the connected domain and the aspect ratio threshold value and calculating the mass center of each residual connected domain; (5) and projecting the centroid of each connected domain into a GPS coordinate system by using a coordinate system conversion array provided by the spaceborne computer and camera attitude information. And eliminating false targets from the connected domain extracted from each frame, and calculating the movement speed and the movement direction of the real moving target in the GPS coordinate system. The method can effectively eliminate background interference, realize satellite-borne real-time detection of the moving target and provide the geographic coordinate and the movement speed of the target.
Description
Technical Field
The invention relates to an image processing method for multi-frame detection of a moving target, in particular to an on-orbit real-time moving target detection and speed estimation method for a satellite-borne early warning camera, which belongs to the technical field of moving target image processing.
Background
The process is that in the process of continuously shooting the ground by a satellite-borne camera, a plurality of frames of continuous images are used for completing the extraction of the interested moving target, the corresponding position and the corresponding movement speed and direction of the target are calculated, then the calculation result is used for carrying out long-time tracking shooting on the target, the analysis of the target is completed, the target information is reported to the ground in time, and the effect of quickly detecting and tracking is achieved.
Due to the fact that the imaging distance is long, the area of the target on the image is from a few to more than ten pixels, the interior of the target has no texture features, the exterior shape information is less, the influence of the background and noise on the target is large, the signal-to-noise ratio of the target is generally low, and the false alarm rate of the algorithm is generally high. The algorithm generally preprocesses a single-frame image to improve the signal-to-noise ratio of a suspicious target, and then performs threshold segmentation on the image and extracts a connected domain; and then, the moving target is detected by using the processing result of the multi-frame image.
The traditional moving target processing algorithm is designed under ideal conditions, the motion of a satellite and the change of the camera attitude are ignored, multi-frame matching of phase-plane coordinates is directly carried out on a preprocessed image, and the method cannot be applied to a real satellite-borne environment. Meanwhile, most methods such as bilateral filtering, wavelet transformation and other algorithms have large calculation amount and cannot perform on-orbit real-time calculation.
Because the phase surface coordinates are directly used for multi-frame target detection, the traditional method has inaccurate setting of the speed and the speed direction threshold value: most methods directly use the displacement of the same connected domain among multiple frames on the phase plane to complete the estimation of the speed and use the displacement direction vector to complete the estimation of the speed direction.
Disclosure of Invention
The technical problem solved by the invention is as follows: the method overcomes the defects of the prior art, provides a satellite-borne moving target detection and speed estimation method, and solves the problems that a satellite-borne camera cannot detect a ground moving target in real time and provide the moving speed and speed of the target.
The technical scheme of the invention is as follows: a satellite-borne moving target detection and speed estimation method comprises the following steps:
(1) carrying out background suppression processing on a plurality of continuously shot satellite-borne earth observation images to obtain background-removed images;
(2) performing threshold segmentation on the background-removed image in the step (1), then performing connected domain extraction, and removing part of false target connected domains to obtain effective connected domains;
specifically, threshold segmentation is carried out on the background-removed image by using a self-adaptive threshold method, and coordinates and gray values of all pixels higher than a threshold value are stored; and extracting connected domains from the stored pixels based on a four-connected domain method, and eliminating partial false target connected domains according to the extracted area and aspect ratio threshold of the connected domains to obtain effective connected domains.
(3) And (3) carrying out centroid extraction on the effective connected domain in the step (2) and solving the accurate coordinate of the centroid of the effective connected domain on the phase plane of the shooting camera.
(4) The coordinate conversion from the accurate coordinate of the centroid on the phase plane of the shooting camera to the GPS coordinate system is completed by utilizing the coordinate conversion matrix and the camera optical axis pointing parameter, and the effective connected domain centroid coordinate under the GPS coordinate system is obtained;
(5) obtaining a geographic position corresponding to the centroid of the corresponding effective connected domain according to the centroid coordinates of the effective connected domain in the GPS coordinate system, calculating the corresponding movement speed and direction of the geographic position, extracting a real target by using a method of comparing multi-frame image connected domains, extracting the connected domain corresponding to the real moving target, finally realizing moving target detection and realizing speed estimation of the moving target.
Preferably, the plurality of frames of satellite-borne earth observation images continuously shot in step (1) are as follows: the early warning camera installed on an orbiting satellite with an orbit height of 500km has a view field angle of 4 degrees.
Preferably, the step (1) performs background suppression processing on the continuously shot multiple frames of satellite-borne earth observation images to obtain a background-removed image, specifically: and respectively carrying out a template-based morphological Top-Hat algorithm on each frame image to obtain a background-removed image.
Preferably, the connected component extraction specifically comprises: and (3) roughly extracting all potential target connected domains based on a four-connected domain method, calculating the area of each connected domain and the maximum length-width ratio of the connected domain, comparing the area and the maximum length-width ratio of the connected domain with a set fixed threshold, and eliminating false target connected domains which do not meet the requirements to obtain the remaining connected domains as effective connected domains.
Preferably, the coordinate of the mass center and the phase plane in the communication domain is converted into the coordinate of the GPS coordinate system, specifically: using the transformation matrix C from the camera body coordinate system to the GPS coordinate system, the satellite displacement between the time and the earth centerRepresenting in a GPS coordinate system, corresponding to the position earth radius R; according to the direction vector of the optical path direction corresponding to the accurate coordinate of the connected domain on the phase surface in the camera body coordinate systemBy such asThe following formula calculates:
in the formulaThe direction vector of the corresponding light path of the pixel under the GPS coordinate system is taken as the direction vector of the pixel; calculated by the following formula:
in the formulaThe direction vector of the object corresponding to the connected domain relative to the earth center under the GPS coordinate system is shown, h is the height of the target, and r is the distance from the satellite to the object corresponding to the connected domain. According to the above formula, the expression of r and direction vector in GPS coordinate system can be solved
Preferably, the method for extracting the real target by comparing the connected domains of the multi-frame images specifically comprises the following steps:
(1) and sequencing the communication domains extracted from each frame of image according to the areas and the energies of the communication domains. Specifically, sorting is carried out according to the area of the connected domains, the largest area is arranged at the front of the queue, and for the connected domains with the same area, the sum of pixel gray values of the connected domains is compared and sorted according to the size;
(2) starting from the second frame, carrying out frame-by-frame connected domain matching with the connected domains in all the previous frames one by one, and storing the successfully matched connected domain combination in the corresponding connected domain of the current frame;
preferably, the connected domain matching specifically includes: extracting connected components one by one from the current frame and performing connectivity detection on the connected components one by one with the matched connected components (namely all result connected components before the current frame), specifically, the method comprises the following steps:
(1) when the length of the matched connected domain is 1, the displacement of the matched connected domain and the displacement of the matched connected domain in the GPS coordinate system and the difference value of the frame where the current connected domain is located and the frame where the matched connected domain is located are used for jointly calculating the motion speed of the corresponding target, the motion speed is compared with a speed threshold, if the motion speed is not within the threshold range, the matching is failed, and then the next connected domain detection is carried out; if the speed is in the threshold range, combining the two connected domains, storing the two connected domains at the position of the current connected domain, adding 1 to the length of the connected domain, storing the corresponding movement speed, calculating and storing a movement direction vector by using the displacement of the two connected domains in a GPS coordinate system, and deleting the data of the matched connected domain at the original position.
(2) When the length of the matched connected domain is greater than 1, the corresponding motion speed and motion direction vectors of the current connected domain and the last frame of connected domain in the matched connected domain are obtained by using a method similar to the method (1), whether the speed meets a threshold condition or not is detected, whether the speed meets the threshold condition or not is determined by using speed direction threshold detection, and if not, the next connected domain detection is carried out; if the two connected domains are satisfied simultaneously, the motion speed and the motion direction vector are subjected to weighted calculation based on the length of the matched connected domain, the weighted calculation is stored in the current connected domain information, the two connected domains are combined and stored at the position of the current connected domain, the length of the connected domain is changed into the length of the matched connected domain plus 1, and the data of the matched connected domain at the original position are deleted.
Preferably, the speed direction threshold detection is: performing dot product operation by using the speed direction vectors of the current frame and the matched connected domain and the corresponding speed direction vector stored in the matched connected domain, and if the calculation result is greater than the threshold value, passing the threshold value detection; otherwise, it is a failure.
Compared with the prior art, the invention has the advantages that:
(1) the template is designed in a targeted manner based on the size of the moving target on the phase plane, so that the algorithm precision is high;
(2) according to the invention, the phase plane coordinates of the connected domain centroid are converted into the GPS coordinate system through the conversion matrix from the camera body coordinate system to the GPS coordinate system and the vector of the satellite relative to the earth, which are provided by the spaceborne computer, so that the size and the direction of the target motion speed can be more accurately calculated;
(3) the invention utilizes two thresholds of speed and speed direction to match the connected domain, and switches the matching method according to the length of the connected domain, thereby greatly reducing the false alarm rate;
(4) the calculation amount of all algorithms used in the method is small, real-time processing can be completed under the hardware condition of the satellite-borne camera, and the method has high practical value;
(5) the invention can accurately extract the suspicious target area under the condition of small calculated amount, and can provide more accurate calculated results of speed and direction, thereby reducing the false alarm rate and improving the detection success rate.
Drawings
FIG. 1 is a flow chart of a method for detecting a satellite-borne moving target and estimating a speed;
FIG. 2 is a specific template of the Top-Hat algorithm;
FIG. 3 is a schematic diagram of solving for a direction vector corresponding to the connected domain relative to the geocenter;
fig. 4 is a schematic diagram illustrating a principle of determining connectivity of a connected domain.
Detailed Description
The present invention is described in further detail below with reference to the attached drawings.
The invention discloses a satellite-borne moving target detection and speed estimation method, which comprises the following steps: first, for each of a plurality of frame images continuously captured, respectively: (1) processing an original star map (namely an original image, and a continuously shot multi-frame satellite-borne earth observation image) to obtain a background-removed image; (2) performing threshold segmentation on the background-removed image by using a self-adaptive threshold method, and storing all pixel coordinates and gray levels which are higher than the threshold; (3) carrying out four-connected domain target extraction on the stored pixels to extract potential targets; (4) removing false targets by using the area of the connected domain and the aspect ratio threshold value and calculating the mass center of each residual connected domain; (5) and projecting the centroid of each connected domain into a GPS coordinate system by using a coordinate system conversion array provided by the spaceborne computer and camera attitude information. And secondly, performing multi-frame comparison on the connected domain extracted from each frame based on the movement speed and the movement direction to remove a false target, and calculating the movement speed and the movement direction of the real moving target in a GPS coordinate system. The method can effectively eliminate background interference, realize satellite-borne real-time detection of the moving target and provide the geographic coordinate and the movement speed of the target.
The method can be used in a satellite-borne early warning camera, realizes on-orbit real-time moving target detection and speed estimation, and provides suspicious target information to a satellite in real time. The invention aims at the problems that the data volume of the image shot by the load camera of the prior remote sensing satellite detection is large, the real-time processing on the orbit cannot be realized, the shot suspicious moving target cannot be provided in real time, and the data transmission to the ground processing cannot generally obtain the action and the purpose of the moving target at the moment. The invention can realize on-orbit detection of the moving target, provides information with timeliness, and enhances the early warning capability of the investigation satellite. The method comprises the following specific operation steps:
(1) background suppression processing is carried out on a plurality of continuously shot satellite-borne earth observation images (namely original images) to obtain background-removed images, and the background-removed images are as follows: in the process that a camera continuously shoots the ground, background and noise of a current frame image are preferably suppressed, the target contrast is improved, and background suppression processing is carried out on a plurality of continuously shot satellite-borne ground observation images;
as shown in fig. 2, preferably usingAnd performing background suppression processing to obtain a background-removed image, wherein TH (f) is a result image (namely background-removed image) matrix, and f is an original image matrix.
The method comprises the following specific steps:
the first step is as follows: solving a maximum image: the original image is subjected to a morphological dilation operation based on the ring-shaped structuring element a,for the sign of the dilation operation, in particular, the dilation operation comprises: and replacing the gray value of the corresponding pixel with the maximum gray value of the pixel in the annular area by taking the pixel as the center and corresponding to the structural element A.
The preferable scheme of the structural element A is that a flat structural element with an internal structural element Bi of 3 x 3 is set(i.e., 3 x 3 matrix, each element in the matrix is 1), and an external structural element B0A 5 × 5 flat structural element (i.e., 5 × 5 matrix, each element in the matrix is 1), and the result a ═ B is obtained by performing pseudo-subtraction of the matrix with the centers of the two elements as the origin0-BiAs a result, a is a ring-shaped structural element, and the structure of a is shown in fig. 2, and has a size of 5 × 5, and an inner 3 × 3 region is empty and has a ring shape.
The second step is that: solving a minimum image: carrying out flat-structure-based element B on the maximum image obtained in the first stepiThe theta is a corrosion operation coincidence, specifically, the corrosion operation includes: replacing the gray value of the corresponding pixel with a structural element B taking the pixel as the centeriCorresponding to the minimum gray value of the pixels within the 3 x 3 region.
The third step: finding the resulting image (i.e., background-free image): and (4) subtracting the original image from the minimum image obtained in the second step, namely, performing gray value subtraction operation on each corresponding pixel of the original image and the minimum image.
A fourth step of: and (3) solving a background image: setting the gray value of the pixel with the gray degree less than 0 in the result image obtained in the third step to be 0, and obtaining a background-removed image;
(2) and performing threshold segmentation on the background-removed image, and then performing connected domain extraction. The method comprises the following specific steps:
and performing threshold segmentation on the background-removed image by using an adaptive threshold method, and storing phase-plane coordinates and gray values of pixels with gray values higher than a threshold value in the image. The adaptive thresholding method includes: calculating a global threshold value with the formula VthIn the formula, μ is a gray level mean value of the background-removed image, σ is a global gray level standard deviation, and α is a set fixed coefficient, the sensitivity (i.e., the detection rate) of the extraction target is determined, and the sensitivity and the false alarm rate are comprehensively considered, preferably 2 to 5, preferably α to 3, so that the balance detection rate and the false alarm rate are improved.
The pixels passing through the threshold value method are subjected to four-connected domain extraction by using a clustering method, and all connected domains with the area larger than 2 and smaller than 10 are preferably stored (due to the fact that the shooting distance is long, the moving target is considered to be not different from the point target, and due to the micro-defocusing effect, the size of the target is the set threshold value, and a user can set the size according to the parameters of the camera optical system), namely the connected domains are effective.
(3) And (4) carrying out centroid extraction on the effective connected domain, and solving the accurate coordinate of the effective connected domain on the phase plane of the shooting camera. The method comprises the following specific steps:
the camera is preferably arranged on a satellite with the orbit height of 500km from the ground, and is connected with the satellite through a holder, and the holder can rotate three axes relative to the satellite, so that a conversion matrix C from a camera body coordinate system to a satellite body coordinate system is providedcsThe angle of view of the camera is theta, the preferred value is 4 DEG, and the coordinate of the centroid in the phase plane is (x)i,yi) I.e. precise coordinates on the camera phase plane, where xiIs a column-wise coordinate of the image, yiAre image line direction coordinates.
(4) And completing the coordinate conversion from the coordinates of the centroid on the phase plane of the shooting camera to the GPS coordinate system by using the conversion matrixes of the coordinate systems and the light path direction vectors of the optical axis of the camera, so as to obtain the effective connected domain centroid coordinates under the GPS coordinate system. The method comprises the following specific steps:
coordinates (x) of the centroid of each connected domain in the phase plane, which are obtained in the transformation matrices C and (3) from the camera body coordinate system to the GPS coordinate systemi,yi) And solving the coordinates of each connected domain in the GPS coordinate system. The conversion matrix C from the camera body coordinate system to the GPS coordinate system consists of three conversion matrices: transformation matrix C from geocentric inertial coordinate system to GPS coordinate systemIeA transformation matrix C from the satellite body coordinate system to the earth center inertial coordinate systemsIFrom the camera body coordinate system to the satellite body coordinate system Ccs. Wherein the optical axis of the camera points to the positive direction of the z axis of the camera body coordinate system, the row direction of the image plane is parallel to the x axis of the camera body coordinate system, the column direction of the image plane is parallel to the y axis of the camera body coordinate system, the field angle of the camera is theta, and the total resolution u x v of the camera, further, the preferable coordinate conversion method is as follows:
the first step is as follows: the optimal conversion matrix C ═ C of the final requirement from the camera body coordinate system to the GPS coordinate system is obtained by utilizing a plurality of conversion matrices obtained by the satellite attitude sensor systemIe·CsI·Ccs；
Second step of: calculating the coordinate of the camera light path direction vector corresponding to the corresponding connected domain under the camera body coordinate system by using the phase plane coordinate and the camera field angle of the corresponding connected domainThe preferred calculation formula is as follows:
in the formula [ theta ]xThe included angle between the projection of the light path direction vector on the XoZ plane and the z axis is calculated by the formula of thetax＝(xi-u/2)/u; similarly, θyThe included angle between the projection of the light path direction vector on the YoZ plane and the z axis is calculated by the following formula: thetay＝(yi-v/2)/v；
The third step: obtaining the coordinates of the light path direction vector under the GPS coordinate system by using the light path direction vector calculated in the second step and the conversion matrix calculated in the first step
The fourth step: coordinates of the satellite centroid-to-geocentric vector provided by the satellite in the GPS coordinate system by using the result of the third stepSolving the equation:wherein R + h is the distance from the position of the moving target to the geocentric, R is the radius of the earth at the corresponding place, h is the altitude of the target, the altitude is a preset value, different heights can be set automatically according to different targets, and the unknown number is the distance R from the camera to the target and the vector of the direction vector from the target to the geocentric under the GPS coordinate systemSpecific practical problems are shown in fig. 3, where the corresponding variables are the same as those referred to herein. Specifically, a triangle cosine theorem formula is utilized:
(R+h)2＝r2+l2-2rlcos(αi)
in the formula of alphaiThe angle between the reverse vector from the camera to the target and the vector from the center of mass to the geocentric of the satellite can be directly calculated because the two quantities are known quantities. Reuse formulaCan obtain the vector of the direction vector from the target to the geocentric under the GPS coordinate systemAnd finally, obtaining a coordinate vector of the target under a GPS coordinate system: li＝(R+h)·lto。
(5) Based on the target speed and direction thresholds, extracting real moving target connected domains, calculating the geographical position of the moving target by using the GPS coordinates, calculating the moving speed and direction of the moving target, setting the moving target speed and direction thresholds, judging the calculated effective connected domains meeting the threshold conditions as the moving target, and otherwise, eliminating the connected domains not meeting the conditions, realizing moving target detection and speed estimation. The method comprises the following specific steps:
the first step is as follows: and after the coordinate calculation of the effective connected domain centroid of each frame under the GPS coordinate system is completed, sequencing the connected domains extracted from the current frame image according to the area and the energy of the connected domains. Sorting according to the area of the communication domains, placing the communication domains with the largest area at the forefront of the queue, comparing the sums of pixel gray values of the communication domains with the same area, and sorting according to the size; if the current ordering is the same, maintaining the current ordering, and storing the coordinates, area, energy and the like of each connected domain under the respective connected domain number. (according to the size of the storage space, the information of the connected domain of the image with the fixed frame number N frames is stored in a rolling way, the information of the initial frame is covered after the frame number is exceeded, and the like) from the second frame, the second step to the fifth step are executed;
the second step is that: and based on the connected domain information of the image shot by the current frame, performing connectivity detection of the connected domains (the current connected domains) one by one and the connected domains (the detected connected domains) one by one frame with the connected domain information stored before. And respectively executing the third step and the fourth step according to the length of the detected communication domain.
If the length of the detected connected domain is 1, executing a third step: making a determination based only on the speed of motion between the two connected domains; if the length of the detected connected domain is larger than 1, executing a fourth step: judging based on the movement speed and the movement direction between the two connected domains;
the third step: when the length of the detected connected domain is only 1, only one criterion of the movement speed between the two connected domains can judge the connectivity of the two connected domains. And obtaining the displacement between the two recorded coordinates under the GPS coordinate system by utilizing the difference between the two coordinates, then obtaining the difference between the frame number of the detected connected domain and the current frame by utilizing the frame number of the detected connected domain, obtaining the time interval between the two frames by utilizing the exposure interval time T, and calculating the corresponding movement speed of the two connected domains. And comparing the calculated movement speed with a set target movement speed threshold. (for example, the movement speed interval of the airplane is preferably 200 m/s-800 m/s, and a certain error tolerance is introduced, preferably set at 90% -110%, the upper limit of the movement speed threshold of the final target is preferably 880m/s, and the lower limit is preferably 180 m/s). when the movement speed is not in the threshold interval, the second step is skipped to continue to execute the loop; when the movement speed accords with the interval, preferably adding 1 to the length of the current connected domain, transferring all information of the detected connected domain to the position of the current connected domain, deleting all contents stored in the detected connected domain, storing the calculated movement speed, and calculating and storing the movement direction vectors corresponding to the two connected domains. And jumping back to the second step, changing the current connected domain into the next connected domain, and entering a new cycle.
The fourth step: when the length of the detected connected domain is larger than 1, the connectivity of the two connected domains can be judged according to the movement speed and the direction of the connected domains. Extracting the connected domain information of the latest frame in the detected connected domain, and combining the extracted connected domain information with the detected connected domain information according to the third stepAnd calculating corresponding motion speed and motion direction vectors. Comparing the calculated movement speed with a set target movement speed threshold, and when the movement speed is within a threshold interval: and performing point multiplication operation on the calculated motion direction vector and the motion direction vector stored in the detected connected domain to obtain a cosine value of an included angle between the two motion direction vectors, and comparing the cosine value with a set cosine threshold. (for example, it is set that the change of the motion direction of the moving object in a short time is not large, so it is considered that the included angle of the motion directions should preferably not exceed 20 ° between multiple frames, the corresponding cosine value is preferably 0.94, that is, the cosine threshold is preferably set to 0.94, when the dot product result of two motion direction vectors is less than the threshold, it is considered that the current connected domain is not the object corresponding to the detected connected domain, and it is determined that the two are not connected) when the comparison result is that the two are not connected, the second step is skipped to continue to execute the loop; and when the comparison result is that the two are communicated, the length of the current connected domain is changed into the length of the detected connected domain plus 1, all information of the detected connected domain is stored at the position of the current connected domain, all contents stored in the detected connected domain are deleted, the calculated movement speed and movement direction are weighted and calculated based on the length of the detected connected domain and the movement speed and movement direction stored in the detected connected domain, and the calculated movement speed and movement direction are stored in the current connected domain. And jumping back to the second step, changing the current into the next connection domain, and entering a new cycle. The principle of connectivity determination is shown in FIG. 4, where uijAnd the numbers of i and j respectively represent the j-th connected domains in the ith frame image, u represents the x-axis coordinate of the centroid of the corresponding connected domain on the phase plane, and v represents the y-axis coordinate of the centroid of the corresponding connected domain on the phase plane.
The fifth step: and after the second step of operation is finished, all connected domains stored in the current frame are considered, and preferably, when the length of the connected domain is more than 2, a data packet with successful detection is sent to the satellite borne computer. The data comprises a successful detection mark, the detected movement speed of the target, the movement direction vector of the target in the GPS coordinate system, the position coordinate of the target in the GPS coordinate system and other information, and the information is used for the satellite-borne computer to judge.
The invention carries out shooting through a prototype and realizes the real-time operation of the algorithm, and the result shows that the method can accurately extract the suspicious moving target in the image in real time and calculate the size and the direction of the moving speed of the target.
The invention is not described in detail and is within the knowledge of a person skilled in the art.
Claims (10)
1. A satellite-borne moving target detection and speed estimation method is characterized by comprising the following steps:
(1) carrying out background suppression processing on a plurality of continuously shot satellite-borne earth observation images to obtain background-removed images;
(2) performing threshold segmentation on the background-removed image in the step (1), then performing connected domain extraction, and removing part of false target connected domains to obtain effective connected domains;
specifically, threshold segmentation is carried out on the background-removed image by using a self-adaptive threshold method, and coordinates and gray values of all pixels higher than a threshold value are stored; extracting connected domains from the stored pixels based on a four-connected domain method, and eliminating partial false target connected domains according to the area and length-width ratio threshold values of the extracted connected domains to obtain effective connected domains;
(3) carrying out centroid extraction on the effective connected domain in the step (2) and solving an accurate coordinate of the centroid of the effective connected domain on the phase plane of the shooting camera;
(4) the coordinate conversion from the accurate coordinate of the centroid on the phase plane of the shooting camera to the GPS coordinate system is completed by utilizing the coordinate conversion matrix and the camera optical axis pointing parameter, and the effective connected domain centroid coordinate under the GPS coordinate system is obtained;
(5) obtaining a geographic position corresponding to the centroid of the corresponding effective connected domain according to the centroid coordinates of the effective connected domain in the GPS coordinate system, calculating the corresponding movement speed and direction of the geographic position, extracting a real target by using a method of comparing multi-frame image connected domains, extracting the connected domain corresponding to the real moving target, finally realizing moving target detection and realizing speed estimation of the moving target.
2. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: the continuously shot multiframe satellite-borne earth observation images in the step (1) require that: the early warning camera installed on an orbiting satellite with an orbit height of 500km shoots, and the view field angle of the early warning camera is 4 degrees.
3. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: the method comprises the following steps of (1) carrying out background suppression processing on a plurality of continuously shot multi-frame satellite-borne earth observation images to obtain background-removed images, wherein the background-removed images specifically comprise the following steps: and respectively processing each frame of image, and performing background suppression to obtain a background-removed image.
4. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: extracting a connected domain, specifically: and (3) roughly extracting all potential target connected domains based on a four-connected domain method, calculating the area of each connected domain and the maximum length-width ratio of the connected domains, comparing the area and the maximum length-width ratio with a set fixed threshold, and eliminating false target connected domains which do not meet the requirements to obtain the remaining connected domains as effective connected domains.
5. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: the method comprises the following steps of converting the coordinates of a communication domain centroid phase plane into the coordinates of a GPS coordinate system, specifically: using the transformation matrix C from the camera body coordinate system to the GPS coordinate system, the satellite displacement between the time and the earth centerRepresenting in a GPS coordinate system, corresponding to the position earth radius R; according to the direction vector of the optical path direction corresponding to the accurate coordinate of the connected domain on the phase surface in the camera body coordinate systemBy the formula:
in the formulaThe direction vector of the corresponding light path of the pixel under the GPS coordinate system is taken as the direction vector of the pixel; by the formula:
in the formulaThe direction vector of the object corresponding to the connected domain relative to the earth center under the GPS coordinate system, h is the height of the target, r is the distance from the satellite to the object corresponding to the connected domain, and the expression of r and the direction vector under the GPS coordinate system is solved according to the formula
6. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: the method for extracting the real target by comparing the connected domains of the multi-frame images specifically comprises the following steps:
(1) sequencing the communication domains extracted from each frame of image according to the areas and the energies of the communication domains; specifically, sorting is carried out according to the area of the connected domains, the largest area is arranged at the front of the queue, and for the connected domains with the same area, the sum of pixel gray values of the connected domains is compared and sorted according to the size;
(2) and starting from the second frame, carrying out frame-by-frame connected domain matching with the connected domains in all the previous frames one by one, and storing the successfully matched connected domain combination in the corresponding connected domain of the current frame.
7. The method according to claim 6, wherein the method comprises: and sequencing the communication domains extracted from each frame of image according to the areas and the energies of the communication domains, specifically, sequencing according to the areas of the communication domains, placing the communication domains with the largest areas at the forefront of the queue, comparing the sums of the gray values of the pixels of the communication domains with the same areas, and sequencing according to the sizes.
8. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: the suspicious target area is accurately extracted under the condition of small calculated amount, and more accurate calculation results of speed and direction can be provided, so that the false alarm rate is reduced, and the detection success rate is improved.
9. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: the method is used in a satellite-borne early warning camera, and realizes on-orbit real-time moving target detection and speed estimation.
10. The method for detecting and estimating the speed of the satellite-borne moving target according to claim 1, wherein the method comprises the following steps: suspicious target information is provided to the satellite in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910684818.2A CN110617802A (en) | 2019-07-26 | 2019-07-26 | Satellite-borne moving target detection and speed estimation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910684818.2A CN110617802A (en) | 2019-07-26 | 2019-07-26 | Satellite-borne moving target detection and speed estimation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110617802A true CN110617802A (en) | 2019-12-27 |
Family
ID=68921577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910684818.2A Pending CN110617802A (en) | 2019-07-26 | 2019-07-26 | Satellite-borne moving target detection and speed estimation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110617802A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111429479A (en) * | 2020-03-26 | 2020-07-17 | 中国科学院长春光学精密机械与物理研究所 | Space target identification method based on image integral mean value |
| CN112669297A (en) * | 2020-12-31 | 2021-04-16 | 中国科学院长春光学精密机械与物理研究所 | Target detection method |
| CN112883865A (en) * | 2021-02-09 | 2021-06-01 | 北京深蓝长盛科技有限公司 | Ball-bearing breakthrough event identification method and device, computer equipment and storage medium |
| CN116740332A (en) * | 2023-06-01 | 2023-09-12 | 南京航空航天大学 | Method for positioning center and measuring angle of space target component on satellite based on region detection |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101825475A (en) * | 2010-05-17 | 2010-09-08 | 哈尔滨工业大学 | Image motion compensation method for space optical remote sensor |
| WO2015199502A1 (en) * | 2014-06-26 | 2015-12-30 | 한국과학기술원 | Apparatus and method for providing augmented reality interaction service |
| CN106709944A (en) * | 2016-12-14 | 2017-05-24 | 上海微小卫星工程中心 | Satellite remote sensing image registration method |
| CN106709914A (en) * | 2017-01-05 | 2017-05-24 | 北方工业大学 | SAR image ship detection false alarm eliminating method based on two-stage DEM sea-land reservoir |
| KR20170125716A (en) * | 2016-05-04 | 2017-11-15 | 임재형 | Apparatus for determining position information of object and method thereof |
| CN107504966A (en) * | 2017-07-10 | 2017-12-22 | 北京控制工程研究所 | There is the method that nautical star asterism extracts under cloud environment in a kind of daytime |
| US20180172822A1 (en) * | 2016-12-21 | 2018-06-21 | The Boeing Company | Method and apparatus for multiple raw sensor image enhancement through georegistration |
| CN108876807A (en) * | 2018-05-31 | 2018-11-23 | 长春博立电子科技有限公司 | A kind of real-time piggyback satellite image motion object detection tracking |
| CN109146963A (en) * | 2017-06-13 | 2019-01-04 | 南京鑫和汇通电子科技有限公司 | One kind being based on the matched image position offsets detection method of swift nature |
-
2019
- 2019-07-26 CN CN201910684818.2A patent/CN110617802A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101825475A (en) * | 2010-05-17 | 2010-09-08 | 哈尔滨工业大学 | Image motion compensation method for space optical remote sensor |
| WO2015199502A1 (en) * | 2014-06-26 | 2015-12-30 | 한국과학기술원 | Apparatus and method for providing augmented reality interaction service |
| KR20170125716A (en) * | 2016-05-04 | 2017-11-15 | 임재형 | Apparatus for determining position information of object and method thereof |
| CN106709944A (en) * | 2016-12-14 | 2017-05-24 | 上海微小卫星工程中心 | Satellite remote sensing image registration method |
| US20180172822A1 (en) * | 2016-12-21 | 2018-06-21 | The Boeing Company | Method and apparatus for multiple raw sensor image enhancement through georegistration |
| CN106709914A (en) * | 2017-01-05 | 2017-05-24 | 北方工业大学 | SAR image ship detection false alarm eliminating method based on two-stage DEM sea-land reservoir |
| CN109146963A (en) * | 2017-06-13 | 2019-01-04 | 南京鑫和汇通电子科技有限公司 | One kind being based on the matched image position offsets detection method of swift nature |
| CN107504966A (en) * | 2017-07-10 | 2017-12-22 | 北京控制工程研究所 | There is the method that nautical star asterism extracts under cloud environment in a kind of daytime |
| CN108876807A (en) * | 2018-05-31 | 2018-11-23 | 长春博立电子科技有限公司 | A kind of real-time piggyback satellite image motion object detection tracking |
Non-Patent Citations (2)
| Title |
|---|
| 李劲东: "《卫星遥感技术》", 31 March 2018 * |
| 王苗苗，毛晓艳，魏春岭: "空间小目标的检测算法", 《空间控制技术与应用》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111429479A (en) * | 2020-03-26 | 2020-07-17 | 中国科学院长春光学精密机械与物理研究所 | Space target identification method based on image integral mean value |
| CN111429479B (en) * | 2020-03-26 | 2022-10-11 | 中国科学院长春光学精密机械与物理研究所 | Space target identification method based on image integral mean value |
| CN112669297A (en) * | 2020-12-31 | 2021-04-16 | 中国科学院长春光学精密机械与物理研究所 | Target detection method |
| CN112669297B (en) * | 2020-12-31 | 2022-05-27 | 中国科学院长春光学精密机械与物理研究所 | Target detection method |
| CN112883865A (en) * | 2021-02-09 | 2021-06-01 | 北京深蓝长盛科技有限公司 | Ball-bearing breakthrough event identification method and device, computer equipment and storage medium |
| CN112883865B (en) * | 2021-02-09 | 2024-01-19 | 北京深蓝长盛科技有限公司 | Identification method and device for break-through event with ball, computer equipment and storage medium |
| CN116740332A (en) * | 2023-06-01 | 2023-09-12 | 南京航空航天大学 | Method for positioning center and measuring angle of space target component on satellite based on region detection |
| CN116740332B (en) * | 2023-06-01 | 2024-04-02 | 南京航空航天大学 | Method for positioning center and measuring angle of space target component on satellite based on region detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111862126B (en) | Non-cooperative target relative pose estimation method combining deep learning and geometric algorithm | |
| US10885328B2 (en) | Determination of position from images and associated camera positions | |
| CN110617802A (en) | Satellite-borne moving target detection and speed estimation method | |
| CN108734103B (en) | Method for detecting and tracking moving target in satellite video | |
| US10694148B1 (en) | Image-based navigation using quality-assured line-of-sight measurements | |
| CN109631911B (en) | Satellite attitude rotation information determination method based on deep learning target recognition algorithm | |
| EP1949339A2 (en) | Tracking using an elastic cluster of trackers | |
| KR101941878B1 (en) | System for unmanned aircraft image auto geometric correction | |
| CN109214254B (en) | Method and device for determining displacement of robot | |
| CN108334885A (en) | A kind of video satellite image space object detection method | |
| CN106022266A (en) | Target tracking method and target tracking apparatus | |
| CN109708627B (en) | Method for rapidly detecting space dynamic point target under moving platform | |
| US8948446B2 (en) | Vision based zero velocity and zero attitude rate update | |
| CN115164900A (en) | Omnidirectional camera based visual aided navigation method and system in urban environment | |
| CN117218054A (en) | Space target detection method under fixed star background | |
| CN116862832A (en) | Three-dimensional live-action model-based operator positioning method | |
| CN114937007B (en) | Unmanned aerial vehicle visual landing runway starting line detection method based on corner features | |
| CN114418913B (en) | ISAR and infrared image pixel level fusion method based on wavelet transformation | |
| CN115856885A (en) | Ship target continuous tracking method based on low-orbit SAR satellite constellation | |
| EP4206607A1 (en) | Imaging system and method for attitude determination | |
| CN116755169B (en) | Small target detection method and system based on star map identification and brightness priori information | |
| CN111508025B (en) | Three-dimensional position estimation device and program | |
| CN112668563B (en) | Method and related device for realizing movement track calculation based on video image | |
| US20220383515A1 (en) | Digital video computing system for vehicle | |
| CN116012450A (en) | Multi-target pose real-time online estimation method, system, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191227 |
|
| RJ01 | Rejection of invention patent application after publication |