CN116883861A - Port large and medium-sized ship activity identification method and system for microsatellite on-orbit application - Google Patents
Port large and medium-sized ship activity identification method and system for microsatellite on-orbit application Download PDFInfo
- Publication number
- CN116883861A CN116883861A CN202310876541.XA CN202310876541A CN116883861A CN 116883861 A CN116883861 A CN 116883861A CN 202310876541 A CN202310876541 A CN 202310876541A CN 116883861 A CN116883861 A CN 116883861A
- Authority
- CN
- China
- Prior art keywords
- ship
- remote sensing
- detection
- area
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 230000000694 effects Effects 0.000 title claims abstract description 40
- 108091092878 Microsatellite Proteins 0.000 title claims abstract description 23
- 101150080778 INPP5D gene Proteins 0.000 title claims description 270
- 238000001514 detection method Methods 0.000 claims abstract description 184
- 238000012937 correction Methods 0.000 claims abstract description 59
- 238000012545 processing Methods 0.000 claims abstract description 36
- 230000008859 change Effects 0.000 claims abstract description 9
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 230000011218 segmentation Effects 0.000 claims description 25
- 238000005520 cutting process Methods 0.000 claims description 22
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 20
- 238000003860 storage Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 238000003384 imaging method Methods 0.000 claims description 11
- 239000007787 solid Substances 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 9
- 238000013461 design Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 claims description 8
- 238000013473 artificial intelligence Methods 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 230000005855 radiation Effects 0.000 claims description 6
- 238000003064 k means clustering Methods 0.000 claims description 4
- 238000012938 design process Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 7
- 230000005540 biological transmission Effects 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 241000208479 Anagallis arvensis Species 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000003653 coastal water Substances 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000008521 reorganization Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Remote Sensing (AREA)
- Astronomy & Astrophysics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a system for identifying the activities of large and medium-sized ships in a port for the on-orbit application of microsatellites, wherein the method comprises the steps of S1, acquiring remote sensing images in a port area; s2, carrying out on-orbit preprocessing on remote sensing images in a port area; s3, acquiring remote sensing images of the wharf and the near-shore area; s4, geometric fine correction of remote sensing images of the wharf and the near-shore area; s5, slicing the remote sensing image of the wharf and the near-shore area after geometric fine correction; s6, primarily detecting and identifying ship targets; s7, final detection and identification of ship targets; s8, comparing the multi-time phase detection recognition results; s9, confirming the change condition of the ship; s10, the ship activities are identified again. The advantages are that: the method can ensure accurate detection and identification of the ship target of interest, and can realize identification of the ship target activity of interest by processing and comparing multi-temporal remote sensing images obtained when the satellite revisits for a plurality of times.
Description
Technical Field
The invention relates to the technical field of intelligent remote sensing interpretation, in particular to a method and a system for identifying activities of large and medium-sized ships in a port for on-orbit application of microsatellites.
Background
The satellite remote sensing mode is used for carrying out large and medium-sized ship activity identification in the port, such as port entry, port departure, coastal navigation and the like, and is an important means for grasping the running condition of the port and carrying out ship behavior analysis.
Currently, earth observation remote sensing satellite systems are developed towards miniaturization and clustering. The continuous increase of the number of satellites and the iterative optimization of constellation configuration enable the coverage frequency of a satellite system to important ground targets to be continuously increased, the revisit period to be continuously shortened, and the space-time resolution capability and the system task capability of the satellite system to be continuously improved. However, the method is limited by the task mode of 'in-orbit observation-ground processing-in-orbit execution' of the current remote sensing satellite, and obvious shortboards exist in the aspects of remote sensing information processing and application, so that the advantages of a new generation earth observation satellite system are difficult to be exerted to the greatest extent. On one hand, more satellite image data need to be downloaded to the ground for processing, and satellite-ground data transmission pressure is higher and higher; on the other hand, intelligent remote sensing interpretation technology is not mature enough, the intelligent degree of remote sensing image processing and application is not enough, the work needing to be manually participated is increased or not, and professional image interpretation force has a shortage trend; on the other hand, the remote sensing information acquisition, analysis and application processes are long, and the information acquisition and application efficiency is low.
In recent years, the rise of artificial intelligence has driven rapid development and technological innovation in the fields of computer vision, natural language processing and the like. The intelligent satellite system combines artificial intelligence with artificial experience, fully utilizes the superposition combination advantages of high-speed efficiency of a machine and high cognition of human beings, performs works such as intelligent remote sensing image interpretation, information reorganization and the like on orbit, develops an intelligent satellite system which does not depend on a ground station and has on-orbit autonomous intelligent processing capability, can greatly reduce satellite-to-ground data transmission quantity, greatly lightens personnel workload, improves the intelligent level of the satellite system and meets the requirements of quick processing and high-timeliness application tasks.
In summary, based on the artificial intelligence method, the on-orbit intelligent remote sensing interpretation and information acquisition technology is developed, the intelligence and timeliness of remote sensing information processing and application are improved, and the method is suitable for urgent requirements and practical requirements of the development of a new generation earth observation satellite system.
The current remote sensing image processing and application are mostly carried out on the ground, and a large amount of remote sensing image data needs to be downloaded to the ground through a satellite-ground data transmission link. However, the satellite-to-ground data transmission link window is very limited, and for a single ground station, there may be only a few minutes of data transmission time per track turn, and remote sensing image data cannot be downloaded at the time. With the development of earth observation systems towards the construction of large-scale constellations, the mode brings huge satellite-earth data transmission pressure, and also severely limits the application of satellite data in high-timeliness demand tasks. Meanwhile, the specific processing mode of the remote sensing image data, such as in-port ship target interpretation and ship activity recognition, still depends on professional interpretation personnel to manually complete the processing mode. Professional interpretation personnel are deeply involved in a large number of repeated basic interpretation works, so that the efficiency is low and the benefit is low. Based on the above, aiming at the task of acquiring the activity information of large and medium-sized ships in the harbor, it is necessary to provide a method and a system for identifying the activities of the ships in the harbor for the application of microsatellites in orbit.
Disclosure of Invention
The invention aims to provide a method and a system for identifying activities of large and medium-sized ships in a port for on-orbit application of microsatellites, so as to solve the problems in the prior art.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a method for identifying the activities of large and medium-sized ships in a port for on-orbit application of microsatellites comprises the following steps,
s1, acquiring remote sensing images of port areas:
the satellite completes remote sensing imaging of the port area according to longitude and latitude geographic coordinates of the target port, the self-posture of the satellite and the imaging load width, and remote sensing images of the port area are obtained;
s2, carrying out on-orbit preprocessing on remote sensing images in port areas:
performing radiation correction and geometric correction on the remote sensing image of the port area on the satellite to obtain a corrected remote sensing image of the port area;
s3, acquiring remote sensing images of wharfs and near-shore areas:
cutting out remote sensing images of the wharf and the near-shore area from the corrected remote sensing images of the harbour area according to geographic longitude and latitude information of minimum envelopes of the wharf and the near-shore area of the harbour prestored on the satellite;
s4, geometric fine correction of remote sensing images of wharfs and near-shore areas:
according to the pre-stored reference images of the wharf and the near-shore area of the harbour on the satellite, performing geometric fine correction on the reference images on the remote sensing images of the wharf and the near-shore area to obtain remote sensing images of the wharf and the near-shore area after the geometric fine correction;
S5, slicing the remote sensing image of the wharf and the near-shore area after geometric fine correction:
according to the port and dock facility layout, a remote sensing image cutting scheme is designed, and the remote sensing images of the dock and the near-shore area after geometric fine correction are sliced based on the remote sensing image cutting scheme;
s6, preliminary detection and identification of ship targets:
sequentially sending the slice images into a ship target detection and identification model to obtain a ship target preliminary detection and identification result;
s7, final detection and identification of ship targets:
combining the primary detection and identification results of the ship targets to obtain the positions of the primary detection and identification results of the ship targets in remote sensing images of wharfs and near-shore areas after geometric fine correction, and performing duplicate removal processing on repeated primary detection and identification results in overlapping areas of the slice images; removing the preliminary detection and identification result of the ship target with errors according to the pre-stored land area segmentation information on the satellite, and obtaining the final detection and identification result of the ship target;
s8, comparing the multi-time phase detection recognition results:
comparing the final detection and identification result of the ship target with the final detection and identification result of the ship target obtained in the last satellite revisit by combining with the geographic position of the wharf to obtain the activity identification result of the ship target;
S9, confirming ship change conditions:
determining whether the wharf ship changes according to the ship target activity recognition result, if so, marking the change condition in remote sensing images of the wharf and the near-shore area, and storing the images;
s10, identifying ship activities again:
the next time the satellite revisits to the port location, S1 to S9 are performed again.
Preferably, step S4 specifically includes,
s41, respectively extracting SIFT angular points in remote sensing images of the wharf and the near-shore area and reference images of the wharf and the near-shore area;
s43, matching SIFT corner points of the two images by using a KNN method;
s44, estimating projection transformation model parameters of the dock and the near-shore area images relative to the dock and near-shore area reference images by using a RANSAC method, and performing projection transformation on the dock and near-shore area remote sensing images by using the projection transformation model;
s45, correcting geographical position information corresponding to each pixel in the remote sensing images of the dock and the near-shore area after projection transformation by using geographical position information of the reference images of the dock and the near-shore area, so as to realize geometric fine correction and obtain the remote sensing images of the dock and the near-shore area after fine correction.
Preferably, the remote sensing image cutting scheme comprises the following principles:
(1) The sizes of the slice images are the same;
(2) The cutting size of each slice image is between 960×960 pixels and 1280×1280 pixels, and should be a multiple of 32;
(3) The sliced image should cover each wharf completely, the land area of the wharf is selected as a boundary, the boundary does not span the berth range of the ship on the wharf, and the berth area is ensured not to be divided;
(4) The adjacent slice images have a certain overlapping area, and the width of the overlapping area is not lower than the width of the horizontal envelope line of the nearby resident ship.
Preferably, the ship target detection and identification model is a deep learning network structure and model parameter file obtained after ground pre-training, and can realize intelligent detection and fine identification of the ship target of interest;
constructing a data set of a ship, wherein the data set comprises a ship sample image, the position of a ship target in the image and ship model type information; dividing the data set into a training set and a testing set; and constructing a ship target rotation detection and identification network based on the combination of YOlOv5 and the annular smooth tag, and training and testing the ship target rotation detection and identification network by utilizing a training set and a testing set to obtain a ship target detection and identification model.
Preferably, the ship target detection and identification model adopts a K-means clustering method, combines the scale attributes of all ship target objects, and designs different candidate envelope frames on the feature graphs of three types of output layers; the specific design process is that,
s61, acquiring size data of all N types of civil ship samples and size data of all M types of large-scale ship samples;
s62, visually representing the acquired size data of the civil ship sample and the large-scale ship sample as two-dimensional data points;
s63, clustering size data of various civil ship samples into 3 clusters by using a K-means method to obtain 3*N civil ship sample points; carrying out de-duplication treatment on the size data of various large-scale ship samples, wherein each large-scale ship only retains 3 sample size data, and 3*M large-scale ship sample points are obtained;
s64, combining 3*N civil ship sample points and 3*M large ship sample points;
s65, clustering the combined sample point data into 12 clusters by using a K-means method, namely generating 12 candidate envelope frames; according to the root mean square value of the length and the width of the candidate envelope frames, sequencing 12 generated candidate envelope frames from small to large; the 4 candidate envelope frames with the smallest size are used in the output layer with the corresponding size of 128 multiplied by 128; 4 candidate envelope frames with middle sizes are used in an output layer with the corresponding size of 64 multiplied by 64; the 4 largest candidate envelope frames are used for the output layer of the corresponding scale 32×32.
Preferably, in step S7, the primary detection and recognition results of each ship target are combined, and the positions of the primary detection and recognition results of each ship target in the remote sensing images of the wharf and the near-shore area after the geometric fine correction are obtained specifically,
a1, when the slice image range is completely in the remote sensing images of the wharf and the near-shore area after the geometric fine correction, adding the position coordinates of the rectangular envelope frame in the primary detection result of the ship target to the coordinates of the origin of the slice image in the remote sensing images of the wharf and the near-shore area after the geometric fine correction, and obtaining the coordinates of the primary detection and identification result of the ship target in the remote sensing images of the wharf and the near-shore area after the geometric fine correction;
a2, when the range of the slice image is close to the edge of the geometrically refined wharf and near-shore area remote sensing image, and the partial area of the slice image exceeds the boundary of the geometrically refined wharf and near-shore area remote sensing image, the part exceeding the area is fully filled with a black image, then the coordinates of the origin of the slice image with respect to the origin of the geometrically refined wharf and near-shore area remote sensing image are updated, and then the coordinates of the rectangular envelope frame in the ship target primary detection result plus the coordinates of the origin of the slice image in the geometrically refined wharf and near-shore area remote sensing image are added, so that the coordinates of the ship target primary detection recognition result in the geometrically refined wharf and near-shore area remote sensing image can be obtained.
Preferably, in step S7, the duplicate removal processing of the repeated preliminary detection and identification result of the existence of the overlapping region of the slice images specifically includes,
b1, importing all ship targets into a primary detection and identification result;
b2, reading a ship target preliminary detection and identification result A which is not verified;
b3, matching the ship target preliminary detection recognition result A with the rest ship target preliminary detection recognition results one by one, calculating the overlapping ratio of the two result enveloping rectangular frames, and when the overlapping ratio is smaller than a preset threshold value, not processing, and continuously matching and comparing the ship target preliminary detection recognition result A with the rest ship target preliminary detection recognition results; if the cross ratio is greater than or equal to a preset threshold value, retaining a primary detection and identification result of one ship target according to a judgment rule;
b4, if the primary detection and identification result A of the ship target is not deleted, returning to the B3; if the ship target preliminary detection recognition result A is deleted, judging whether all the rest ship target preliminary detection recognition results are checked, and if the ship target preliminary detection recognition results are not checked, returning to the step B2; and if all the results are verified, outputting a ship detection primary identification result after the duplication removal treatment.
Preferably, the decision rule in step B3 is,
if the areas of the enveloping rectangular frames of the primary detection and identification results of the two ship targets are different, retaining the primary detection and identification results of the ship targets with large areas, and deleting the primary detection and identification results of the other ship target overlapped with the primary detection and identification results of the ship targets; if the enveloping rectangular areas of the primary detection and identification results of the two ship targets are the same, the primary detection and identification results of the ship targets with high confidence are reserved, and the primary detection and identification results of the other ship targets overlapped with the primary detection and identification results of the ship targets are deleted.
Preferably, in step S7, the primary detection and identification result of the ship target with errors is removed specifically includes the following content,
c1, finishing binarization distinguishing treatment of land and water area ranges through manual design to form a land and water area segmentation map; performing binarization processing on the envelope rectangular frame range of the preliminary detection and identification result of each ship target after the duplication removal to form a ship target envelope segmentation map;
c2, representing land and water area segmentation maps and each ship target envelope segmentation map as a matrix;
and C3, multiplying the matrix corresponding to each ship target envelope segmentation map with the matrix corresponding to the land and water area segmentation map, summing the elements of the result matrix, judging that the ship target preliminary detection and identification result after the duplication removal is an error detection result if the sum result is larger than the preset percentage of the ship target envelope segmentation map, deleting the error detection result, and otherwise, reserving the ship target preliminary detection and identification result after the duplication removal.
The invention also aims to provide a port large and medium-sized ship activity recognition system facing microsatellite on-orbit application, which comprises,
high-speed data acquisition and storage interface unit: the system is used for providing high-speed interface support for acquisition and reading of original remote sensing image data and providing high-speed interface support for data reading and writing and storage between the embedded artificial intelligent processing unit and the memory and the solid state disk;
embedded artificial intelligence processing unit: consisting of a CPU and a GPU on which a Linux kernel operating system runs for implementing the method according to any of claims 1 to 9;
memory: the method comprises the steps of caching read-in original remote sensing image data;
solid state disk: the method is used for storing basic files required in the running process of the method and related images and results generated after the method is run;
a power supply control unit: the power supply system is used for carrying out voltage adjustment on an external power supply provided by a satellite platform end and providing required power supply voltage and current for each unit in the system.
The beneficial effects of the invention are as follows: 1. compared with the traditional image change detection method, the method can cope with the characteristics and difficulties that the background of the offshore port environment is complex, the illumination conditions are various (under different time in daytime), the angles of ship targets in remote sensing images can be random, multiple ship targets can be densely parked and the like, can ensure accurate detection and identification of the ship targets of interest, and can realize identification of the ship target activities of interest by processing and comparing multi-time-phase remote sensing images obtained when satellites are revisited for multiple times. 2. Compared with the traditional complicated data processing and application processes such as remote sensing data downloading, ground processing, uplink control and the like, the method can greatly compress the remote sensing image data transmission quantity for carrying out the tasks and the time consumption required by task completion. 3. The system has simple composition and lower cost, can provide a modularized solution for the microsatellite to realize the on-orbit interpretation application of the port ships, and has good portability and compatibility. 4. The timeliness and the intelligence of the remote sensing task are improved, the satellite-ground data transmission pressure is greatly reduced, and the comprehensive benefit of the remote sensing task is remarkably improved.
Drawings
FIG. 1 is a flow chart of a method of identifying ship activity in an embodiment of the invention;
FIG. 2 is a diagram showing the contrast of an original remote sensing image, a preprocessed image, a cropped image, and a geometrically refined corrected image in an embodiment of the present invention;
FIG. 3 is a flow chart of the geometric rectification operation of the newly acquired image by using the reference image according to the embodiment of the invention;
FIG. 4 is a diagram of a personalized cutting pattern in an embodiment of the invention;
FIG. 5 is a block diagram of a ship target detection recognition model in an embodiment of the invention;
FIG. 6 is a flow chart of a method for designing a candidate block for detecting and identifying a network in an embodiment of the invention;
FIG. 7 is a view of the reduction of slice images to quay and near shore areas in accordance with an embodiment of the present invention;
FIG. 8 is a flow chart of a ship identification result process in detecting and identifying overlapping areas in an embodiment of the invention;
FIG. 9 is a view of the amphibious segmentation of dock and near shore area images after fine correction in an embodiment of the invention;
fig. 10 is a block diagram of a ship activity recognition system in an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the detailed description is presented by way of example only and is not intended to limit the invention.
In the embodiment, a method and a system for identifying the activities of large and medium-sized ships in a port for on-orbit application of microsatellites are provided, and according to the characteristics of relatively fixed berths and types of the ships on the code heads of the port, the method for detecting and identifying the targets of the ships in visible light remote sensing images is designed to realize the detection and fine identification of the targets of the ships in the visible light remote sensing images, fully exert the high-energy-efficiency edge calculation force of the remote sensing low-orbit microsatellites, autonomously develop the identification tasks of the activities of the large and medium-sized ships in the port in orbit, and overcome the defects of large data transmission requirement and low timeliness of information generation in the traditional task mode.
Example 1
In this embodiment, by combining fine-grained target detection and identification with priori geographic knowledge, a method for accurately judging and identifying the targets of the intraport ships and the activities of the intraport ships, which can be deployed on a low-orbit remote sensing small satellite, is provided, as shown in fig. 1, and specifically comprises the following steps:
1. and (3) acquiring remote sensing images in a port area:
the satellite completes remote sensing imaging of the port area according to longitude and latitude geographic coordinates of the target port (namely priori guiding information of satellite imaging photographing), the self-posture of the satellite, imaging load breadth and the like, and remote sensing images of the port area are obtained, as shown in fig. 2 (a). The imaging area only ensures coverage of the port area, and the image data amount is controlled as much as possible.
In this embodiment, the rough geographic coordinates of the center of the satellite imaging region can be calculated from the satellite orbit positioning data and the satellite attitude. Therefore, firstly, according to satellite orbit positioning information, the satellite attitude is controlled, so that the rough position of a satellite imaging center is in the center of a port and a dock. And then, carrying out remote sensing imaging on the port area and acquiring remote sensing data. The satellite has a large breadth, and the imaging center is close to the center of the wharf, so that the complete remote sensing image of the port area can be obtained.
2. In-orbit preprocessing of remote sensing images in port areas:
performing radiation correction and geometric correction on the remote sensing image of the port area on the satellite to obtain a corrected remote sensing image of the port area, as shown in fig. 2 (b).
3. Remote sensing image acquisition in wharf and near-shore area:
and cutting out the remote sensing images of the wharf and the near-shore area from the corrected remote sensing images of the port area according to the geographical longitude and latitude information of the minimum envelopes of the wharf and the near-shore area of the port, which is stored in advance on the satellite, as shown in fig. 2 (c). The cut image includes the port and dock and a small part of the land area and water area near the port and dock.
4. Geometric fine correction of remote sensing images in wharf and near-shore areas:
and (3) performing geometric fine correction (namely relative correction) on the remote sensing images of the wharf and the near-shore area according to the reference images of the wharf and the near-shore area of the harbor stored in advance on the satellite, and obtaining the remote sensing images of the wharf and the near-shore area after the geometric fine correction, as shown in fig. 2 (d).
As shown in fig. 3, in particular, the geometric fine correction includes in particular,
1. respectively extracting SIFT angular points in remote sensing images of the wharf and the near-shore area and reference images of the wharf and the near-shore area;
3. matching SIFT corner points of the two images by using a KNN method (K adjacent algorithm);
4. estimating projection transformation model parameters of the dock and near-shore region images relative to the dock and near-shore region reference images by using a RANSAC method (random consistency sampling algorithm), and performing projection transformation on the dock and near-shore region remote sensing images by using the projection transformation model;
5. and correcting the geographical position information corresponding to each pixel in the remote sensing images of the dock and the near-shore area after projection transformation by using the geographical position information of the reference images of the dock and the near-shore area, so as to realize geometric fine correction and obtain the remote sensing images of the dock and the near-shore area after fine correction.
5. Slicing the remote sensing image of the wharf and the near shore area after geometric fine correction:
because the remote sensing images of the wharf and the near-shore area are large in size, a personalized remote sensing image cutting scheme is designed according to the wharf facility layout of the port so as to support ship target detection and identification in the slice images with more isolated cutting ranges; and slicing the remote sensing image of the wharf and the near-shore area after the geometric fine correction based on the remote sensing image cutting scheme.
On the one hand, in the personalized cutting case shown in fig. 4, considering that the range of the port and the dock and the near-shore area is generally larger, the remote sensing image is larger in size, if the whole image is sent into the deep learning detection recognition network, the calculation time consumption is exponentially increased, and the precision cannot be ensured; on the other hand, considering the identification of the ship target activity, the area range of important attention is the wharf and the coastal water area, and the detection and identification of the partial water area and the land area do not need to consume calculation power and time; therefore, a personalized cutting scheme is designed, a plurality of slices are cut out from remote sensing images of wharfs and near-shore water areas by taking the wharfs as the center, and ship target detection and identification are carried out on the slice images. Personalized cutting requires designers to be familiar with port and dock and ship berths. The personalized remote sensing image cutting scheme comprises the following principles:
(1) The sizes of the slice images are the same;
(2) The size of each slice image is proper, the method is mainly suitable for the visible light remote sensing image with the resolution of 0.8-0.5 m, and the cutting size of each slice image is 960X 960 pixels to 1280X 1280 pixels and is a multiple of 32;
(3) The sliced image takes complete coverage of each wharf as a main principle, a wharf land area is selected as a boundary as far as possible, the boundary does not span the berth range of the ship on the wharf, and the berth area is ensured not to be divided;
(4) The adjacent slice images have a certain overlapping area, and the width of the overlapping area is not lower than the width of the horizontal envelope line of the nearby resident ship.
The land area and the water area far away from the wharf are not subjected to cutting area design and ship target detection, so that the calculation load is reduced as much as possible, and the detection and identification of the wharf and the ships in the adjacent water area can be ensured.
6. Preliminary detection and identification of ship targets:
and sequentially sending the slice images into a ship target detection and identification model to obtain a ship target preliminary detection and identification result. The preliminary detection and identification result is given through a rotary rectangular envelope frame, so that the minimum envelope of the ship target area is realized.
The ship target detection and identification model is a deep learning network structure and model parameter file obtained after ground pre-training, and can realize intelligent detection and fine identification of the ship target of interest;
constructing a data set of a ship, wherein the data set comprises a ship sample image, the positions of ship targets in the image (given by four vertex coordinates of a rotary rectangular envelope frame) and ship model type information; dividing the data set into a training set and a testing set; and constructing a ship target rotation detection and identification network based on the combination of YOlOv5 and annular smooth labels (CSL), and training and testing the ship target rotation detection and identification network by utilizing a training set and a testing set to obtain a ship target detection and identification model.
As shown in fig. 5, the ship target detection and identification model is improved based on the yolo_v5 network. Compared with the YOLO_v5 network, the deep learning network for large and medium-sized ship target detection and identification mainly has the following improvements: (1) the dimension of the network input image is 1024 x 3; (2) The target rotation angle prediction is increased by utilizing the Circular Smooth Label (CSL) rotation target detection idea, and the envelope of the ship target is realized by rotating the rectangular frame; meanwhile, the method for calculating the network predicted loss value and the rotation target Non-maximum suppression algorithm (Non-Maximum Suppression, NMS) are adaptively improved by referring to the CSL rotation target detection idea. (3) The prediction output layer scale is improved to 128×128, 64×64, 32×32. (4) Different candidate envelope frames are designed on the feature graphs of the three types of output layers by using a K-means clustering method and combining the scale attributes of all ship target objects, and 4 candidate envelope frames are designed on each layer of feature graph, namely 12 candidate envelope frames; on the feature map with larger scale, the size of the corresponding candidate envelope frame is smaller; and vice versa. Each pixel point of each output layer feature map corresponds to 4 prediction result parameters (about 4 candidate envelope frames, 1 prediction result each), and each prediction result comprises K+6 parameters (K category parameters, 1 confidence coefficient parameter, 1 rotation angle parameter and 4 position coordinate parameters). Therefore, the dimensions of the three types of output layer output data are [4× (k+6) ]×128×128, [4× (k+6) ]×64×64, [4× (k+6) ]×32×32, respectively.
As shown in fig. 6, in this embodiment, the specific design process of the ship target detection and identification model for designing different candidate envelope frames by adopting the K-means clustering method is that,
1. acquiring size data of all N types of civil ship samples and size data of all M types of large ship samples;
2. the acquired size data (length and width) of the civil ship sample and the large-scale ship sample are visually represented as two-dimensional data points;
3. the civil ships have the characteristics that the number of various ships is large, various sizes (such as the size of a bulk carrier can be large or small) possibly exist in various ships, and the number of various ships is unbalanced, in order to avoid the situation that the sizes of candidate frames of the civil ships with large tendency number and large size types are not representative and universal, the size data of various civil ship samples are clustered into 3 clusters by using a K-means method, so as to obtain 3*N civil ship sample points;
the large-scale ships have the characteristics of relatively fixed various sizes and unbalanced quantity of various ships, and in order to avoid large-scale ships with large size tendency of the candidate frames, the size of the candidate frames is not representative and universal, so that the size data of various large-scale ship samples are subjected to de-duplication treatment, and only 3 sample size data are reserved for each large-scale ship, so as to obtain 3*M large-scale ship sample points.
For a certain large-sized ship with fixed size, the corresponding 3 sample size data are all of the fixed size; for a large ship, where there may be small differences in size, the corresponding 3 sample size data are arithmetic averages of different sizes.
4. Combining 3*N civil ship sample points and 3*M large ship sample points;
5. clustering the combined sample point data into 12 clusters by using a K-means method, namely generating 12 candidate envelope frames; according to the root mean square value of the length and the width of the candidate envelope frames, sequencing 12 generated candidate envelope frames from small to large; the 4 candidate envelope frames with the smallest size are used in the output layer with the corresponding size of 128 multiplied by 128; 4 candidate envelope frames with middle sizes are used in an output layer with the corresponding size of 64 multiplied by 64; the 4 largest candidate envelope frames are used for the output layer of the corresponding scale 32×32.
By the candidate frame design method, the designed candidate frames can be guaranteed to have universality and can cover and care all ship targets to be detected and identified, so that the designed candidate envelope frames are prevented from being more prone to ship targets with more samples, and the ship target detection and identification model can have relatively balanced and stronger prediction capability on all types of ships.
7. Final detection and identification of ship targets:
combining the primary detection and identification results of the ship targets to obtain the positions of the primary detection and identification results of the ship targets in remote sensing images of wharfs and near-shore areas after geometric fine correction, and performing duplicate removal processing on repeated primary detection and identification results in overlapping areas of the slice images; and removing the preliminary detection and identification result of the ship target with errors (namely the preliminary detection and identification result of the ship target on land) according to the pre-stored land and water area segmentation information on the satellite, and obtaining the final detection and identification result of the ship target.
In the present embodiment, see FIG. 7,O C1 、O C2 、O C3 O is the origin of a single detection region image (slice image) R The origin of the corrected wharf and near-shore region image is obtained.
Combining the primary detection and identification results of the ship targets to obtain the positions of the primary detection and identification results of the ship targets in the remote sensing images of the wharf and the near-shore area after the geometric fine correction,
a1, when the slice image range is completely in the remote sensing images of the wharf and the near-shore area after the geometric fine correction, adding the position coordinates of the rectangular envelope frame in the primary detection result of the ship target to the coordinates of the origin of the slice image in the remote sensing images of the wharf and the near-shore area after the geometric fine correction, and obtaining the coordinates of the primary detection and identification result of the ship target in the remote sensing images of the wharf and the near-shore area after the geometric fine correction.
Such as with O C3 As the slice image of the origin, the position coordinates of the rectangular envelope frame in the ship detection and identification result are added with O C3 At O R X R Y R And (3) obtaining the coordinates of the ship detection and identification result in the finely corrected wharf and near-shore region images.
A2, when the range of the slice image is close to the edge of the remote sensing image of the geometrically refined wharf and the near-shore region, and the partial region of the slice image exceeds the boundary of the remote sensing image of the geometrically refined wharf and the near-shore region, the excess region is complemented into a black image, and then the origin (such as O C2 And O C1 ) Regarding the origin (i.e., O) of the remote sensing image of the wharf and near-shore region after the geometric fine correction R ) The coordinate calculation of the subsequent ship detection and identification result in the geometrically precisely corrected wharf and near-shore region image is the same as A1 (namely, the ship is further subjected toAnd adding the coordinates of the rectangular envelope frame in the ship target preliminary detection result and the coordinates of the origin of the slice image in the remote sensing images of the wharf and the near-shore area after the geometric fine correction to obtain the coordinates of the ship target preliminary detection recognition result in the remote sensing images of the wharf and the near-shore area after the geometric fine correction.
In this embodiment, according to the personalized design scheme of the clipping region, a certain overlapping region exists between adjacent clipping slice images, so that the ship identification result in the overlapping region needs to be subjected to de-duplication processing. As shown in fig. 8, the duplicate removal processing for the repeated preliminary detection recognition result of the existence of the overlapping region of the slice images specifically includes the following,
B1, importing all ship target preliminary detection and identification results (four vertex coordinates of an envelope frame) which are output after each slice image is predicted by a ship target detection and identification model;
b2, reading a ship target preliminary detection and identification result A which is not verified;
b3, matching the ship target preliminary detection recognition result A with the rest ship target preliminary detection recognition results one by one, calculating the overlapping ratio of the two result enveloping rectangular frames, and when the overlapping ratio (IOU) is smaller than a preset threshold (which can be set according to actual conditions and is set to 0.3 in the embodiment), performing no processing, and continuously matching and comparing the ship target preliminary detection recognition result A with the rest ship target preliminary detection recognition results; and if the cross ratio is greater than or equal to a preset threshold value, reserving a primary detection and identification result of one of the ship targets according to a judgment rule.
The judgment rule is as follows: if the areas of the enveloping rectangular frames of the primary detection and identification results of the two ship targets are different, retaining the primary detection and identification results of the ship targets with large areas, and deleting the primary detection and identification results of the other ship target overlapped with the primary detection and identification results of the ship targets; if the enveloping rectangular areas of the primary detection and identification results of the two ship targets are the same, the primary detection and identification results of the ship targets with high confidence are reserved, and the primary detection and identification results of the other ship targets overlapped with the primary detection and identification results of the ship targets are deleted.
B4, if the primary detection and identification result A of the ship target is not deleted, returning to the B3; if the ship target preliminary detection recognition result A is deleted, judging whether all the rest ship target preliminary detection recognition results are checked, and if the ship target preliminary detection recognition results are not checked, returning to the step B2; and if all the results are verified, outputting a ship detection primary identification result after the duplication removal treatment.
In the embodiment, the primary detection and identification result of the ship target with error is removed specifically comprises the following content,
c1, performing binarization distinguishing processing on land and water areas through manual design to form a land and water area segmentation diagram, wherein as shown in fig. 9 (b), pixels corresponding to the land areas are marked as '1', and the diagram is shown as white; the pixels corresponding to the water area are marked as '0', and are shown as black in the figure; binarizing the envelope rectangular frame range of the preliminary detection and identification result of each ship target after the duplication removal to form a ship target envelope segmentation diagram, wherein as shown in fig. 9 (c), the pixels corresponding to the ship envelope range are marked as '1', the pixels are displayed as white, and the rest are marked as '0', and the pixels are displayed as black.
C2, representing land and water area segmentation maps and each ship target envelope segmentation map as a matrix;
and C3, multiplying the matrix corresponding to each ship target envelope segmentation map with the matrix corresponding to the land and water area segmentation map, summing the elements of the result matrix, if the sum result is larger than the preset percentage of the ship target envelope segmentation map (which can be set according to actual requirements, and is 60% in the embodiment), judging that the primary detection and identification result of the ship target after the duplication removal is an error detection result, deleting the error detection result, and otherwise, reserving the primary detection and identification result of the ship target after the duplication removal.
Thus, the possible land misidentification ship targets can be removed.
8. Comparing the multi-phase detection and identification results:
and comparing the final detection and identification result of the ship target with the final detection and identification result of the ship target obtained in the last satellite revisit by combining with the geographical position of the wharf, and giving the movement conditions of the ship of interest on the offshore surface and on each wharf, such as the ship leaving the wharf, the ship entering the wharf and the ship moving in the water area close to the wharf.
9. And (3) confirming ship change conditions:
And determining whether the wharf ship changes according to the ship target activity recognition result, if so, marking the change condition in remote sensing images of the wharf and the near-shore area, and storing the images.
10. Vessel activity is identified again:
the next time the satellite revisits to the port location, steps one through nine are again performed.
Example two
In this embodiment, a system for identifying activities of large and medium-sized ships in a port for on-orbit application of microsatellite is provided, as shown in fig. 10, and includes a power control unit, an embedded artificial intelligence processing unit (including a CPU and a GPU processor), a high-speed data acquisition and storage interface unit (FPGA), a memory (RAM) and a Solid State Disk (SSD), an original remote sensing image data read-in bus, an external control bus, an external data bus, and the like.
(1) The high-speed data acquisition and storage interface unit (FPGA) can provide high-speed interface support for acquisition and reading of original remote sensing image data on one hand, and can provide high-speed interface support for data reading, writing, storage and the like between the CPU or the GPU and storage equipment (memory RAM, solid state disk SSD) on the other hand.
The high-speed bus can be provided for reading in original remote sensing image data, and the internal high-speed bus can be provided for reading and writing data between the embedded artificial intelligent processing unit and the memory RAM or the solid state disk SSD.
(2) The Solid State Disk (SSD) is used for storing basic files such as an operating system required by program operation, images generated after the program operation, extracted ship target slice images and the like, and mainly comprises: the deep learning network structure and model parameter file are used for carrying out ship target detection and identification; the method is used for developing newly acquired reference images and the like required by the radiation correction and geometric correction related parameter files and geometric fine correction operation of the remote sensing images of the ports; an operating system, a software library and the like required by program operation; the remote sensing image of the port and the dock and the near-shore area after pretreatment (radiation correction, geometric correction and geometric fine correction); and after the program is run, cutting out result data such as ship target slice images.
(3) The memory RAM is used for caching the read-in original remote sensing image data, and the data is stored in the memory RAM after passing through the data reading bus and the high-speed data acquisition and storage interface unit by the imaging load.
(4) The embedded artificial intelligent computing unit mainly comprises a CPU and a GPU, a Linux kernel operating system is operated on the CPU, and functions of image preprocessing (radiation correction, geometric correction and geometric fine correction), ship target intelligent detection and identification related application programs, program control management with higher capability, data flow processing, image processing parallel computing and the like are provided; the data in the RAM can be read and written into the SSD through the high-speed data acquisition and storage interface unit; the control bus and the data bus are provided, and can perform task control, data transmission and other interactive operations with the satellite platform.
The remote sensing image preprocessing, the intelligent detection and recognition of the ship target and the calculation processing of the intelligent algorithm related to the ship target activity recognition are undertaken, and a control bus and a data bus are provided for the interaction between the system and a satellite platform end.
(5) The power supply control unit can provide power supply voltage and current required by each unit in the system through voltage adjustment to an external power supply provided by a satellite platform end.
In this embodiment, the data exchange is performed between the embedded artificial intelligence processing unit and the high-speed data acquisition and storage interface unit, and between the high-speed data acquisition and storage interface unit and the internal memory RAM, and between the high-speed data acquisition and storage interface unit and the solid state disk through the internal bus. The system can be used as a subsystem of a satellite, is compatible with various remote sensing satellite platforms, and has good portability and strong compatibility.
By adopting the technical scheme disclosed by the invention, the following beneficial effects are obtained:
compared with the traditional image change detection method, the method can cope with the characteristics and difficulties that the background of the near-shore port environment is complex, the illumination conditions are various (under different time in daytime), the angles of ship targets in remote sensing images can be random, multiple ship targets can be densely parked and the like, can ensure accurate detection and identification of the ship targets of interest, and can realize the identification of the ship targets of interest by processing and comparing multi-time remote sensing images obtained when satellites are revisited for multiple times. Compared with the traditional complicated data processing and application processes such as remote sensing data downloading, ground processing, uplink control and the like, the method can greatly compress the remote sensing image data transmission quantity for carrying out the tasks and the time consumption required by task completion. The system has simple composition and lower cost, can provide a modularized solution for the microsatellite to realize the on-orbit interpretation application of the port ships, and has good portability and compatibility. The timeliness and the intelligence of the remote sensing task are improved, the satellite-ground data transmission pressure is greatly reduced, and the comprehensive benefit of the remote sensing task is remarkably improved.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which is also intended to be covered by the present invention.
Claims (10)
1. A method for identifying the activities of large and medium-sized ships in a port for on-orbit application of microsatellites is characterized by comprising the following steps: comprises the following steps of the method,
s1, acquiring remote sensing images of port areas:
the satellite completes remote sensing imaging of the port area according to longitude and latitude geographic coordinates of the target port, the self-posture of the satellite and the imaging load width, and remote sensing images of the port area are obtained;
s2, carrying out on-orbit preprocessing on remote sensing images in port areas:
performing radiation correction and geometric correction on the remote sensing image of the port area on the satellite to obtain a corrected remote sensing image of the port area;
s3, acquiring remote sensing images of wharfs and near-shore areas:
cutting out remote sensing images of the wharf and the near-shore area from the corrected remote sensing images of the harbour area according to geographic longitude and latitude information of minimum envelopes of the wharf and the near-shore area of the harbour prestored on the satellite;
s4, geometric fine correction of remote sensing images of wharfs and near-shore areas:
According to the pre-stored reference images of the wharf and the near-shore area of the harbour on the satellite, performing geometric fine correction on the reference images on the remote sensing images of the wharf and the near-shore area to obtain remote sensing images of the wharf and the near-shore area after the geometric fine correction;
s5, slicing the remote sensing image of the wharf and the near-shore area after geometric fine correction:
according to the port and dock facility layout, a remote sensing image cutting scheme is designed, and the remote sensing images of the dock and the near-shore area after geometric fine correction are sliced based on the remote sensing image cutting scheme;
s6, preliminary detection and identification of ship targets:
sequentially sending the slice images into a ship target detection and identification model to obtain a ship target preliminary detection and identification result;
s7, final detection and identification of ship targets:
combining the primary detection and identification results of the ship targets to obtain the positions of the primary detection and identification results of the ship targets in remote sensing images of wharfs and near-shore areas after geometric fine correction, and performing duplicate removal processing on repeated primary detection and identification results in overlapping areas of the slice images; removing the preliminary detection and identification result of the ship target with errors according to the pre-stored land area segmentation information on the satellite, and obtaining the final detection and identification result of the ship target;
S8, comparing the multi-time phase detection recognition results:
comparing the final detection and identification result of the ship target with the final detection and identification result of the ship target obtained in the last satellite revisit by combining with the geographic position of the wharf to obtain the activity identification result of the ship target;
s9, confirming ship change conditions:
determining whether the wharf ship changes according to the ship target activity recognition result, if so, marking the change condition in remote sensing images of the wharf and the near-shore area, and storing the images;
s10, identifying ship activities again:
the next time the satellite revisits to the port location, S1 to S9 are performed again.
2. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: step S4 specifically includes the following,
s41, respectively extracting SIFT angular points in remote sensing images of the wharf and the near-shore area and reference images of the wharf and the near-shore area;
s43, matching SIFT corner points of the two images by using a KNN method;
s44, estimating projection transformation model parameters of the dock and the near-shore area images relative to the dock and near-shore area reference images by using a RANSAC method, and performing projection transformation on the dock and near-shore area remote sensing images by using the projection transformation model;
S45, correcting geographical position information corresponding to each pixel in the remote sensing images of the dock and the near-shore area after projection transformation by using geographical position information of the reference images of the dock and the near-shore area, so as to realize geometric fine correction and obtain the remote sensing images of the dock and the near-shore area after fine correction.
3. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: the remote sensing image cutting scheme comprises the following principles:
(1) The sizes of the slice images are the same;
(2) The cutting size of each slice image is between 960×960 pixels and 1280×1280 pixels, and should be a multiple of 32;
(3) The sliced image should cover each wharf completely, the land area of the wharf is selected as a boundary, the boundary does not span the berth range of the ship on the wharf, and the berth area is ensured not to be divided;
(4) The adjacent slice images have a certain overlapping area, and the width of the overlapping area is not lower than the width of the horizontal envelope line of the nearby resident ship.
4. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: the ship target detection and identification model is a deep learning network structure and model parameter file obtained after ground pre-training, and can realize intelligent detection and fine identification of the ship target of interest;
Constructing a data set of a ship, wherein the data set comprises a ship sample image, the position of a ship target in the image and ship model type information; dividing the data set into a training set and a testing set; and constructing a ship target rotation detection and identification network based on the combination of YOlOv5 and the annular smooth tag, and training and testing the ship target rotation detection and identification network by utilizing a training set and a testing set to obtain a ship target detection and identification model.
5. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 4, which is characterized in that: the ship target detection and identification model adopts a K-means clustering method, combines the scale attributes of all ship target objects, and designs different candidate envelope frames on the characteristic diagrams of the three output layers; the specific design process is that,
s61, acquiring size data of all N types of civil ship samples and size data of all M types of large-scale ship samples;
s62, visually representing the acquired size data of the civil ship sample and the large-scale ship sample as two-dimensional data points;
s63, clustering size data of various civil ship samples into 3 clusters by using a K-means method to obtain 3*N civil ship sample points; carrying out de-duplication treatment on the size data of various large-scale ship samples, wherein each large-scale ship only retains 3 sample size data, and 3*M large-scale ship sample points are obtained;
S64, combining 3*N civil ship sample points and 3*M large ship sample points;
s65, clustering the combined sample point data into 12 clusters by using a K-means method, namely generating 12 candidate envelope frames; according to the root mean square value of the length and the width of the candidate envelope frames, sequencing 12 generated candidate envelope frames from small to large; the 4 candidate envelope frames with the smallest size are used in the output layer with the corresponding size of 128 multiplied by 128; 4 candidate envelope frames with middle sizes are used in an output layer with the corresponding size of 64 multiplied by 64; the 4 largest candidate envelope frames are used for the output layer of the corresponding scale 32×32.
6. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: in step S7, combining the primary detection and identification results of each ship target, and obtaining the positions of the primary detection and identification results of each ship target in the remote sensing images of the wharf and the near-shore area after the geometric fine correction is specifically,
a1, when the slice image range is completely in the remote sensing images of the wharf and the near-shore area after the geometric fine correction, adding the position coordinates of the rectangular envelope frame in the primary detection result of the ship target to the coordinates of the origin of the slice image in the remote sensing images of the wharf and the near-shore area after the geometric fine correction, and obtaining the coordinates of the primary detection and identification result of the ship target in the remote sensing images of the wharf and the near-shore area after the geometric fine correction;
A2, when the range of the slice image is close to the edge of the geometrically refined wharf and near-shore area remote sensing image, and the partial area of the slice image exceeds the boundary of the geometrically refined wharf and near-shore area remote sensing image, the part exceeding the area is fully filled with a black image, then the coordinates of the origin of the slice image with respect to the origin of the geometrically refined wharf and near-shore area remote sensing image are updated, and then the coordinates of the rectangular envelope frame in the ship target primary detection result plus the coordinates of the origin of the slice image in the geometrically refined wharf and near-shore area remote sensing image are added, so that the coordinates of the ship target primary detection recognition result in the geometrically refined wharf and near-shore area remote sensing image can be obtained.
7. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: in step S7, the duplicate removal processing for the repeated preliminary detection and identification result of the existence of the overlapping region of the slice images specifically includes the following,
b1, importing all ship targets into a primary detection and identification result;
b2, reading a ship target preliminary detection and identification result A which is not verified;
B3, matching the ship target preliminary detection recognition result A with the rest ship target preliminary detection recognition results one by one, calculating the overlapping ratio of the two result enveloping rectangular frames, and when the overlapping ratio is smaller than a preset threshold value, not processing, and continuously matching and comparing the ship target preliminary detection recognition result A with the rest ship target preliminary detection recognition results; if the cross ratio is greater than or equal to a preset threshold value, retaining a primary detection and identification result of one ship target according to a judgment rule;
b4, if the primary detection and identification result A of the ship target is not deleted, returning to the B3; if the ship target preliminary detection recognition result A is deleted, judging whether all the rest ship target preliminary detection recognition results are checked, and if the ship target preliminary detection recognition results are not checked, returning to the step B2; and if all the results are verified, outputting a ship detection primary identification result after the duplication removal treatment.
8. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: the decision rule in step B3 is that,
if the areas of the enveloping rectangular frames of the primary detection and identification results of the two ship targets are different, retaining the primary detection and identification results of the ship targets with large areas, and deleting the primary detection and identification results of the other ship target overlapped with the primary detection and identification results of the ship targets; if the enveloping rectangular areas of the primary detection and identification results of the two ship targets are the same, the primary detection and identification results of the ship targets with high confidence are reserved, and the primary detection and identification results of the other ship targets overlapped with the primary detection and identification results of the ship targets are deleted.
9. The method for identifying the activities of large and medium-sized ships in ports for on-orbit application of microsatellites according to claim 1, which is characterized in that: in step S7, the primary detection and identification result of the ship target with errors is removed, which comprises the following contents,
c1, finishing binarization distinguishing treatment of land and water area ranges through manual design to form a land and water area segmentation map; performing binarization processing on the envelope rectangular frame range of the preliminary detection and identification result of each ship target after the duplication removal to form a ship target envelope segmentation map;
c2, representing land and water area segmentation maps and each ship target envelope segmentation map as a matrix;
and C3, multiplying the matrix corresponding to each ship target envelope segmentation map with the matrix corresponding to the land and water area segmentation map, summing the elements of the result matrix, judging that the ship target preliminary detection and identification result after the duplication removal is an error detection result if the sum result is larger than the preset percentage of the ship target envelope segmentation map, deleting the error detection result, and otherwise, reserving the ship target preliminary detection and identification result after the duplication removal.
10. A port large and medium-sized ship activity recognition system for microsatellite on-orbit application is characterized in that: comprising the steps of (a) a step of,
High-speed data acquisition and storage interface unit: the system is used for providing high-speed interface support for acquisition and reading of original remote sensing image data and providing high-speed interface support for data reading and writing and storage between the embedded artificial intelligent processing unit and the memory and the solid state disk;
embedded artificial intelligence processing unit: consisting of a CPU and a GPU on which a Linux kernel operating system runs for implementing the method according to any of claims 1 to 9;
memory: the method comprises the steps of caching read-in original remote sensing image data;
solid state disk: the method is used for storing basic files required in the running process of the method and related images and results generated after the method is run;
a power supply control unit: the power supply system is used for carrying out voltage adjustment on an external power supply provided by a satellite platform end and providing required power supply voltage and current for each unit in the system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310876541.XA CN116883861B (en) | 2023-07-17 | 2023-07-17 | Port large and medium-sized ship activity identification method and system for microsatellite on-orbit application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310876541.XA CN116883861B (en) | 2023-07-17 | 2023-07-17 | Port large and medium-sized ship activity identification method and system for microsatellite on-orbit application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116883861A true CN116883861A (en) | 2023-10-13 |
| CN116883861B CN116883861B (en) | 2024-01-26 |
Family
ID=88261792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310876541.XA Active CN116883861B (en) | 2023-07-17 | 2023-07-17 | Port large and medium-sized ship activity identification method and system for microsatellite on-orbit application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116883861B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118135426A (en) * | 2024-05-07 | 2024-06-04 | 浙江四港联动发展有限公司 | Port water area traffic condition identification method based on satellite image and SAM |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679138A (en) * | 2013-11-15 | 2014-03-26 | 中国科学院遥感与数字地球研究所 | Ship and port prior knowledge supported large-scale ship detection method |
| CN108256419A (en) * | 2017-12-05 | 2018-07-06 | 交通运输部规划研究院 | A kind of method for extracting port and pier image using multispectral interpretation |
| CN110414509A (en) * | 2019-07-25 | 2019-11-05 | 中国电子科技集团公司第五十四研究所 | Stop Ship Detection in harbour based on the segmentation of extra large land and feature pyramid network |
| US20200365035A1 (en) * | 2019-05-13 | 2020-11-19 | Hoseo University Academic Cooperation Foundation | Method and system for recognition of objects near ship by using deep neural network |
| CN112329542A (en) * | 2020-10-10 | 2021-02-05 | 中国人民解放军战略支援部队航天工程大学 | SAR image ship target detection method based on feature refined network model |
| CN113920436A (en) * | 2021-11-22 | 2022-01-11 | 江苏科技大学 | Remote sensing image marine vessel recognition system and method based on improved YOLOv4 algorithm |
| WO2022075530A1 (en) * | 2020-10-06 | 2022-04-14 | 주식회사 스카이시스 | Automatic vessel recognition and monitoring system using artificial intelligence image processing, and method for providing same |
| CN114494829A (en) * | 2022-01-16 | 2022-05-13 | 西安电子科技大学 | SAR image multi-scale ship target detection method |
-
2023
- 2023-07-17 CN CN202310876541.XA patent/CN116883861B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679138A (en) * | 2013-11-15 | 2014-03-26 | 中国科学院遥感与数字地球研究所 | Ship and port prior knowledge supported large-scale ship detection method |
| CN108256419A (en) * | 2017-12-05 | 2018-07-06 | 交通运输部规划研究院 | A kind of method for extracting port and pier image using multispectral interpretation |
| US20200365035A1 (en) * | 2019-05-13 | 2020-11-19 | Hoseo University Academic Cooperation Foundation | Method and system for recognition of objects near ship by using deep neural network |
| CN110414509A (en) * | 2019-07-25 | 2019-11-05 | 中国电子科技集团公司第五十四研究所 | Stop Ship Detection in harbour based on the segmentation of extra large land and feature pyramid network |
| WO2022075530A1 (en) * | 2020-10-06 | 2022-04-14 | 주식회사 스카이시스 | Automatic vessel recognition and monitoring system using artificial intelligence image processing, and method for providing same |
| CN112329542A (en) * | 2020-10-10 | 2021-02-05 | 中国人民解放军战略支援部队航天工程大学 | SAR image ship target detection method based on feature refined network model |
| CN113920436A (en) * | 2021-11-22 | 2022-01-11 | 江苏科技大学 | Remote sensing image marine vessel recognition system and method based on improved YOLOv4 algorithm |
| CN114494829A (en) * | 2022-01-16 | 2022-05-13 | 西安电子科技大学 | SAR image multi-scale ship target detection method |
Non-Patent Citations (3)
| Title |
|---|
| HUIXUAN FU ET AL.: "《Ship detection based on Improved YOLO Algorithm》", 《2021 PROCEEDINGS OF THE 40TH CHINESE CONTROL CONFERENCE (CCC)》, pages 8181 - 8186 * |
| QUANCHENG ZHOU ET AL.: "《Harbor Ship Detection Based on Channel Weighting and Spatial Information Fusion》", 《2020 2ND INTERNATIONAL CONFERENCE ON ELECTRONICS AND COMMUNICATION, NETWORK AND COMPUTER TECHNOLOGY (ECNCT)》, vol. 1738, pages 1742 - 6596 * |
| 张雪阳等: "《基于运动信息和深度学习的遥感视频多目标检测跟踪方法》", 《第三届中国空天安全会议论文集》, pages 295 - 303 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118135426A (en) * | 2024-05-07 | 2024-06-04 | 浙江四港联动发展有限公司 | Port water area traffic condition identification method based on satellite image and SAM |
| CN118135426B (en) * | 2024-05-07 | 2024-07-05 | 浙江四港联动发展有限公司 | Port water area traffic condition identification method based on satellite image and SAM |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116883861B (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lam et al. | xview: Objects in context in overhead imagery | |
| CN116883861B (en) | Port large and medium-sized ship activity identification method and system for microsatellite on-orbit application | |
| CN110889198B (en) | Space-time probability distribution prediction method and system based on multi-factor joint learning | |
| CN111582218A (en) | Remote sensing image island extraction method based on deep learning | |
| CN111241970A (en) | SAR image sea surface ship detection method based on yolov3 algorithm and sliding window strategy | |
| CN111160065A (en) | Remote sensing image ship detection method, device, equipment and storage medium thereof | |
| CN112883900B (en) | Method and device for bare-ground inversion of visible images of remote sensing images | |
| CN116310800B (en) | Terrace automatic extraction method and device based on deep learning | |
| CN117274826B (en) | River and lake management violation problem remote sensing monitoring method based on large model and prompt guidance | |
| CN113962473A (en) | Ship route planning method and device, electronic equipment and storage medium | |
| CN114373009A (en) | Building shadow height measurement intelligent calculation method based on high-resolution remote sensing image | |
| CN113011295A (en) | Method, computer equipment and medium for identifying photovoltaic power station based on remote sensing image | |
| CN110428425A (en) | A kind of SAR image sea land separation method based on coastline vector data | |
| CN113805178A (en) | Method for detecting static obstructive objects on water surface | |
| CN114898213A (en) | AIS knowledge assistance-based remote sensing image rotating ship target detection method | |
| CN112966558A (en) | Port automatic identification method and system based on optimized SSD target detection model | |
| Li et al. | Sea–sky line detection using gray variation differences in the time domain for unmanned surface vehicles | |
| CN111274878A (en) | Satellite cloud picture classification method and system | |
| Xu et al. | Deep learning guided building reconstruction from satellite imagery-derived point clouds | |
| CN112116628A (en) | Dynamic fitting method and device for marine ship track curve and storage medium | |
| CN114445572B (en) | DeeplabV3+ based method for immediately positioning and constructing map of obstacle in strange sea area | |
| CN115631426A (en) | Target detection method based on key point positioning in remote sensing image | |
| CN115019107A (en) | Sonar simulation image generation method, system and medium based on style migration | |
| CN113705567A (en) | Ship crack detection method, system, equipment and computer readable storage medium | |
| Li et al. | Automatic detection of beacon towers in historical aerial images using improved FCOS |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |