CN116883861A - Method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites - Google Patents
Method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites Download PDFInfo
- Publication number
- CN116883861A CN116883861A CN202310876541.XA CN202310876541A CN116883861A CN 116883861 A CN116883861 A CN 116883861A CN 202310876541 A CN202310876541 A CN 202310876541A CN 116883861 A CN116883861 A CN 116883861A
- Authority
- CN
- China
- Prior art keywords
- ship
- remote sensing
- detection
- area
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000000694 effects Effects 0.000 title claims abstract description 42
- 108091092878 Microsatellite Proteins 0.000 title claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 180
- 238000012937 correction Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 33
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 230000008859 change Effects 0.000 claims abstract description 5
- 101150080778 INPP5D gene Proteins 0.000 claims description 259
- 230000011218 segmentation Effects 0.000 claims description 25
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 17
- 238000013461 design Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000003384 imaging method Methods 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 11
- 238000013473 artificial intelligence Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 238000005520 cutting process Methods 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 230000005855 radiation Effects 0.000 claims description 6
- 239000007787 solid Substances 0.000 claims description 5
- 238000003064 k means clustering Methods 0.000 claims description 4
- 238000012938 design process Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 6
- 239000000523 sample Substances 0.000 description 16
- 230000005540 biological transmission Effects 0.000 description 12
- 230000000717 retained effect Effects 0.000 description 10
- 230000006872 improvement Effects 0.000 description 5
- 239000003643 water by type Substances 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 239000003653 coastal water Substances 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Remote Sensing (AREA)
- Astronomy & Astrophysics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Processing (AREA)
Abstract
Description
技术领域Technical field
本发明涉及智能遥感解译技术领域,尤其涉及一种面向微小卫星在轨应用的港口大中型舰船活动识别方法及系统。The present invention relates to the technical field of intelligent remote sensing interpretation, and in particular to a method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites.
背景技术Background technique
利用卫星遥感方式,开展港口内大中型舰船活动识别,如进港、离港和近岸航行等,是掌握港口运行状况、开展舰船行为分析的重要手段。Using satellite remote sensing to identify the activities of large and medium-sized ships in the port, such as entry, departure, and offshore navigation, is an important means to grasp the operating status of the port and conduct analysis of ship behavior.
当前对地观测遥感卫星系统朝着小型化、集群化的方向发展。卫星数量的不断增加、星座构型的迭代优化,使得卫星系统对重要地面目标的覆盖频次不断增加、重访周期不断缩短,卫星系统的时空分辨能力、体系任务能力不断提升。但是,受限于当前遥感卫星“在轨观测—地面处理—在轨执行”的任务模式,遥感信息处理与应用方面还存在明显短板,难以最大程度地发挥新一代对地观测卫星系统的优势。一方面,更多的卫星影像数据需要下传到地面进行处理,星地数据传输压力越来越大;另一方面,智能遥感解译技术还不够成熟,遥感影像处理与应用的智能化程度不够,需要人工参与的工作有增无减,而专业的图像判读力量有存在短缺趋势;再一方面,遥感信息获取、分析与应用流程繁长,信息获取与应用效率较低。Currently, Earth observation and remote sensing satellite systems are developing in the direction of miniaturization and clustering. The continuous increase in the number of satellites and the iterative optimization of constellation configurations have led to the continuous increase in the frequency of coverage of important ground targets by the satellite system, the continuous shortening of revisit cycles, and the continuous improvement of the satellite system's spatial and temporal resolution capabilities and system mission capabilities. However, limited by the current mission model of "on-orbit observation - ground processing - on-orbit execution" of remote sensing satellites, there are still obvious shortcomings in remote sensing information processing and application, making it difficult to maximize the advantages of the new generation of earth observation satellite systems. . On the one hand, more satellite image data needs to be transmitted to the ground for processing, and the pressure on satellite-to-ground data transmission is increasing; on the other hand, intelligent remote sensing interpretation technology is not mature enough, and remote sensing image processing and application are not intelligent enough. , the work requiring manual participation continues to increase, and there is a trend of shortage of professional image interpretation capabilities; on the other hand, the remote sensing information acquisition, analysis and application processes are lengthy, and the information acquisition and application efficiency is low.
近年来,人工智能的兴起带动了计算机视觉、自然语言处理等领域的快速发展和技术革新。将人工智能与人工经验相结合,充分利用机器高速效能与人类高等认知的叠加组合优势,在轨开展智能遥感图像解译和信息整编等工作,发展不依赖于地面站、具备在轨自主智能处理能力的智能卫星系统,能够大幅减少星地数据传输量、大幅减轻人员工作负担,提升卫星系统智能化水平,满足快速处理和高时效应用任务的需求。In recent years, the rise of artificial intelligence has led to rapid development and technological innovation in fields such as computer vision and natural language processing. Combining artificial intelligence with artificial experience, making full use of the combined advantages of high-speed machine performance and advanced human cognition, we can carry out intelligent remote sensing image interpretation and information compilation on-orbit, and develop autonomous intelligence on-orbit that does not rely on ground stations. Intelligent satellite systems with high processing capabilities can significantly reduce the amount of satellite-to-ground data transmission, significantly reduce personnel workload, improve the intelligence level of satellite systems, and meet the needs of fast processing and high-time application tasks.
综上所述,基于人工智能方法,发展在轨智能遥感解译与信息获取技术,提升遥感信息处理与应用的智能性和时效性,是适应新一代对地观测卫星系统发展的迫切需求和切实要求。In summary, based on artificial intelligence methods, developing on-orbit intelligent remote sensing interpretation and information acquisition technology, and improving the intelligence and timeliness of remote sensing information processing and application are urgent needs and practical to adapt to the development of a new generation of earth observation satellite systems. Require.
当前遥感影像的处理与应用多在地面进行,大量遥感影像数据需要通过星地数传链路下传到地面。然而,星地数传链路窗口是十分有限的,对于单一地面站而言,可能每个轨道圈次仅有几分钟数传时间,遥感影像数据无法实现及时下传。随着对地观测系统朝着构建大规模星座的方向发展,该模式不仅带来了巨大的星地数传压力,也严重限制了卫星数据在高时效性需求任务中的应用。同时,遥感影像数据的具体处理方式,如港内舰船目标判读、舰船活动识别,仍多依赖于专业判读人员人工完成。专业判读人员深陷于大量、重复性的基础判读工作,效率较低、效益不高。基于此,针对港内大中型舰船活动信息获取任务,有必要提供一种面向微小卫星在轨应用的港内舰船活动识别方法及系统。At present, the processing and application of remote sensing images are mostly carried out on the ground, and a large amount of remote sensing image data needs to be transmitted to the ground through satellite-to-ground digital transmission links. However, the satellite-to-ground data transmission link window is very limited. For a single ground station, there may be only a few minutes of data transmission time for each orbital lap, and remote sensing image data cannot be downloaded in a timely manner. As the Earth observation system develops towards the construction of large-scale constellations, this model not only brings huge pressure on satellite-to-ground data transmission, but also severely limits the application of satellite data in tasks with high time-sensitive requirements. At the same time, the specific processing methods of remote sensing image data, such as ship target interpretation and ship activity identification in the harbor, still rely on manual interpretation by professional interpretation personnel. Professional interpretation personnel are stuck in a large number of repetitive basic interpretation tasks, which are inefficient and ineffective. Based on this, for the task of obtaining information on the activities of large and medium-sized ships in the harbor, it is necessary to provide a method and system for identifying ship activities in the harbor for micro-satellite on-orbit applications.
发明内容Contents of the invention
本发明的目的在于提供一种面向微小卫星在轨应用的港口大中型舰船活动识别方法及系统,从而解决现有技术中存在的前述问题。The purpose of the present invention is to provide a method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites, thereby solving the aforementioned problems existing in the existing technology.
为了实现上述目的,本发明采用的技术方案如下:In order to achieve the above objects, the technical solutions adopted by the present invention are as follows:
一种面向微小卫星在轨应用的港口大中型舰船活动识别方法,包括如下步骤,A method for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites, including the following steps:
S1、港口区域遥感影像获取:S1. Acquisition of remote sensing images of the port area:
卫星根据目标港口的经纬度地理坐标、卫星自身姿态以及成像载荷幅宽,完成港口区域的遥感成像,获取港口区域遥感影像;The satellite completes remote sensing imaging of the port area and obtains remote sensing images of the port area based on the longitude and latitude geographical coordinates of the target port, the satellite's own attitude, and the width of the imaging load;
S2、港口区域遥感影像在轨预处理:S2. On-orbit preprocessing of remote sensing images of the port area:
星上对港口区域遥感影像进行辐射校正和几何校正,获取校正后的港口区域遥感影像;The satellite performs radiation correction and geometric correction on the remote sensing images of the port area, and obtains the corrected remote sensing images of the port area;
S3、码头及近岸区域遥感影像获取:S3. Acquisition of remote sensing images of docks and nearshore areas:
根据星上预先存储的该港口的码头及近岸区域最小包络的地理经纬度信息,从校正后的港口区域遥感影像中裁剪出码头及近岸区域遥感影像;Based on the geographical longitude and latitude information of the minimum envelope of the port's terminal and near-shore area pre-stored on the satellite, the remote sensing image of the terminal and near-shore area is cropped from the corrected remote sensing image of the port area;
S4、码头及近岸区域遥感影像几何精纠正:S4. Geometric precision correction of remote sensing images of docks and nearshore areas:
根据星上预先存储的该港口的码头及近岸区域参考基准影像,对码头及近岸区域遥感影像进行关于参考基准影像的几何精纠正,获取几何精纠正后的码头及近岸区域遥感影像;According to the reference datum image of the port and nearshore area pre-stored on the satellite, perform geometric precision correction on the remote sensing image of the wharf and nearshore area with respect to the reference datum image, and obtain the geometrically finely corrected remote sensing image of the wharf and nearshore area;
S5、几何精纠正后的码头及近岸区域遥感影像的切片:S5. Slices of remote sensing images of docks and nearshore areas after geometric correction:
根据港口码头设施布局,设计遥感影像裁切方案,基于遥感影像裁切方案对几何精纠正后的码头及近岸区域遥感影像进行切片;Based on the layout of port terminal facilities, design a remote sensing image cropping scheme, and slice the geometrically finely corrected remote sensing images of the terminal and nearshore areas based on the remote sensing image cropping scheme;
S6、舰船目标初步检测识别:S6. Preliminary detection and identification of ship targets:
将切片影像依次送入舰船目标检测识别模型中,获取舰船目标初步检测识别结果;Send the sliced images to the ship target detection and recognition model in sequence to obtain preliminary ship target detection and recognition results;
S7、舰船目标最终检测识别:S7. Final detection and identification of ship targets:
对各个舰船目标初步检测识别结果进行合并,获取各舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的位置,并对切片影像重叠区域存在的重复初步检测识别结果进行去重处理;根据星上预先存储的水陆区域分割信息,剔除掉存在错误的舰船目标初步检测识别结果,获取舰船目标最终检测识别结果;The preliminary detection and recognition results of each ship target are combined to obtain the position of the preliminary detection and recognition results of each ship target in the geometrically finely corrected remote sensing images of the wharf and offshore areas, and the repeated preliminary detection and identification of overlapping areas of the sliced images are obtained The results are deduplicated; based on the pre-stored water and land area segmentation information on the satellite, the erroneous preliminary detection and identification results of ship targets are eliminated, and the final detection and identification results of ship targets are obtained;
S8、多时相检测识别结果比对:S8. Comparison of multi-temporal detection and recognition results:
结合码头地理位置,将本次舰船目标最终检测识别结果与上次卫星重访时得到的舰船目标最终检测识别结果进行比对,获取舰船目标活动识别结果;Combined with the geographical location of the dock, compare the final detection and recognition results of this ship target with the final detection and recognition results of the ship target obtained during the last satellite revisit, and obtain the ship target activity recognition results;
S9、舰船变化情况确认:S9. Confirmation of ship changes:
根据舰船目标活动识别结果确定码头舰船是否存在变化,若是,则在码头及近岸区域遥感影像中标出变化情况,并保存影像;Determine whether there are changes in the dock ship based on the ship target activity recognition results. If so, mark the changes in the remote sensing images of the dock and nearshore areas, and save the images;
S10、舰船活动再次识别:S10. Re-identification of ship activities:
下一次卫星重访至该港口位置时,再次执行S1至S9。The next time the satellite revisits the port position, S1 to S9 will be executed again.
优选的,步骤S4具体包括如下内容,Preferably, step S4 specifically includes the following content:
S41、分别提取码头及近岸区域遥感影像和码头及近岸区域参考基准影像中的SIFT角点;S41. Extract SIFT corner points from the remote sensing images of the dock and near-shore area and the reference images of the dock and near-shore area respectively;
S43、使用KNN方法对两幅图像的SIFT角点进行匹配;S43. Use the KNN method to match the SIFT corner points of the two images;
S44、利用RANSAC方法估计码头及近岸区域影像关于码头及近岸区域参考基准影像的投影变换模型参数,利用该投影变换模型对码头及近岸区域遥感影像进行投影变换;S44. Use the RANSAC method to estimate the projection transformation model parameters of the dock and nearshore area images with respect to the dock and nearshore area reference benchmark images, and use the projection transformation model to perform projection transformation on the dock and nearshore area remote sensing images;
S45、利用码头及近岸区域参考基准影像的地理位置信息修正投影变换后的码头及近岸区域遥感影像中各像素对应的地理位置信息,实现几何精纠正,获取精纠正后的码头及近岸区域遥感影像。S45. Use the geographical location information of the reference base image of the dock and nearshore area to correct the geographical location information corresponding to each pixel in the remote sensing image of the dock and nearshore area after projection transformation, achieve geometric precision correction, and obtain the precisely corrected dock and nearshore area. Regional remote sensing images.
优选的,所述遥感影像裁切方案包括如下原则:Preferably, the remote sensing image cropping scheme includes the following principles:
(1)各切片影像的尺寸相同;(1) The size of each slice image is the same;
(2)各切片影像的裁切尺寸为960×960像素至1280×1280像素之间,且应为32的倍数;(2) The crop size of each slice image is between 960×960 pixels and 1280×1280 pixels, and should be a multiple of 32;
(3)切片影像应当完整覆盖各码头,选取码头陆地区域为分界线、分界线不跨越码头上的舰船泊位范围,保证泊位区域不被分割;(3) The sliced image should completely cover each dock, and the land area of the dock should be selected as the dividing line. The dividing line should not cross the ship berth range on the dock to ensure that the berth area is not divided;
(4)相邻切片影像存在一定的重叠区域,重叠区域的宽度不低于附近常驻舰船水平包络线的宽度。(4) There is a certain overlap area in adjacent slice images, and the width of the overlap area is not less than the width of the horizontal envelope of nearby resident ships.
优选的,所述舰船目标检测识别模型为经地面预先训练后得到的深度学习网络结构及模型参数文件,能够实现对感兴趣的舰船目标进行智能检测和精细识别;Preferably, the ship target detection and recognition model is a deep learning network structure and model parameter file obtained after pre-training on the ground, which can realize intelligent detection and fine identification of ship targets of interest;
构建舰船的数据集,所述数据集中包括舰船样本图像、图像中舰船目标的位置以及舰船型号类别信息;将数据集划分为训练集和测试集;构建基于YOlOv5与环形平滑标签相结合的舰船目标旋转检测识别网络,利用训练集和测试集训练并测试舰船目标旋转检测识别网络,以获取舰船目标检测识别模型。Construct a data set of ships, which includes ship sample images, the position of ship targets in the images, and ship model category information; divide the data set into a training set and a test set; construct a data set based on YOlOv5 and annular smooth labels Combined with the ship target rotation detection and recognition network, the training set and test set are used to train and test the ship target rotation detection and recognition network to obtain the ship target detection and recognition model.
优选的,所述舰船目标检测识别模型采用K-means聚类方法,并结合所有舰船目标对象的尺度属性,在三类输出层的特征图上设计不同的候选包络框;具体设计过程为,Preferably, the ship target detection and recognition model adopts the K-means clustering method and combines the scale attributes of all ship target objects to design different candidate envelope boxes on the feature maps of the three types of output layers; the specific design process. for,
S61、获取所有N类民用船舶样本的尺寸数据、所有M型大型舰艇样本的尺寸数据;S61. Obtain the dimensional data of all N-type civilian ship samples and the dimensional data of all M-type large ship samples;
S62、将获取到的民用船舶样本和大型舰艇样本的尺寸数据形象化表示为二维数据点;S62. Visualize the obtained dimensional data of the civilian ship samples and large ship samples as two-dimensional data points;
S63、对各类民用船舶样本的尺寸数据利用K-means方法聚类为3簇,得到3*N个民用船舶样本点;对各型大型舰艇样本的尺寸数据,将进行去重处理,每型大型舰艇仅保留3个样本尺寸数据,得到3*M个大型舰艇样本点;S63. Use the K-means method to cluster the size data of various types of civilian ship samples into 3 clusters to obtain 3*N civilian ship sample points. The size data of various types of large ship samples will be deduplicated. Each type Only 3 sample size data are retained for large ships, and 3*M large ship sample points are obtained;
S64、将3*N个民用船舶样本点和3*M个大型舰艇样本点进行合并;S64. Merge 3*N civilian ship sample points and 3*M large ship sample points;
S65、利用K-means方法,将上述合并的样本点数据聚类为12簇,即生成12种候选包络框;根据候选包络框长和宽的均方根值大小,将生成的12种候选包络框由小到大排序;尺寸最小的4种候选包络框,对应尺度128×128的输出层使用;尺寸中间的4种候选包络框,对应尺度64×64的输出层使用;尺寸最大的4种候选包络框,对应尺度32×32的输出层使用。S65. Use the K-means method to cluster the above merged sample point data into 12 clusters, that is, generate 12 types of candidate envelope boxes; according to the root mean square value of the length and width of the candidate envelope boxes, the generated 12 types The candidate envelope boxes are sorted from small to large; the four candidate envelope boxes with the smallest size are used for the output layer of the scale 128×128; the four candidate envelope boxes with the middle size are used for the output layer of the scale 64×64; The four largest candidate envelope boxes are used for the output layer with a scale of 32×32.
优选的,步骤S7中,对各个舰船目标初步检测识别结果进行合并,获取各舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的位置具体为,Preferably, in step S7, the preliminary detection and recognition results of each ship target are merged, and the position of each ship target's preliminary detection and recognition result in the geometrically finely corrected remote sensing image of the dock and near-shore area is obtained. Specifically,
A1、当切片影像范围完全处于几何精纠正后的码头及近岸区域遥感影像中时,则将舰船目标初步检测结果中矩形包络框的位置坐标加上切片影像的原点在几何精纠正后的码头及近岸区域遥感影像中的坐标,即可得到舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的坐标;A1. When the range of the sliced image is completely within the remote sensing image of the wharf and near-shore area after geometric fine correction, then the position coordinates of the rectangular envelope box in the preliminary detection result of the ship target are added to the origin of the slice image after geometric fine correction. By using the coordinates in the remote sensing images of the wharf and near-shore area, we can obtain the coordinates of the preliminary detection and recognition results of the ship target in the geometrically corrected remote sensing images of the wharf and near-shore area;
A2、当切片影像范围靠近几何精纠正后的码头及近岸区域遥感影像的边缘,且切片影像的部分区域超出几何精纠正后的码头及近岸区域遥感影像的边界时,先将超出区域的部分补全为黑色图像,再更新切片影像的原点关于几何精纠正后的码头及近岸区域遥感影像的原点的坐标,再将舰船目标初步检测结果中矩形包络框的位置坐标加上切片影像的原点在几何精纠正后的码头及近岸区域遥感影像中的坐标,即可得到舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的坐标。A2. When the range of the sliced image is close to the edge of the geometrically finely corrected remote sensing image of the pier and nearshore area, and part of the sliced image exceeds the boundary of the geometrically finely corrected remote sensing image of the pier and nearshore area, the areas outside the area are first removed. Part of it is completed as a black image, and then the coordinates of the origin of the sliced image are updated with respect to the origin of the geometrically finely corrected remote sensing image of the wharf and offshore area, and then the position coordinates of the rectangular envelope box in the preliminary detection results of the ship target are added to the slice The coordinates of the origin of the image in the geometrically finely corrected remote sensing images of the dock and coastal areas can be used to obtain the coordinates of the initial detection and identification results of the ship target in the geometrically finely corrected remote sensing images of the docks and nearshore areas.
优选的,步骤S7中,对切片影像重叠区域存在的重复初步检测识别结果进行去重处理具体包括如下内容,Preferably, in step S7, the deduplication process on the duplicate preliminary detection and identification results existing in the overlapping area of the sliced images specifically includes the following content:
B1、导入所有的舰船目标初步检测识别结果;B1. Import all preliminary detection and identification results of ship targets;
B2、读取一个未经过校验的舰船目标初步检测识别结果A;B2. Read an unverified preliminary ship target detection and identification result A;
B3、将该舰船目标初步检测识别结果A与剩余的舰船目标初步检测识别结果逐个进行匹配,计算两个结果包络矩形框重叠的交并比值,当交并比值小于预设阈值时,则不做处理,继续将该舰船目标初步检测识别结果A与剩下的其他舰船目标初步检测识别结果进行匹配比较;若交并比值大于等于预设阈值时,则根据判定规则保留其中一个舰船目标初步检测识别结果;B3. Match the initial detection and recognition result A of the ship target with the remaining preliminary detection and recognition results of the ship target one by one, and calculate the intersection and union ratio of the overlapping envelope rectangles of the two results. When the intersection and union ratio is less than the preset threshold, No processing will be done, and the preliminary detection and identification result A of the ship target will continue to be matched and compared with the remaining preliminary detection and identification results of other ship targets; if the intersection ratio is greater than or equal to the preset threshold, one of them will be retained according to the judgment rules Preliminary detection and identification results of ship targets;
B4、若该舰船目标初步检测识别结果A未被删除,则返回B3;若该舰船目标初步检测识别结果A被删除,则判断剩下的所有舰船目标初步检测识别结果是否均已校验完毕,若还有结果未被校验,则返回步骤B2;若所有结果均已校验完毕,则输出去重处理后的舰船检测初步识别结果。B4. If the preliminary detection and identification result A of the ship target has not been deleted, return to B3; if the preliminary detection and identification result A of the ship target is deleted, determine whether all the remaining preliminary detection and identification results of the ship target have been corrected. After the verification is completed, if there are still results that have not been verified, return to step B2; if all results have been verified, output the preliminary identification results of ship detection after deduplication processing.
优选的,步骤B3中的判定规则为,Preferably, the determination rule in step B3 is,
若两个舰船目标初步检测识别结果的包络矩形框面积不同,则保留面积大的舰船目标初步检测识别结果,删除与之存在重叠的另一个舰船目标初步检测识别结果;若两个舰船目标初步检测识别结果的包络矩形面积相同,则保留置信度高的舰船目标初步检测识别结果,删除与之存在重叠的另一个舰船目标初步检测识别结果。If the areas of the enveloping rectangles of the two preliminary detection and recognition results of ship targets are different, the preliminary detection and recognition results of the ship target with the larger area are retained, and the preliminary detection and recognition results of the other ship target that overlap with it are deleted; if the two If the envelope rectangular areas of the preliminary detection and recognition results of ship targets are the same, the preliminary detection and recognition results of ship targets with high confidence are retained, and the preliminary detection and recognition results of another ship target that overlap with them are deleted.
优选的,步骤S7中,剔除掉存在错误的舰船目标初步检测识别结果具体包括如下内容,Preferably, in step S7, the preliminary detection and identification results of eliminating erroneous ship targets include the following:
C1、通过人工设计,完成陆地和水域范围的二值化区分处理,形成陆地和水域分割图;将去重后的各个舰船目标初步检测识别结果的包络矩形框范围进行二值化处理,形成舰船目标包络分割图;C1. Through manual design, complete the binary distinction processing of land and water ranges to form a land and water segmentation map; perform binarization processing on the envelope rectangular frame range of the preliminary detection and identification results of each ship target after deduplication, Form a ship target envelope segmentation diagram;
C2、将陆地和水域分割图、各舰船目标包络分割图表示为矩阵;C2. Express the land and water segmentation map and the target envelope segmentation map of each ship as a matrix;
C3、将各舰船目标包络分割图对应的矩阵与陆地和水域分割图对应的矩阵点乘后再将结果矩阵的元素求和,若求和结果大于舰船目标包络分割图尺寸的预设百分比,则判定该去重后的舰船目标初步检测识别结果为错误检测结果,删除该错误检测结果,否则保留该去重后的舰船目标初步检测识别结果。C3. Dot-multiply the matrix corresponding to each ship's target envelope segmentation map with the matrix corresponding to the land and water segmentation map, and then sum the elements of the result matrix. If the summation result is larger than the predetermined size of the ship's target envelope segmentation map, Assuming a percentage, it is determined that the preliminary detection and recognition result of the ship target after deduplication is an erroneous detection result, and the erroneous detection result is deleted. Otherwise, the preliminary detection and recognition result of the ship target after deduplication is retained.
本发明的目的还在于提供一种面向微小卫星在轨应用的港口大中型舰船活动识别系统,包括,The purpose of the present invention is also to provide a port large and medium-sized ship activity identification system for micro-satellite in-orbit applications, including:
高速数据采集与存储接口单元:用于为原始遥感影像数据的采集读入提供高速接口支持,并为嵌入式人工智能处理单元与内存和固态硬盘间的数据读写、存储提供高速接口支持;High-speed data acquisition and storage interface unit: used to provide high-speed interface support for the collection and reading of original remote sensing image data, and to provide high-speed interface support for data reading, writing, and storage between the embedded artificial intelligence processing unit and memory and solid-state drives;
嵌入式人工智能处理单元:由CPU和GPU组成,其上运行Linux内核操作系统,用于实现如权利要求1至9任一所述的方法;Embedded artificial intelligence processing unit: composed of a CPU and a GPU, running a Linux kernel operating system on it, used to implement the method described in any one of claims 1 to 9;
内存:用于缓存读入的原始遥感影像数据;Memory: used to cache the read original remote sensing image data;
固态硬盘:用于存储所述方法运行过程中所需的基础文件以及所述方法运行后生成的相关影像及结果;Solid state drive: used to store the basic files required during the operation of the method and the related images and results generated after the method is run;
电源控制单元:用于对卫星平台端提供的外部电源进行电压调整,为系统内各单元提供所需的供电电压和电流。Power control unit: used to adjust the voltage of the external power supply provided by the satellite platform to provide the required power supply voltage and current for each unit in the system.
本发明的有益效果是:1、相比传统的图像变化检测方法,本方法能够应对近岸港口环境背景复杂、光照条件多样(白天不同时间下)、舰船目标呈现在遥感影像中的角度可能是任意的、多艘舰船目标可能密集停靠等特点和难点,能保证对感兴趣舰船目标进行准确的检测识别,通过对卫星先后多次重访时获得的多时相遥感影像进行处理和比对,实现感兴趣舰船目标活动的识别。2、相比传统遥感数据下传、地面处理、上行控制等繁琐的数据处理与应用流程,该方法能够大幅压缩开展该类任务的遥感影像数据传输量和任务完成所需耗时。3、该系统组成简单、成本较低,能够为微小卫星实现港口舰船的在轨解译应用提供模块化的解决方案,系统移植性、兼容性好。4、在提高遥感任务时效性、智能性的同时,也大幅降低了星地数据传输压力,实现遥感任务综合效益的显著提升。The beneficial effects of the present invention are: 1. Compared with traditional image change detection methods, this method can cope with the complex background of the nearshore port environment, diverse lighting conditions (at different times during the day), and the possible angles of ship targets appearing in remote sensing images. It is arbitrary and has the characteristics and difficulties of multiple ship targets possibly docking intensively. It can ensure accurate detection and identification of ship targets of interest by processing and comparing multi-temporal remote sensing images obtained by satellites during multiple revisits. Yes, to achieve the identification of target activities of ships of interest. 2. Compared with traditional remote sensing data downlink, ground processing, uplink control and other cumbersome data processing and application processes, this method can significantly reduce the amount of remote sensing image data transmission and the time required to complete this type of task. 3. The system has simple composition and low cost. It can provide a modular solution for the on-orbit interpretation application of port ships by micro-satellites. The system has good portability and compatibility. 4. While improving the timeliness and intelligence of remote sensing missions, it also significantly reduces the pressure on satellite-to-ground data transmission, achieving a significant improvement in the comprehensive benefits of remote sensing missions.
附图说明Description of the drawings
图1是本发明实施例中舰船活动识别方法的流程图;Figure 1 is a flow chart of a ship activity recognition method in an embodiment of the present invention;
图2是本发明实施例中原始遥感影像、预处理影像、裁剪影像和几何精校正后影像的对比图;Figure 2 is a comparison diagram of original remote sensing images, pre-processed images, cropped images and geometrically corrected images in the embodiment of the present invention;
图3是本发明实施例中利用参考基准影像进行新获取影像的几何精纠正操作流程;Figure 3 is a geometric fine correction operation process of using a reference datum image to perform newly acquired images in an embodiment of the present invention;
图4是本发明实施例中个性化裁切案例图;Figure 4 is a diagram of a personalized cutting case in an embodiment of the present invention;
图5是本发明实施例中舰船目标检测识别模型的结构图;Figure 5 is a structural diagram of the ship target detection and recognition model in the embodiment of the present invention;
图6是本发明实施例中检测识别网络候选框设计方法流程图;Figure 6 is a flow chart of a detection and recognition network candidate frame design method in an embodiment of the present invention;
图7是本发明实施例中由切片影像还原到码头及近岸区域影像图;Figure 7 is an image diagram of the dock and offshore area restored from the sliced image in the embodiment of the present invention;
图8是本发明实施例中检测识别重叠区域中的舰船识别结果处理流程;Figure 8 is a processing flow of ship recognition results in the overlapping area of detection and recognition in the embodiment of the present invention;
图9是本发明实施例中精纠正后码头及近岸区域影像的水陆分割图;Figure 9 is a water and land segmentation diagram of the pier and nearshore area images after fine correction in the embodiment of the present invention;
图10是本发明实施例中舰船活动识别系统的结构图。Figure 10 is a structural diagram of the ship activity recognition system in the embodiment of the present invention.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施方式仅仅用以解释本发明,并不用于限定本发明。In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.
实施例中,提供了一种面向微小卫星在轨应用的港口大中型舰船活动识别方法及系统,根据港口码头上的舰船泊位及舰船类型相对固定等特点,设计轻量的深度学习目标检测识别方法实现可见光遥感影像中的舰船目标检测和精细识别,并充分发挥遥感低轨小卫星高能效的边缘算力,在轨自主开展港内大中型舰船活动的识别任务,改善传统任务模式数传需求量大、信息生成时效性不高的弊端。In the embodiment, a method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites are provided. Based on the characteristics of relatively fixed ship berths and ship types at the port terminal, a lightweight deep learning target is designed. The detection and identification method realizes the detection and fine identification of ship targets in visible light remote sensing images, and gives full play to the energy-efficient edge computing power of remote sensing low-orbit small satellites to independently carry out identification tasks of large and medium-sized ship activities in the port in orbit, improving the traditional task mode. The disadvantages of large demand for digital transmission and low timeliness of information generation.
实施例一Embodiment 1
本实施例中,通过结合细粒度目标检测识别和先验地理知识,提供了一种高效、可部署在低轨遥感小卫星上的港内舰船目标精细判读及其活动智能识别方法,如图1所示,具体包括如下步骤:In this embodiment, by combining fine-grained target detection and recognition with prior geographical knowledge, an efficient and deployable method for fine interpretation of ship targets in harbor and intelligent identification of their activities is provided, which can be deployed on low-orbit remote sensing small satellites, as shown in Figure 1 As shown, it specifically includes the following steps:
一、港口区域遥感影像获取:1. Obtaining remote sensing images of the port area:
卫星根据目标港口的经纬度地理坐标(即卫星成像拍照的先验引导信息)、卫星自身姿态以及成像载荷幅宽等,完成港口区域的遥感成像,获取港口区域遥感影像,如图2(a)所示。成像区域仅保证对港口区域范围的覆盖,尽可能地控制影像数据量。The satellite completes the remote sensing imaging of the port area and obtains the remote sensing image of the port area based on the longitude and latitude geographical coordinates of the target port (i.e., the prior guidance information for satellite imaging and photography), the attitude of the satellite, and the width of the imaging load, etc., as shown in Figure 2(a) Show. The imaging area only ensures coverage of the port area and controls the amount of image data as much as possible.
本实施例中,卫星成像区域中心的粗略地理坐标,可根据卫星轨道定位数据和卫星姿态计算出。因此,首先根据卫星轨道定位信息,控制卫星姿态,使卫星成像中心的粗略位置在港口码头中央。随后,对港口区域进行遥感成像并获取遥感数据。由于卫星具有较大幅宽且成像中心靠近码头中央,因此能够获取完整的港口区域遥感影像。In this embodiment, the rough geographical coordinates of the center of the satellite imaging area can be calculated based on the satellite orbit positioning data and satellite attitude. Therefore, first, based on the satellite orbit positioning information, the satellite attitude is controlled so that the rough position of the satellite imaging center is in the center of the port terminal. Subsequently, remote sensing imaging was performed on the port area and remote sensing data were obtained. Since the satellite has a large width and the imaging center is close to the center of the pier, it can obtain a complete remote sensing image of the port area.
二、港口区域遥感影像在轨预处理:2. On-orbit preprocessing of remote sensing images of the port area:
星上对港口区域遥感影像进行辐射校正和几何校正,获取校正后的港口区域遥感影像,如图2(b)所示。The satellite performs radiation correction and geometric correction on the remote sensing image of the port area, and obtains the corrected remote sensing image of the port area, as shown in Figure 2(b).
三、码头及近岸区域遥感影像获取:3. Acquisition of remote sensing images of docks and nearshore areas:
根据星上预先存储的该港口的码头及近岸区域最小包络的地理经纬度信息,从校正后的港口区域遥感影像中裁剪出码头及近岸区域遥感影像,如图2(c)所示。裁剪出的影像中包含了港口码头及其少部分近岸陆域、水域。According to the geographical longitude and latitude information of the minimum envelope of the port's terminal and near-shore area pre-stored on the satellite, the remote sensing image of the terminal and near-shore area is cropped from the corrected remote sensing image of the port area, as shown in Figure 2(c). The cropped image includes the port terminal and a small part of the offshore land and water areas.
四、码头及近岸区域遥感影像几何精纠正:4. Geometric precision correction of remote sensing images in docks and nearshore areas:
根据星上预先存储的该港口的码头及近岸区域参考基准影像,对码头及近岸区域遥感影像进行关于参考基准影像的几何精纠正(即相对纠正),获取几何精纠正后的码头及近岸区域遥感影像,如图2(d)所示。According to the reference datum image of the port and nearshore area pre-stored on the satellite, the remote sensing images of the wharf and nearshore area are geometrically finely corrected (i.e., relative correction) with respect to the reference datum image, and the geometrically finely corrected wharf and nearshore area are obtained. The remote sensing image of the shore area is shown in Figure 2(d).
如图3所示,具体地,几何精纠正的具体包括如下内容,As shown in Figure 3, specifically, the geometric precision correction includes the following contents,
1、分别提取码头及近岸区域遥感影像和码头及近岸区域参考基准影像中的SIFT角点;1. Extract SIFT corner points from the remote sensing images of the dock and near-shore area and the reference benchmark images of the dock and near-shore area respectively;
3、使用KNN方法(K邻近算法)对两幅图像的SIFT角点进行匹配;3. Use the KNN method (K neighbor algorithm) to match the SIFT corner points of the two images;
4、利用RANSAC方法(随机一致性采样算法)估计码头及近岸区域影像关于码头及近岸区域参考基准影像的投影变换模型参数,利用该投影变换模型对码头及近岸区域遥感影像进行投影变换;4. Use the RANSAC method (Random Consistent Sampling Algorithm) to estimate the projection transformation model parameters of the dock and nearshore area images with respect to the dock and nearshore area reference benchmark images, and use this projection transformation model to project the remote sensing images of the dock and nearshore area. ;
5、利用码头及近岸区域参考基准影像的地理位置信息修正投影变换后的码头及近岸区域遥感影像中各像素对应的地理位置信息,实现几何精纠正,获取精纠正后的码头及近岸区域遥感影像。5. Use the geographical location information of the reference benchmark image of the dock and nearshore area to correct the geographical location information corresponding to each pixel in the remote sensing image of the dock and nearshore area after projection transformation, achieve geometric precision correction, and obtain the precisely corrected dock and nearshore area. Regional remote sensing images.
五、几何精纠正后的码头及近岸区域遥感影像的切片:5. Slices of remote sensing images of docks and offshore areas after geometric correction:
由于码头及近岸区域遥感影像尺寸较大,因此根据港口码头设施布局,设计个性化的遥感影像裁切方案,以支持在尺寸大小,裁切范围更隔离的切片影像中进行舰船目标检测识别;基于遥感影像裁切方案对几何精纠正后的码头及近岸区域遥感影像进行切片。Due to the large size of remote sensing images in docks and offshore areas, a personalized remote sensing image cropping scheme is designed based on the layout of port terminal facilities to support ship target detection and recognition in sliced images with larger sizes and more isolated cropping ranges. ; Based on the remote sensing image cropping scheme, the geometrically finely corrected remote sensing images of the dock and offshore areas are sliced.
如图4所示的个性化裁切案例,一方面,考虑到港口码头及近岸区域的范围一般较大,其遥感影像尺寸较大,若将整图送入深度学习检测识别网络,计算耗时将指数增长且精度无法保证;另一方面,考虑到对舰船目标活动的识别,重点关注的区域范围为码头及近岸水域,不需要再耗费计算力和时间对部分水域和陆地区域进行检测识别;因此,设计个性化的裁切方案,从码头及近岸水域遥感影像中,以码头为中心裁剪出多个切片,对这些切片影像进行舰船目标检测识别。个性化裁切需要设计者对港口码头及舰船泊位比较熟悉。个性化的遥感影像裁切方案包括如下原则:As shown in the personalized cropping case shown in Figure 4, on the one hand, considering that the scope of port terminals and near-shore areas is generally large, and the size of their remote sensing images is large, if the entire image is sent to the deep learning detection and recognition network, the calculation cost will be very high. The time will increase exponentially and the accuracy cannot be guaranteed; on the other hand, considering the identification of ship target activities, the key areas of focus are docks and offshore waters, and there is no need to waste computing power and time on some waters and land areas. Detection and identification; therefore, a personalized cutting plan is designed to cut out multiple slices centered on the dock from remote sensing images of docks and coastal waters, and detect and identify ship targets on these slice images. Personalized tailoring requires designers to be familiar with port terminals and ship berths. The personalized remote sensing image cropping plan includes the following principles:
(1)各切片影像的尺寸相同;(1) The size of each slice image is the same;
(2)各切片影像的尺寸要适宜,该方法主要适用于分辨率0.8米至0.5米的可见光遥感影像,各切片影像的裁切尺寸为960×960像素至1280×1280像素之间,且应为32的倍数;(2) The size of each slice image must be appropriate. This method is mainly suitable for visible light remote sensing images with a resolution of 0.8 meters to 0.5 meters. The crop size of each slice image is between 960×960 pixels and 1280×1280 pixels, and should be is a multiple of 32;
(3)切片影像以完整覆盖各码头为主要原则,尽可能选取码头陆地区域为分界线、分界线不跨越码头上的舰船泊位范围,保证泊位区域不被分割;(3) The main principle of the sliced image is to completely cover each dock. Try to select the land area of the dock as the dividing line. The dividing line does not cross the ship berth range on the dock to ensure that the berth area is not divided;
(4)相邻切片影像存在一定的重叠区域,重叠区域的宽度不低于附近常驻舰船水平包络线的宽度。(4) There is a certain overlap area in adjacent slice images, and the width of the overlap area is not less than the width of the horizontal envelope of nearby resident ships.
陆地区域、离码头较远的水域不进行裁剪区域设计、不进行舰船目标检测,尽量减少计算负担的同时,又能保证对码头及邻近水域舰船的检测识别。There is no clipping area design or ship target detection in land areas and waters far away from the dock. This can reduce the computational burden as much as possible while ensuring the detection and identification of ships in the dock and adjacent waters.
六、舰船目标初步检测识别:6. Preliminary detection and identification of ship targets:
将切片影像依次送入舰船目标检测识别模型中,获取舰船目标初步检测识别结果。初步检测识别结果通过旋转矩形包络框给出,实现对舰船目标区域的最小包络。The sliced images are sequentially sent to the ship target detection and recognition model to obtain preliminary ship target detection and recognition results. The preliminary detection and recognition results are given by rotating the rectangular envelope frame to achieve the minimum envelope of the ship target area.
所述舰船目标检测识别模型为经地面预先训练后得到的深度学习网络结构及模型参数文件,能够实现对感兴趣的舰船目标进行智能检测和精细识别;The ship target detection and recognition model is a deep learning network structure and model parameter file obtained after pre-training on the ground, which can realize intelligent detection and fine identification of ship targets of interest;
构建舰船的数据集,所述数据集中包括舰船样本图像、图像中舰船目标的位置(以旋转矩形包络框的四个顶点坐标给出)以及舰船型号类别信息;将数据集划分为训练集和测试集;构建基于YOlOv5与环形平滑标签(CSL)相结合的舰船目标旋转检测识别网络,利用训练集和测试集训练并测试舰船目标旋转检测识别网络,以获取舰船目标检测识别模型。Construct a data set of ships, which includes ship sample images, the position of the ship target in the image (given as the four vertex coordinates of the rotated rectangular envelope box), and ship model category information; divide the data set For the training set and test set; build a ship target rotation detection and recognition network based on YOlOv5 combined with Circular Smooth Label (CSL), use the training set and test set to train and test the ship target rotation detection and recognition network to obtain the ship target Detection and recognition models.
如图5所示,舰船目标检测识别模型基于YOLO_v5网络改进得到。相比YOLO_v5网络,面向大中型舰船目标检测识别的深度学习网络主要有以下几方面的改进:(1)网络输入影像的尺度为1024×1024×3;(2)利用Circular Smooth Label(CSL)旋转目标检测思想,增加了对目标旋转角度的预测,以旋转矩形框实现了对舰船目标的包络;同时,参考CSL旋转目标检测思想对网络预测损失值的计算方法、旋转目标非极大值抑制算法(Non-MaximumSuppression,NMS)也进行了适应性改进。(3)预测输出层尺度改进为128×128,64×64,32×32三种。(4)利用K-means聚类方法,并结合所有舰船目标对象的尺度属性,在三类输出层的特征图上设计不同的候选包络框,每层特征图上设计4种候选包络框,一共12种;尺度越大的特征图上,对应的候选包络框尺寸越小;反之亦然。各输出层特征图每个像素点对应有4个预测结果参数(关于4种候选包络框,各有1个预测结果),每个预测结果包含K+6个参数(K个类别参数,1个置信度参数,1个旋转角度参数,4个位置坐标参数)。因此,三类输出层输出数据的维度分别为[4*(K+6)]×128×128、[4*(K+6)]×64×64、[4*(K+6)]×32×32。As shown in Figure 5, the ship target detection and recognition model is improved based on the YOLO_v5 network. Compared with the YOLO_v5 network, the deep learning network for large and medium-sized ship target detection and recognition mainly has the following improvements: (1) The scale of the network input image is 1024×1024×3; (2) Circular Smooth Label (CSL) is used The rotating target detection idea adds the prediction of the target rotation angle, and uses a rotating rectangular frame to achieve the envelope of the ship target; at the same time, refer to the CSL rotating target detection idea to calculate the network prediction loss value, and the rotating target non-maximum The value suppression algorithm (Non-MaximumSuppression, NMS) has also been adaptively improved. (3) The prediction output layer scale is improved to 128×128, 64×64, and 32×32. (4) Use the K-means clustering method and combine the scale attributes of all ship target objects to design different candidate envelope boxes on the feature maps of the three types of output layers, and design 4 types of candidate envelopes on each layer of feature maps. There are 12 types of boxes in total; the larger the scale of the feature map, the smaller the size of the corresponding candidate envelope box; vice versa. Each pixel of each output layer feature map corresponds to 4 prediction result parameters (one prediction result for each of the 4 candidate envelope boxes), and each prediction result contains K+6 parameters (K category parameters, 1 confidence parameters, 1 rotation angle parameter, 4 position coordinate parameters). Therefore, the dimensions of the output data of the three types of output layers are [4*(K+6)]×128×128, [4*(K+6)]×64×64, [4*(K+6)]× 32×32.
如图6所示,本实施例中,舰船目标检测识别模型采用K-means聚类方法设计不同的候选包络框的具体设计过程为,As shown in Figure 6, in this embodiment, the ship target detection and recognition model uses the K-means clustering method to design different candidate envelope frames. The specific design process is as follows:
1、获取所有N类民用船舶样本的尺寸数据、所有M型大型舰艇样本的尺寸数据;1. Obtain the dimensional data of all N-type civilian ship samples and the dimensional data of all M-type large ship samples;
2、将获取到的民用船舶样本和大型舰艇样本的尺寸数据(长度和宽度)形象化表示为二维数据点;2. Visualize the obtained dimensional data (length and width) of civilian ship samples and large ship samples as two-dimensional data points;
3、民用船舶具有各型船舶数量多、各型船舶中有可能存在多种尺寸(如散装货船的尺寸可以有大有小)、各型船舶数量不均衡的特点,为避免候选框尺寸倾向数量多、尺寸类型多的民用船舶,造成候选框尺寸不具备代表性和普适应,所以对各类民用船舶样本的尺寸数据利用K-means方法聚类为3簇,得到3*N个民用船舶样本点;3. Civilian ships have the characteristics of a large number of various types of ships, various sizes of ships (such as bulk carriers can be large or small), and an unbalanced number of various types of ships. In order to avoid the candidate box size being biased towards quantity, There are many civil ships with many size types, resulting in the candidate frame size not being representative and universally adaptable. Therefore, the size data of various types of civil ship samples are clustered into 3 clusters using the K-means method, and 3*N civil ship samples are obtained. point;
大型舰艇具有各型尺寸相对固定、各型舰艇数量不均衡的特点,为避免候选框尺寸倾向数量多的大型舰艇,造成候选框尺寸不具备代表性和普适性,所以对各型大型舰艇样本的尺寸数据,将进行去重处理,每型大型舰艇仅保留3个样本尺寸数据,得到3*M个大型舰艇样本点。Large ships have the characteristics of relatively fixed sizes of various types and unbalanced numbers of various types of ships. In order to avoid the candidate box size being biased towards large numbers of large ships, resulting in the candidate box size not being representative and universal, so samples of various types of large ships are The size data will be deduplicated, and only 3 sample size data will be retained for each type of large ship, resulting in 3*M large ship sample points.
其中,对于尺寸固定的某大型舰艇,对应的3个样本尺寸数据均为该固定尺寸;对于尺寸可能存在微小差别的某大型舰艇,对应的3个样本尺寸数据均为不同尺寸的算术平均值。Among them, for a certain large ship with a fixed size, the corresponding three sample size data are all the fixed size; for a certain large ship with slight differences in size, the corresponding three sample size data are all the arithmetic average of different sizes.
4、将3*N个民用船舶样本点和3*M个大型舰艇样本点进行合并;4. Merge 3*N civilian ship sample points and 3*M large ship sample points;
5、利用K-means方法,将上述合并的样本点数据聚类为12簇,即生成12种候选包络框;根据候选包络框长和宽的均方根值大小,将生成的12种候选包络框由小到大排序;尺寸最小的4种候选包络框,对应尺度128×128的输出层使用;尺寸中间的4种候选包络框,对应尺度64×64的输出层使用;尺寸最大的4种候选包络框,对应尺度32×32的输出层使用。5. Use the K-means method to cluster the above-mentioned merged sample point data into 12 clusters, that is, generate 12 types of candidate envelope boxes; according to the root mean square value of the length and width of the candidate envelope boxes, the generated 12 types The candidate envelope boxes are sorted from small to large; the four candidate envelope boxes with the smallest size are used for the output layer of the scale 128×128; the four candidate envelope boxes with the middle size are used for the output layer of the scale 64×64; The four largest candidate envelope boxes are used for the output layer with a scale of 32×32.
通过上述候选框设计方法,能够保证设计的候选框具有普适性且能覆盖照顾到所有待检测识别的舰船目标,从而避免设计出的候选包络框更倾向于样本数量更多的舰船目标,使得舰船目标检测识别模型对所有类型舰船都能够有相对均衡的、较强的预测能力。Through the above candidate frame design method, it can be ensured that the designed candidate frame is universal and can cover all ship targets to be detected and identified, thereby preventing the designed candidate envelope frame from being more inclined to ships with a larger number of samples. target, so that the ship target detection and recognition model can have a relatively balanced and strong prediction ability for all types of ships.
七、舰船目标最终检测识别:7. Final detection and identification of ship targets:
对各个舰船目标初步检测识别结果进行合并,获取各舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的位置,并对切片影像重叠区域存在的重复初步检测识别结果进行去重处理;根据星上预先存储的水陆区域分割信息,剔除掉存在错误的舰船目标初步检测识别结果(即位于陆地上的舰船目标初步检测识别结果),获取舰船目标最终检测识别结果。The preliminary detection and recognition results of each ship target are combined to obtain the position of the preliminary detection and recognition results of each ship target in the geometrically finely corrected remote sensing images of the wharf and offshore areas, and the repeated preliminary detection and identification of overlapping areas of the sliced images are obtained The results are deduplicated; based on the pre-stored water and land area segmentation information on the satellite, the erroneous preliminary detection and identification results of ship targets (that is, the preliminary detection and identification results of ship targets on land) are eliminated, and the final detection of ship targets is obtained. Recognition results.
本实施例中,参见图7,OC1、OC2、OC3为单次检测区域图像(切片影像)的原点,OR为精纠正后的码头及近岸区域图像的原点。In this embodiment, referring to Figure 7, O C1 , O C2 , and O C3 are the origins of the single detection area image (slice image), and OR is the origin of the finely corrected pier and nearshore area images.
对各个舰船目标初步检测识别结果进行合并,获取各舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的位置具体为,The preliminary detection and recognition results of each ship target are combined to obtain the position of the preliminary detection and recognition results of each ship target in the geometrically finely corrected remote sensing images of the dock and coastal areas. The specific positions are as follows:
A1、当切片影像范围完全处于几何精纠正后的码头及近岸区域遥感影像中时,则将舰船目标初步检测结果中矩形包络框的位置坐标加上切片影像的原点在几何精纠正后的码头及近岸区域遥感影像中的坐标,即可得到舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的坐标。A1. When the range of the sliced image is completely within the remote sensing image of the wharf and near-shore area after geometric fine correction, then the position coordinates of the rectangular envelope box in the preliminary detection result of the ship target are added to the origin of the slice image after geometric fine correction. By using the coordinates in the remote sensing images of the dock and near-shore area, we can obtain the coordinates of the preliminary detection and identification results of the ship target in the geometrically corrected remote sensing images of the dock and near-shore area.
如以OC3为原点的切片影像,舰船检测识别结果中矩形包络框的位置坐标加上OC3在ORXRYR中的坐标,即可得到舰船检测识别结果在精纠正后的码头及近岸区域图像中的坐标。For example, for a sliced image with O C3 as the origin, the position coordinates of the rectangular envelope box in the ship detection and recognition results are added to the coordinates of O C3 in O R X R Y R , and the ship detection and recognition results can be obtained after fine correction The coordinates in the image of the pier and nearshore area.
A2、当切片影像范围靠近几何精纠正后的码头及近岸区域遥感影像的边缘,且切片影像的部分区域超出几何精纠正后的码头及近岸区域遥感影像的边界时,先将超出区域的部分补全为黑色图像,再更新切片影像的原点(如OC2和OC1)关于几何精纠正后的码头及近岸区域遥感影像的原点(即OR)的坐标,后续舰船检测识别结果在几何精纠正的码头及近岸区域图像中的坐标计算与A1相同(即再将舰船目标初步检测结果中矩形包络框的位置坐标加上切片影像的原点在几何精纠正后的码头及近岸区域遥感影像中的坐标,即可得到舰船目标初步检测识别结果在几何精纠正后的码头及近岸区域遥感影像中的坐标)。A2. When the range of the sliced image is close to the edge of the geometrically finely corrected remote sensing image of the pier and nearshore area, and part of the sliced image exceeds the boundary of the geometrically finely corrected remote sensing image of the pier and nearshore area, the areas outside the area are first removed. Partially complete the image as a black image, and then update the coordinates of the origin of the sliced image (such as O C2 and O C1 ) with respect to the origin of the geometrically corrected remote sensing image of the dock and offshore area (i.e. OR ), and the subsequent ship detection and recognition results The coordinate calculation in the geometrically finely corrected dock and nearshore area images is the same as A1 (that is, the position coordinates of the rectangular envelope box in the preliminary ship target detection results are added to the origin of the sliced image in the geometrically finely corrected dock and coastal area images). The coordinates in the remote sensing images of the near-shore area can be used to obtain the coordinates of the preliminary detection and recognition results of the ship target in the geometrically corrected remote sensing images of the dock and near-shore area).
本实施例中,根据个性化的裁剪区域设计方案,相邻裁剪切片影像间存在一定的重叠区域,因此需要对重叠区域中的舰船识别结果进行去重处理。如图8所示,对切片影像重叠区域存在的重复初步检测识别结果进行去重处理具体包括如下内容,In this embodiment, according to the personalized cropping area design scheme, there is a certain overlapping area between adjacent cropped images, so the ship recognition results in the overlapping area need to be deduplicated. As shown in Figure 8, the deduplication process for the duplicate preliminary detection and identification results that exist in the overlapping area of the sliced images specifically includes the following contents:
B1、导入所有的舰船目标初步检测识别结果(包络框的四个顶点坐标),这些结果是各切片影像经舰船目标检测识别模型预测后的输出;B1. Import all preliminary ship target detection and recognition results (the four vertex coordinates of the envelope box). These results are the output of each sliced image predicted by the ship target detection and recognition model;
B2、读取一个未经过校验的舰船目标初步检测识别结果A;B2. Read an unverified preliminary ship target detection and identification result A;
B3、将该舰船目标初步检测识别结果A与剩余的舰船目标初步检测识别结果逐个进行匹配,计算两个结果包络矩形框重叠的交并比值,当交并比值(IOU)小于预设阈值(可以根据实际情况进行设置,本实施例中设置为0.3)时,则不做处理,继续将该舰船目标初步检测识别结果A与剩下的其他舰船目标初步检测识别结果进行匹配比较;若交并比值大于等于预设阈值时,则根据判定规则保留其中一个舰船目标初步检测识别结果。B3. Match the preliminary detection and identification results A of the ship target with the remaining preliminary detection and identification results of the ship target one by one, and calculate the intersection and union ratio of the overlap of the envelope rectangular frames of the two results. When the intersection and union ratio (IOU) is less than the preset When the threshold (can be set according to the actual situation, set to 0.3 in this embodiment), no processing is performed, and the preliminary detection and identification result A of the ship target is continued to be matched and compared with the remaining preliminary detection and identification results of other ship targets. ; If the intersection ratio is greater than or equal to the preset threshold, one of the ship target preliminary detection and identification results will be retained according to the judgment rules.
判定规则为:若两个舰船目标初步检测识别结果的包络矩形框面积不同,则保留面积大的舰船目标初步检测识别结果,删除与之存在重叠的另一个舰船目标初步检测识别结果;若两个舰船目标初步检测识别结果的包络矩形面积相同,则保留置信度高的舰船目标初步检测识别结果,删除与之存在重叠的另一个舰船目标初步检测识别结果。The judgment rule is: if the areas of the enveloping rectangles of the two preliminary detection and recognition results of ship targets are different, the preliminary detection and recognition result of the ship target with the larger area is retained, and the preliminary detection and recognition result of the other ship target that overlaps with it is deleted. ; If the envelope rectangular areas of the two preliminary detection and recognition results of ship targets are the same, the preliminary detection and recognition result of the ship target with high confidence is retained, and the preliminary detection and recognition result of the other ship target that overlaps with it is deleted.
B4、若该舰船目标初步检测识别结果A未被删除,则返回B3;若该舰船目标初步检测识别结果A被删除,则判断剩下的所有舰船目标初步检测识别结果是否均已校验完毕,若还有结果未被校验,则返回步骤B2;若所有结果均已校验完毕,则输出去重处理后的舰船检测初步识别结果。B4. If the preliminary detection and identification result A of the ship target has not been deleted, return to B3; if the preliminary detection and identification result A of the ship target is deleted, determine whether all the remaining preliminary detection and identification results of the ship target have been corrected. After the verification is completed, if there are still results that have not been verified, return to step B2; if all results have been verified, output the preliminary identification results of ship detection after deduplication processing.
本实施例中,剔除掉存在错误的舰船目标初步检测识别结果具体包括如下内容,In this embodiment, the preliminary detection and identification results of eliminating erroneous ship targets include the following:
C1、通过人工设计,完成陆地和水域范围的二值化区分处理,形成陆地和水域分割图,如图9(b)所示,陆地范围对应的像素标记为“1”,图中显示为白色;水域范围对应的像素标记为“0”,图中显示为黑色;将去重后的各个舰船目标初步检测识别结果的包络矩形框范围进行二值化处理,形成舰船目标包络分割图,如图9(c)所示,舰船包络范围对应的像素标记为“1”,图中显示为白色,其余部分标记“0”,图中显示为黑色。C1. Through manual design, complete the binary distinction processing of land and water ranges to form a land and water segmentation map, as shown in Figure 9(b). The pixels corresponding to the land range are marked as "1" and are displayed in white in the figure. ; The pixels corresponding to the water area are marked as "0" and are displayed as black in the figure; the envelope rectangular frame range of the initial detection and recognition results of each ship target after deduplication is binarized to form the ship target envelope segmentation As shown in Figure 9(c), the pixels corresponding to the ship envelope range are marked as "1" and are displayed in white in the image, while the remaining parts are marked as "0" and are displayed in black in the image.
C2、将陆地和水域分割图、各舰船目标包络分割图表示为矩阵;C2. Express the land and water segmentation map and the target envelope segmentation map of each ship as a matrix;
C3、将各舰船目标包络分割图对应的矩阵与陆地和水域分割图对应的矩阵点乘后再将结果矩阵的元素求和,若求和结果大于舰船目标包络分割图尺寸的预设百分比(可以根据实际需求进行设置,本实施例中设置为60%),则判定该去重后的舰船目标初步检测识别结果为错误检测结果,删除该错误检测结果,否则保留该去重后的舰船目标初步检测识别结果。C3. Dot-multiply the matrix corresponding to each ship's target envelope segmentation map with the matrix corresponding to the land and water segmentation map, and then sum the elements of the result matrix. If the summation result is larger than the predetermined size of the ship's target envelope segmentation map, Assuming a percentage (which can be set according to actual needs, and is set to 60% in this embodiment), it is determined that the initial detection and identification result of the ship target after deduplication is an erroneous detection result, and the erroneous detection result is deleted, otherwise the deduplication is retained. The subsequent preliminary detection and recognition results of ship targets.
由此,可将可能存在的陆地上的误识别舰船目标进行剔除。Thus, possible misidentified ship targets on land can be eliminated.
八、多时相检测识别结果比对:8. Comparison of multi-temporal detection and recognition results:
结合码头地理位置,将本次舰船目标最终检测识别结果与上次卫星重访时得到的舰船目标最终检测识别结果进行比对,给出感兴趣的舰船在近岸海面上以及各码头上的活动情况,如舰船离开码头、舰船进入码头、有舰船在靠近码头水域活动。Combined with the geographical location of the dock, the final detection and identification results of this ship target are compared with the final detection and identification results of the ship target obtained during the last satellite revisit, and the locations of the ships of interest on the coastal sea and at each dock are given. Activities on the dock, such as ships leaving the dock, ships entering the dock, and ships operating in the waters near the dock.
九、舰船变化情况确认:9. Confirmation of ship changes:
根据舰船目标活动识别结果确定码头舰船是否存在变化,若是,则在码头及近岸区域遥感影像中标出变化情况,并保存影像。Based on the ship target activity recognition results, it is determined whether there are changes in the dock ships. If so, the changes are marked in the remote sensing images of the dock and nearshore areas, and the images are saved.
十、舰船活动再次识别:10. Re-identification of ship activities:
下一次卫星重访至该港口位置时,再次执行一至九的步骤。The next time the satellite revisits the port location, perform steps one to nine again.
实施例二Embodiment 2
本实施例中,提供了一种面向微小卫星在轨应用的港口大中型舰船活动识别系统,如图10所示,包括电源控制单元、嵌入式人工智能处理单元(包括CPU和GPU处理器)、高速数据采集与存储接口单元(FPGA)、内存(RAM)和固态硬盘(SSD)、原始遥感影像数据读入总线、外部控制总线和外部数据总线等。In this embodiment, a port large and medium-sized ship activity identification system for on-orbit applications of micro-satellites is provided, as shown in Figure 10, including a power control unit and an embedded artificial intelligence processing unit (including a CPU and a GPU processor) , high-speed data acquisition and storage interface unit (FPGA), memory (RAM) and solid-state drive (SSD), raw remote sensing image data read-in bus, external control bus and external data bus, etc.
(1)高速数据采集与存储接口单元(FPGA),一方面能够为原始遥感影像数据的采集读入提供高速接口支持,另一方面能够为CPU或GPU与存储设备(内存RAM、固态硬盘SSD)间的数据读写、存储等提供高速接口支持。(1) High-speed data acquisition and storage interface unit (FPGA), on the one hand, it can provide high-speed interface support for the acquisition and reading of original remote sensing image data, on the other hand, it can provide high-speed interface support for CPU or GPU and storage devices (memory RAM, solid state drive SSD) Provides high-speed interface support for data reading, writing, and storage between devices.
能够为原是遥感影像数据的读入提供高速总线,也能够为嵌入式人工智能处理单元实现与内存RAM或固态硬盘SSD间的数据读写提供内部高速总线。It can provide a high-speed bus for reading the original remote sensing image data, and it can also provide an internal high-speed bus for the embedded artificial intelligence processing unit to read and write data with the memory RAM or solid-state drive SSD.
(2)固态硬盘(SSD)用于存储程序运行所需的操作系统等基础文件,以及程序运行后生成的图像、提取的舰船目标切片影像等,主要包含:用于开展舰船目标检测识别的深度学习网络结构及模型参数文件;用于开展新获取港口遥感影像辐射校正和几何校正相关参数文件、几何精纠正操作所需的参考基准影像等;程序运行所需的操作系统、软件库等;经过预处理(辐射校正、几何校正、几何精纠正)后的港口码头及近岸区域遥感影像;程序运行后,裁剪出的舰船目标切片影像等结果数据。(2) Solid-state drive (SSD) is used to store basic files such as the operating system required for program running, as well as images generated after the program is run, extracted ship target slice images, etc., mainly including: used for ship target detection and recognition Deep learning network structure and model parameter files; used to carry out newly acquired port remote sensing image radiation correction and geometric correction related parameter files, reference benchmark images required for geometric fine correction operations, etc.; operating system, software library, etc. required for program operation ;Remote sensing images of port terminals and nearshore areas after preprocessing (radiation correction, geometric correction, geometric fine correction); after the program is run, the result data such as cropped ship target slice images.
(3)内存RAM用于缓存读入的原始遥感影像数据,该数据由成像载荷经数据读取总线、高速数据采集与存储接口单元后,存进内存RAM中。(3) The memory RAM is used to cache the read original remote sensing image data. The data is stored in the memory RAM by the imaging load after passing through the data reading bus and the high-speed data acquisition and storage interface unit.
(4)嵌入式人工智能计算单元主要由CPU和GPU组成,其上运行Linux内核操作系统,以及图像预处理(辐射校正、几何校正、几何精纠正)、舰船目标智能检测识别相关应用程序,提供了较强能力的程序控制管理、数据流处理、图像处理并行计算等功能;经高速数据采集与存储接口单元,可读取内存RAM中数据、读写固态硬盘SSD中数据;提供了控制总线和数据总线,能与卫星平台进行任务控制、数据传输等交互操作。(4) The embedded artificial intelligence computing unit is mainly composed of CPU and GPU, which runs the Linux kernel operating system, as well as image preprocessing (radiation correction, geometric correction, geometric fine correction), and ship target intelligent detection and recognition related applications. It provides powerful program control management, data flow processing, image processing and parallel computing functions; through the high-speed data acquisition and storage interface unit, it can read data in the memory RAM and read and write data in the solid state drive SSD; provides a control bus and data bus, which can interact with the satellite platform for mission control, data transmission and other operations.
承担了遥感影像预处理、舰船目标智能检测识别、舰船目标活动识别相关智能算法的计算处理,同时也为本系统与卫星平台端进行交互提供了控制总线和数据总线。It is responsible for the calculation and processing of intelligent algorithms related to remote sensing image preprocessing, intelligent detection and identification of ship targets, and identification of ship target activities. It also provides a control bus and data bus for the interaction between this system and the satellite platform.
(5)电源控制单元,通过对卫星平台端提供的外部电源进行电压调整,能够为系统内各单元提供其所需的供电电压和电流。(5) The power control unit can provide the required power supply voltage and current for each unit in the system by adjusting the voltage of the external power supply provided by the satellite platform.
本实施例中,该系统的嵌入式人工智能处理单元和高速数据采集与存储接口单元之间,高速数据采集与存储接口单元和内存RAM、固态硬盘之间,是通过内部总线进行数据交换的。该系统可作为卫星的一个子系统,兼容各类遥感卫星平台,移植性好、兼容性强。In this embodiment, data is exchanged through an internal bus between the system's embedded artificial intelligence processing unit and the high-speed data collection and storage interface unit, and between the high-speed data collection and storage interface unit and the memory RAM and solid-state hard drive. This system can be used as a subsystem of the satellite and is compatible with various remote sensing satellite platforms, with good portability and strong compatibility.
通过采用本发明公开的上述技术方案,得到了如下有益的效果:By adopting the above technical solutions disclosed in the present invention, the following beneficial effects are obtained:
本发明提供了一种面向微小类型在轨应用的港口大中型舰船活动识别方法及系统,相比传统的图像变化检测方法,本方法能够应对近岸港口环境背景复杂、光照条件多样(白天不同时间下)、舰船目标呈现在遥感影像中的角度可能是任意的、多艘舰船目标可能密集停靠等特点和难点,能保证对感兴趣舰船目标进行准确的检测识别,通过对卫星先后多次重访时获得的多时相遥感影像进行处理和比对,实现感兴趣舰船目标活动的识别。相比传统遥感数据下传、地面处理、上行控制等繁琐的数据处理与应用流程,该方法能够大幅压缩开展该类任务的遥感影像数据传输量和任务完成所需耗时。该系统组成简单、成本较低,能够为微小卫星实现港口舰船的在轨解译应用提供模块化的解决方案,系统移植性、兼容性好。在提高遥感任务时效性、智能性的同时,也大幅降低了星地数据传输压力,实现遥感任务综合效益的显著提升。The present invention provides a method and system for identifying large and medium-sized ship activities in ports for micro-type on-orbit applications. Compared with traditional image change detection methods, this method can cope with the complex background of the near-shore port environment and diverse lighting conditions (different during the day). time), the angle at which ship targets appear in remote sensing images may be arbitrary, and multiple ship targets may be docked densely. This can ensure accurate detection and identification of ship targets of interest. By successively analyzing satellites Multi-temporal remote sensing images obtained during multiple revisits are processed and compared to identify the target activities of ships of interest. Compared with traditional remote sensing data downlink, ground processing, uplink control and other cumbersome data processing and application processes, this method can significantly reduce the amount of remote sensing image data transmission and the time required to complete the task. The system has a simple composition and low cost. It can provide a modular solution for micro-satellites to implement on-orbit interpretation applications for port ships. The system has good portability and compatibility. While improving the timeliness and intelligence of remote sensing missions, it also significantly reduces the pressure on satellite-to-ground data transmission and significantly improves the comprehensive benefits of remote sensing missions.
以上所述仅是本发明的优选实施方式,应当指出,对于本技术领域的普通技术人员来说,在不脱离本发明原理的前提下,还可以做出若干改进和润饰,这些改进和润饰也应视本发明的保护范围。The above are only preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications can also be made. The scope of protection of the present invention should be considered.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310876541.XA CN116883861B (en) | 2023-07-17 | 2023-07-17 | Method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310876541.XA CN116883861B (en) | 2023-07-17 | 2023-07-17 | Method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites |
Publications (2)
Publication Number | Publication Date |
---|---|
CN116883861A true CN116883861A (en) | 2023-10-13 |
CN116883861B CN116883861B (en) | 2024-01-26 |
Family
ID=88261792
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310876541.XA Active CN116883861B (en) | 2023-07-17 | 2023-07-17 | Method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116883861B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118135426A (en) * | 2024-05-07 | 2024-06-04 | 浙江四港联动发展有限公司 | Port water area traffic condition identification method based on satellite image and SAM |
CN119168841A (en) * | 2024-11-12 | 2024-12-20 | 陕西航天技术应用研究院有限公司 | A ship extraction acceleration method based on wide-area satellite images |
CN119291737A (en) * | 2024-09-27 | 2025-01-10 | 中国人民解放军军事航天部队航天工程大学 | A method and system for relay tracking application of large and medium-sized ships in the open sea |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103679138A (en) * | 2013-11-15 | 2014-03-26 | 中国科学院遥感与数字地球研究所 | Ship and port prior knowledge supported large-scale ship detection method |
CN108256419A (en) * | 2017-12-05 | 2018-07-06 | 交通运输部规划研究院 | A kind of method for extracting port and pier image using multispectral interpretation |
CN110414509A (en) * | 2019-07-25 | 2019-11-05 | 中国电子科技集团公司第五十四研究所 | Stop Ship Detection in harbour based on the segmentation of extra large land and feature pyramid network |
US20200365035A1 (en) * | 2019-05-13 | 2020-11-19 | Hoseo University Academic Cooperation Foundation | Method and system for recognition of objects near ship by using deep neural network |
CN112329542A (en) * | 2020-10-10 | 2021-02-05 | 中国人民解放军战略支援部队航天工程大学 | SAR image ship target detection method based on feature refinement network model |
CN113920436A (en) * | 2021-11-22 | 2022-01-11 | 江苏科技大学 | A remote sensing image marine ship identification system and method based on improved YOLOv4 algorithm |
WO2022075530A1 (en) * | 2020-10-06 | 2022-04-14 | 주식회사 스카이시스 | Automatic vessel recognition and monitoring system using artificial intelligence image processing, and method for providing same |
CN114494829A (en) * | 2022-01-16 | 2022-05-13 | 西安电子科技大学 | SAR image multi-scale ship target detection method |
-
2023
- 2023-07-17 CN CN202310876541.XA patent/CN116883861B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103679138A (en) * | 2013-11-15 | 2014-03-26 | 中国科学院遥感与数字地球研究所 | Ship and port prior knowledge supported large-scale ship detection method |
CN108256419A (en) * | 2017-12-05 | 2018-07-06 | 交通运输部规划研究院 | A kind of method for extracting port and pier image using multispectral interpretation |
US20200365035A1 (en) * | 2019-05-13 | 2020-11-19 | Hoseo University Academic Cooperation Foundation | Method and system for recognition of objects near ship by using deep neural network |
CN110414509A (en) * | 2019-07-25 | 2019-11-05 | 中国电子科技集团公司第五十四研究所 | Stop Ship Detection in harbour based on the segmentation of extra large land and feature pyramid network |
WO2022075530A1 (en) * | 2020-10-06 | 2022-04-14 | 주식회사 스카이시스 | Automatic vessel recognition and monitoring system using artificial intelligence image processing, and method for providing same |
CN112329542A (en) * | 2020-10-10 | 2021-02-05 | 中国人民解放军战略支援部队航天工程大学 | SAR image ship target detection method based on feature refinement network model |
CN113920436A (en) * | 2021-11-22 | 2022-01-11 | 江苏科技大学 | A remote sensing image marine ship identification system and method based on improved YOLOv4 algorithm |
CN114494829A (en) * | 2022-01-16 | 2022-05-13 | 西安电子科技大学 | SAR image multi-scale ship target detection method |
Non-Patent Citations (3)
Title |
---|
HUIXUAN FU ET AL.: "《Ship detection based on Improved YOLO Algorithm》", 《2021 PROCEEDINGS OF THE 40TH CHINESE CONTROL CONFERENCE (CCC)》, pages 8181 - 8186 * |
QUANCHENG ZHOU ET AL.: "《Harbor Ship Detection Based on Channel Weighting and Spatial Information Fusion》", 《2020 2ND INTERNATIONAL CONFERENCE ON ELECTRONICS AND COMMUNICATION, NETWORK AND COMPUTER TECHNOLOGY (ECNCT)》, vol. 1738, pages 1742 - 6596 * |
张雪阳等: "《基于运动信息和深度学习的遥感视频多目标检测跟踪方法》", 《第三届中国空天安全会议论文集》, pages 295 - 303 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118135426A (en) * | 2024-05-07 | 2024-06-04 | 浙江四港联动发展有限公司 | Port water area traffic condition identification method based on satellite image and SAM |
CN118135426B (en) * | 2024-05-07 | 2024-07-05 | 浙江四港联动发展有限公司 | Port water area traffic condition identification method based on satellite image and SAM |
CN119291737A (en) * | 2024-09-27 | 2025-01-10 | 中国人民解放军军事航天部队航天工程大学 | A method and system for relay tracking application of large and medium-sized ships in the open sea |
CN119291737B (en) * | 2024-09-27 | 2025-06-24 | 中国人民解放军军事航天部队航天工程大学 | A method and system for relay tracking application of large and medium-sized ships in the open sea |
CN119168841A (en) * | 2024-11-12 | 2024-12-20 | 陕西航天技术应用研究院有限公司 | A ship extraction acceleration method based on wide-area satellite images |
Also Published As
Publication number | Publication date |
---|---|
CN116883861B (en) | 2024-01-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN116883861B (en) | Method and system for identifying large and medium-sized ship activities in ports for on-orbit applications of micro-satellites | |
Hodúl et al. | Satellite derived photogrammetric bathymetry | |
CN111985376A (en) | A deep learning-based method for extracting ship contours from remote sensing images | |
CN110889198B (en) | A method and system for predicting flight position probability distribution based on multi-factor joint learning | |
Thompson et al. | Efficient lidar-based object segmentation and mapping for maritime environments | |
Heidarsson et al. | Obstacle detection from overhead imagery using self-supervised learning for autonomous surface vehicles | |
CN114241377A (en) | Ship target detection method, device, equipment and medium based on improved YOLOX | |
CN211787203U (en) | Agricultural insurance survey unmanned aerial vehicle device, rotor and fixed wing unmanned aerial vehicle flight platform | |
CN117274826A (en) | River and lake management violation problem remote sensing monitoring method based on large model and prompt guidance | |
CN113962473A (en) | Ship route planning method and device, electronic equipment and storage medium | |
CN113011295A (en) | Method, computer equipment and medium for identifying photovoltaic power station based on remote sensing image | |
Xu et al. | Deep learning guided building reconstruction from satellite imagery-derived point clouds | |
Thompson | Maritime object detection, tracking, and classification using lidar and vision-based sensor fusion | |
WO2024153703A1 (en) | Flood segmentation method using synthetic aperture radar images and digital elevation models | |
Ruiz-Lendínez et al. | Deep learning methods applied to digital elevation models: state of the art | |
Jiang et al. | Pavement crack measurement based on aerial 3D reconstruction and learning-based segmentation method | |
US20220214689A1 (en) | Autonomous Vessel and Infrastructure for Supporting an Autonomous Vessel on Inland Waterways | |
CN114445572B (en) | A method for real-time positioning and mapping of obstacles in unfamiliar waters based on DeeplabV3+ | |
CN109116351A (en) | A kind of spaceborne InSAR positioning analytical algorithm | |
CN117788572A (en) | Fused ship precise positioning method and system of optical image and laser point cloud | |
Feng et al. | Multi-scale building maps from aerial imagery | |
CN117152532A (en) | Power transmission line section labeling method, device, computer equipment and storage medium | |
CN113240026B (en) | Vector-grid combined inland water surface floater batch identification and extraction method | |
Liu et al. | Visual Localization Method for Unmanned Aerial Vehicles in Urban Scenes Based on Shape and Spatial Relationship Matching of Buildings | |
Li et al. | Automatic detection of beacon towers in historical aerial images using improved FCOS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |