CN117853932A - Sea surface target detection method, detection platform and system based on photoelectric pod - Google Patents

Sea surface target detection method, detection platform and system based on photoelectric pod Download PDF

Info

Publication number
CN117853932A
CN117853932A CN202410248872.3A CN202410248872A CN117853932A CN 117853932 A CN117853932 A CN 117853932A CN 202410248872 A CN202410248872 A CN 202410248872A CN 117853932 A CN117853932 A CN 117853932A
Authority
CN
China
Prior art keywords
sea surface
image
target detection
image data
surface target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410248872.3A
Other languages
Chinese (zh)
Other versions
CN117853932B (en
Inventor
鲁辛凯
陈立群
颜露新
李佼佼
朱颖盼
王震宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202410248872.3A priority Critical patent/CN117853932B/en
Publication of CN117853932A publication Critical patent/CN117853932A/en
Application granted granted Critical
Publication of CN117853932B publication Critical patent/CN117853932B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07Target detection
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a sea surface target detection method, a detection platform and a system based on a photoelectric pod, which belong to the field of sea surface target detection and comprise the following steps: for the aggregated multiband sea surface image data transmitted by the optoelectronic pod, performing: depolymerizing it into a plurality of single-pass image data; respectively executing sea clutter self-adaptive suppression steps on each single-path image data to obtain a denoised image; respectively carrying out target detection on the denoised images under the same focal length, and fusing detection results; the sea clutter self-adaptive suppression step comprises the following steps: performing top hat transformation on the image data to be processed, calculating gradients of the image data in different directions, and taking the minimum pixel value of a plurality of gradient images at the same position as the pixel value of the denoised image at the position; and adjusting the pose of the tele lens by using the detection result under the wide-angle lens. The invention can fully utilize the multichannel sea surface image data acquired by the photoelectric pod, inhibit sea surface clutter therein and improve the detection precision of sea surface targets.

Description

一种基于光电吊舱的海面目标检测方法、检测平台及系统A sea surface target detection method, detection platform and system based on optoelectronic pod

技术领域Technical Field

本发明属于海面目标检测领域,更具体地,涉及一种基于光电吊舱的海面目标检测方法、检测平台及系统。The present invention belongs to the field of sea surface target detection, and more specifically, relates to a sea surface target detection method, a detection platform and a system based on an optoelectronic pod.

背景技术Background technique

海上目标检测技术在民用和军用等领域中都有较大的研究意义,一方面,随着台风等气候因素以及海洋油气开发等活动的不断扩展深入,水上交通事故的数量呈上升趋势,恶劣的海上环境极其容易造成重大经济损失和人员的伤亡,因此快速识别与跟踪海上遇险目标,保障安全的航行环境至关重要。另一方面,海上防护意义重大,快速定位监控和打击入侵船只才能更好的掌控海上主动权,因此需要对复杂的海面环境进行高性能、全方位的监测,对海上目标实现智能检测是保护海防安全的关键。Maritime target detection technology has great research significance in both civil and military fields. On the one hand, with the continuous expansion and deepening of climate factors such as typhoons and marine oil and gas development activities, the number of water traffic accidents is on the rise. The harsh marine environment is extremely likely to cause major economic losses and casualties. Therefore, it is very important to quickly identify and track targets in distress at sea and ensure a safe navigation environment. On the other hand, maritime protection is of great significance. Rapid positioning, monitoring and combating of invading ships can better control the initiative at sea. Therefore, high-performance and all-round monitoring of the complex sea environment is required. Intelligent detection of maritime targets is the key to protecting coastal defense security.

海上目标检测通常存在背景复杂的问题,海面背景的复杂杂波中不仅有高亮度的海面亮条、密集的海面鱼鳞状亮斑,还有海天异质复杂背景和岛屿杂波等背景杂波噪声,使得目标混淆于复杂海面杂波背景噪声从而增加了海面目标的检测难度。Marine target detection usually has the problem of complex background. The complex clutter of the sea surface background includes not only high-brightness sea surface bright strips and dense sea surface fish-scale bright spots, but also background clutter noise such as the heterogeneous and complex background of the sea and sky and island clutter. The target is confused by the complex sea surface clutter background noise, thereby increasing the difficulty of sea surface target detection.

随着短波红外、可见光、中波红外、长波红外等光电探测器的小型化和多样化,可以更加方便的获取不同波段的图像数据。而不同波段图像中信息也存在一些差异性,比如红外光的穿透性使得红外图像可以根据辐射差异将目标与背景区分开,因此红外图像在夜间能获取到更有效的信息,而可见光可以获取更符合人类视觉系统的纹理和颜色信息。同样对于镜头的选择也更丰富多样:广角镜头具有焦距短、视角大的特点。可以捕捉到更多的景象,而长焦镜头焦距长、视角小,可以清楚的成像远处的目标。对于不同波段的探测器和不同的镜头组合,如何对其图像数据进行处理可以更加高效的利用多通道图像的特性进而得到更精准的检测与识别结果尤为重要。With the miniaturization and diversification of photoelectric detectors such as short-wave infrared, visible light, medium-wave infrared, and long-wave infrared, it is easier to obtain image data in different bands. There are also some differences in the information in images of different bands. For example, the penetration of infrared light allows infrared images to distinguish targets from backgrounds based on radiation differences. Therefore, infrared images can obtain more effective information at night, while visible light can obtain texture and color information that is more in line with the human visual system. Similarly, the choice of lenses is more diverse: wide-angle lenses have the characteristics of short focal length and large viewing angle. It can capture more scenes, while telephoto lenses have long focal lengths and small viewing angles, and can clearly image distant targets. For detectors of different bands and different lens combinations, it is particularly important to process their image data so that the characteristics of multi-channel images can be more efficiently utilized to obtain more accurate detection and recognition results.

光电吊舱可根据任务需要灵活配置,可分别采用短波红外、可见光、长波红外探测器与长焦、广角镜头的不同组合配置,采集到不同波段、不同焦距的海面图像数据。在申请公布号为CN116248705A的专利文件中,公开了一种微型光电吊舱多通道图像传输与处理系统,如图1所示,其包括多路光电探测器、FPGA单元A以及图像聚合单元,多路光电探测器用于采集不同波段的光子信号,通过光电转换电路生成图像电信号,并将多路图像信号输出到FPGA单元A;FPGA单元A用于实现多路图像信号的采集,生成多路图像数据并输出到图像聚合单元;图像聚合单元将多路图像数据聚合为聚合图像数据后输出到连接微型光电吊舱和远端数据处理平台(例如飞机平台)的单通道高速链路,由此实现多路探测器图像数据的同步实时传输、处理、转发或存储,具有实时性好、传输速率高、链路成本低、灵活性好等特点。利用该微型光电吊舱采集海面图像数据,有利于提高海面目标检测效果,但是如何对多路图像数据进行有效处理以准确完成海面目标检测,仍缺乏有效的方法。The optoelectronic pod can be flexibly configured according to the mission requirements, and can respectively adopt different combinations of short-wave infrared, visible light, and long-wave infrared detectors and telephoto and wide-angle lenses to collect sea surface image data of different bands and different focal lengths. In the patent document with application publication number CN116248705A, a multi-channel image transmission and processing system for a micro optoelectronic pod is disclosed, as shown in FIG1, which includes a multi-channel photoelectric detector, an FPGA unit A, and an image aggregation unit. The multi-channel photoelectric detector is used to collect photon signals of different bands, generate image electrical signals through a photoelectric conversion circuit, and output the multi-channel image signals to the FPGA unit A; the FPGA unit A is used to realize the collection of multi-channel image signals, generate multi-channel image data and output it to the image aggregation unit; the image aggregation unit aggregates the multi-channel image data into aggregated image data and then outputs it to a single-channel high-speed link connecting the micro optoelectronic pod and a remote data processing platform (such as an aircraft platform), thereby realizing the synchronous real-time transmission, processing, forwarding or storage of multi-channel detector image data, which has the characteristics of good real-time performance, high transmission rate, low link cost, and good flexibility. Using the micro optoelectronic pod to collect sea surface image data is beneficial to improving the sea surface target detection effect, but there is still a lack of effective methods for effectively processing multi-channel image data to accurately complete sea surface target detection.

发明内容Summary of the invention

针对现有技术的缺陷和改进需求,本发明提供了一种基于光电吊舱的海面目标检测方法、检测平台及系统,其目的在于,充分利用光电吊舱所采集的多通道海面图像数据,并抑制其中的海面杂波,以提高海面目标的检测精度。In view of the defects of the prior art and the need for improvement, the present invention provides a sea surface target detection method, detection platform and system based on an optoelectronic pod, the purpose of which is to make full use of the multi-channel sea surface image data collected by the optoelectronic pod and suppress the sea surface clutter therein to improve the detection accuracy of sea surface targets.

为实现上述目的,按照本发明的一个方面,提供了一种基于光电吊舱的海面目标检测方法,包括:To achieve the above object, according to one aspect of the present invention, a method for detecting sea surface targets based on an optoelectronic pod is provided, comprising:

持续接收由光电吊舱发送的海面图像数据;对于所接收到的每一帧聚合后的多波段海面图像数据,执行以下步骤:Continuously receive the sea surface image data sent by the optoelectronic pod; for each frame of aggregated multi-band sea surface image data received, perform the following steps:

(S1)将其解聚为多个单路图像数据;每一个单路图像数据对应光电吊舱采集的一个焦距下的一个波段的图像数据;(S1) deaggregating it into a plurality of single-channel image data; each single-channel image data corresponds to image data of a band at a focal length collected by the optoelectronic pod;

(S2)对各单路图像数据分别执行海面杂波自适应抑制步骤,得到对应的去噪后图像;海面杂波自适应抑制步骤包括:(S2) performing a sea surface clutter adaptive suppression step on each single channel image data to obtain a corresponding denoised image; the sea surface clutter adaptive suppression step includes:

(S21)对待处理的图像数据进行顶帽变换,得到初步去噪图像;(S21) performing a top-hat transformation on the image data to be processed to obtain a preliminary denoised image;

(S22)计算初步去噪图像在不同方向的梯度,得到多张梯度图像;(S22) calculating the gradient of the preliminary denoised image in different directions to obtain a plurality of gradient images;

(S23)将多张梯度图像在相同位置的最小像素值作为去噪后图像在该位置的像素值,得到待处理的图像所对应的去噪后图像;(S23) taking the minimum pixel value of the multiple gradient images at the same position as the pixel value of the denoised image at the position, and obtaining the denoised image corresponding to the image to be processed;

(S3)对于同一焦距下的去噪后图像,分别进行目标检测后,将检测结果融合,将融合结果作为相应焦距下的海面目标检测结果。(S3) After performing target detection on the denoised images at the same focal length, the detection results are fused and the fused results are used as the sea surface target detection results at the corresponding focal length.

进一步地,步骤(S22)包括:Further, step (S22) includes:

对初步去噪图像进行连通域标记,将小于预设阈值的连通域剔除后,计算剩余连通域的平均大小CMark the connected domains of the preliminary denoised image, remove the connected domains smaller than the preset threshold, and calculate the average size C of the remaining connected domains;

按照ks=C//M计算梯度卷积核大小ks,并按照所计算的梯度卷积核大小确定各方向下的梯度卷积核;Calculate the gradient convolution kernel size ks according to ks = C // M , and determine the downward gradient convolution kernels in all directions according to the calculated gradient convolution kernel size;

利用各方向下的梯度卷积核对初步去噪图像进行卷积操作以计算初步去噪图像在不同方向的梯度,得到多张梯度图像;The preliminary denoised image is convolved using the downward gradient convolution kernel in all directions to calculate the gradient of the preliminary denoised image in different directions, thereby obtaining multiple gradient images;

其中,“//”表示整除运算;M为预设正整数。Wherein, “//” represents integer division operation; M is a preset positive integer.

进一步地,M=5。Furthermore, M =5.

进一步地,本发明提供的基于光电吊舱的海面目标检测方法,在步骤(S3)之后还包括:Furthermore, the method for detecting sea surface targets based on an optoelectronic pod provided by the present invention further comprises, after step (S3):

(S4)根据较短焦距下海面目标检测结果计算海面目标的中心位置,并计算海面目标的中心位置相对于图像中心的偏移量;(S4) calculating the center position of the sea surface target according to the sea surface target detection result at the shorter focal length, and calculating the offset of the center position of the sea surface target relative to the center of the image;

(S5)向光电吊舱发送指令,以使光电吊舱按照偏移量调整其中较长焦距的镜头的位姿,使该焦距下海面目标的中心位置与图像中心位置重合。(S5) Sending a command to the optoelectronic pod so that the optoelectronic pod adjusts the posture of the lens with the longer focal length according to the offset, so that the center position of the sea surface target at the focal length coincides with the center position of the image.

进一步地,步骤(S3)中,将同一焦距下的目标检测结果融合,包括:Furthermore, in step (S3), the target detection results at the same focal length are fused, including:

计算该焦距下,各去噪后图像经目标检测所得检测框之间的重叠度,将重叠度大于预设的高阈值的检测框作为可信目标输出,将重叠度小于预设的低阈值的检测框作为虚警剔除;Calculate the overlap between the detection frames obtained by target detection in each denoised image at the focal length, output the detection frames with an overlap greater than a preset high threshold as credible targets, and remove the detection frames with an overlap less than a preset low threshold as false alarms;

根据当前帧之前的海面图像数据的海面目标检测结果序列确定海面目标的轨迹,对于重叠度位于低阈值和高阈值之间的检测框,若其位于海面目标的轨迹上,则作为可信目标输出,否则,作为虚警剔除。The trajectory of the sea surface target is determined according to the sea surface target detection result sequence of the sea surface image data before the current frame. For the detection frame with an overlap between the low threshold and the high threshold, if it is located on the trajectory of the sea surface target, it is output as a credible target, otherwise it is rejected as a false alarm.

进一步地,步骤(S3)中,若去噪后图像为可见光波段图像,则采用YOLOv5进行目标检测;若去噪后图像为红外波段图像,则采用多尺度块对比度测度算法进行检测。Furthermore, in step (S3), if the denoised image is a visible light band image, YOLOv5 is used for target detection; if the denoised image is an infrared band image, a multi-scale block contrast measurement algorithm is used for detection.

按照本发明的又一个方面,提供了一种基于光电吊舱的海面目标检测平台,包括:According to another aspect of the present invention, a sea surface target detection platform based on an optoelectronic pod is provided, comprising:

计算机可读存储介质,用于存储计算机程序;A computer-readable storage medium for storing a computer program;

以及处理器,用于读取计算机可读存储介质中存储的计算机程序,执行本发明提供的基于光电吊舱的海面目标检测方法。and a processor, which is used to read a computer program stored in a computer-readable storage medium and execute the sea surface target detection method based on an optoelectronic pod provided by the present invention.

按照本发明的又一个方面,提供了一种海面目标检测系统,包括:According to another aspect of the present invention, there is provided a sea surface target detection system, comprising:

本发明提供的上述基于光电吊舱的海面目标检测平台;The above-mentioned sea surface target detection platform based on optoelectronic pod provided by the present invention;

以及光电吊舱;and optoelectronic pods;

光电吊舱与海面目标检测平台之间通过单通道链路传输图像数据流。The image data stream is transmitted between the optoelectronic pod and the sea surface target detection platform via a single-channel link.

总体而言,通过本发明所构思的以上技术方案,能够取得以下有益效果。In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects.

(1)本发明对于由光电吊舱所采集的海面图像数据,首先对其进行顶帽变换,可有效过滤掉海面上的海面鱼鳞波、亮条等亮度均匀、纹理简单的海杂波噪声,之后基于方向性的梯度计算确定图像中各像素的具体取值,由此能够有效抑制掉图像中与目标尺寸接近但形状不一致的噪声,二者相互结合能够有效抑制海面图像数据中的海杂波噪声,在此基础上,对去噪后的多路图像数据分别进行目标检测,并将同一焦距下不同波段图像的目标检测结果作为相应焦距下的海面目标检测结果,由此能够充分利用不同波段、不同焦距下图像的信息,有效提高海面目标检测的精度。(1) The present invention first performs a top-hat transformation on the sea surface image data collected by the optoelectronic pod, which can effectively filter out sea clutter noise with uniform brightness and simple texture, such as fish scale waves and bright stripes on the sea surface. Then, the specific value of each pixel in the image is determined based on the directional gradient calculation, thereby effectively suppressing the noise in the image that is close to the target size but inconsistent with the shape. The combination of the two can effectively suppress the sea clutter noise in the sea surface image data. On this basis, target detection is performed on the denoised multi-channel image data respectively, and the target detection results of images of different bands at the same focal length are used as the sea surface target detection results at the corresponding focal length. In this way, the information of images at different bands and focal lengths can be fully utilized, and the accuracy of sea surface target detection can be effectively improved.

(2)在本发明的优选方案中,在基于方向性的梯度计算抑制图像中与目标尺寸接近但形状不一致的噪声时,先对图像进行连通域标记,并过滤掉尺寸过小的连通域,再基于剩余连通域的平均大小确定用于计算梯度的梯度卷积核大小,由此能够在不同场景下自适应待检测的海面目标的大小,有效提高海面目标检测的精度和鲁棒性。在本发明进一步的优选方案中,进一步设定梯度卷积核为连通域平均大小整除5后的结果,实验数据表明,基于该梯度卷积核大小计算梯度,能够最大程度上抑制海面背景杂波。(2) In the preferred embodiment of the present invention, when suppressing noise in an image that is close in size to the target but inconsistent in shape based on directional gradient calculation, the image is first marked for connected domains, and connected domains that are too small are filtered out. Then, the size of the gradient convolution kernel used to calculate the gradient is determined based on the average size of the remaining connected domains. This can adapt to the size of the sea surface target to be detected in different scenarios, effectively improving the accuracy and robustness of sea surface target detection. In a further preferred embodiment of the present invention, the gradient convolution kernel is further set to the result of the average size of the connected domain divided by 5. Experimental data show that the gradient calculated based on the gradient convolution kernel size can suppress the sea surface background clutter to the greatest extent.

(3)较短焦距下,视场较大且噪声和干扰物较多,目标尺寸小,检测难度大,而较长焦距下,视场较小,噪声和干扰物较小,目标清晰,检测难度小,但海面目标有可能位于视场之外,在本发明的优选方案中,会利用较短焦距下海面目标检测结果所计算的海面目标中心相对于图像中心的偏移量来调整较长焦距的镜头位姿,由此能够利用长焦镜头获得更高质量的目标图像数据,进而得到更加准确的长焦距下海面目标检测结果。(3) At a shorter focal length, the field of view is larger and there are more noise and interference, the target size is small, and the detection difficulty is large. At a longer focal length, the field of view is smaller, the noise and interference are smaller, the target is clear, and the detection difficulty is small. However, the sea surface target may be located outside the field of view. In the preferred embodiment of the present invention, the offset of the sea surface target center relative to the image center calculated from the sea surface target detection result at a shorter focal length is used to adjust the lens posture of the longer focal length, thereby obtaining higher quality target image data using a telephoto lens, thereby obtaining more accurate sea surface target detection results at a longer focal length.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1为现有的光电吊舱与飞机平台间数据连接关系示意图。FIG. 1 is a schematic diagram of the data connection relationship between the existing optoelectronic pod and the aircraft platform.

图2为本发明实施例提供的基于光电吊舱的海面目标检测方法示意图。FIG2 is a schematic diagram of a method for detecting sea surface targets based on an optoelectronic pod according to an embodiment of the present invention.

图3为本发明实施例提供的顶帽变换前的原始图像。FIG. 3 is an original image before top-hat transformation provided by an embodiment of the present invention.

图4为图3所示原始图像经顶帽变换后的图像。FIG. 4 is an image of the original image shown in FIG. 3 after being top-hat transformed.

图5为本发明实施例提供的8个方向下的梯度卷积核示意图。FIG5 is a schematic diagram of gradient convolution kernels in eight directions provided by an embodiment of the present invention.

图6为图4所示图像在0°方向的梯度图。FIG. 6 is a gradient diagram of the image shown in FIG. 4 in the 0° direction.

图7为图4所示图像在90°方向的梯度图。FIG. 7 is a gradient diagram of the image shown in FIG. 4 in the 90° direction.

具体实施方式Detailed ways

为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。此外,下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

在本发明中,本发明及附图中的术语“第一”、“第二”等(如果存在)是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。In the present invention, the terms "first", "second", etc. (if any) in the present invention and the drawings are used to distinguish similar objects but not necessarily to describe a specific order or sequence.

为了有效提高海面目标的检测精度,本发明提供了一种基于光电吊舱的海面目标检测方法、检测平台及系统,其整体思路在于,利用光电吊舱采集不同波段、不同焦距下的海面图像,并基于海面图像中海面杂波的特性,提出有效的海面杂波抑制方法,从而在有效抑制海面杂波的情况下充分利用多通道图像数据的信息,提高海面目标检测的精度。In order to effectively improve the detection accuracy of sea surface targets, the present invention provides a sea surface target detection method, detection platform and system based on an optoelectronic pod. The overall idea is to use the optoelectronic pod to collect sea surface images in different bands and focal lengths, and based on the characteristics of sea surface clutter in the sea surface images, an effective sea surface clutter suppression method is proposed, thereby making full use of the information of multi-channel image data while effectively suppressing the sea surface clutter, thereby improving the accuracy of sea surface target detection.

在实际应用中,光电吊舱多为微型或小型结构,其中的多路光电探测器用于采集不同波段的光子信号,对于不同波段的输入光子信号,通过光电转换电路生成图像电信号,然后通过图像聚合单元将多个单路图像数据聚合为一路高速图像数据,通过单通道高速链路实现对于图像数据流的高速传输。其光电吊舱中的光电探测器可根据任务需要灵活配置,可分别采用短波红外、可见光、长波红外等不同波段的探测器与广角镜头、长焦镜头等不同焦距镜头的组合。不失一般性地,在以下实施例中,采用的光电吊舱中,包括四个光电探测器,分别是可见光探测器与广角镜头的组合、红外探测器与广角镜头的组合、可见光探测器与长焦镜头的组合,以及红外探测器与长焦镜头的组合,因此,该光电吊舱一次可采集4路不同的海面图像数据。为便于描述,在以下实施例中,这四路探测器分别记为探测器1、探测器2、探测器3和探测器4。应当说明的是,此处的多路探测器仅为示例性的说明,在本发明其他的一些实施例中,也可采用具有不同的多路探测器的光电吊舱采集海面图像数据。In practical applications, optoelectronic pods are mostly micro or small structures, in which the multi-channel photoelectric detectors are used to collect photon signals of different bands. For input photon signals of different bands, image electrical signals are generated through photoelectric conversion circuits, and then multiple single-channel image data are aggregated into one high-speed image data through an image aggregation unit, and high-speed transmission of image data streams is achieved through a single-channel high-speed link. The photoelectric detectors in the optoelectronic pod can be flexibly configured according to task requirements, and can respectively use combinations of detectors of different bands such as short-wave infrared, visible light, and long-wave infrared and lenses of different focal lengths such as wide-angle lenses and telephoto lenses. Without loss of generality, in the following embodiments, the optoelectronic pod used includes four photoelectric detectors, namely, a combination of a visible light detector and a wide-angle lens, a combination of an infrared detector and a wide-angle lens, a combination of a visible light detector and a telephoto lens, and a combination of an infrared detector and a telephoto lens. Therefore, the optoelectronic pod can collect 4 different sea surface image data at a time. For ease of description, in the following embodiments, these four detectors are respectively recorded as detector 1, detector 2, detector 3, and detector 4. It should be noted that the multi-channel detectors herein are merely exemplary. In other embodiments of the present invention, an optoelectronic pod with different multi-channel detectors may also be used to collect sea surface image data.

以下为实施例。The following are examples.

实施例1:Embodiment 1:

一种基于光电吊舱的海面目标检测方法,如图2所示,包括:A method for detecting sea surface targets based on an optoelectronic pod, as shown in FIG2 , includes:

持续接收由光电吊舱发送的海面图像数据;对于所接收到的每一帧聚合后的多波段海面图像数据,执行以下步骤(S1)~(S3)。Continuously receive the sea surface image data sent by the optoelectronic pod; for each frame of aggregated multi-band sea surface image data received, perform the following steps (S1) to (S3).

本实施例的步骤(S1)包括:将其解聚为多个单路图像数据;每一个单路图像数据对应光电吊舱采集的一个焦距下的一个波段的图像数据;本实施例中,对于接收到的一帧海面图像数据进行解聚后,可得到四路图像数据,分别是可见光波段广角图像、红外波段广角图像、可见光波段长焦图像和红外波段长焦图像。可选地,本实施例中,可见光图像尺寸为1920*1080,红外图像尺寸为640*512。The step (S1) of this embodiment includes: deaggregating it into multiple single-channel image data; each single-channel image data corresponds to image data of a band at a focal length collected by the optoelectronic pod; in this embodiment, after deaggregating a frame of sea surface image data received, four channels of image data can be obtained, namely, a wide-angle image of a visible light band, a wide-angle image of an infrared band, a telephoto image of a visible light band, and a telephoto image of an infrared band. Optionally, in this embodiment, the size of the visible light image is 1920*1080, and the size of the infrared image is 640*512.

本实施例的步骤(S2)包括:对各单路图像数据分别执行海面杂波自适应抑制步骤,得到对应的去噪后图像。The step (S2) of this embodiment includes: performing a sea surface clutter adaptive suppression step on each single channel of image data to obtain a corresponding denoised image.

海面上亮度均匀、纹理简单的海杂波噪声,如亮条、海面鱼鳞波等,是海面杂波中的主要成分,会严重干扰海面目标的检测,因此,本实施例会首先抑制这些噪声,分析及相关的实验数据表明,顶帽变换可有效抑制这样的海杂波噪声;海面图像中亮度均匀、纹理简单的海杂波噪声被抑制后,图像中仍会存在部分与目标尺寸接近但形状不一致的噪声,这些噪声仍会干扰海面目标检测结果,本发明进一步分析发现,由于背景杂波仅在某个方向上有较大的梯度,而目标具各向同性,周围不同方向的梯度变化较小,基于这一发现,本发明提出在亮条、海面鱼鳞波噪声得到抑制的基础上,计算各个方向的梯度,并将各梯度图中同一像素位置的最小值作为图像上该位置的像素值,从而达到对于背景杂波有效抑制的效果。Sea clutter noise with uniform brightness and simple texture on the sea surface, such as bright stripes and sea fish scale waves, is the main component of sea clutter and will seriously interfere with the detection of sea surface targets. Therefore, this embodiment will first suppress these noises. Analysis and related experimental data show that top hat transformation can effectively suppress such sea clutter noise. After the sea clutter noise with uniform brightness and simple texture in the sea surface image is suppressed, there will still be some noise in the image that is close to the target size but inconsistent in shape. These noises will still interfere with the sea surface target detection result. The present invention further analyzes and finds that since the background clutter has a large gradient only in a certain direction, and the target is isotropic, the gradient changes in different directions around it are small. Based on this finding, the present invention proposes to calculate the gradients in each direction on the basis of suppressing the bright stripes and sea fish scale wave noises, and take the minimum value of the same pixel position in each gradient map as the pixel value of the position on the image, so as to achieve the effect of effectively suppressing the background clutter.

基于以上分析,如图2所示,为了有效抑制海面杂波,本实施例中,海面杂波自适应抑制步骤包括:Based on the above analysis, as shown in FIG2 , in order to effectively suppress sea surface clutter, in this embodiment, the sea surface clutter adaptive suppression step includes:

(S21)对待处理的图像数据进行顶帽变换,得到初步去噪图像;(S21) performing a top-hat transformation on the image data to be processed to obtain a preliminary denoised image;

(S22)计算初步去噪图像在不同方向的梯度,得到多张梯度图像;(S22) calculating the gradient of the preliminary denoised image in different directions to obtain a plurality of gradient images;

(S23)将多张梯度图像在相同位置的最小像素值作为去噪后图像在该位置的像素值,得到待处理的图像所对应的去噪后图像。(S23) The minimum pixel value of the multiple gradient images at the same position is used as the pixel value of the denoised image at the position, so as to obtain the denoised image corresponding to the image to be processed.

本实施例进一步考虑到不同场景下目标尺寸大小不一,因此,设计了自适应的梯度卷积核大小来应对不同的场景,进一步提高海面杂波的抑制效果。具体地,步骤(S22)包括:This embodiment further takes into account that the target sizes vary in different scenarios, and therefore designs an adaptive gradient convolution kernel size to cope with different scenarios, further improving the sea clutter suppression effect. Specifically, step (S22) includes:

对初步去噪图像进行连通域标记,将小于预设阈值的连通域剔除后,计算剩余连通域的平均大小CMark the connected domains of the preliminary denoised image, remove the connected domains smaller than the preset threshold, and calculate the average size C of the remaining connected domains;

按照ks=C//M计算梯度卷积核大小ks,并按照所计算的梯度卷积核大小确定各方向下的梯度卷积核;Calculate the gradient convolution kernel size ks according to ks = C // M , and determine the downward gradient convolution kernels in all directions according to the calculated gradient convolution kernel size;

利用各方向下的梯度卷积核对初步去噪图像进行卷积操作以计算初步去噪图像在不同方向的梯度,得到多张梯度图像;The preliminary denoised image is convolved using the downward gradient convolution kernel in all directions to calculate the gradient of the preliminary denoised image in different directions, thereby obtaining multiple gradient images;

其中,“//”表示整除运算;M为预设正整数,作为一种优选的实施方式,本实施例中,M=5,实验数据表明,由此设置梯度卷积核大小与连通域平均大小间的关系,可以最大程度上大小抑制海面杂波的效果。Wherein, “//” represents integer division operation; M is a preset positive integer. As a preferred implementation, in this embodiment, M = 5. Experimental data show that by setting the relationship between the gradient convolution kernel size and the average size of the connected domain in this way, the effect of suppressing sea surface clutter can be maximized.

本实施例中,自适应的基于方向性的梯度计算,其目的在于抑制掉图像中与目标大小接近而形状不同的噪声,经过连通域标记后,每一个连通域即对应一个噪声或者海面目标,因此,先根据预设阈值剔除掉过小的噪声,能够保证连通域平均大小与海面目标的大小接近;可选地,本实施例中,考虑到实际的海面目标多为海上舰船、低空无人机等,基于这些目标的经验尺寸大小,本实施例将该预设阈值设置为10。在其他的一些实施例中,也可根据实际检测目标的尺寸设置为其他值,确保可所剔除的连通域明显小于目标尺寸即可。In this embodiment, the adaptive directional gradient calculation is aimed at suppressing the noise in the image that is close to the target size but different in shape. After the connected domain is marked, each connected domain corresponds to a noise or a sea surface target. Therefore, the noise that is too small is first removed according to the preset threshold, which can ensure that the average size of the connected domain is close to the size of the sea surface target; optionally, in this embodiment, considering that most of the actual sea surface targets are ships at sea, low-altitude drones, etc., based on the empirical size of these targets, this embodiment sets the preset threshold to 10. In some other embodiments, it can also be set to other values according to the size of the actual detected target, ensuring that the connected domain that can be removed is significantly smaller than the target size.

以下以一幅实际图像的处理过程为例,对上述海面杂波自适应抑制步骤进行进一步的解释说明。图3所示,为一幅原始的海面图像数据,图中,方框内的对象为待检测的海面目标,按照本实施例提出的海面杂波自适应抑制步骤对其进行处理。图3所示海面图像经顶帽变换后,所得图像如图4所示,对比图3和图4可以明显看出,原始图像中的亮条、海面鱼鳞波等都得到了有效抑制。对图4所示图像进行二值化后,对其进行连通域标记,并剔除其中小于10的连通域,按照ks=C//5计算得到梯度卷积核大小为7,分别计算0°、45°、90°、135°、180°、225°、270°以及315°这8个方向下的梯度卷积核如图5所示,利用不同方向下的梯度卷积核即可计算得到相应方向下的梯度图。其中,对图4所示图像进行梯度计算后,0°和90°方向下的梯度图分别如图6和图7所示。对于8个不同方向下的梯度图,将各梯度图在同一像素位置的最小像素值作为图像中相应位置的像素值,即可得到海面杂波得到有效抑制的去噪后图像。The following is a further explanation of the above-mentioned sea clutter adaptive suppression step by taking the processing process of an actual image as an example. As shown in FIG3, it is an original sea surface image data. In the figure, the object in the box is the sea surface target to be detected, and it is processed according to the sea surface clutter adaptive suppression step proposed in this embodiment. After the sea surface image shown in FIG3 is top-hat transformed, the resulting image is shown in FIG4. By comparing FIG3 and FIG4, it can be clearly seen that the bright bars and sea surface fish scale waves in the original image are effectively suppressed. After binarization of the image shown in FIG4, the connected domain is marked, and the connected domains less than 10 are eliminated. According to ks = C //5, the gradient convolution kernel size is calculated to be 7, and the gradient convolution kernels in the eight directions of 0°, 45°, 90°, 135°, 180°, 225°, 270° and 315° are calculated as shown in FIG5. The gradient map in the corresponding direction can be calculated using the gradient convolution kernels in different directions. Among them, after the gradient calculation is performed on the image shown in Figure 4, the gradient maps at 0° and 90° directions are shown in Figures 6 and 7 respectively. For the gradient maps at 8 different directions, the minimum pixel value of each gradient map at the same pixel position is used as the pixel value of the corresponding position in the image, and the denoised image with effectively suppressed sea clutter can be obtained.

本实施例中,步骤(S3)包括:对于同一焦距下的去噪后图像,分别进行目标检测后,将检测结果融合,将融合结果作为相应焦距下的海面目标检测结果。In this embodiment, step (S3) includes: for the denoised images at the same focal length, after performing target detection respectively, fusing the detection results, and using the fusion results as the sea surface target detection results at the corresponding focal length.

对于各路图像进行目标检测时,可根据相应波段下图像的特点选取具体的目标检测算法。本实施例中,可见光图像尺寸为1920*1080,红外图像尺寸为640*512,由于可见光图像中存在更多的信息,因此选用YOLOv5作为可见光目标检测算法,而选用传统的多尺度块对比度测度算法(Multiscale Patch Contrast Measurement ,MPCM)实现对于红外图像目标的检测。在本发明其他的一些实施例中,对于具体图像的目标检测算法也可灵活选取为其他的算法。When performing target detection on each image, a specific target detection algorithm can be selected according to the characteristics of the image in the corresponding band. In this embodiment, the size of the visible light image is 1920*1080, and the size of the infrared image is 640*512. Since there is more information in the visible light image, YOLOv5 is selected as the visible light target detection algorithm, and the traditional multiscale patch contrast measurement algorithm (Multiscale Patch Contrast Measurement, MPCM) is selected to realize the detection of infrared image targets. In some other embodiments of the present invention, the target detection algorithm for a specific image can also be flexibly selected as other algorithms.

通过将同一焦距下的不同波段图像的目标检测结果融合到一起,可充分利用不同波段下图像所携带的不同信息,实现多波段图像协同处理,进一步提高海面目标的检测精度。由于同一焦距下不同波段的图像具有相同的视场,因此对于两张图像的检测结果进行检测框的匹配,为了有效实现同一焦距下的不同波段图像检测结果的融合,本实施例采用了高低双阈值的融合方式,具体地,对于某一焦距下的图像,将目标检测结果融合,包括:By fusing the target detection results of images of different bands at the same focal length, the different information carried by images of different bands can be fully utilized to achieve multi-band image collaborative processing, and further improve the detection accuracy of sea surface targets. Since images of different bands at the same focal length have the same field of view, the detection frames of the detection results of the two images are matched. In order to effectively achieve the fusion of the detection results of images of different bands at the same focal length, this embodiment adopts a high and low dual threshold fusion method. Specifically, for an image at a certain focal length, the target detection results are fused, including:

计算该焦距下,各去噪后图像经目标检测所得检测框之间的重叠度(IOU),将重叠度大于预设的高阈值(例如0.7)的检测框作为可信目标输出,将重叠度小于预设的低阈值(例如0.3)的检测框作为虚警剔除;Calculate the overlap (IOU) between the detection frames obtained by target detection in each denoised image at the focal length, output the detection frames with an overlap greater than a preset high threshold (e.g. 0.7) as credible targets, and remove the detection frames with an overlap less than a preset low threshold (e.g. 0.3) as false alarms;

根据当前帧之前的海面图像数据的海面目标检测结果序列确定海面目标的轨迹,对于重叠度位于低阈值和高阈值之间的检测框,若其位于海面目标的轨迹上,则作为可信目标输出,否则,作为虚警剔除。The trajectory of the sea surface target is determined according to the sea surface target detection result sequence of the sea surface image data before the current frame. For the detection frame with an overlap between the low threshold and the high threshold, if it is located on the trajectory of the sea surface target, it is output as a credible target, otherwise it is rejected as a false alarm.

其中,根据当前帧之前的海面图像数据的海面目标检测结果序列确定海面目标的轨迹,可基于现有的目标跟踪手段完成。Among them, determining the trajectory of the sea surface target according to the sea surface target detection result sequence of the sea surface image data before the current frame can be completed based on the existing target tracking means.

由于不同焦距的镜头的视场不同,其中,较短焦距下,视场较大且噪声和干扰物较多,目标尺寸小,检测难度大,而较长焦距下,视场较小,噪声和干扰物较小,目标清晰,检测难度小,但海面目标有可能位于视场之外,基于此,本实施例在同时获取到较短焦距下的海面目标检测结果和较长焦距下的海面目标检测结果后,可直接取其中较长焦距下的海面目标检测结果作为最终的海面目标检测结果;否则,若仅获取到一个焦距下的海面目标检测结果,则将其作为最终的海面目标检测结果。Since the fields of view of lenses with different focal lengths are different, at a shorter focal length, the field of view is larger and there are more noise and interference, the target size is small, and the detection difficulty is large, while at a longer focal length, the field of view is smaller, the noise and interference are smaller, the target is clear, and the detection difficulty is small, but the sea surface target may be outside the field of view. Based on this, after simultaneously obtaining the sea surface target detection results at the shorter focal length and the sea surface target detection results at the longer focal length, this embodiment can directly take the sea surface target detection result at the longer focal length as the final sea surface target detection result; otherwise, if only the sea surface target detection result at one focal length is obtained, it is used as the final sea surface target detection result.

为了进一步提高海面目标检测结果的检测精度,本发明进一步提出,将较短焦距下的海面目标检测结果视为低置信度结果并以此来计算目标中心相对于图像中心的偏移量信息,并以该信息作为光电吊舱端调整较长镜头位姿的指导信息,从而得到更高质量的目标图像数据,进而得到更加准确的长焦图像目标检测结果。相应地,本实施例在步骤(S3)之后还包括:In order to further improve the detection accuracy of the sea surface target detection results, the present invention further proposes to regard the sea surface target detection results at a shorter focal length as low confidence results and use this to calculate the offset information of the target center relative to the image center, and use this information as guidance information for adjusting the longer lens posture at the optoelectronic pod end, so as to obtain higher quality target image data, and then obtain more accurate telephoto image target detection results. Accordingly, this embodiment further includes after step (S3):

(S4)根据较短焦距下海面目标检测结果计算海面目标的中心位置,并计算海面目标的中心位置相对于图像中心的偏移量;(S4) calculating the center position of the sea surface target according to the sea surface target detection result at the shorter focal length, and calculating the offset of the center position of the sea surface target relative to the center of the image;

(S5)向光电吊舱发送指令,以使光电吊舱按照偏移量调整其中较长焦距的镜头的位姿,使该焦距下海面目标的中心位置与图像中心位置重合。(S5) Sending a command to the optoelectronic pod so that the optoelectronic pod adjusts the posture of the lens with the longer focal length according to the offset, so that the center position of the sea surface target at the focal length coincides with the center position of the image.

总的来说,本实施例以光电吊舱作为海面数据采集端,能够采集到不同焦距、不同波段的多路图像数据,为精确检测海面目标提供了更充分的数据基础;基于海杂波的特性,提出了海面杂波自适应抑制步骤,能够有效抑制海面图像数据中的海杂波噪声,从而有效提高海面目标的检测精度。在此基础上,利用较短镜头下的目标检测结果指导光电吊舱中较长焦距下镜头的位姿调整,可获取到更高质量的目标图像数据,进而得到更加准确的长焦图像目标检测结果。In general, this embodiment uses the optoelectronic pod as the sea surface data collection end, which can collect multi-channel image data of different focal lengths and different bands, providing a more sufficient data basis for the accurate detection of sea surface targets; based on the characteristics of sea clutter, an adaptive suppression step for sea surface clutter is proposed, which can effectively suppress the sea clutter noise in the sea surface image data, thereby effectively improving the detection accuracy of sea surface targets. On this basis, the target detection results under the shorter lens are used to guide the posture adjustment of the lens under the longer focal length in the optoelectronic pod, so that higher quality target image data can be obtained, and then more accurate telephoto image target detection results can be obtained.

实施例2:Embodiment 2:

一种基于光电吊舱的海面目标检测平台,包括:A sea surface target detection platform based on an optoelectronic pod, comprising:

计算机可读存储介质,用于存储计算机程序;A computer-readable storage medium for storing a computer program;

以及处理器,用于读取计算机可读存储介质中存储的计算机程序,执行上述实施例1提供的基于光电吊舱的海面目标检测方法。and a processor, for reading a computer program stored in a computer-readable storage medium, and executing the sea surface target detection method based on an optoelectronic pod provided in the above-mentioned embodiment 1.

本实施例提供的基于光电吊舱的海面目标检测平台,可以是飞机平台、远程控制平台等任意一种可与光电吊舱进行数据传输且具备数据处理能力的平台。The sea surface target detection platform based on the optoelectronic pod provided in this embodiment can be any platform such as an aircraft platform, a remote control platform, etc. that can transmit data with the optoelectronic pod and has data processing capabilities.

实施例3:Embodiment 3:

一种海面目标检测系统,包括:A sea surface target detection system, comprising:

上述实施例2提供的基于光电吊舱的海面目标检测平台;The sea surface target detection platform based on the optoelectronic pod provided in the above-mentioned embodiment 2;

以及光电吊舱;and optoelectronic pods;

光电吊舱与海面目标检测平台之间通过单通道链路传输图像数据流。The image data stream is transmitted between the optoelectronic pod and the sea surface target detection platform via a single-channel link.

本领域的技术人员容易理解,以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (8)

1.一种基于光电吊舱的海面目标检测方法,其特征在于,包括:1. A method for detecting sea surface targets based on an optoelectronic pod, comprising: 持续接收由所述光电吊舱发送的海面图像数据;对于所接收到的每一帧聚合后的多波段海面图像数据,执行以下步骤:Continuously receiving the sea surface image data sent by the optoelectronic pod; for each frame of the received aggregated multi-band sea surface image data, performing the following steps: (S1)将其解聚为多个单路图像数据;每一个单路图像数据对应所述光电吊舱采集的一个焦距下的一个波段的图像数据;(S1) deaggregating it into a plurality of single-channel image data; each single-channel image data corresponds to image data of a band at a focal length collected by the optoelectronic pod; (S2)对各单路图像数据分别执行海面杂波自适应抑制步骤,得到对应的去噪后图像;所述海面杂波自适应抑制步骤包括:(S2) performing a sea surface clutter adaptive suppression step on each single channel of image data to obtain a corresponding denoised image; the sea surface clutter adaptive suppression step comprises: (S21)对待处理的图像数据进行顶帽变换,得到初步去噪图像;(S21) performing a top-hat transformation on the image data to be processed to obtain a preliminary denoised image; (S22)计算所述初步去噪图像在不同方向的梯度,得到多张梯度图像;(S22) calculating the gradients of the preliminary denoised image in different directions to obtain a plurality of gradient images; (S23)将所述多张梯度图像在相同位置的最小像素值作为去噪后图像在该位置的像素值,得到待处理的图像所对应的去噪后图像;(S23) taking the minimum pixel value of the plurality of gradient images at the same position as the pixel value of the denoised image at the position, and obtaining the denoised image corresponding to the image to be processed; (S3)对于同一焦距下的去噪后图像,分别进行目标检测后,将检测结果融合,将融合结果作为相应焦距下的海面目标检测结果。(S3) After performing target detection on the denoised images at the same focal length, the detection results are fused and the fused results are used as the sea surface target detection results at the corresponding focal length. 2.如权利要求1所述的基于光电吊舱的海面目标检测方法,其特征在于,所述步骤(S22)包括:2. The method for detecting sea surface targets based on an optoelectronic pod according to claim 1, wherein the step (S22) comprises: 对所述初步去噪图像进行二值化后,进行连通域标记,将小于预设阈值的连通域剔除后,计算剩余连通域的平均大小CAfter binarizing the preliminary denoised image, connected domains are marked, connected domains smaller than a preset threshold are removed, and the average size C of the remaining connected domains is calculated; 按照ks=C//M计算梯度卷积核大小ks,并按照所计算的梯度卷积核大小确定各方向下的梯度卷积核;Calculate the gradient convolution kernel size ks according to ks = C // M , and determine the downward gradient convolution kernels in all directions according to the calculated gradient convolution kernel size; 利用各方向下的梯度卷积核对所述初步去噪图像进行卷积操作以计算所述二值化图像在不同方向的梯度,得到多张梯度图像;Using downward gradient convolution kernels in all directions to perform a convolution operation on the preliminary denoised image to calculate the gradients of the binary image in different directions to obtain a plurality of gradient images; 其中,“//”表示整除运算;M为预设正整数。Wherein, “//” represents integer division operation; M is a preset positive integer. 3.如权利要求2所述的基于光电吊舱的海面目标检测方法,其特征在于,M=5。3. The sea surface target detection method based on the optoelectronic pod as described in claim 2 is characterized in that M = 5. 4.如权利要求1~3任一项所述的基于光电吊舱的海面目标检测方法,其特征在于,在步骤(S3)之后还包括:4. The method for detecting sea surface targets based on an optoelectronic pod according to any one of claims 1 to 3, characterized in that after step (S3), it also includes: (S4)根据较短焦距下海面目标检测结果计算海面目标的中心位置,并计算海面目标的中心位置相对于图像中心的偏移量;(S4) calculating the center position of the sea surface target according to the sea surface target detection result at the shorter focal length, and calculating the offset of the center position of the sea surface target relative to the center of the image; (S5)向所述光电吊舱发送指令,以使所述光电吊舱按照所述偏移量调整其中较长焦距的镜头的位姿,使该焦距下海面目标的中心位置与图像中心位置重合。(S5) Sending a command to the optoelectronic pod so that the optoelectronic pod adjusts the posture of the lens with the longer focal length according to the offset so that the center position of the sea surface target at the focal length coincides with the center position of the image. 5.如权利要求1~3任一项所述的基于光电吊舱的海面目标检测方法,其特征在于,所述步骤(S3)中,将同一焦距下的目标检测结果融合,包括:5. The method for detecting sea surface targets based on an optoelectronic pod according to any one of claims 1 to 3, characterized in that in the step (S3), the target detection results at the same focal length are fused, comprising: 计算该焦距下,各去噪后图像经目标检测所得检测框之间的重叠度,将重叠度大于预设的高阈值的检测框作为可信目标输出,将重叠度小于预设的低阈值的检测框作为虚警剔除;Calculate the overlap between the detection frames obtained by target detection in each denoised image at the focal length, output the detection frames with an overlap greater than a preset high threshold as credible targets, and remove the detection frames with an overlap less than a preset low threshold as false alarms; 根据当前帧之前的海面图像数据的海面目标检测结果序列确定海面目标的轨迹,对于重叠度位于所述低阈值和所述高阈值之间的检测框,若其位于海面目标的轨迹上,则作为可信目标输出,否则,作为虚警剔除。The trajectory of the sea surface target is determined according to the sea surface target detection result sequence of the sea surface image data before the current frame. For the detection frame whose overlap degree is between the low threshold and the high threshold, if it is located on the trajectory of the sea surface target, it is output as a credible target, otherwise, it is eliminated as a false alarm. 6.如权利要求5所述的基于光电吊舱的海面目标检测方法,其特征在于,所述步骤(S3)中,若去噪后图像为可见光波段图像,则采用YOLOv5进行目标检测;若去噪后图像为红外波段图像,则采用多尺度块对比度测度算法进行检测。6. The method for sea surface target detection based on an optoelectronic pod as described in claim 5 is characterized in that, in the step (S3), if the denoised image is a visible light band image, YOLOv5 is used for target detection; if the denoised image is an infrared band image, a multi-scale block contrast measurement algorithm is used for detection. 7.一种基于光电吊舱的海面目标检测平台,其特征在于,包括:7. A sea surface target detection platform based on an optoelectronic pod, characterized by comprising: 计算机可读存储介质,用于存储计算机程序;A computer-readable storage medium for storing a computer program; 以及处理器,用于读取所述计算机可读存储介质中存储的计算机程序,执行权利要求1~6任一项所述的基于光电吊舱的海面目标检测方法。and a processor, configured to read the computer program stored in the computer-readable storage medium and execute the sea surface target detection method based on the optoelectronic pod according to any one of claims 1 to 6. 8.一种海面目标检测系统,其特征在于,包括:8. A sea surface target detection system, comprising: 权利要求7所述的基于光电吊舱的海面目标检测平台;The sea surface target detection platform based on the optoelectronic pod as described in claim 7; 以及光电吊舱;and optoelectronic pods; 所述光电吊舱与海面目标检测平台之间通过单通道链路传输图像数据流。The image data stream is transmitted between the optoelectronic pod and the sea surface target detection platform via a single-channel link.
CN202410248872.3A 2024-03-05 2024-03-05 Sea surface target detection method, detection platform and system based on photoelectric pod Active CN117853932B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410248872.3A CN117853932B (en) 2024-03-05 2024-03-05 Sea surface target detection method, detection platform and system based on photoelectric pod

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410248872.3A CN117853932B (en) 2024-03-05 2024-03-05 Sea surface target detection method, detection platform and system based on photoelectric pod

Publications (2)

Publication Number Publication Date
CN117853932A true CN117853932A (en) 2024-04-09
CN117853932B CN117853932B (en) 2024-05-14

Family

ID=90538545

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410248872.3A Active CN117853932B (en) 2024-03-05 2024-03-05 Sea surface target detection method, detection platform and system based on photoelectric pod

Country Status (1)

Country Link
CN (1) CN117853932B (en)

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070110322A1 (en) * 2005-09-02 2007-05-17 Alan Yuille System and method for detecting text in real-world color images
US20070110319A1 (en) * 2005-11-15 2007-05-17 Kabushiki Kaisha Toshiba Image processor, method, and program
US20130051665A1 (en) * 2011-08-31 2013-02-28 Hirotaka SHINOZAKI Image processing apparatus, image processing method, and program
CN104504670A (en) * 2014-12-11 2015-04-08 上海理工大学 Multi-scale gradient domain image fusion algorithm
CN104751443A (en) * 2014-12-12 2015-07-01 郑州轻工业学院 Cotton fault detecting and identifying method based on multi-spectrum technology
CN108140239A (en) * 2015-09-23 2018-06-08 皇家飞利浦有限公司 For organizing the method and apparatus of identification
CN108364277A (en) * 2017-12-20 2018-08-03 南昌航空大学 A kind of infrared small target detection method of two-hand infrared image fusion
CN108764325A (en) * 2018-05-23 2018-11-06 腾讯科技(深圳)有限公司 Image-recognizing method, device, computer equipment and storage medium
CN109785314A (en) * 2019-01-22 2019-05-21 中科院金华信息技术有限公司 A kind of pck count detection system and method based on u-net network
CN111179334A (en) * 2019-11-14 2020-05-19 青岛理工大学 Sea surface small-area oil spilling area detection system and detection method based on multi-sensor fusion
CN111696107A (en) * 2020-08-05 2020-09-22 南京知谱光电科技有限公司 Molten pool contour image extraction method for realizing closed connected domain
US20210118144A1 (en) * 2018-09-15 2021-04-22 Beijing Sensetime Technology Development Co., Ltd. Image processing method, electronic device, and storage medium
CN112967305A (en) * 2021-03-24 2021-06-15 南京莱斯电子设备有限公司 Image cloud background detection method under complex sky scene
CN114972140A (en) * 2022-04-11 2022-08-30 大连海事大学 Sea surface infrared and visible light image fusion method based on target segmentation
CN115690402A (en) * 2022-11-15 2023-02-03 吉林大学 OCC image recognition decoding method based on gradient corner comprehensive detection
CN116248705A (en) * 2022-11-29 2023-06-09 宜昌测试技术研究所 Multichannel image transmission and processing system of miniature photoelectric pod
CN117274251A (en) * 2023-11-20 2023-12-22 山东鲁抗医药集团赛特有限责任公司 Tablet quality detection method in medicine production process based on image data

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070110322A1 (en) * 2005-09-02 2007-05-17 Alan Yuille System and method for detecting text in real-world color images
US20070110319A1 (en) * 2005-11-15 2007-05-17 Kabushiki Kaisha Toshiba Image processor, method, and program
US20130051665A1 (en) * 2011-08-31 2013-02-28 Hirotaka SHINOZAKI Image processing apparatus, image processing method, and program
CN104504670A (en) * 2014-12-11 2015-04-08 上海理工大学 Multi-scale gradient domain image fusion algorithm
CN104751443A (en) * 2014-12-12 2015-07-01 郑州轻工业学院 Cotton fault detecting and identifying method based on multi-spectrum technology
CN108140239A (en) * 2015-09-23 2018-06-08 皇家飞利浦有限公司 For organizing the method and apparatus of identification
CN108364277A (en) * 2017-12-20 2018-08-03 南昌航空大学 A kind of infrared small target detection method of two-hand infrared image fusion
CN108764325A (en) * 2018-05-23 2018-11-06 腾讯科技(深圳)有限公司 Image-recognizing method, device, computer equipment and storage medium
US20210118144A1 (en) * 2018-09-15 2021-04-22 Beijing Sensetime Technology Development Co., Ltd. Image processing method, electronic device, and storage medium
CN109785314A (en) * 2019-01-22 2019-05-21 中科院金华信息技术有限公司 A kind of pck count detection system and method based on u-net network
CN111179334A (en) * 2019-11-14 2020-05-19 青岛理工大学 Sea surface small-area oil spilling area detection system and detection method based on multi-sensor fusion
CN111696107A (en) * 2020-08-05 2020-09-22 南京知谱光电科技有限公司 Molten pool contour image extraction method for realizing closed connected domain
CN112967305A (en) * 2021-03-24 2021-06-15 南京莱斯电子设备有限公司 Image cloud background detection method under complex sky scene
CN114972140A (en) * 2022-04-11 2022-08-30 大连海事大学 Sea surface infrared and visible light image fusion method based on target segmentation
CN115690402A (en) * 2022-11-15 2023-02-03 吉林大学 OCC image recognition decoding method based on gradient corner comprehensive detection
CN116248705A (en) * 2022-11-29 2023-06-09 宜昌测试技术研究所 Multichannel image transmission and processing system of miniature photoelectric pod
CN117274251A (en) * 2023-11-20 2023-12-22 山东鲁抗医药集团赛特有限责任公司 Tablet quality detection method in medicine production process based on image data

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JYOTSNA DOGRA, ET AL: "Gradient-based kernel selection technique for tumour detection and extraction of medical images using graph cut", IET IMAGE PROCESSING, 2020 - WILEY ONLINE LIBRARY, 1 January 2020 (2020-01-01), pages 84 - 93, XP006088231, DOI: 10.1049/iet-ipr.2018.6615 *
胡文魁: "基于深度卷积神经网络的桥梁裂缝检测方法研究", 广东:华南理工大学, 15 July 2023 (2023-07-15), pages 1 - 125 *

Also Published As

Publication number Publication date
CN117853932B (en) 2024-05-14

Similar Documents

Publication Publication Date Title
US9398235B2 (en) Apparatus and method for fusing images
CN108154494A (en) A kind of image fusion system based on low-light and infrared sensor
CN102521814B (en) Wireless sensor network image fusion method based on multi-focus fusion and image splicing
CN105607251B (en) Multi-mode infrared imaging system and method facing weak target detection
CN108921803B (en) Defogging method based on millimeter wave and visible light image fusion
CN114694011A (en) A method and device for detecting fog-penetrating targets based on multi-sensor fusion
JPWO2011055772A1 (en) Image target identification device, image target identification method, and image target identification program
CN113223065B (en) Automatic matching method for SAR satellite image and optical image
CN112966635B (en) Low-resolution time sequence remote sensing image-oriented moving ship detection method and device
CN117974792B (en) Ship target detection positioning method based on vision and AIS data cooperative training
CN105730705A (en) Aircraft camera shooting positioning system
JP2015508584A (en) Method for 3D reconstruction of scenes that rely on asynchronous sensors
CN115219026A (en) Polarization IntelliSense System and Perception Method
CN112991246B (en) Visible light and infrared video image fusion method and device
CN106204496A (en) A kind of fish eye images processing method and processing device of cradle head camera
CN112857587A (en) Short wave infrared detector imaging control method based on scene definition evaluation
CN117011296A (en) Method, equipment and storage medium for quickly detecting tracking precision of photoelectric pod
CN117853932B (en) Sea surface target detection method, detection platform and system based on photoelectric pod
EP4350638A1 (en) Detection method and system for base-station feeder line, and related apparatus
CN112884803A (en) Real-time intelligent monitoring target detection method and device based on DSP
CN115348364A (en) A curved surface bionic compound eye large field of view imaging device and imaging method
CN113033355B (en) Abnormal target identification method and device based on intensive power transmission channel
CN118644761A (en) A construction site safety helmet detection method, computer equipment and storage medium
JP5664551B2 (en) Image target identification device, image target identification method, and image target identification program
Su et al. Research on a multi-dimensional image information fusion algorithm based on NSCT transform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant