WO2018120445A1 - 一种动目标多维度多尺度红外光谱特征测量方法及系统 - Google Patents

一种动目标多维度多尺度红外光谱特征测量方法及系统 Download PDF

Info

Publication number
WO2018120445A1
WO2018120445A1 PCT/CN2017/077104 CN2017077104W WO2018120445A1 WO 2018120445 A1 WO2018120445 A1 WO 2018120445A1 CN 2017077104 W CN2017077104 W CN 2017077104W WO 2018120445 A1 WO2018120445 A1 WO 2018120445A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
infrared
infrared spectrum
image
scale
Prior art date
Application number
PCT/CN2017/077104
Other languages
English (en)
French (fr)
Inventor
张天序
喻洪涛
姚守悝
戴小兵
Original Assignee
华中科技大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华中科技大学 filed Critical 华中科技大学
Publication of WO2018120445A1 publication Critical patent/WO2018120445A1/zh
Priority to US16/458,219 priority Critical patent/US10839527B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/02—Constructional details
    • G01J5/08—Optical arrangements
    • G01J5/0808—Convex mirrors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/50—Systems of measurement based on relative movement of target
    • G01S17/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J2005/0077—Imaging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the invention belongs to the field of image processing, motion control and infrared spectral features, and particularly relates to a moving target multi-dimensional multi-scale infrared spectrum feature measuring method and system.
  • Imaging infrared spectrometer is one of the achievements in the development of remote sensing. Humans use optical technology to identify the target mainly by optical imaging technology and optical spectroscopy. Optical imaging technology mainly acquires geometric features and grayscale features of the target. However, the single image information cannot obtain the infrared spectrum information of the target component, and qualitative analysis cannot be performed under low altitude conditions, and quantitative analysis cannot be performed in the space environment. Infrared spectroscopy is based on the principle of infrared spectroscopy to obtain target components and state information, but lacks target image feature information. Imaging infrared spectroscopy combines imaging and spectroscopy techniques to capture image features of the target, as well as target infrared spectral features.
  • an imaging infrared spectrometer was proposed to combine imaging and spectroscopy techniques.
  • the main disadvantage of the device is that the amount of invalid infrared spectroscopy data is too large to process infrared spectral data in real time.
  • Target infrared spectroscopy and background infrared spectroscopy Mixed, the target infrared spectral features cannot be accurately obtained.
  • the present invention aims to provide an infrared spectral feature measuring method capable of overcoming the shortcomings of an infrared spectrum imaging instrument that cannot acquire target multi-dimensional multi-scale information in real time and cannot satisfy the short-term acquisition target precise feature model.
  • the present invention provides a moving target multi-dimensional multi-scale infrared spectrum feature measuring method, comprising the following steps:
  • the tracking target obtains the target pixel difference between the two frames and the target moving direction, and then performs target motion compensation according to the target pixel difference between the two frames;
  • step (3) Scanning the target identified in step (2), after successfully capturing the tracking target image, the control inner frame points to each target of interest, and according to the target moving direction information, in a direction shifted by 90° with respect to the moving direction.
  • N pixel size motion open the spectrum measurement module, record the distance information between the measurement device and the target at this time, measure the azimuth angle information, scale information, time dimension information, and input the above information into the object side obtained in step (1).
  • the multi-dimensional multi-scale model of the target infrared spectrum features, and the multi-dimensional multi-scale model of the infrared spectrum features of the image-like target is obtained.
  • step (1) further comprises the following substeps:
  • target infrared spectrum radiation measurement characteristics of the object side are expressed as follows:
  • F rad () represents the radiation intensity of the target in the case where the target is in the earth coordinate system (x, y, z), the measurement angle ⁇ , the scale S, and the time dimension T;
  • the radiation measurement characteristic f rad1 as a point-like, plaque target is expressed as follows:
  • ⁇ is the infrared spectral variable
  • r is the distance between the measuring instrument and the target
  • the radiation measurement characteristic f rad2 as a surface target is expressed as follows:
  • ⁇ is the infrared spectral variable and r is the distance between the measuring instrument and the target.
  • step (2) comprises the following sub-steps:
  • Thread 1 super pixel segmentation obtains sky background, ground background, target area, etc., according to the segmentation result and measures the target area, the grayscale feature recognition background area, as a negative sample;
  • Thread 2 Extract the full-image HOG feature, distinguish the background from the target region according to the sliding window method, and obtain the suspected target as a positive sample;
  • Thread 3 using a full convolutional neural network to detect the target image as a positive sample
  • step (2.4) detecting two frame target image position information according to step (2.2) and processing, obtaining frame difference and moving target direction information of two frame image targets; performing target according to target pixel difference between two frames obtained in step (2.3) Motion compensation.
  • the present invention also provides a measurement system for the above measurement method, which comprises: an industrial computer, a rotating mirror, a beam splitter, a medium wave lens, a long wave lens, a non-infrared infrared spectrum measuring unit, and a long wave infrared.
  • the imaging unit is connected to the rotating mirror of the control interface of the industrial computer;
  • the medium wave lens is mounted on the non-infrared infrared spectrum measuring unit, the output end of the non-infrared infrared spectrum measuring unit is connected to the input end of the industrial computer;
  • the long wave lens is mounted on the long wave infrared imaging unit.
  • the output end of the long wave infrared imaging unit is connected to the input end of the industrial computer;
  • the rotating mirror is used to reflect the light of the target of interest to the beam splitter, and the beam splitter is used to divide the received reflected light into a medium wave and a long wave, respectively, to the medium wave lens and the long wave lens, and then respectively
  • the medium-wave lens and the long-wave lens are transmitted to the non-infrared infrared spectrum measuring unit and the long-wave infrared imaging unit for processing;
  • the non-infrared infrared spectrum measuring unit is configured to process the received medium wave into infrared spectrum data and transmit it to the industrial computer;
  • the imaging unit is configured to process the received long wave into image data and transmit it to the industrial computer;
  • the industrial computer is used to control the steering of the rotating mirror, and process the received infrared spectral data and image data to obtain a multi-dimensional multi-scale infrared spectrum of the moving target. feature.
  • the rotating mirror adopts four-frame servo control, and includes a mirror, an inner tilt frame, an inner azimuth frame, an outer pitch frame, and an outer azimuth frame which are sequentially arranged from the inside to the outside.
  • the invention establishes a three-dimensional model in advance, combines and contrasts the three-dimensional model with the measured infrared spectrum, establishes a multi-dimensional multi-scale feature representation formula of the target infrared spectrum, and proposes an accurate model of the target infrared spectrum, and proposes The precise positioning method of the target of interest identifies the position of each component of the target of interest, and solves the problem that the existing target infrared spectrum features are incomplete, inaccurate, and the infrared spectrum cannot be measured or measured.
  • Figure 1 is a general flow chart of the present invention
  • Figure 2 is an infrared simulation image of the F16 fighter
  • Figure 3 is a partial image of the F16 aircraft simulation image library with a resolution of 1.25 m in the 8 ⁇ m to 14 ⁇ m band;
  • Figure 4 is a map of the area of interest of the aircraft
  • Figure 5 is a diagram showing the composition of the infrared spectrum of the aircraft
  • Figure 6 is a graph of infrared spectrum data of an aircraft measured by different azimuth angles
  • Figure 7 is a multi-frame servo structure diagram
  • Figure 8 is a schematic structural diagram of the system
  • Figure 9 is a diagram showing the detection results of the region of interest of the actual infrared image
  • FIG. 1 is a general flowchart of the present invention.
  • the invention comprises the following steps:
  • each point (x, y, z) of the target has infrared spectral characteristics, each point has a temperature, and the emissivity may be different.
  • the target motion causes the target infrared spectral characteristics. change.
  • the radiation source of the infrared spectrum mainly includes the reflected solar radiation, the reflected earth radiation, the engine intake component radiation, the aerodynamic heating skin radiation, the engine thermal component radiation, and the tail. Flame radiation.
  • the infrared spectrum of each region of interest is shown in Figure 5.
  • the infrared spectrum characteristic distribution of the aircraft can be obtained at a certain angle.
  • the infrared spectrum of the target can be measured from different angles, and the infrared spectrum characteristics of the target at different angles can be obtained, as shown in Fig. 6. It can be seen that the infrared spectral characteristics of the aircraft vary greatly in different observed azimuths.
  • the aircraft's target radiation characteristics can be expressed as:
  • F rad () represents the radiation intensity of the target in the case where the target is in the earth coordinate system (x, y, z), the measurement angle ⁇ , the scale S, and the time dimension T.
  • the image of the aircraft target radiation measurement characteristic function has the following two cases.
  • the radiation measurement characteristic f rad1 as a point-like, plaque target, and the azimuth angle of the measuring instrument Related, can vary with azimuth.
  • ⁇ is the infrared spectral variable and r is the distance.
  • the radiation measurement characteristic f rad2 as a surface target is related to the measurement azimuth and the target spatial distribution, and varies with the distance, with the azimuth angle. Variety. Target space distribution Varying radiance characteristics:
  • the tracking target obtains the target pixel difference between two frames and obtains the target moving direction, and controls the two-axis four-frame servo to perform target motion compensation according to the target pixel difference between the two frames.
  • Thread 1 Superpixel segmentation obtains the sky background, the ground background, the target region, etc., and the background area is determined according to the segmentation result and the target area and the grayscale feature are identified as negative samples.
  • Thread 2 Extract the full-image HOG feature (Histogram of Oriented Gradient, HOG), distinguish the background from the target region according to the sliding window method, and obtain the suspected target as a positive sample.
  • HOG Heistogram of Oriented Gradient
  • Thread 3 Using the full convolutional neural network, the target is detected as a positive sample.
  • the result obtained by the above thread is input into the pre-trained SVM classifier to obtain the target image position information (x, y); wherein the SVM classifier part training sample is shown in FIGS. 2 and 3.
  • the actual detection effect is shown in Figure 9.
  • the SVM is called the Support Vector Machine and supports vector machines.
  • the target tracking module After detecting the target position, the target tracking module is started to obtain the frame difference and the moving target direction information of the two image targets.
  • the servo system of this system adopts four-frame structure servo system.
  • the schematic diagram of the structure of the mirror is shown in Figure 7.
  • the frame in the picture is responsible for large-scale target capture and tracking; specifically, the outer frame is motion-compensated for moving targets, and the target moving pixels between two frames of images
  • the differential input servo control system the control system responds quickly to the error input, and compensates the moving target so that the target is always at the center of the field of view.
  • the control inner frame After successfully capturing the tracking target image, the control inner frame points to each target of interest, and according to the target moving direction information, an offset motion of N pixel sizes in a direction of 90° with respect to the moving direction, that is, a scanning direction and a moving direction Similarly, after each scan, offset N pixels perpendicular to the direction of motion, and then continue scanning in the direction of motion; turn on the spectrum module, record the distance information between the measuring device and the target, measure the azimuth angle information, scale information Time dimension information, the above information is input into the multi-dimensional multi-scale model of the target infrared spectrum feature obtained in step (1), and the final target infrared spectrum model is obtained.
  • the scanning schematic diagram is shown in FIG. 10, and the actual effect diagram is shown in FIG.
  • the invention also provides a measurement system, that is, a target tracking spectrum measurement system for realizing the above method and capable of combining image information and infrared spectrum information, and the structure of the system is shown in FIG. 8.
  • the target infrared spectrum information is measured according to the target multi-dimensional multi-scale infrared spectrum model.
  • the device acquires the ROI (region of interest) position information and motion information through the imaging system, controls the multi-frame two-axis servo to perform target motion compensation, scans the target specific region to obtain the target infrared spectrum, and establishes the target infrared spectrum model. Mainly used for time sensitive target recognition.

Abstract

一种动目标多尺度多维度红外光谱特征测量方法及系统，属于图像处理、红外光谱测量领域。主要为解决对动目标红外光谱特征测不到，测不准的问题。其中方法包括：(1)建立目标红外光谱多维度多尺度特征表示公式，从中提取目标红外光谱精确模型，(2)从中提取感兴趣区域精确定位方法，识别感兴趣区域各部件位置信息，(3)从中提取多框架伺服控制方法，解决目标红外光谱测不到问题。该方法将红外光谱数据及图像数据结合进行处理，从而实现测准。

Description

一种动目标多维度多尺度红外光谱特征测量方法及系统 【技术领域】
本发明属于图像处理、运动控制、红外光谱特征的交叉领域，具体涉及一种动目标多维度多尺度红外光谱特征测量方法及系统。
【背景技术】
成像红外光谱仪是遥感发展的成就之一，人类利用光学技术对目标识别的方法主要有光学成像技术和光学成谱技术。光学成像技术主要获取目标的几何特征、灰度特征。但单一的图像信息无法获取目标成分红外光谱信息，在低空条件下无法进行定性分析，在太空环境下无法进行定量分析。红外光谱分析技术是根据红外光谱学相关原理获得目标成分、状态信息，但是缺乏目标图像特征信息。成像红外光谱仪器将成像和成谱技术结合起来，可以获取目标的图像特征，以及目标红外光谱特征。因此在20世纪八十年代提出成像红外光谱仪，将成像和成谱技术结合的测量设备，目前设备主要缺点是无效红外光谱数据量过大，无法实时处理红外光谱数据，目标红外光谱与背景红外光谱混杂，无法精确获取目标红外光谱特征。
【发明内容】
本发明旨在提供一种能够克服红外光谱成像仪器无法实时获取目标多维度多尺度信息、无法满足获取目标精确特征模型的缺点的红外光谱特征测量方法。
为了实现上述目的，本发明提供了一种动目标多维度多尺度红外光谱特征测量方法，包括如下步骤：
(1)建立物方目标红外光谱特征多维度多尺度模型，从中提取物方感兴趣区域测量模型；
(2)对实测红外图像进行目标检测，识别目标各感兴趣区域位置信息； 跟踪目标获得两帧之间目标像素差以及目标运动方向，然后根据两帧之间目标像素差进行目标运动补偿；
(3)对步骤(2)识别出的目标作扫描，成功捕获跟踪目标图像后，控制内框指向各感兴趣目标，并根据目标运动方向信息，在相对于运动方向偏移90°的方向上作N个像素大小的运动，开启测谱模块，记录此时测量设备与目标之间距离信息，测量方位立体角信息、尺度信息、时间维信息，将上述信息输入步骤(1)获得的物方目标红外光谱特征多维度多尺度模型，得到像方目标红外光谱特征多维度多尺度模型。
进一步地，步骤(1)还包括如下子步骤：
(1.1)对需要测量的目标建立三维模型；
(1.2)从三维模型中确定测量目标的感兴趣区域部位，并对上述部位的三维模型进行材质划分，以确定辐射源；
(1.3)对辐射源红外光谱特征进行测量，得到特定角度下物方红外光谱特征分布。
进一步地，物方的目标红外光谱辐射测量特性表达如下：
Frad(x,y,z,ω,S,T)   (1)
其中，Frad()表示目标在地球坐标系下位置(x,y,z)、测量角度ω、尺度S、时间维度T的情况下目标的辐射强度；
像方的目标辐射测量特性函数有如下两种情况：
作为点状、斑状目标的辐射测量特性frad1表达如下：
Figure PCTCN2017077104-appb-000001
其中，
Figure PCTCN2017077104-appb-000002
为测量仪器所处的方位立体角，λ为红外光谱变量，r为测量仪器与目标的距离；
作为面目标的辐射测量特性frad2表达如下：
Figure PCTCN2017077104-appb-000003
其中，
Figure PCTCN2017077104-appb-000004
为目标空间分布，
Figure PCTCN2017077104-appb-000005
为测量仪器所处的方位立体角，λ为红外光谱变量，r为测量仪器与目标的距离。
进一步地，步骤(2)包括如下子步骤：
(2.1)对输入的实测红外图像进行多线程操作，包括：
线程1：超像素分割获得天空背景、地面背景、目标区域等，根据分割结果并测量目标面积、灰度特征识别背景区域，作为负样本；
线程2：提取全图HOG特征，根据滑动窗口方法区分背景与目标区域，获得疑似目标，作为正样本；
线程3：利用全卷积神经网络，对输入图像检测出目标，作为正样本；
(2.2)将上述线程得到的结果，输入已预先训练好的支持向量机分类器中获得目标图像位置信息(x,y)；
(2.3)对目标图像做高斯金字塔获得图像多尺度信息后输入已训练的卷积神经网络，获得目标各感兴趣区域相对于目标中心位置的像素差
Figure PCTCN2017077104-appb-000006
(2.4)按照步骤(2.2)检测出两帧目标图像位置信息并进行处理，获得两帧图像目标的帧差、运动目标方向信息；按照步骤(2.3)得到的两帧之间目标像素差进行目标运动补偿。
为了实现上述目的，本发明还提供了用于上述测量方法的测量系统，其特征在于，包括：工控机、转镜、分光镜、中波镜头、长波镜头、非成像红外光谱测量单元、长波红外成像单元；工控机的控制接口连接转镜；中波镜头安装在非成像红外光谱测量单元上，非成像红外光谱测量单元的输出端连接工控机的输入端；长波镜头安装在长波红外成像单元上，长波红外成像单元的输出端连接工控机的输入端；
其中，转镜用于将感兴趣目标的光线反射给分光镜，分光镜用于将接收到的反射光线分成中波和长波分别传递给中波镜头和长波镜头，再分别 通过中波镜头和长波镜头传递给非成像红外光谱测量单元和长波红外成像单元进行处理；非成像红外光谱测量单元用于将接收到的中波处理成红外光谱数据并传输到工控机；长波红外成像单元用于将接收到的长波处理成图像数据并传输到工控机；工控机用于控制转镜的转向，以及处理接收到的红外光谱数据和图像数据，得到动目标多维度多尺度红外光谱特征。
进一步地，转镜采用四框伺服控制，包括由内到外依次设置的反射镜、内俯仰框、内方位框、外俯仰框、外方位框。
与现有技术相比，本发明通过预先建立三维模型，并将三维模型与实测处理后的红外光谱结合、对比，建立目标红外光谱多维度多尺度特征表示公式，提出目标红外光谱精确模型，提出感兴趣目标精确定位方法，识别感兴趣目标各部件位置，解决现有目标红外光谱特征表达不完整、不准确以及红外光谱测不到、测不准的问题。
【附图说明】
图1为本发明的总体流程图；
图2为F16战斗机红外仿真图像；
图3为8μm～14μm波段，1.25米分辨率的F16飞机仿真图像库部分图像；
图4为飞机感兴趣区域标记图；
图5为飞机红外光谱特征组成图；
图6为通过不同方位角测得的飞机红外光谱数据图；
图7为多框架伺服结构图；
图8为系统结构示意图；
图9为实际红外图感兴趣区域检测结果图；
[根据细则91更正 18.05.2017]　
图10为伺服内框跟踪系统扫描目标的示意图；
图11为实际红外图跟踪测谱结果图。
【具体实施方式】
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体 实施例仅仅用以解释本发明，并不用于限定本发明。此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
请参照图1，为本发明的总体流程图。本发明包括如下步骤：
(1)建立物方目标红外光谱特征多维度多尺度模型，从中提取感兴趣区域测量的过程模型。此模型主要目的为满足目标精确测谱需求。
(1.1)对需要测量的目标建立三维模型；
(1.2)确定测量目标的感兴趣区域部位，并对三维模型进行材质划分。对于立体目标而言，目标每个点(x,y,z)都有红外光谱特征，每个点都有温度，发射率可能不同，对于时敏目标而言，目标发生运动会导致目标红外光谱特征改变。以飞机为例，建模如图4所示，其红外光谱特征的辐射源主要包括，反射太阳辐射、反射地球辐射、发动机进气部件辐射、气动加热的蒙皮辐射、发动机热部件辐射、尾焰辐射。各感兴趣区域测量红外光谱如图5所示。
(1.3)根据飞机辐射源红外光谱特征分布测量获得数据，可得到特定角度下飞机红外光谱特征分布，从不同角度测量目标红外光谱，可得到不同角度下目标红外光谱特征，由图6所示，可知飞机的红外光谱特性在不同的观测方位角上变化很大。
物方的飞机目标辐射特性可表达为：
Frad(x,y,z,ω,S,T)    (1)
上式表明飞机目标是三维的，不同部位具有随空间位置不同的温度分布和辐射特性。其中Frad()表示目标在地球坐标系下位置(x,y,z)、测量角度ω、尺度S、时间维度T的情况下目标的辐射强度。
由实际实验结果，像方的飞机目标辐射测量特性函数有如下两种情况。
作为点状、斑状目标的辐射测量特性frad1，与测量仪器所处的方位角
Figure PCTCN2017077104-appb-000007
有关，可随方位角变化。空间平均辐射亮度(辐射强度)特性：
Figure PCTCN2017077104-appb-000008
式中，
Figure PCTCN2017077104-appb-000009
为测量的方位立体角，λ为红外光谱变量，r为距离。
作为面目标的辐射测量特性frad2与测量方位、目标空间分布相关，随距离变化，随方位角度
Figure PCTCN2017077104-appb-000010
变化。目标空间分布
Figure PCTCN2017077104-appb-000011
变化的辐射亮度特性：
Figure PCTCN2017077104-appb-000012
(2)对实测红外图像进行目标检测，识别目标各区域位置信息。跟踪目标获得两帧之间目标像素差以及获得目标运动方向，根据两帧之间目标像素差控制两轴四框架伺服进行目标运动补偿。
启动对输入图像进行多线程操作，执行步骤如下：
线程1：超像素分割获得天空背景、地面背景、目标区域等，根据分割结果并测量目标面积、灰度特征识别背景区域，作为负样本。
线程2：提取全图HOG特征(方向梯度直方图，Histogram of Oriented Gradient，HOG)，根据滑动窗口方法区分背景与目标区域，获得疑似目标，作为正样本。
线程3：利用全卷积神经网络，对输入图像检测出目标，作为正样本。
将上述线程得到的结果，输入已预先训练好的SVM分类器中获得目标图像位置信息(x,y)；其中SVM分类器部分训练样本如图2，3所示。实际检测效果如图9所示。SVM全称为Support Vector Machine，支持向量机。
对目标图像执行如下操作：输入目标图像，对图像做高斯金字塔获得图像多尺度信息后输入已训练的CNN网络(卷积神经网络，Convolutional Neural Network，CNN)，获得目标各感兴趣区域相对于目标中心位置的像素差
Figure PCTCN2017077104-appb-000013
检测出目标位置后，启动目标跟踪模块，获得两帧图像目标的帧差、运动目标方向信息。
本系统伺服采用四框架结构伺服系统，其转镜结构示意图如图7所示，图中外框负责大范围目标捕获跟踪；具体为外框对运动目标作运动补偿，将两帧图像间目标运动像素差输入伺服控制系统，控制系统快速响应误差输入，对运动目标作运动补偿，使目标始终位于视场中心位置。
(3)启动伺服内框跟踪系统，对目标作扫描。
成功捕获跟踪目标图像后，控制内框指向各感兴趣目标，并根据目标运动方向信息，在相对于运动方向90°的方向上作N个像素大小的偏移运动，即，扫描方向与运动方向相同，每扫描一次之后，垂直于运动方向偏移N个像素，然后再沿运动方向继续扫描；开启测谱模块，记录此时测量设备与目标之间距离信息，测量方位立体角信息、尺度信息、时间维信息，将上述信息输入步骤(1)获得的目标红外光谱特征多维度多尺度模型，得到最终的目标红外光谱模型，扫描示意图如图10，实际效果图如图11所示。
本发明还提出一种测量系统，即用于实现上述方法、能够将图像信息、红外光谱信息结合的图谱一体化的目标跟踪测谱系统，该系统结构示意图如图8所示。根据目标多维度多尺度红外光谱模型测量目标红外光谱信息。该设备通过成像系统获取目标的感兴趣区域(ROI，region of interest)位置信息、运动信息，控制多框架两轴伺服进行目标运动补偿，扫描目标特定区域获得目标红外光谱，建立目标红外光谱模型。主要应用于时敏目标识别。
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (6)

  1. 一种动目标多维度多尺度红外光谱特征测量方法，其特征在于，包括如下步骤：
    (1)建立物方目标红外光谱特征多维度多尺度模型，从中提取物方感兴趣区域测量模型；
    (2)对实测红外图像进行目标检测，识别目标各感兴趣区域位置信息；跟踪目标获得两帧之间目标像素差以及目标运动方向，然后根据两帧之间目标像素差进行目标运动补偿；
    (3)对步骤(2)识别出的目标作扫描，成功捕获跟踪目标图像后，控制内框指向各感兴趣目标，并根据目标运动方向信息，在相对于运动方向偏移90°的方向上作N个像素大小的运动，开启测谱模块，记录此时测量设备与目标之间距离信息，测量方位立体角信息、尺度信息、时间维信息，将上述信息输入步骤(1)获得的物方目标红外光谱特征多维度多尺度模型，得到像方目标红外光谱特征多维度多尺度模型。
  2. 如权利要求1所述的一种动目标多维度多尺度红外光谱特征测量方法，其特征在于，步骤(1)还包括如下子步骤：
    (1.1)对需要测量的目标建立三维模型；
    (1.2)从三维模型中确定测量目标的感兴趣区域部位，并对上述部位的三维模型进行材质划分，以确定辐射源；
    (1.3)对辐射源红外光谱特征进行测量，得到特定角度下物方红外光谱特征分布。
  3. 如权利要求2所述的一种动目标多维度多尺度红外光谱特征测量方法，其特征在于，物方的目标红外光谱辐射测量特性表达如下：
    Frad(x,y,z,ω,S,T)          (1)
    其中，Frad()表示目标在地球坐标系下位置(x,y,z)、测量角度ω、尺度 S、时间维度T的情况下目标的辐射强度；
    像方的目标辐射测量特性函数有如下两种情况：
    作为点状、斑状目标的辐射测量特性frad1表达如下：
    Figure PCTCN2017077104-appb-100001
    其中，
    Figure PCTCN2017077104-appb-100002
    为测量仪器所处的方位立体角，λ为红外光谱变量，r为测量仪器与目标的距离；
    作为面目标的辐射测量特性frad2表达如下：
    Figure PCTCN2017077104-appb-100003
    其中，
    Figure PCTCN2017077104-appb-100004
    为目标空间分布，
    Figure PCTCN2017077104-appb-100005
    为测量仪器所处的方位立体角，λ为红外光谱变量，r为测量仪器与目标的距离。
  4. 如权利要求1所述的一种动目标多维度多尺度红外光谱特征测量方法，其特征在于，步骤(2)包括如下子步骤：
    (2.1)对输入的实测红外图像进行多线程操作，包括：
    线程1：超像素分割获得天空背景、地面背景、目标区域等，根据分割结果并测量目标面积、灰度特征识别背景区域，作为负样本；
    线程2：提取全图HOG特征，根据滑动窗口方法区分背景与目标区域，获得疑似目标，作为正样本；
    线程3：利用全卷积神经网络，对输入图像检测出目标，作为正样本；
    (2.2)将上述线程得到的结果，输入已预先训练好的支持向量机分类器中获得目标图像位置信息(x,y)；
    (2.3)对目标图像做高斯金字塔获得图像多尺度信息后输入已训练的卷积神经网络，获得目标各感兴趣区域相对于目标中心位置的像素差
    Figure PCTCN2017077104-appb-100006
    (2.4)按照步骤(2.2)检测出两帧目标图像位置信息并进行处理，获得两帧图像目标的帧差、运动目标方向信息；按照步骤(2.3)得到的两帧 之间目标像素差进行目标运动补偿。
  5. 用于权利要求1所述的一种动目标多维度多尺度红外光谱特征测量方法的测量系统，其特征在于，包括：工控机、转镜、分光镜、中波镜头、长波镜头、非成像红外光谱测量单元、长波红外成像单元；工控机的控制接口连接转镜；中波镜头安装在非成像红外光谱测量单元上，非成像红外光谱测量单元的输出端连接工控机的输入端；长波镜头安装在长波红外成像单元上，长波红外成像单元的输出端连接工控机的输入端；
    其中，转镜用于将感兴趣目标的光线反射给分光镜，分光镜用于将接收到的反射光线分成中波和长波分别传递给中波镜头和长波镜头，再分别通过中波镜头和长波镜头传递给非成像红外光谱测量单元和长波红外成像单元进行处理；非成像红外光谱测量单元用于将接收到的中波处理成红外光谱数据并传输到工控机；长波红外成像单元用于将接收到的长波处理成图像数据并传输到工控机；工控机用于控制转镜的转向，以及处理接收到的红外光谱数据和图像数据，得到动目标多维度多尺度红外光谱特征。
  6. 如权利要求5所述的测量系统，其特征在于，转镜采用四框伺服控制，包括由内到外依次设置的反射镜、内俯仰框、内方位框、外俯仰框、外方位框。
PCT/CN2017/077104 2016-12-31 2017-03-17 一种动目标多维度多尺度红外光谱特征测量方法及系统 WO2018120445A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/458,219 US10839527B2 (en) 2016-12-31 2019-07-01 Method and device of measuring infrared spectral characteristics of moving target

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611268920.7A CN106772417B (zh) 2016-12-31 2016-12-31 一种动目标多维度多尺度红外光谱特征测量方法及系统
CN201611268920.7 2016-12-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/458,219 Continuation-In-Part US10839527B2 (en) 2016-12-31 2019-07-01 Method and device of measuring infrared spectral characteristics of moving target

Publications (1)

Publication Number Publication Date
WO2018120445A1 true WO2018120445A1 (zh) 2018-07-05

Family

ID=58951843

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/077104 WO2018120445A1 (zh) 2016-12-31 2017-03-17 一种动目标多维度多尺度红外光谱特征测量方法及系统

Country Status (3)

Country Link
US (1) US10839527B2 (zh)
CN (1) CN106772417B (zh)
WO (1) WO2018120445A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111915616A (zh) * 2020-05-26 2020-11-10 华瑞新智科技（北京）有限公司 一种基于弱监督图像分割进行红外测温的方法及装置

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3430595B1 (en) * 2017-05-23 2020-10-28 Brainlab AG Determining the relative position between a point cloud generating camera and another camera
CN107101720B (zh) * 2017-06-07 2018-11-13 中国科学技术大学 一种用于红外天光背景测量的装置和方法
CN108305290B (zh) * 2017-12-29 2020-12-08 华中科技大学 一种动目标的精确测谱方法
CN110033038B (zh) * 2019-04-09 2022-08-05 电子科技大学 气动热试验数据的智能选取算法
CN110815213B (zh) * 2019-10-21 2021-03-19 华中科技大学 一种基于多维度特征融合的零件识别装配方法和装置
CN110796687B (zh) * 2019-10-30 2022-04-01 电子科技大学 天空背景红外成像多目标跟踪方法
CN112287796B (zh) * 2020-10-23 2022-03-25 电子科技大学 基于VMD-Teager能量算子的辐射源识别方法
CN116310713B (zh) * 2023-02-16 2024-03-19 嘉洋智慧安全科技(北京)股份有限公司 红外图像识别方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050077424A1 (en) * 2003-05-30 2005-04-14 Schneider Arthur J. System and method for locating a target and guiding a vehicle toward the target
CN101465071A (zh) * 2009-01-08 2009-06-24 上海交通大学 多平台目标跟踪与分布交互仿真系统
CN101702021A (zh) * 2009-11-06 2010-05-05 华中科技大学 一种图谱一体化的时变对象光谱信息获取方法与装置
CN101738619A (zh) * 2009-11-27 2010-06-16 华中科技大学 双波段红外光学系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8462323B2 (en) * 2007-03-27 2013-06-11 Metrolaser, Inc. Integrated multi-sensor surveilance and tracking system
US7710545B2 (en) * 2008-02-13 2010-05-04 The Boeing Company Scanned laser detection and ranging apparatus
BRPI0921770A2 (pt) * 2008-11-04 2016-01-05 Univ Rice William M espectrômetros de mapeamento de imagem
CN102564589B (zh) * 2011-12-20 2013-07-24 华中科技大学 一种多波段动目标光谱特征探测识别方法和装置
US9258550B1 (en) * 2012-04-08 2016-02-09 Sr2 Group, Llc System and method for adaptively conformed imaging of work pieces having disparate configuration
WO2015195746A1 (en) * 2014-06-18 2015-12-23 Innopix, Inc. Spectral imaging system for remote and noninvasive detection of target substances using spectral filter arrays and image capture arrays

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050077424A1 (en) * 2003-05-30 2005-04-14 Schneider Arthur J. System and method for locating a target and guiding a vehicle toward the target
CN101465071A (zh) * 2009-01-08 2009-06-24 上海交通大学 多平台目标跟踪与分布交互仿真系统
CN101702021A (zh) * 2009-11-06 2010-05-05 华中科技大学 一种图谱一体化的时变对象光谱信息获取方法与装置
CN101738619A (zh) * 2009-11-27 2010-06-16 华中科技大学 双波段红外光学系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111915616A (zh) * 2020-05-26 2020-11-10 华瑞新智科技（北京）有限公司 一种基于弱监督图像分割进行红外测温的方法及装置

Also Published As

Publication number Publication date
US10839527B2 (en) 2020-11-17
CN106772417B (zh) 2017-11-14
CN106772417A (zh) 2017-05-31
US20190325586A1 (en) 2019-10-24

Similar Documents

Publication Publication Date Title
WO2018120445A1 (zh) 一种动目标多维度多尺度红外光谱特征测量方法及系统
WO2015101062A1 (zh) 一种多波段共光路图谱联合遥感测量系统及方法
CN110208771B (zh) 一种移动二维激光雷达的点云强度改正方法
CN113359097A (zh) 一种毫米波雷达和相机联合标定的方法
CN109269525B (zh) 一种空间探测器起飞或着陆过程光学测量系统及方法
CN112508903B (zh) 一种卫星望远镜镜片表面疵病轮廓检测方法
CN110245566B (zh) 一种基于背景特征的红外目标远距离追踪方法
Li et al. Space target detection in complicated situations for wide-field surveillance
Knyaz Multimodal data fusion for object recognition
Kölling et al. Aircraft-based stereographic reconstruction of 3-D cloud geometry
Liu et al. Multispectral registration method based on stellar trajectory fitting
O'Byrne et al. A comparison of image based 3D recovery methods for underwater inspections
CN105403886B (zh) 一种机载sar定标器图像位置自动提取方法
CN109990902B (zh) 一种图谱关联的分谱段估计目标亮温的方法
CN106353032B (zh) 一种欠光照条件下的天体形心快速检测方法
CN108305290B (zh) 一种动目标的精确测谱方法
CN109781259B (zh) 一种图谱关联的精准测量空中运动小目标红外光谱的方法
Liu et al. A method for restraining gyroscope drift using horizon detection in infrared video
CN115082555A (zh) 一种rgbd单目相机的高精度位移实时测量系统及方法
Sease et al. Long-integration star tracker image processing for combined attitude-attitude rate estimation
Khodaei et al. 3D surface generation from aerial thermal imagery
CN117113284B (zh) 多传感器融合的数据处理方法、装置与多传感器融合方法
Brown et al. Experiments in multiple-waveband passive polarimetric and active infrared imaging for material classification
KR20180092089A (ko) 적외선 모델러 검증 장치 및 방법
Kim et al. Height-adaptive vehicle detection in aerial imagery using metadata of eo sensor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17888445

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17888445

Country of ref document: EP

Kind code of ref document: A1