WO2021248270A1 - 一种异源图像配准方法及系统 - Google Patents

一种异源图像配准方法及系统 Download PDF

Info

Publication number
WO2021248270A1
WO2021248270A1 PCT/CN2020/094892 CN2020094892W WO2021248270A1 WO 2021248270 A1 WO2021248270 A1 WO 2021248270A1 CN 2020094892 W CN2020094892 W CN 2020094892W WO 2021248270 A1 WO2021248270 A1 WO 2021248270A1
Authority
WO
WIPO (PCT)
Prior art keywords
point
contour
points
feature
curvature
Prior art date
Application number
PCT/CN2020/094892
Other languages
English (en)
French (fr)
Inventor
刘亚东
严英杰
姜骞
裴凌
李喆
徐鹏
苏磊
傅晓飞
江秀臣
Original Assignee
上海交通大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海交通大学 filed Critical 上海交通大学
Priority to CN202080097152.1A priority Critical patent/CN115176274A/zh
Priority to US17/598,310 priority patent/US20220319011A1/en
Priority to PCT/CN2020/094892 priority patent/WO2021248270A1/zh
Publication of WO2021248270A1 publication Critical patent/WO2021248270A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/149—Segmentation; Edge detection involving deformable models, e.g. active contour models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods

Definitions

  • the present invention relates to the technical field of image distribution, in particular to a method and system for heterogeneous image registration.
  • Inspection robots and drones are equipped with infrared and visible light binocular cameras, which are responsible for photographing the operating status of electrical equipment under working conditions, and upload the image data to the database of the maintenance unit for status monitoring and fault diagnosis.
  • These automated inspection equipment mainly use multi-mode imaging sensors and image processing technology to work, so the development of efficient and reliable image processing technology for power equipment is very important for the construction of diagnostic systems, such as image recognition, fusion and distribution Technology etc.
  • image distribution technology is the prelude to the application of many image technologies, especially image stitching, fusion and target detection.
  • the purpose of image distribution is to align multiple images of the same scene taken by different sensors, different moments, and different perspectives.
  • the visible light image can capture the shape information of the power equipment, while the infrared image can capture the heat radiation information of the power equipment. Therefore, the infrared and visible image distribution and recognition technology of power equipment can present various information of power equipment in one image, which greatly facilitates the fault detection of power equipment.
  • the present invention provides a heterogeneous image registration method and system, which can solve the problem of accurate provision of infrared and visible light images of power equipment.
  • the present invention provides the following technical solutions: including, using the Canny edge detection operator to perform edge detection on the collected images, combining with the curvature scale space strategy to extract contour curve segments in the edge image; based on global and local curvature features
  • the point detection strategy detects the feature points in the contour curve segment, and obtains the nearest local minimum curvature values of the feature points pointing to the start and end directions of the contour respectively; calculate the number of neighborhood sampling points and the number of sampling points according to the local curvature minimum
  • the neighborhood auxiliary feature points of the neighborhood on both sides of the feature point use the neighborhood auxiliary feature point and the feature point to form a feature triangle, and calculate the angle bisector vector and main direction corresponding to the feature point in the feature triangle;
  • the main direction points to the concave side of the contour of the feature point to complete the main direction distribution.
  • detecting the feature point includes detecting the edge binary image of the image using the Canny edge detection operator, and extracting the The contour curve segment in the edge image; calculate the curvature of each point in the contour curve segment, use the curvature maximum point on the contour as a candidate feature point and store the nearest local extremes at both ends of each maximum point Small value points; calculate the average curvature value of each candidate feature point in the neighborhood of the contour curve segment; set a curvature multiple threshold, when the curvature of the candidate feature point is less than the average curvature value and the threshold When multiplying the product, reject the candidate feature point; when the curvature of the candidate feature point is greater than the product of the average curvature value and the threshold, calculate whether the candidate feature point is on a circular contour, and if so, reject it, If not, keep it; calculate the positional relationship of the retained candidate feature points, and define the candidate if the angle between a certain feature point and two adjacent feature points
  • the local curvatures of the contour curve segment that are closest to the feature point to the starting point and the end point of the contour are respectively detected to be extremely small Value point; traverse from the first point to the last point of each contour, if the curvature of a certain point is less than the curvature of the left and right points at the same time, it is defined as the local curvature minimum Value points; the characteristic points and their corresponding local curvature minimum points pointing to the starting and ending points of the contour are respectively denoted as and
  • calculating the number of sampling points in the neighborhood includes:
  • f ⁇ [2, n], n the number of feature points included in each contour
  • ⁇ L and ⁇ R are respectively the number of neighborhood sampling points of the feature point.
  • the neighborhood auxiliary feature points include: The abscissa and ordinate of the point are respectively Gaussian weighted, as follows,
  • G ⁇ one-dimensional Gaussian kernel
  • calculating the angle bisector vector includes:
  • calculating the main direction includes:
  • the direction of the contour angle is the main direction of the feature point in the distribution.
  • performing edge detection on the image includes performing Gaussian filtering processing on the gray image; and calculating the gradient in the filtered image Matrix and direction matrix; non-maximum value suppression is performed on the gradient matrix to obtain a non-maximum value suppression image; a dual-threshold strategy is used to detect and connect image edge contours.
  • extracting the contour curve segment includes: starting from the first row and the first column of the edge image, iteratively traverses and searches in units of rows; If a certain point is found to be an edge point, mark it as the first contour point and set it to 0; find out whether there is the edge point in the neighborhood where the side length of the contour point is 1, and if so, add it to all As the second contour point in the contour neighborhood, set it to 0; repeat the search until there is no such edge point in a certain point neighborhood, and define the stopping point as the last point of the contour; each contour It is a set of the edge points, and all the contours obtained are integrated to form a contour set, as follows,
  • ⁇ j the j-th contour curve segment in the set S
  • n the number of pixels contained in the contour ⁇ j
  • N s the total number of contour curve segments in the set S.
  • the heterogeneous image registration system of the present invention includes: an information collection module for collecting the image data information and acquiring characteristic information; a data processing center module for receiving, Calculate, store, and output the data information to be processed, which includes an arithmetic unit, a database, and an input and output management unit.
  • the arithmetic unit is connected to the acquisition module and is used to receive the data information acquired by the information acquisition module to perform operations Processing, calculating the local curvature minimum, the number of neighborhood sampling points, the neighborhood auxiliary feature points, the angle bisector vector and the main direction, the database is connected to each module, and is used to store all received Data information provides deployment and supply services for the data processing center module, the input and output management unit is used to receive the information of each module and output the calculation results of the calculation unit; the distribution module is connected to the data processing center module, which It is used to read the calculation result of the arithmetic unit, control the main direction to point to the concave side of the contour of the feature point, complete the main direction distribution, and achieve precise matching.
  • the method of the present invention calculates the main direction through the contour features between the images.
  • the difference in the resolution, spectrum and viewing angle of infrared and visible images is large, and it is impossible to realize the configuration when these three scenes coexist.
  • each feature point is given a unique direction parameter to perform accurate description calculations, so that the extracted feature points are significant and accurate when matching;
  • the infrared and visible light images of the power equipment are both It has extremely prominent contour characteristics, so the method of the present invention has higher adaptability to image registration scenes of power equipment.
  • FIG. 1 is a schematic flowchart of a method for registration of heterogeneous images according to the first embodiment of the present invention
  • FIG. 2 is a schematic diagram of an image contour set extraction process of a heterogeneous image registration method according to the first embodiment of the present invention
  • FIG. 3 is a schematic diagram of the CAO calculation of the main direction of the contour angle of a heterogeneous image registration method according to the first embodiment of the present invention
  • FIG. 4 is an infrared schematic diagram of a power device of a heterogeneous image registration method according to the first embodiment of the present invention
  • FIG. 5 is a visible light schematic diagram of a power device of a heterogeneous image registration method according to the first embodiment of the present invention
  • FIG. 6 is a schematic diagram of the quantitative comparison result of the main direction experiment of a heterogeneous image registration method according to the first embodiment of the present invention.
  • FIG. 7 is a schematic diagram of image comparison results when the rotation angle of a heterogeneous image registration method according to the first embodiment of the present invention is 0;
  • FIG. 8 is a schematic diagram of the module structure distribution of a heterogeneous image registration system according to the second embodiment of the present invention.
  • the “one embodiment” or “embodiment” referred to herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation of the present invention.
  • the appearances of "in one embodiment” in different places in this specification do not all refer to the same embodiment, nor are they separate or selectively mutually exclusive embodiments with other embodiments.
  • connection should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection.
  • the connection can also be indirectly connected through an intermediate medium, or it can be the internal communication between two components.
  • the specific meaning of the above-mentioned terms in the present invention can be understood in specific situations.
  • the point feature-based allocation method is very adaptable to image viewing angle changes and image registration scenes shot at different times, so it is widely used and researched.
  • the point feature-based method still has obvious problems in registering infrared and visible light images. For example, the resolution, spectrum, and viewing angle of infrared and visible light images are quite different.
  • the existing point feature-based methods cannot achieve registration when these three scenes coexist. The main problem is that the feature points are not accurately described. .
  • This embodiment provides a heterogeneous image registration method, including:
  • S1 Use the Canny edge detection operator to perform edge detection on the collected image, and combine the curvature scale space strategy to extract the contour curve segment in the edge image. It should be noted that the edge detection of an image includes:
  • Scan image 2 when encountering a non-zero gray pixel p(x,y), trace the contour line starting with p(x,y) until the end point q(x,y) of the contour line ;
  • the search for the next contour is repeated in sequence, until no new contour is found, the search is stopped, and the edge detection of the canny operator is completed.
  • extracting contour curve segments includes:
  • Each contour is a collection of edge points, and all the contours obtained are integrated to form a contour set, as follows,
  • ⁇ j the j-th contour curve segment in the set S
  • n the number of pixels contained in the contour ⁇ j
  • N s the total number of contour curve segments in the set S.
  • the feature point detection strategy based on global and local curvature detects the feature points in the contour curve segment, and obtains the nearest local minimum curvature points of the feature points pointing to the starting and ending points of the contour respectively.
  • the detection feature points include:
  • the candidate feature point is defined as a feature point, otherwise, the candidate is eliminated Feature points;
  • the first point and the last point of each unclosed contour are regarded as feature points, and the feature point extraction ends.
  • S3 Calculate the number of neighborhood sampling points and the neighborhood auxiliary feature points of the neighborhood on both sides of the feature point according to the local curvature minimum. It should also be noted that calculating the number of sampling points in the neighborhood includes:
  • f ⁇ [2, n], n the number of feature points included in each contour
  • ⁇ L and ⁇ R are respectively the number of neighborhood sampling points of the feature point.
  • the neighborhood auxiliary feature points include:
  • Gaussian weighting is performed on the abscissa and ordinate of all points between the kLth point and the fth point in the jth contour, as follows,
  • G ⁇ one-dimensional Gaussian kernel
  • the main direction of calculation includes:
  • the direction of the contour angle is the main direction of the feature point in the distribution.
  • Each feature point and its neighboring auxiliary feature points form a feature triangle
  • the characteristic triangles corresponding to the same feature point in infrared and visible light images are similar triangles.
  • the angle corresponding to the angle bisector vector is selected as the main direction to ensure that the main direction of the same feature point in different images is the same;
  • the corresponding image registration strategy can successfully register images with different viewing angles.
  • this embodiment uses three existing registration methods as examples to better understand the method of the present invention.
  • the method based on the calibration parameters of infrared and visible cameras requires prior knowledge of the camera. It can only align a set of images taken by the camera at the same time, and cannot register images taken at different times and locations. Once calibrated, the registration error at different distances is constant, so this method only adapts to the scene In limited circumstances, image registration at different locations at different times cannot be achieved; 2.
  • the region-based infrared and visible image registration method mainly uses some similarity measurement algorithms to compare the similarity of certain regions in infrared and visible images. Make judgments to find the global optimal transformation parameters.
  • This method is only suitable for situations where there is no obvious viewing angle difference between infrared and visible images and the overlap area between the images is relatively large. Its application scenarios are still limited; three, feature-based
  • the registration method mainly processes some point, line and surface features in the image to register the image.
  • the resolution, spectrum and viewing angle of infrared and visible light images are quite different, and the existing methods based on point features cannot be realized.
  • the description of the feature points is not accurate enough.
  • the main direction assignment of feature points is one of the most important steps in the description of feature points. Its purpose is to assign unique directional parameters to each feature point, highlighting the image characteristics of the point, so as to extract the feature description.
  • the existing main direction allocation SIFT, PIIFD, and SURF algorithms are all based on the gradient characteristics of the image to perform the main direction allocation, but the infrared and visible light image spectrums are very different, and the extracted gradient features are not as good as the correlation between the homology images. High performance; and the method of the present invention mainly relies on the contour features between the images for main direction calculation. Since the infrared and visible light images of the power equipment have extremely prominent contour features, the method of the present invention is adapted to the image registration scene of the power equipment High sex.
  • this embodiment chooses to use the traditional SIFT method, PIIFD method, SURF method and the method of the present invention for comparative testing, to scientifically demonstrate The method compares the test results to verify the true effect of the method of the present invention; the traditional SIFT method, the PIIFD method and the SURF method have low accuracy and low adaptability for the main direction allocation.
  • the traditional method and the method of the present invention will be used to perform a distribution comparison test on one infrared image and six visible light images of the same electric device;
  • Test conditions (1) One infrared image (resolution 120 ⁇ 280) and six visible light images of the same electrical equipment are used as contrast images;
  • the rotation angle of the viewing angle of the six visible light images relative to the infrared image is 0° (resolution 120 ⁇ 280), 60° (resolution 252 ⁇ 203), 120° (resolution 252 ⁇ 203), 180°( Resolution 120 ⁇ 280), 240°(resolution 252 ⁇ 203), 300°(resolution 252 ⁇ 203);
  • the comparison curve of the main direction distribution test results between the traditional method and the method of the present invention (ie CAO), the output curve of the method of the present invention is much higher than that of the three traditional methods at a rotation angle of 0° to 300°
  • Figure 7 is a schematic diagram of image comparison results when the rotation angle is 0°. It can be intuitively seen that the method of the present invention shows the most correct points under different rotation angles, and the method of the present invention obtains The correct number of points is three times that of the traditional method. According to the image results, the main directions of the feature points of the method of the present invention all point to the concave side of the contour, which verifies that the method of the present invention has high accuracy and adaptability.
  • FIG. 8 it is the second embodiment of the present invention.
  • This embodiment is different from the first embodiment in that it provides a heterogeneous image registration system, including:
  • the information collection module 100 is used to collect image data information and obtain characteristic information; the data collection is a bridge connecting the computer and the external physical world, and it is composed of sensors and controllers.
  • the data processing center module 200 is used to receive, calculate, store, and output data information to be processed. It includes a computing unit 201, a database 202, and an input and output management unit 203.
  • the computing unit 201 is connected to the collection module 100 for receiving information
  • the data information acquired by the module 100 is collected for arithmetic processing to calculate the minimum local curvature, the number of neighborhood sampling points, the neighborhood auxiliary feature points, the angle bisector vector and the main direction.
  • the database 202 is connected to each module for storing the received All data information provides deployment and supply services for the data processing center module 200, and the input and output management unit 203 is used to receive the information of each module and output the calculation result of the calculation unit 201.
  • the distribution module 300 is connected to the data processing center module 200, and is used to read the calculation result of the calculation unit 201, control the main direction to point to the concave side of the feature point contour, complete main direction distribution, and achieve precise matching.
  • the information collection module 100 is a communication module based on a remote data collection platform, which integrates a communication chip and a memory chip on a circuit board, so that it can send and receive information, communication, and data transmission through the remote data collection platform.
  • Function, and the computer, single-chip microcomputer, and ARM are connected with the remote data acquisition platform through the RS232 serial port, and the information acquisition module 100 is controlled by the AI command to realize the data communication function.
  • the data processing center module 200 is mainly divided into three levels, including a control layer, a computing layer, and a storage layer.
  • the control layer is the command and control center of the data processing center module 200.
  • the decoder ID and the operation controller OC are composed.
  • the control layer can take out each instruction from the memory in turn according to the program programmed by the user, put it in the instruction register IR, analyze and determine by the instruction decoder, and notify the operation controller OC performs operations and sends micro-operation control signals to the corresponding components according to the determined timing;
  • the arithmetic layer is the core of the data processing center module 200, which can perform arithmetic operations (such as addition, subtraction, multiplication, division and additional operations) and logical operations (such as shifting) , Logic test or comparison of two values), which is connected to the control layer, and performs arithmetic operations by receiving control signals from the control layer;
  • the storage layer is the database of the data processing center module 200, which can store data (to be processed and processed data) ).
  • the embodiments of the present invention can be realized or implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory.
  • the method can be implemented in a computer program using standard programming techniques-including a non-transitory computer readable storage medium configured with a computer program, where the storage medium so configured allows the computer to operate in a specific and predefined manner-according to the specific
  • Each program can be implemented in a high-level process or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed application specific integrated circuit for this purpose.
  • the processes (or variants and/or combinations thereof) described herein can be executed under the control of one or more computer systems configured with executable instructions, and can be used as codes that are executed collectively on one or more processors (e.g., , Executable instructions, one or more computer programs, or one or more applications), implemented by hardware or a combination thereof.
  • the computer program includes a plurality of instructions executable by one or more processors.
  • the method can be implemented in any type of computing platform that is operably connected to a suitable computing platform, including but not limited to a personal computer, a mini computer, a main frame, a workstation, a network or a distributed computing environment, a separate or integrated computer Platform, or communication with charged particle tools or other imaging devices, etc.
  • a suitable computing platform including but not limited to a personal computer, a mini computer, a main frame, a workstation, a network or a distributed computing environment, a separate or integrated computer Platform, or communication with charged particle tools or other imaging devices, etc.
  • Aspects of the present invention can be implemented by machine-readable codes stored on non-transitory storage media or devices, whether removable or integrated into computing platforms, such as hard disks, optical reading and/or writing storage media, RAM, ROM, etc., so that they can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein.
  • the machine-readable code, or part of it can be transmitted over a wired or wireless network.
  • the invention described herein includes these and other different types of non-transitory computer-readable storage media.
  • the present invention also includes the computer itself.
  • a computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data that is stored in non-volatile memory.
  • the output information can also be applied to one or more output devices such as displays.
  • the converted data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on the display.
  • a component may be, but is not limited to: a process running on a processor, a processor, an object, an executable file, an executing thread, a program, and/or a computer.
  • an application running on a computing device and the computing device may be components.
  • One or more components may exist in an executing process and/or thread, and the components may be located in one computer and/or distributed between two or more computers.
  • these components can execute from various computer-readable media having various data structures thereon.
  • These components can be based on, for example, having one or more data packets (for example, data from a component that interacts with another component in a local system, a distributed system, and/or via signals such as the Internet).
  • the network interacts with other systems) signals to communicate in a local and/or remote process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种异源图像配准方法及系统，包括，利用Canny边缘检测算子对采集的图像进行边缘检测，结合曲率尺度空间策略提取边缘图像中的轮廓曲线段；基于全局和局部曲率的特征点检测策略检测轮廓曲线段中的特征点，分别得到特征点指向轮廓起点和终点方向的最近局部曲率极小值；根据局部曲率极小值计算邻域采样点数和特征点两侧邻域的邻域辅助特征点利用邻域辅助特征点与特征点构成特征三角形并计算特征三角形中特征点对应的角平分线向量和主方向；主方向指向特征点轮廓的凹侧，完成主方向分配。该方法具有显著性、精准性，对电力设备的图像配准场景具有较高的适应性。

Description

一种异源图像配准方法及系统 技术领域
本发明涉及图像分配的技术领域，尤其涉及一种异源图像配准方法及系统。
背景技术
由于自动化巡检降低了电网高压设备巡检工作对人力资源的需求，并且能提高巡检效率，因此基于巡检机器人和无人机的自动化巡检技术已成为电网设备智能诊断系统的重要组成部分。巡检机器人和无人机均装配有红外和可见光双目相机，负责拍摄工况下的电力设备运行状态，并将图像数据上传到检修单位的数据库用于状态监测和故障诊断。这些自动化的巡检设备主要是利用多模成像传感器和图像处理技术进行工作的，因此研发高效可靠的电力设备图像处理技术对于诊断系统的构建来说是非常重要的，例如图像识别、融合和分配技术等。其中图像分配技术是多种图像技术应用的前序工作，尤其是图像拼接、融合以及目标探测。图像分配的目的在于对齐同一场景在不同传感器、不同时刻、不同视角拍摄下的多幅图像。可见光图像能捕捉电力设备的外形信息，同时红外图像能捕捉电力设备的热辐射信息。因此，对电力设备红外和可见光的图像分配与识别技术可以将电力设备的多种信息呈现在一张图像中，大大方便电力设备的故障检测。
然而，实现电力设备红外和可见光图像的精确分配在目前来说是一项非常具有挑战性的工作，并且现有的算法很难做到精确对齐。因为红外和可见光图像的分辨率、光谱以及视角差异均有明显差异。
发明内容
本部分的目的在于概述本发明的实施例的一些方面以及简要介绍一些较佳实施例。在本部分以及本申请的说明书摘要和发明名称中可能会做些简化或省略以避免使本部分、说明书摘要和发明名称的目的模糊，而这种简化或省略不能用于限制本发明的范围。
鉴于上述现有存在的问题，提出了本发明。
因此，本发明提供了一种异源图像配准方法及系统，能够解决电力设备红外和可见光图像的精准配备问题。
为解决上述技术问题，本发明提供如下技术方案：包括，利用Canny边缘检测算子对采集的图像进行边缘检测，结合曲率尺度空间策略提取边缘图像中 的轮廓曲线段；基于全局和局部曲率的特征点检测策略检测所述轮廓曲线段中的特征点，分别得到所述特征点指向轮廓起点和终点方向的最近局部曲率极小值；根据所述局部曲率极小值计算邻域采样点数和所述特征点两侧邻域的邻域辅助特征点；利用所述邻域辅助特征点与所述特征点构成特征三角形并计算所述特征三角形中所述特征点对应的角平分线向量和主方向；所述主方向指向所述特征点轮廓的凹侧，完成主方向分配。
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：检测所述特征点包括，利用所述Canny边缘检测算子检测所述图像的边缘二值图像，提取所述边缘图像中的所述轮廓曲线段；计算所述轮廓曲线段中每个点的曲率，将轮廓上的曲率极大值点作为候选特征点并存储检测每个极大值点两端的最近局部极小值点；计算每个所述候选特征点在所述轮廓曲线段邻域的平均曲率值；设定曲率倍数阈值，当所述候选特征点的曲率小于所述平均曲率值与所述阈值的乘积时，剔除所述候选特征点；当所述候选特征点的曲率大于所述平均曲率值与所述阈值的乘积时，计算所述候选特征点是否处于圆形轮廓上，若是，则剔除，若否，则保留；对保留的所述候选特征点进行位置关系计算，若同一轮廓上某个特征点与其相邻的两个特征点之间的夹角小于所述阈值，则定义所述候选特征点为所述特征点，反之，则剔除所述候选特征点；将每条非闭合轮廓的第一个点和最后一个点作为所述特征点，所述特征点提取结束。
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：分别检测出所述轮廓曲线段内离所述特征点最近的指向轮廓起点和终点的所述局部曲率极小值点；从每条轮廓的所述第一个点遍历到所述最后一个点，若某个点的所述曲率同时小于左右两个点的所述曲率，则定义为所述局部曲率极小值点；所述特征点及其对应的指向轮廓起点和终点的所述局部曲率极小值点分别记为
Figure PCTCN2020094892-appb-000001
和
Figure PCTCN2020094892-appb-000002
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：计算所述邻域采样点数包括，
λ L＝f-kL
λ R＝kR-f
其中，f∈[2，n]，n：每条轮廓包括的特征点数目，λ L和λ R分别是所述特征点的邻域采样点数。
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：所述邻域辅助特征点包括，对第j条轮廓中第kL个点到第f个点之间的所有点的横坐标和纵坐标分别进行高斯加权，如下，
Figure PCTCN2020094892-appb-000003
Figure PCTCN2020094892-appb-000004
Figure PCTCN2020094892-appb-000005
其中，G σ：一维高斯内核，
Figure PCTCN2020094892-appb-000006
和
Figure PCTCN2020094892-appb-000007
所述邻域辅助特征点，当f＝1或n时，计算如下，
Figure PCTCN2020094892-appb-000008
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：计算所述角平分线向量包括，
Figure PCTCN2020094892-appb-000009
Figure PCTCN2020094892-appb-000010
Figure PCTCN2020094892-appb-000011
其中，
Figure PCTCN2020094892-appb-000012
分别为所述特征点
Figure PCTCN2020094892-appb-000013
的坐标，
Figure PCTCN2020094892-appb-000014
所述角平分线向量。
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：计算所述主方向包括，
Figure PCTCN2020094892-appb-000015
其中，
Figure PCTCN2020094892-appb-000016
特征点
Figure PCTCN2020094892-appb-000017
的轮廓角方向，在分配中该轮廓角方向即为特征点的主方向。
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：对所述图像进行边缘检测包括，对灰度图像进行高斯滤波处理；计算滤波后的所述图像中的梯度矩阵和方向矩阵；对所述梯度矩阵进行非极大值抑制，得到非极大值抑制图像；利用双阈值策略检测、连接图像边缘轮廓。
作为本发明所述的一种异源图像配准方法的一种优选方案，其中：提取所述轮廓曲线段包括，从所述边缘图像的第一行第一列开始以行为单位循环遍历查找；若查找到某一个点为边缘点，则标记为第一个轮廓点，且设为0；查找所述轮廓点的边长为1的邻域内是否有所述边缘点，若有，则加入所述轮廓邻域内作为第二个所述轮廓点，设为0；重复进行查找，直至某个点邻域内没有所述边缘点为止，定义停止的点为轮廓的所述最后一个点；每条轮廓都是一个所述边缘点的集合，整合获取的全部轮廓，形成轮廓集合，如下，
Figure PCTCN2020094892-appb-000018
其中，Γ j：集合S中的第j条轮廓曲线段，n：轮廓Γ j包含的像素点个数，N s：集合S中的轮廓曲线段总数目。
作为本发明所述的一种异源图像配准系统的一种优选方案，其中：包括，信息采集模块，用于采集所述图像数据信息、获取特征信息；数据处理中心模块，用于接收、计算、存储、输出待处理的数据信息，其包括运算单元、数据库和输入输出管理单元，所述运算单元与所述采集模块相连接，用于接收所述信息采集模块获取的数据信息以进行运算处理，计算所述局部曲率极小值、所述邻域采样点数、所述邻域辅助特征点及所述角平分线向量和主方向，所述数据库连接于各个模块，用于存储接收的所有数据信息，为所述数据处理中心模 块提供调配供应服务，所述输入输出管理单元用于接收各个模块的信息并输出所述运算单元的运算结果；分配模块连接于所述数据处理中心模块，其用于读取所述运算单元的运算结果，控制所述主方向指向所述特征点轮廓的凹侧，完成主方向分配，达到精准匹配。
本发明的有益效果：本发明方法通过图像之间的轮廓特征进行主方向计算，一方面针对红外和可见光图像的分辨率、光谱和视角差异较大而无法实现对这三种场景共存时的配准问题，对每个特征点赋予独特的方向参数以进行精准的描述运算，使得提取到的特征点具有显著性、匹配时具有精准性；另一方面，由于电力设备的红外和可见光图中都具有极其显著的轮廓特性，因此本发明方法对电力设备的图像配准场景具有较高的适应性。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其它的附图。其中：
图1为本发明第一个实施例所述的一种异源图像配准方法的流程示意图；
图2为本发明第一个实施例所述的一种异源图像配准方法的图像轮廓集合提取流程示意图；
图3为本发明第一个实施例所述的一种异源图像配准方法的轮廓角主方向CAO计算示意图；
图4为本发明第一个实施例所述的一种异源图像配准方法的电力设备红外示意图；
图5为本发明第一个实施例所述的一种异源图像配准方法的电力设备可见光示意图；
图6为本发明第一个实施例所述的一种异源图像配准方法的主方向实验定量对比结果示意图；
图7为本发明第一个实施例所述的一种异源图像配准方法的旋转角度为0时的图像对比结果示意图；
图8为本发明第二个实施例所述的一种异源图像配准系统的模块结构分布示意图。
具体实施方式
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合说明书附图对本发明的具体实施方式做详细的说明，显然所描述的实施例是本发明的一部分实施例，而不是全部实施例。基于本发明中的实施例，本领域普通人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明的保护的范围。
在下面的描述中阐述了很多具体细节以便于充分理解本发明，但是本发明还可以采用其他不同于在此描述的其它方式来实施，本领域技术人员可以在不违背本发明内涵的情况下做类似推广，因此本发明不受下面公开的具体实施例的限制。
其次，此处所称的“一个实施例”或“实施例”是指可包含于本发明至少一个实现方式中的特定特征、结构或特性。在本说明书中不同地方出现的“在一个实施例中”并非均指同一个实施例，也不是单独的或选择性的与其他实施例互相排斥的实施例。
本发明结合示意图进行详细描述，在详述本发明实施例时，为便于说明，表示器件结构的剖面图会不依一般比例作局部放大，而且所述示意图只是示例，其在此不应限制本发明保护的范围。此外，在实际制作中应包含长度、宽度及深度的三维空间尺寸。
同时在本发明的描述中，需要说明的是，术语中的“上、下、内和外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。此外，术语“第一、第二或第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。
本发明中除非另有明确的规定和限定，术语“安装、相连、连接”应做广义理解，例如：可以是固定连接、可拆卸连接或一体式连接；同样可以是机械连接、电连接或直接连接，也可以通过中间媒介间接相连，也可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本发明中的具体含义。
实施例1
基于点特征的分配方法对于图像视角变换和不同时刻拍摄的图像配准场 景的适应性非常强，因此被广泛使用和研究，但是基于点特征的方法在配准红外和可见光图像依旧存在明显问题，例如，红外和可见光图像的分辨率、光谱和视角差异比较大，现有的基于点特征的方法无法实现对这三种场景共存时的配准，其主要问题在于对特征点进行描述时不够精确。
参照图1～图7，为本发明的第一个实施例，该实施例提供了一种异源图像配准方法，包括：
S1：利用Canny边缘检测算子对采集的图像进行边缘检测，结合曲率尺度空间策略提取边缘图像中的轮廓曲线段。其中需要说明的是，对图像进行边缘检测包括：
对灰度图像进行高斯滤波处理；
计算滤波后的图像中的梯度矩阵和方向矩阵，分别使用Sx和Sy与图像卷积，得到两个相同的图像矩阵G x和G y，并计算该矩阵相同位置的元素平方和(即梯度)与正负切函数值(方向)，分别得到梯度矩阵和方向矩阵，如下：
Figure PCTCN2020094892-appb-000019
Figure PCTCN2020094892-appb-000020
θ＝arctan(G y/G x)
对梯度矩阵进行非极大值抑制，查找梯度矩阵中的极大值(如果一个点的梯度大于其所在的边长为3的正方形邻域内所有点的梯度则为极大值)，得到非极大值抑制图像；
利用双阈值策略检测、连接图像边缘轮廓，对非极大值抑制图像设定两个阈值th1和th2，两者关系th1＝0.4th2，将梯度值小于th1的像素的灰度值设为0，得到图像1，将梯度值小于th2的像素的灰度值设为0，得到图像2(由于图像2的阈值较高，去除大部分噪音，但同时也损失了有用的边缘信息，而图像1的阈值较低，保留了较多的信息，以图像2为基础，以图像1为补充来连结图像的边缘)。
具体的，包括：
对图像2进行扫描，当遇到一个非零灰度的像素p(x,y)时，跟踪以p(x,y)为开始点的轮廓线，直到轮廓线的终点q(x,y)；
观察图像1中与图像2中q(x,y)点位置对应的点s(x,y)的邻近区域；
若在s(x,y)点的邻近区域中有非零像素s(x,y)存在，则将其融合到图像2中，作为r(x,y)点；
重复进行上述步骤，直至无法继续在图像1和图像2中进行扫描；
当完成对包含p(x,y)的轮廓线的连结之后，标记这条轮廓线为已经访问；
依次重复进行寻找下一条轮廓线，直至找不到新的轮廓线时停止寻找，完成canny算子的边缘检测。
进一步的，参照图2，提取轮廓曲线段包括：
从边缘图像的第一行第一列开始以行为单位循环遍历查找；
若查找到某一个点为边缘点，则标记为第一个轮廓点，且设为0；
查找轮廓点的边长为1的邻域内是否有边缘点，若有，则加入轮廓邻域内作为第二个轮廓点，设为0；
重复进行查找，直至某个点邻域内没有边缘点为止，定义停止的点为轮廓的最后一个点；
每条轮廓都是一个边缘点的集合，整合获取的全部轮廓，形成轮廓集合，如下，
Figure PCTCN2020094892-appb-000021
其中，Γ j：集合S中的第j条轮廓曲线段，n：轮廓Γ j包含的像素点个数，N s：集合S中的轮廓曲线段总数目。
S2：基于全局和局部曲率的特征点检测策略检测轮廓曲线段中的特征点，分别得到特征点指向轮廓起点和终点方向的最近局部曲率极小值点。本步骤需要说明的是，检测特征点包括：
利用Canny边缘检测算子检测图像的边缘二值图像，提取边缘图像中的轮廓曲线段；
计算轮廓曲线段中每个点的曲率，将轮廓上的曲率极大值点作为候选特征点并存储检测每个极大值点两端的局部极小值点；
计算每个候选特征点在轮廓曲线段邻域的平均曲率值Kmean；
设定曲率倍数阈值C，当候选特征点的曲率小于平均曲率值Kmean与阈值C的乘积时，剔除候选特征点；
当候选特征点的曲率大于平均曲率值Kmean与阈值C的乘积时，计算候选特征点是否处于圆形轮廓上，若是，则剔除，若否，则保留；
对保留的候选特征点进行位置关系计算，若同一轮廓上某个特征点与其相邻的两个特征点之间的夹角小于阈值θ，则定义候选特征点为特征点，反之，则剔除候选特征点；
将每条非闭合轮廓的第一个点和最后一个点作为特征点，特征点提取结束。
进一步的是：
分别检测出轮廓曲线段内离特征点
Figure PCTCN2020094892-appb-000022
最近的指向轮廓起点和终点的局部曲率极小值
Figure PCTCN2020094892-appb-000023
和
Figure PCTCN2020094892-appb-000024
从每条轮廓的第一个点遍历到最后一个点，若某个点的曲率同时小于左右两个点的曲率，则定义为局部曲率极小值点。
S3：根据局部曲率极小值计算邻域采样点数和特征点两侧邻域的邻域辅助特征点。其中还需要说明的是，计算邻域采样点数包括：
λ L＝f-kL
λ R＝kR-f
其中，f∈[2，n]，n：每条轮廓包括的特征点数目，λ L和λ R分别是特征点的邻域采样点数。
具体的，邻域辅助特征点包括：
对第j条轮廓中第kL个点到第f个点之间的所有点的横坐标和纵坐标分别进行高斯加权，如下，
Figure PCTCN2020094892-appb-000025
Figure PCTCN2020094892-appb-000026
Figure PCTCN2020094892-appb-000027
其中，G σ：一维高斯内核，
Figure PCTCN2020094892-appb-000028
和
Figure PCTCN2020094892-appb-000029
邻域辅助特征点，当f＝1或n时，计算如下，
Figure PCTCN2020094892-appb-000030
Figure PCTCN2020094892-appb-000031
S4：利用邻域辅助特征点与特征点构成特征三角形并计算特征三角形中特征点对应的角平分线向量和主方向。本步骤还需要说明的是，参照图3，计算角平分线向量包括：
Figure PCTCN2020094892-appb-000032
Figure PCTCN2020094892-appb-000033
Figure PCTCN2020094892-appb-000034
其中，
Figure PCTCN2020094892-appb-000035
分别为特征点
Figure PCTCN2020094892-appb-000036
的坐标，
Figure PCTCN2020094892-appb-000037
角平分线向量。
再进一步的，计算主方向包括：
Figure PCTCN2020094892-appb-000038
其中，
Figure PCTCN2020094892-appb-000039
特征点
Figure PCTCN2020094892-appb-000040
的轮廓角方向，在分配中该轮廓角方向即为特征点的主方向。
较佳的，还需要说明的是：
每个特征点与其邻域辅助特征点构成了一个特征三角形；
红外和可见光图像中的同一个特征点对应的特征三角形是相似三角形，此时选取角平分线向量对应的角度作为主方向以保证不同图像中的同一个特征点的主方向是相同的；
当主方向相同时，对应的图像配准策略才能成功配准具有视角差异的图像。
S5：主方向指向特征点轮廓的凹侧，完成主方向分配。
还需要说明的是，本实施例以现有的三种配准方法进行举例说明，以更好的理解本发明方法，一、基于红外和可见光相机标定参数的方法，其需要已知相机的先验参数，且只能对齐相机同一时刻拍摄的一组图像，无法配准不同时刻、不同位置拍摄的图像，若一经标定，则不同距离的配准误差是恒定的，因此该方法只适应场景有限的情况，无法做到不同时刻不同位置的图像配准；二、基于区域的红外和可见光图像配准方法，主要是利用一些相似度度量算法对红外和可见光图像中的某些区域的相似度进行判定，以寻找到全局最优的变换参数，该方法只适合红外和可见光图像之间没有明显视角差异并且图像之间的重叠区域比较大的情况，其应用场景仍然有限；三、基于特征的配准方法，主要对图像中的一些点、线和面特征进行处理以配准图像，但是，红外和可见光图像的分辨率、光谱和视角差异比较大，现有的基于点特征的方法无法实现对这三种场景共存时的配准，对特征点进行描述时不够精确。
通俗的说，特征点的主方向分配是对特征点描述时最为重要的步骤之一，其目的在于为每个特征点赋予独特的方向参数，突出该点的图像特征，以便提取到的特征描述符具有显著性，现有的主方向分配SIFT、PIIFD、SURF算法都基于图像的梯度特征进行主方向分配，而红外和可见光图像光谱差异大，提取到的梯度特征不如同源图像之间的关联性高；而本发明方法主要依赖图像之间的轮廓特征进行主方向计算，由于电力设备的红外和可见光图像都具有极其显著的轮廓特征，因此本发明方法对电力设备的图像配准场景的适应性较高。
优选的，参照图4和图5，为了对本发明方法中采用的技术效果加以验证说明，本实施例选择以传统的SIFT方法、PIIFD方法、SURF方法与本发明方法进行对比测试，以科学论证的手段对比试验结果，以验证本发明方法所具有的真实效果；传统的SIFT方法、PIIFD方法及SURF方法对于主方向分配的准确度不高，适应性不强，为验证本发明方法相对传统方法较高的准确度，本实施例中将采用传统方法与本发明方法分别对同一电力设备的一张红外图像和六张可见光图像进行分配对比测试；
测试条件：(1)采用同一个电力设备的一张红外图像(分辨率120×280)和六张可见光图像作为对比图像；
(2)六张可见光图像相对红外图像的视角的旋转角度分别为0°(分辨率120×280)、60°(分辨率252×203)、120°(分辨率252×203)、180°(分辨率120×280)、240°(分辨率252×203)、300°(分辨率252×203)；
(3)开启自动化测试设备并运用MATLB仿真运行输出结果曲线。
参照图6，为传统方法与本发明方法(即CAO)的主方向分配测试结果对比曲线图，在0°至300°的旋转角度中，本发明方法输出的曲线远高于三种传统方法分配的特征点数目；参照图7，为旋转角度为0°时的图像对比结果示意图，能够直观的看到本发明方法在不同的旋转角度下都表现出最多的正确点数，并且本发明方法得到的正确点数是传统方法的三倍，根据图像结果，本发明方法的特征点主方向都指向轮廓的凹侧，验证了本发明方法具有较高的准确度和适应性。
实施例2
参照图8，为本发明的第二个实施例，该实施例不同于第一个实施例的是，提供了一种异源图像配准系统，包括：
信息采集模块100，用于采集图像数据信息、获取特征信息；数据采集是计算机与外部物理世界连接的桥梁，其由传感器、控制器组成。
数据处理中心模块200，用于接收、计算、存储、输出待处理的数据信息，其包括运算单元201、数据库202和输入输出管理单元203，运算单元201与采集模块100相连接，用于接收信息采集模块100获取的数据信息以进行运算处理，计算局部曲率极小值、邻域采样点数、邻域辅助特征点及角平分线向量和主方向，数据库202连接于各个模块，用于存储接收的所有数据信息，为数据处理中心模块200提供调配供应服务，输入输出管理单元203用于接收各个模块的信息并输出运算单元201的运算结果。
分配模块300连接于数据处理中心模块200，其用于读取运算单元201的运算结果，控制主方向指向特征点轮廓的凹侧，完成主方向分配，达到精准匹配。
较佳的是，信息采集模块100是基于远程数据采集平台的通信模块，其将通信芯片、存储芯片集成在一块电路板上，使其具有发送通过远程数据采集平台收发信息、通信、数据传输的功能，且电脑、单片机、ARM通过RS232串口与远程数据采集平台相连接，通过AI指令控制信息采集模块100实现数据 通信功能。
优选的，还需要说明的是，数据处理中心模块200主要分为三个层次，包括控制层、运算层及存储层，控制层是数据处理中心模块200的指挥控制中心，由指令寄存器IR、指令译码器ID和操作控制器OC组成，控制层能够根据用户预先编好的程序，依次从存储器中取出各条指令，放在指令寄存器IR中，通过指令译码器分析确定，通知操作控制器OC进行操作，按照确定的时序向相应的部件发出微操作控制信号；运算层是数据处理中心模块200的核心，能够执行算术运算(如加减乘除及其附加运算)和逻辑运算(如移位、逻辑测试或两个值比较)，其连接于控制层，通过接受控制层的控制信号进行运算操作；存储层是数据处理中心模块200的数据库，能够存放数据(待处理及已经处理过的数据)。
应当认识到，本发明的实施例可以由计算机硬件、硬件和软件的组合、或者通过存储在非暂时性计算机可读存储器中的计算机指令来实现或实施。所述方法可以使用标准编程技术-包括配置有计算机程序的非暂时性计算机可读存储介质在计算机程序中实现，其中如此配置的存储介质使得计算机以特定和预定义的方式操作——根据在具体实施例中描述的方法和附图。每个程序可以以高级过程或面向对象的编程语言来实现以与计算机系统通信。然而，若需要，该程序可以以汇编或机器语言实现。在任何情况下，该语言可以是编译或解释的语言。此外，为此目的该程序能够在编程的专用集成电路上运行。
此外，可按任何合适的顺序来执行本文描述的过程的操作，除非本文另外指示或以其他方式明显地与上下文矛盾。本文描述的过程(或变型和/或其组合)可在配置有可执行指令的一个或多个计算机系统的控制下执行，并且可作为共同地在一个或多个处理器上执行的代码(例如，可执行指令、一个或多个计算机程序或一个或多个应用)、由硬件或其组合来实现。所述计算机程序包括可由一个或多个处理器执行的多个指令。
进一步，所述方法可以在可操作地连接至合适的任何类型的计算平台中实现，包括但不限于个人电脑、迷你计算机、主框架、工作站、网络或分布式计算环境、单独的或集成的计算机平台、或者与带电粒子工具或其它成像装置通信等等。本发明的各方面可以以存储在非暂时性存储介质或设备上的机器可读代码来实现，无论是可移动的还是集成至计算平台，如硬盘、光学读取和/或 写入存储介质、RAM、ROM等，使得其可由可编程计算机读取，当存储介质或设备由计算机读取时可用于配置和操作计算机以执行在此所描述的过程。此外，机器可读代码，或其部分可以通过有线或无线网络传输。当此类媒体包括结合微处理器或其他数据处理器实现上文所述步骤的指令或程序时，本文所述的发明包括这些和其他不同类型的非暂时性计算机可读存储介质。当根据本发明所述的方法和技术编程时，本发明还包括计算机本身。计算机程序能够应用于输入数据以执行本文所述的功能，从而转换输入数据以生成存储至非易失性存储器的输出数据。输出信息还可以应用于一个或多个输出设备如显示器。在本发明优选的实施例中，转换的数据表示物理和有形的对象，包括显示器上产生的物理和有形对象的特定视觉描绘。
如在本申请所使用的，术语“组件”、“模块”、“系统”等等旨在指代计算机相关实体，该计算机相关实体可以是硬件、固件、硬件和软件的结合、软件或者运行中的软件。例如，组件可以是，但不限于是：在处理器上运行的处理、处理器、对象、可执行文件、执行中的线程、程序和/或计算机。作为示例，在计算设备上运行的应用和该计算设备都可以是组件。一个或多个组件可以存在于执行中的过程和/或线程中，并且组件可以位于一个计算机中以及/或者分布在两个或更多个计算机之间。此外，这些组件能够从在其上具有各种数据结构的各种计算机可读介质中执行。这些组件可以通过诸如根据具有一个或多个数据分组(例如，来自一个组件的数据，该组件与本地系统、分布式系统中的另一个组件进行交互和/或以信号的方式通过诸如互联网之类的网络与其它系统进行交互)的信号，以本地和/或远程过程的方式进行通信。
应说明的是，以上实施例仅用以说明本发明的技术方案而非限制，尽管参照较佳实施例对本发明进行了详细说明，本领域的普通技术人员应当理解，可以对本发明的技术方案进行修改或者等同替换，而不脱离本发明技术方案的精神和范围，其均应涵盖在本发明的权利要求范围当中。

Claims (10)

  1. 一种异源图像配准方法，其特征在于：包括，
    利用Canny边缘检测算子对采集的图像进行边缘检测，结合曲率尺度空间策略提取边缘图像中的轮廓曲线段；
    基于全局和局部曲率的特征点检测策略检测所述轮廓曲线段中的特征点，分别得到所述特征点指向轮廓起点和终点方向的最近局部曲率极小值；
    根据所述局部曲率极小值计算邻域采样点数和所述特征点两侧邻域的邻域辅助特征点；
    利用所述邻域辅助特征点与所述特征点构成特征三角形并计算所述特征三角形中所述特征点对应的角平分线向量和主方向；
    所述主方向指向所述特征点轮廓的凹侧，完成主方向分配。
  2. 根据权利要求1所述的异源图像配准方法，其特征在于：检测所述特征点包括，
    利用所述Canny边缘检测算子检测所述图像的边缘二值图像，提取所述边缘图像中的所述轮廓曲线段；
    计算所述轮廓曲线段中每个点的曲率，将轮廓上的曲率极大值点作为候选特征点并存储检测每个极大值点两端的最近局部极小值点；
    计算每个所述候选特征点在所述轮廓曲线段邻域的平均曲率值；
    设定曲率倍数阈值，当所述候选特征点的曲率小于所述平均曲率值与所述阈值的乘积时，剔除所述候选特征点；
    当所述候选特征点的曲率大于所述平均曲率值与所述阈值的乘积时，计算所述候选特征点是否处于圆形轮廓上，若是，则剔除，若否，则保留；
    对保留的所述候选特征点进行位置关系计算，若同一轮廓上某个特征点与其相邻的两个特征点之间的夹角小于所述阈值，则定义所述候选特征点为所述特征点，反之，则剔除所述候选特征点；
    将每条非闭合轮廓的第一个点和最后一个点作为所述特征点，所述特征点提取结束。
  3. 根据权利要求1或2所述的异源图像配准方法，其特征在于：分别检测出所述轮廓曲线段内离所述特征点最近的指向轮廓起点和终点的所述局部曲率极小值点；
    从每条轮廓的所述第一个点遍历到所述最后一个点，若某个点的所述曲率 同时小于左右两个点的所述曲率，则定义为所述局部曲率极小值点；
    所述特征点及其对应的指向轮廓起点和终点的所述局部曲率极小值点分别记为
    Figure PCTCN2020094892-appb-100001
    和
    Figure PCTCN2020094892-appb-100002
  4. 根据权利要求3所述的异源图像配准方法，其特征在于：计算所述邻域采样点数包括，
    λ L＝f-kL
    λ R＝kR-f
    其中，f∈[2，n]，n：每条轮廓包括的特征点数目，λ L和λ R分别是所述特征点的邻域采样点数。
  5. 根据权利要求4所述的异源图像配准方法，其特征在于：所述邻域辅助特征点包括，
    对第j条轮廓中第kL个点到第f个点之间的所有点的横坐标和纵坐标分别进行高斯加权，如下，
    Figure PCTCN2020094892-appb-100003
    Figure PCTCN2020094892-appb-100004
    Figure PCTCN2020094892-appb-100005
    其中，G σ：一维高斯内核，
    Figure PCTCN2020094892-appb-100006
    和
    Figure PCTCN2020094892-appb-100007
    所述邻域辅助特征点，当f＝1或n时，计算如下，
    Figure PCTCN2020094892-appb-100008
  6. 根据权利要求5所述的异源图像配准方法，其特征在于：计算所述角平分线向量包括，
    Figure PCTCN2020094892-appb-100009
    Figure PCTCN2020094892-appb-100010
    Figure PCTCN2020094892-appb-100011
    其中，
    Figure PCTCN2020094892-appb-100012
    分别为所述特征点
    Figure PCTCN2020094892-appb-100013
    的坐标，
    Figure PCTCN2020094892-appb-100014
    所述角平分线向量。
  7. 根据权利要求6所述的异源图像配准方法，其特征在于：计算所述主方向包括，
    Figure PCTCN2020094892-appb-100015
    其中，
    Figure PCTCN2020094892-appb-100016
    特征点
    Figure PCTCN2020094892-appb-100017
    的轮廓角方向，在分配中该轮廓角方向即为特征点的主方向。
  8. 根据权利要求1或7所述的异源图像配准方法，其特征在于：对所述图像进行边缘检测包括，
    对灰度图像进行高斯滤波处理；
    计算滤波后的所述图像中的梯度矩阵和方向矩阵；
    对所述梯度矩阵进行非极大值抑制，得到非极大值抑制图像；
    利用双阈值策略检测、连接图像边缘轮廓。
  9. 根据权利要求8所述的异源图像配准方法，其特征在于：提取所述轮廓曲线段包括，
    从所述边缘图像的第一行第一列开始以行为单位循环遍历查找；
    若查找到某一个点为边缘点，则标记为第一个轮廓点，且设为0；
    查找所述轮廓点的边长为1的邻域内是否有所述边缘点，若有，则加入所述轮廓邻域内作为第二个所述轮廓点，设为0；
    重复进行查找，直至某个点邻域内没有所述边缘点为止，定义停止的点为轮廓的所述最后一个点；
    每条轮廓都是一个所述边缘点的集合，整合获取的全部轮廓，形成轮廓集合，如下，
    Figure PCTCN2020094892-appb-100018
    其中，Γ j：集合S中的第j条轮廓曲线段，n：轮廓Γ j包含的像素点个数，N s：集合S中的轮廓曲线段总数目。
  10. 一种异源图像配准系统，其特征在于：包括，
    信息采集模块(100)，用于采集所述图像数据信息、获取特征信息；
    数据处理中心模块(200)，用于接收、计算、存储、输出待处理的数据信息，其包括运算单元(201)、数据库(202)和输入输出管理单元(203)，所述运算单元(201)与所述采集模块(100)相连接，用于接收所述信息采集模块(100)获取的数据信息以进行运算处理，计算所述局部曲率极小值、所述邻域采样点数、所述邻域辅助特征点及所述角平分线向量和主方向，所述数据库(202)连接于各个模块，用于存储接收的所有数据信息，为所述数据处理中心模块(200)提供调配供应服务，所述输入输出管理单元(203)用于接收各个模块的信息并输出所述运算单元(201)的运算结果；
    分配模块(300)连接于所述数据处理中心模块(200)，其用于读取所述运算单元(201)的运算结果，控制所述主方向指向所述特征点轮廓的凹侧，完成主方向分配，达到精准匹配。
PCT/CN2020/094892 2020-06-08 2020-06-08 一种异源图像配准方法及系统 WO2021248270A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202080097152.1A CN115176274A (zh) 2020-06-08 2020-06-08 一种异源图像配准方法及系统
US17/598,310 US20220319011A1 (en) 2020-06-08 2020-06-08 Heterogeneous Image Registration Method and System
PCT/CN2020/094892 WO2021248270A1 (zh) 2020-06-08 2020-06-08 一种异源图像配准方法及系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/094892 WO2021248270A1 (zh) 2020-06-08 2020-06-08 一种异源图像配准方法及系统

Publications (1)

Publication Number Publication Date
WO2021248270A1 true WO2021248270A1 (zh) 2021-12-16

Family

ID=78846967

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/094892 WO2021248270A1 (zh) 2020-06-08 2020-06-08 一种异源图像配准方法及系统

Country Status (3)

Country Link
US (1) US20220319011A1 (zh)
CN (1) CN115176274A (zh)
WO (1) WO2021248270A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114723770A (zh) * 2022-05-16 2022-07-08 中国人民解放军96901部队 一种基于特征空间关系的异源图像匹配方法
CN114998127A (zh) * 2022-05-25 2022-09-02 江南大学 一种用于无人机高动态遥感的测绘图像清晰度优化方法
CN116703903A (zh) * 2023-08-04 2023-09-05 山东济宁运河煤矿有限责任公司 一种基于机器视觉的锚杆修复质量检测方法
CN116862909A (zh) * 2023-09-01 2023-10-10 泰州市航宇电器有限公司 一种基于三维成像的金属封装外壳检测方法及系统

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114821114B (zh) * 2022-03-28 2024-04-30 南京业恒达智能系统有限公司 一种基于视觉系统的坡口切割机器人图像处理方法
CN115775269B (zh) * 2023-02-10 2023-05-02 西南交通大学 一种基于线特征的列车图像精确配准方法
CN116385449B (zh) * 2023-06-07 2023-08-01 江阴市华明电力发展集团有限公司 光伏基站全天候智能化视频监控系统
CN117372790B (zh) * 2023-12-08 2024-03-08 浙江托普云农科技股份有限公司 植物叶片形状分类的方法、系统及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100310181A1 (en) * 2006-12-22 2010-12-09 Art Advanced Research Technologies Inc. Registration of optical images of turbid media
CN102800097A (zh) * 2012-07-19 2012-11-28 中国科学院自动化研究所 多特征多级别的可见光与红外图像高精度配准方法
CN104318548A (zh) * 2014-10-10 2015-01-28 西安电子科技大学 一种基于空间稀疏度和sift特征提取的快速图像配准实现方法
CN104978709A (zh) * 2015-06-24 2015-10-14 北京邮电大学 描述符生成方法及装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105930858B (zh) * 2016-04-06 2021-03-26 吴晓军 一种带旋转、缩放的快速高精度几何模板匹配方法
CN106257535B (zh) * 2016-08-11 2019-05-21 河海大学常州校区 基于surf算子的电气设备红外与可见光图像配准方法
CN107977924A (zh) * 2016-10-21 2018-05-01 杭州海康威视数字技术股份有限公司 一种基于双传感器成像的图像处理方法、系统
CN107169947B (zh) * 2017-04-26 2021-02-12 国网上海市电力公司 一种基于特征点定位和边缘检测的图像融合实验方法
CN107507208B (zh) * 2017-07-12 2020-04-10 天津大学 一种基于轮廓上曲率估计的图像特征点提取方法
CN108254077B (zh) * 2018-01-02 2020-04-28 国网上海市电力公司 基于局部与全局特征信息融合的gis热故障诊断方法
CN109409292B (zh) * 2018-10-26 2021-09-03 西安电子科技大学 基于精细化特征优化提取的异源图像匹配方法
WO2020113408A1 (zh) * 2018-12-04 2020-06-11 深圳市大疆创新科技有限公司 一种图像处理方法、设备、无人机、系统及存储介质
CN111145228B (zh) * 2019-12-23 2023-05-26 西安电子科技大学 基于局部轮廓点与形状特征融合的异源图像配准方法
CN111223133B (zh) * 2020-01-07 2022-10-11 上海交通大学 一种异源图像的配准方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100310181A1 (en) * 2006-12-22 2010-12-09 Art Advanced Research Technologies Inc. Registration of optical images of turbid media
CN102800097A (zh) * 2012-07-19 2012-11-28 中国科学院自动化研究所 多特征多级别的可见光与红外图像高精度配准方法
CN104318548A (zh) * 2014-10-10 2015-01-28 西安电子科技大学 一种基于空间稀疏度和sift特征提取的快速图像配准实现方法
CN104978709A (zh) * 2015-06-24 2015-10-14 北京邮电大学 描述符生成方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DU QINGLEI; FAN AOXIANG; MA YONG; FAN FAN; HUANG JUN; MEI XIAOGUANG: "Infrared and Visible Image Registration Based on Scale-Invariant PIIFD Feature and Locality Preserving Matching", IEEE ACCESS, IEEE, USA, vol. 6, 1 January 1900 (1900-01-01), USA , pages 64107 - 64121, XP011697976, DOI: 10.1109/ACCESS.2018.2877642 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114723770A (zh) * 2022-05-16 2022-07-08 中国人民解放军96901部队 一种基于特征空间关系的异源图像匹配方法
CN114723770B (zh) * 2022-05-16 2022-08-09 中国人民解放军96901部队 一种基于特征空间关系的异源图像匹配方法
CN114998127A (zh) * 2022-05-25 2022-09-02 江南大学 一种用于无人机高动态遥感的测绘图像清晰度优化方法
CN114998127B (zh) * 2022-05-25 2024-03-29 江南大学 一种用于无人机高动态遥感的测绘图像清晰度优化方法
CN116703903A (zh) * 2023-08-04 2023-09-05 山东济宁运河煤矿有限责任公司 一种基于机器视觉的锚杆修复质量检测方法
CN116703903B (zh) * 2023-08-04 2023-10-20 山东济宁运河煤矿有限责任公司 一种基于机器视觉的锚杆修复质量检测方法
CN116862909A (zh) * 2023-09-01 2023-10-10 泰州市航宇电器有限公司 一种基于三维成像的金属封装外壳检测方法及系统
CN116862909B (zh) * 2023-09-01 2023-11-24 泰州市航宇电器有限公司 一种基于三维成像的金属封装外壳检测方法及系统

Also Published As

Publication number Publication date
US20220319011A1 (en) 2022-10-06
CN115176274A (zh) 2022-10-11

Similar Documents

Publication Publication Date Title
WO2021248270A1 (zh) 一种异源图像配准方法及系统
Shan et al. A stereovision-based crack width detection approach for concrete surface assessment
CN104200461B (zh) 基于互信息图像选块和sift特征的遥感图像配准方法
Han et al. Visible and infrared image registration in man-made environments employing hybrid visual features
WO2019105044A1 (zh) 一种镜头畸变矫正和特征提取的方法及系统
WO2016062159A1 (zh) 图像匹配方法及手机应用测试平台
CN108272154B (zh) 一种服装尺寸测量方法和装置
CN109961399B (zh) 基于图像距离变换的最佳缝合线搜寻方法
JP6648925B2 (ja) 画像処理方法、画像処理装置、画像処理システム、生産装置、プログラム及び記録媒体
JP5538868B2 (ja) 画像処理装置、その画像処理方法及びプログラム
CN104134208B (zh) 利用几何结构特征从粗到精的红外与可见光图像配准方法
US8666170B2 (en) Computer system and method of matching for images and graphs
JP6899189B2 (ja) ビジョンシステムで画像内のプローブを効率的に採点するためのシステム及び方法
CN105701827A (zh) 可见光相机与红外相机的参数联合标定方法及装置
KR102073468B1 (ko) 비전 시스템에서 컬러 이미지에 대해 컬러 후보 포즈들의 점수화를 위한 시스템 및 방법
CN110400315A (zh) 一种缺陷检测方法、装置及系统
CN105335973A (zh) 运用于带钢加工生产线的视觉处理方法
CN108362205B (zh) 基于条纹投影的空间测距方法
CN106897995A (zh) 一种面向机械装配过程的零部件自动识别方法
CN106023187A (zh) 一种基于sift特征和角度相对距离的图像配准方法
CN108986152A (zh) 一种基于差分图像的异物检测方法及装置
CN107133967B (zh) 提取目标图像的方法及装置
CN110954555A (zh) Wdt 3d视觉检测系统
CN115049821A (zh) 一种基于多传感器融合的三维环境目标检测方法
AU2022373777A1 (en) Three-dimensional building model generation based on classification of image elements

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20939628

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20939628

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20939628

Country of ref document: EP

Kind code of ref document: A1