WO2024067435A1 - 一种基于视频的多目标位移跟踪监测方法及装置 - Google Patents

一种基于视频的多目标位移跟踪监测方法及装置 Download PDF

Info

Publication number
WO2024067435A1
WO2024067435A1 PCT/CN2023/120925 CN2023120925W WO2024067435A1 WO 2024067435 A1 WO2024067435 A1 WO 2024067435A1 CN 2023120925 W CN2023120925 W CN 2023120925W WO 2024067435 A1 WO2024067435 A1 WO 2024067435A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
displacement
monitoring
video
tracking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/120925
Other languages
English (en)
French (fr)
Inventor
舒江鹏
张从广
周姝康
马亥波
段元锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University
Original Assignee
Zhejiang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University filed Critical Zhejiang University
Publication of WO2024067435A1 publication Critical patent/WO2024067435A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/269—Analysis of motion using gradient-based methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention relates to the field of structural health monitoring, and in particular to a video-based multi-target displacement tracking monitoring method and device.
  • the displacement and vibration monitoring of existing bridge structures and super-high-rise structures is an important part of structural damage identification and health monitoring, which can provide basic information for structural safety assessment and maintenance.
  • the traditional method of monitoring by arranging sensors has a cumbersome deployment process, difficult maintenance, low efficiency and high cost, and is not convenient for long-term multi-point monitoring of structures.
  • the non-contact visual measurement method developed in recent years can make full use of common cameras such as surveillance cameras and mobile phone cameras to track and monitor key nodes of the structure, allowing maintenance personnel to monitor locations that are difficult to reach or dangerous with traditional methods.
  • most current methods still require manual calibration using calibration plates in advance, and rely on artificial markers during the monitoring process, making on-site monitoring very difficult.
  • Patent document CN112686879A discloses a structural vibration analysis and device based on video images, including: acquiring video images of the object to be tested vibrating along the vibration direction frame by frame, and converting each frame of the video image into a grayscale image, intercepting a specified area of each frame of the grayscale image, and calculating the maximum pixel value in the specified area of each frame of the grayscale image, and analyzing the vibration frequency and/or vibration amplitude of the object to be tested according to the maximum pixel value in the specified area of each frame of the grayscale image, wherein the specified area of each frame of the grayscale image has the same position on each frame of the grayscale image and contains a boundary in the vibration direction of the object to be tested and a column of pixel points in the neighborhood of the boundary.
  • This method requires the arrangement of the camera calibration plate in advance, which is difficult for some super-high buildings.
  • Patent document CN114964384A discloses a health monitoring system for large-scale earthquake simulation research facilities, including sensors, data acquisition modules, cloud servers, data processing modules, remote monitoring terminals and early warning modules;
  • the sensors include dynamic strain gauges, displacement sensors, acceleration/velocity sensors, which are used to obtain strain, displacement, acceleration and velocity information of large-scale earthquake simulation research facilities;
  • the data acquisition module converts the signals collected by the sensors into real-time data and uploads them to the cloud server;
  • the data processing module processes, analyzes and distinguishes the uploaded data, and when the collected signal exceeds the set alarm threshold, the early warning module alarms.
  • This method uses a variety of sensors to identify various types of data of the structure to be monitored, but this method is more dependent on the accuracy of the sensor, and is easily damaged during use, and is inconvenient for daily maintenance.
  • the present invention provides a multi-target tracking and monitoring method with convenient deployment of monitoring equipment and no need for advance manual calibration.
  • the method can improve the accuracy and reliability of physical displacement acquisition.
  • a multi-target displacement tracking and monitoring method based on video comprising:
  • Step 1 Obtain the structural parameters of the target building and establish the corresponding structural three-dimensional model, and estimate the corresponding camera parameters based on the pinhole camera model;
  • Step 2 according to the structural three-dimensional model and camera parameters of step 1, set the monitoring target to be monitored in the target building, and output the region of interest containing at least one target feature point based on the time axis sequence of the video;
  • Step 3 Use the pyramid streamer method to track the displacement of all target feature points in the region of interest, remove the target feature points with abnormal displacement data in the tracking results, and calculate the displacement average of all target feature points that are not removed as the pixel displacement of the monitoring target in the region of interest;
  • Step 4 According to the camera parameters, the pixel displacement of the monitoring target in all the regions of interest is reversely mapped to the three-dimensional model to obtain the physical displacement of the monitoring target in the real environment.
  • the physical displacement is used to calculate the vibration of the target building and is an important indicator in structural health monitoring.
  • the present invention uses the surface texture features of the structure to complete the tracking measurement of the monitoring target in the form of target feature point tracking. At the same time, it adopts a forward-reverse error mechanism and a method of eliminating noise points based on the statistical characteristics of the displacement of the target feature points, further ensuring the reliability and accuracy of the calculation results.
  • the structural three-dimensional model is established based on the actual size and spatial position relationship of the target building.
  • the monitoring targets include: node plates of truss beam bridges, corner points of roofs and floor windows of super high-rise buildings, intersections of bridge railings and bridges, anchorage points of cable-stayed bridges and suspension bridge cables, and intersections of truss arch bridge members.
  • the camera parameters include distortion parameters and camera intrinsic and extrinsic parameters, which are calculated using a direct linear transformation algorithm and a Levenberg-Marquardt algorithm.
  • the specific process of obtaining the camera parameters is as follows: first, the camera distortion is ignored and the initial values of the camera intrinsic and extrinsic parameters are obtained based on direct linear transformation, and then the distortion parameters and the optimized camera intrinsic and extrinsic parameters are obtained by minimizing the reprojection error of the target feature points through the Levenberg-Marquardt algorithm.
  • the region of interest is obtained by projecting the monitoring target in the three-dimensional model onto the image plane and capturing it with a window of size w ⁇ h, centered on the projection of the monitoring target in the image plane, where w is the width of the window and h is the height of the window.
  • the projection formula is as follows:
  • [u, v, 1] T represents the projection point on the image plane.
  • the target feature points are obtained by identifying the region of interest in the first frame image after the video image sequence is corrected using an image gradient automated detection method.
  • the target feature points include Harris corner points, SURF corner points and Shi-Tomasi corner points.
  • step 3 the forward-backward error mechanism and the statistical characteristics of feature point displacement are used to eliminate target feature points with abnormal displacement data.
  • the forward-backward error mechanism is as follows:
  • the method of using the characteristic point displacement statistical characteristics is as follows:
  • the physical displacement is obtained by the intersection of the computer camera ray and the monitoring target motion plane.
  • xt is the position of the monitoring point in It
  • C is the camera center
  • K and R are the camera intrinsic parameter matrix and extrinsic parameter matrix respectively
  • ⁇ and e r are the depth value and direction vector of the ray
  • the present invention also provides a multi-target position tracking and monitoring device, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, the device is used to implement the claim.
  • a multi-target displacement tracking and monitoring method as described in any one of 1-9, wherein the specific steps are as follows: obtaining the structural parameters and structural three-dimensional model of the target building, identifying and analyzing the designated monitoring structure through the multi-target displacement tracking and monitoring method, and outputting the actual physical displacement of the target building.
  • the present invention has the following beneficial effects:
  • the camera parameters are estimated based on the geometric information of the target building without the need to arrange the calibration plate in advance.
  • the forward-reverse error mechanism and the method of eliminating noise points based on the statistical characteristics of the characteristic point displacement further ensure the reliability and accuracy of the calculation results.
  • FIG1 is a schematic diagram of a truss beam model, a camera model and monitoring points provided in this embodiment
  • FIG2 is a technical flow chart of a video-based multi-target displacement tracking and monitoring method provided in this embodiment
  • FIG3 is a schematic diagram of partial ROI of a truss beam model and target feature point extraction results
  • FIG. 4 is a schematic diagram of the displacement monitoring results of No. 1, No. 7 and No. 8 in the embodiment.
  • a truss beam bridge is taken as a sample, and the vibrations of monitoring targets 1-16 of the upper chord, lower chord and web members of the truss beam bridge are monitored.
  • a video-based multi-target tracking and monitoring method includes:
  • Step 1 Set up the camera so that the targets to be monitored are within the camera's field of view. Then, build a three-dimensional model based on the structure's geometric information to meet the requirements of camera parameter estimation and target monitoring point picking. By picking nodes 1-24 on the three-dimensional model, their three-dimensional coordinates can be obtained.
  • the feature points used for camera calibration are features such as corner points or center points of a structure
  • algorithms such as Harris corner point recognition, circle detection and fitting can be used to automatically complete the image coordinate determination problem.
  • the initial values of the camera's intrinsic and extrinsic parameters R and K are calculated based on the direct linear transformation algorithm, and then the LM algorithm is used to optimize the feature point reprojection error to obtain the final camera parameters R, K and distortion parameters.
  • the method for calculating the initial values of the camera's internal and external parameters based on direct linear transformation is to solve the following homogeneous linear equations by singular value decomposition:
  • p mi [u i , vi ,1] T is the feature point in the image, that is, the pixel coordinates of nodes 1-24 in the image;
  • R and K are the camera extrinsic matrix and intrinsic matrix
  • C is the camera center coordinate, and is the right singular vector corresponding to the minimum singular value of the projection matrix P.
  • M is the first 3 rows and 3 columns of P.
  • the objective function of optimizing the feature point reprojection error using the LM algorithm is:
  • f(p wi , ⁇ ) is the projection point of the feature point in the three-dimensional space on the image plane based on the camera parameter ⁇
  • p mi is the corresponding feature point on the image
  • ⁇ includes the camera's intrinsic parameter matrix K, extrinsic parameter matrix R, and radial distortion coefficients ⁇ 1 and ⁇ 2 .
  • Step 2 region of interest (ROI) extraction: In the three-dimensional model established in step (1), pick up the monitoring target to be tracked and measured and input the window size. According to the camera parameters of step (1), automatically determine the ROI of the monitoring target in the image;
  • Feature point recognition according to the camera parameters of step (1), the video image sequence is rectified, and in the ROI determined by step (2) on the first frame image, feature points are automatically detected and extracted based on the image gradient;
  • the video image needs to be dedistorted.
  • feature points are automatically extracted based on the image gradient.
  • a suitable feature detector can be selected according to the texture features of the tracked object, such as Harris corner points, SURF corner points, and Shi-Tomasi corner points.
  • the Shi-Tomasi corner point detector is used to extract feature points in each ROI, which can obtain a more suitable quality and quantity of feature points.
  • Step 3 Use the pyramid LK optical flow method to As the reference image, track the feature points in all ROIs in the image sequence
  • the average displacement of the reliable feature points in the ROI can be As The image displacement of the target monitoring node;
  • the forward-backward error mechanism is as follows: let I t be the image in the video sequence at time t, x t be the position of the feature point in I t , first use the pyramid optical flow method to calculate the position of the feature point in I t+1 from I t to I t+1 Then based on Calculate the estimated position of the feature point in I t in reverse from I t+1 to I t Finally, based on the forward-backward error The size of the evaluation tracking results;
  • the specific method of filtering noise points based on the statistical characteristics of feature point displacement is as follows: is the displacement of the jth feature point in the i-th ROI calculated by the pyramid optical flow method at time t, is the mean and standard deviation of displacement in the i-th ROI at time t, if If the tracking result is reliable, then it is unreliable and discarded.
  • Step 4 According to the camera parameters, the pixel displacement of the monitoring target in all the regions of interest is reversely mapped to the 3D model to obtain the physical displacement of the monitoring target in the real environment:
  • Displacement backprojection Based on the position parameters of the camera in step (3), the tracking results of the displacement in the image are backprojected into the three-dimensional space to obtain the physical displacement in the real world.
  • detection nodes 1-16 mainly vibrate in the vertical direction, that is, they move on the target plane in Figure 1.
  • xt be the position of the monitoring point in It
  • C be the camera center
  • K and R be the camera intrinsic matrix and extrinsic matrix respectively
  • ⁇ and e r are the depth value and direction vector of the ray.
  • c p is a point on the plane
  • e r is the normal vector of the plane
  • the image needs to be shifted
  • the real physical displacement of the monitoring point can be obtained as
  • This embodiment also provides a multi-target position tracking and monitoring device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above
  • the multi-target displacement tracking monitoring method described above has the following specific steps: obtaining the structural parameters and structural three-dimensional model of the target building, identifying and analyzing the designated monitoring structure through the multi-target displacement tracking monitoring method, and outputting the actual physical displacement of the target building.
  • FIG4 there are schematic diagrams of displacement monitoring results No. 1, No. 7 and No. 8 in the output results.
  • vibration monitoring of truss nodes is used for illustration in the examples, but the present invention is not limited to this, and can also be used for displacement monitoring of key nodes of structures such as super high-rise buildings, beam bridges, arch bridges, cable-stayed bridges and suspension bridges.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于视频的多目标位移跟踪监测方法，包括：步骤1、获取目标建筑物的结构参数并建立对应的结构三维模型，获得对应的相机参数；步骤2、设定目标建筑物中需要监测的监测目标，输出包含至少一个目标特征点的感兴趣区域，步骤3、采用金字塔流光法对感兴趣区域内的所有目标特征点进行位移跟踪，将跟踪结果中位移数据异常的目标特征点进行剔除，并计算所有未剔除目标特征点的位移平均值作为感兴趣区域中监测目标的像素位移；步骤4、根据相机参数，将所有监测目标的像素位移反向映射至三维模型中，获得监测目标真实环境中的物理位移。本发明还提供了一种多目标位移跟踪监测装置。本发明的方法可以提高物理位移获取的准确性和可靠性。

Description

一种基于视频的多目标位移跟踪监测方法及装置 技术领域
本发明涉及结构健康监测领域，尤其涉及一种基于视频的多目标位移跟踪监测方法及装置。
背景技术
现役桥梁结构、超高层结构的位移、振动监测是结构损伤识别与健康监测的重要内容，其能够为结构安全评估和维护提供基本的信息。传统通过布置传感器进行监测的方法，其部署过程繁琐、维护困难、效率低且成本高，不便于对结构进行长期多点监测。近年来发展的非接触式视觉测量方法，能够充分利用监控摄像头、手机摄像头等常见相机进行结构关键节点的跟踪监测，使得维护人员能够监测到传统方法难以达到或危险的位置。但是目前多数方法仍需要采用标定板提前进行人工标定，且在监测过程中依赖人工标记物，导致现场监测非常困难。
专利文献CN112686879A公开了一种基于视频图像的结构振动分析及装置，包括：逐帧获取待测物沿振动方向振动的视频图像，并将各帧视频图像转化为灰度图像，截取各帧灰度图像的指定区域，并计算所述各帧灰度图像的指定区域中的最大像素值，根据所述各帧灰度图像的指定区域中最大像素值分析待测物的振动频率和/或振动幅值，其中，所述各帧灰度图像的指定区域在各帧灰度图像上的位置相同且均包含待测物振动方向上边界及所述边界邻域的一列像素点，该方法需要提前做好相机的标定板布置，对于一些超高建筑就存在布置困难的问题。
专利文献CN114964384A公开了一种用于大型地震模拟研究设施的健康监测系统，包括传感器、数据采集模块、云服务器、数据处理模块、远程监控终端及预警模块；所述传感器包括动态应变仪、位移传感器、加速度/速度传感器，用于获取大型地震模拟研究设施的应变、位移、加速度及速度信息；所述数据采集模块将传感器采集的信号转换成实时数据，并上传到云服务器；所述数据处理模块对上传的数据进行处理、分析及判别，当采集信号超过设定的报警阈值时，预警模块进行报警。该方法通过多种传感器去识别待监测结构的各类数据，但该方法较依赖传感器准确度，且容易在使用过程损坏，日常维护不便。
发明内容
为了解决上述问题，本发明提供了一种监测设备部署方便、无需提前人工标定的多目标跟踪监测方法，该方法可以提高物理位移获取的准确性和可靠性。
一种基于视频的多目标位移跟踪监测方法，包括：
步骤1、获取目标建筑物的结构参数并建立对应的结构三维模型，基于针孔相机模型估计对应的相机参数；
步骤2、根据步骤1的结构三维模型与相机参数，设定目标建筑物中需要监测的监测目标，并基于视频的时间轴顺序，输出包含至少一个目标特征点的感兴趣区域；
步骤3、采用金字塔流光法对感兴趣区域内的所有目标特征点进行位移跟踪，将跟踪结果中位移数据异常的目标特征点进行剔除，并计算所有未剔除目标特征点的位移平均值作为所述感兴趣区域中监测目标的像素位移；
步骤4、根据相机参数，将所有感兴趣区域中监测目标的像素位移反向映射至三维模型中，获得真实环境中监测目标的物理位移。
所述物理位移用于计算目标建筑物的振动情况，是结构健康监测中的重要指标。
本发明借助于结构表面纹理特征，以目标特征点跟踪的方式完成监测目标的跟踪测量，同时采用前向-反向误差机制和基于目标特征点位移统计特性剔除噪点的方法，进一步确保了计算结果的可靠性和准确性。
优选的，所述结构三维模型基于目标建筑的实际尺寸和空间位置关系建立，在估计相机参数时无需提前设置标定物，仅需目标监测结构关键点，线和面的几何尺寸并结合空间位置对相机位置进行标定。
具体的，所述监测目标包括：桁架梁桥的节点板，超高层建筑的屋顶的角点、楼层窗户的角点，桥梁栏杆与桥梁的交点，斜拉桥、悬索桥索的锚固点，桁架拱桥杆件的交点。
优选的，在步骤1中，所述相机参数包括畸变参数和相机内外参数，采用直接线性变换算法与Levenberg-Marquardt算法计算得到。
具体的，所述相机参数的具体获取过程如下：首选忽略相机畸变并基于直接线性变换得到相机内外参数初始值，再通过Levenberg-Marquardt算法最小化目标特征点重投影误差得到畸变参数和优化后的相机内外参数。
优选的，在步骤1中，所述感兴趣区域通过将三维模型中的监测目标投影至图像平面，并以监测目标在图像平面中的投影为中心，利用尺寸为w×h的窗口截取获得，其中w为窗口的宽度，h为窗口的高度。
具体的，所述投影的公式如下：
其中，[xw,yw,zw,1]T,(i＝1,2,…,n)表示三维空间拾取的监测目标，[u,v,1]T表示图像平面的投影点。
优选的，步骤2中，所述目标特征点采用图像梯度自动化检测法对视频图像序列校正后第一帧图像中的感兴趣区域识别获得。
具体的，在步骤2中，所述目标特征点包括Harris角点、SURF角点以及Shi-Tomasi角点。
优选的，在步骤3中，采用前向-后向误差机制和特征点位移统计特性对位移数据异常的目标特征点进行剔除。
具体的，所述前向-后向误差机制具体如下：
设It为t时刻视频序列中的图像，xt为It中特征点的位置，首先利用金字塔光流法从It到It+1计算特征点在It+1中的位置然后在基于反向从It+1到It计算特征点在It中的位置估计值最后基于前向-反向误差的大小评估跟踪结果的好坏；
设置阈值eth，若efb<eth，则判定跟踪结果可靠，否则舍弃该特征点的监测结果。
具体的，所述特征点位移统计特性的使用方法如下：
设为第i个ROI中第j个特征点在t时刻由金字塔光流法计算所得的位移， 为t时刻第i个ROI中位移均值和标准差；
如果则该跟踪结果可靠，否则不可靠并舍弃。
优选的，在步骤4中，所述物理位移通过计算机相机射线与监测目标运动平面的交点获得。
具体的，所述物理位移的具体获取过程如下：
设xt为It中监测点的位置，C为相机中心，K，R分别相机内参矩阵和外参矩阵，则相机射线可以表达为：
r(λ)＝C+λR-1K-1xt＝C+λer
其中，λ和er为射线的深度值和方向向量；
若平面为其中cp为平面上一点，为该平面的法向量，则
由此，xt在平面中的反向投影点为进一步可得监测点的位移为
本发明还提供了一种多目标位跟踪监测装置，包括存储器和一个或多个处理器，所述存储器中存储有可执行代码，所述一个或多个处理器执行所述可执行代码时，用于实现权利要 求1-9任一所述的多目标位移跟踪监测方法，其具体步骤如下：获取目标建筑的结构参数与结构三维模型，通过多目标位移跟踪监测方法对指定监测结构进行识别分析，输出目标建筑的实际物理位移。
与现有技术相比，本发明的有益效果：
(1)基于目标建筑的几何信息完成相机参数的估计，无需提前布置标定板。
(2)在监测目标的位移跟踪时，采用的前向-反向误差机制和基于特征点位移统计特性剔除噪点的方法进一步确保了计算结果的可靠性和准确性。
附图说明
图1为本实施例提供的桁架梁模型、相机模型及监测点的示意图；
图2为本实施例提供的基于视频的多目标位移跟踪监测方法的技术流程图；
图3为桁架梁模型的部分ROI以及目标特征点提取结果示意图；
图4为实施例中1号，7号和8号位移监测结果示意图。
具体实施方式
如图1所示，以一座桁架梁桥为样本，对该桁架梁桥的上弦杆、下弦杆以及腹杆的1-16号监测目标的振动进行监测。
如图2所示，为实施例提供的一种基于视频的多目标位跟踪监测方法，包括：
步骤1、架设好相机使得待监测的目标均在相机视野内，之后应根据结构的几何信息建立其三维模型，以满足相机参数估计和目标监测点拾取的要求，通过在三维模型上拾取1-24节点，可以获取他们的三维坐标。
接着，在启动相机开始监测后可以获取视频图像序列Ii,(i＝0,1,…,t)，其中I0为视频的第一帧图像。由于上下弦杆和腹杆交叉处是较大的节点板，这里采用手动拾取的方式在I0中获取1-24号节点的图像坐标。
需要注意的是，在其他实际应用过程，若用于相机标定的特征点为结构的角点或者圆心等特征，可以采用Harris角点识别、圆检测与拟合等算法自动完成图像坐标确定问题。
获取用于相机标定的1-24节点的三维坐标和图像坐标后，基于直接线性变换算法计算相机内外参数R和K的初始值，再采用LM算法优化特征点重投影误差，得到最终的相机参数R、K以及畸变参数。
基于直接线性变换计算相机内外参数初始值的方法具体为通过奇异值分解求解如下齐次线性方程组：
其中，pwi＝[xwi,ywi,zwi]T,(i＝1,2,…,n)为三维空间的特征点，即三维模型中1-24节点坐标，pmi＝[ui,vi,1]T为图像中的特征点，即图像中的1-24节点像素坐标；
其中R，K为相机外参矩阵和内参矩阵，C为相机中心坐标，是投影矩阵P的最小奇异值对应的右奇异向量。
M为P的前3行前3列，通过对M进行QR分解便可以得到相机参数R和K。
采用LM算法优化特征点重投影误差的目标函数为：
其中f(pwi,Φ)为三维空间中的特征点基于相机参数Φ在图像平面上的投影点，pmi为与之对应的图像上的特征点，Φ包括相机的内参矩阵K，外参矩阵R以及径向畸变系数κ1和κ2。
步骤2、感兴趣区域(region of interest，ROI)提取：在步骤(1)所建立的三维模型中，拾取需要跟踪测量的监测目标并输入窗口大小，根据步骤(1)的相机参数，自动在图像中确定监测目标在图像中的ROI；
特征点识别：根据步骤(1)的相机参数，进行视频图像序列矫正，并在第一帧图像上由步骤(2)确定的ROI中，基于图像梯度，自动化检测提取特征点；
为保证监测结果的准确性，需要视频图像进行去畸变，同时为加快计算效率，这里首先根据步骤(1)的相机参数计算原始图像和矫正图像之间的映射关系，然后基于这个关系将视频图像序列Ii,(i＝1,2,…,t)转换为无畸变图像序列图像矫正后，在第一帧图像上由步骤(2)确定的ROI中，基于图像梯度自动化提取特征点。在实际应用中，可以根据跟踪对象的纹理特征，选择合适的特征检测器，如Harris角点、SURF角点以及Shi-Tomasi角点等。本实例中，采用Shi-Tomasi角点检测器在每一个ROI中提取特征点，能够得到较为合适的特征点质量和数量。
步骤3、采用金字塔LK光流法以为参考图像，跟踪所有ROI中特征点在图像序列中的位置xt，则特征点的图像位移为dt＝xt-xt-1,(t＝1,2,3,…)，经过前向-反向误差机制验证和基于特征点位移统计特性过滤噪点位移后，可以将ROI中可靠特征点位移的平均值作为 该目标监测节点的图像位移；
前向-反向误差机制具体为：设It为t时刻视频序列中的图像，xt为It中特征点的位置，首先利用金字塔光流法从It到It+1中计算特征点在It+1中的位置然后在基于反向从It+1到It计算特征点在It中的位置估计值最后基于前向-反向误差的大小评估跟踪结果的好坏；
设置阈值eth，若efb<eth，则判定跟踪结果可靠，否则舍弃该特征点的监测结果。
根据特征点位移统计特性过滤噪点的方法具体为：设为第i个ROI中第j个特征点在t时刻由金字塔光流法计算所得的位移，为t时刻第i个ROI中位移均值和标准差，如果则该跟踪结果可靠，否则不可靠并舍弃。
如图3所示，所有“×”特征点均在背景上，在视频图像序列中并没有移动，因此他们的位移始终接近为零，很容易被剔除。
步骤4、根据相机参数，将所有感兴趣区域中监测目标的像素位移反向映射至三维模型中，获得真实环境中监测目标的物理位移：
位移反向投影：根据步骤(3)中相机的位置参数，将图像中位移的跟踪结果反向投影到三维空间得到真实世界中的物理位移。
本实例中，1-16号检测节点主要在竖直方向振动，即在图1中的目标平面上运动。设xt为It中监测点的位置，C为相机中心，K，R分别相机内参矩阵和外参矩阵，则相机射线可以表达为
r(λ)＝C+λR-1K-1xt＝C+λer
其中，λ和er为射线的深度值和方向向量。设目标平面为其中cp为平面上一点，为该平面的法向量，则由此，xt在平面中的反向投影点为
因此，需要将图像位移施加在步骤(2)得到的目标监测点在图像中的投影点上，得到t时刻目标监测点在图像上的坐标并将其按照上述方法反向投影至目标平面上得到时刻t目标监测点在目标平面上的反向投影点进一步可得监测点的真实物理位移为
本实施例中还提供了一种多目标位跟踪监测装置，包括存储器和一个或多个处理器，所述存储器中存储有可执行代码，所述一个或多个处理器执行所述可执行代码时，用于实现上 述的多目标位移跟踪监测方法，其具体步骤如下：获取目标建筑的结构参数与结构三维模型，通过多目标位移跟踪监测方法对指定监测结构进行识别分析，输出目标建筑的实际物理位移。
如图4所示，输出结果中的1号，7号和8号位移监测结果示意图。
需要指出的是，示例中采用桁架节点的振动监测进行说明，但本发明不限于这些，也可以用于超高层建筑、梁桥、拱桥、斜拉桥以及悬索桥等结构的关键节点位移监测。
以上所述仅为本发明的较佳实施例，用以对本发明进行详细说明，并不用以限制本发明，凡在本发明的精神和原则之内所做的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种基于视频的多目标位移跟踪监测方法，其特征在于，包括：
    步骤1、获取目标建筑物的结构参数并建立对应的结构三维模型，基于针孔相机模型估计对应的相机参数；
    步骤2、根据步骤1的结构三维模型与相机参数，设定目标建筑物中需要监测的监测目标，并基于视频的时间轴顺序，输出包含至少一个目标特征点的感兴趣区域；
    步骤3、采用金字塔流光法对感兴趣区域内的所有目标特征点进行位移跟踪，将跟踪结果中位移数据异常的目标特征点进行剔除，并计算所有未剔除目标特征点的位移平均值作为所述感兴趣区域中监测目标的像素位移；
    步骤4、根据相机参数，将所有感兴趣区域中监测目标的像素位移反向映射至三维模型中，获得监测目标真实环境中的物理位移。
  2. 根据权利要求1所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步骤1中，所述结构三维模型基于目标建筑的实际尺寸和空间位置关系建立。
  3. 根据权利要求1所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步骤1中，所述相机参数包括畸变参数和相机内外参数，采用直接线性变换算法与Levenberg-Marquardt算法计算得到。
  4. 根据权利要求1所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步骤2中，所述感兴趣区域通过将三维模型中的监测目标投影至图像平面，并以监测目标在图像平面中的投影点为中心，利用尺寸为w×h的窗口截取获得。
  5. 根据权利要求4所述的基于视频的多目标位移跟踪监测方法，其特征在于，所述投影的公式如下：
    其中，[xw,yw,zw,1]T,(i＝1,2,…,n)表示三维空间拾取的监测目标，[u,v,1]T表示图像平面的投影点。
  6. 根据权利要求1所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步骤2中，所述目标特征点采用图像梯度自动化检测法对视频图像序列校正后第一帧图像中的感兴趣区域识别获得。
  7. 根据权利要求1或6所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步 骤2中，所述目标特征点包括Harris角点、SURF角点以及Shi-Tomasi角点。
  8. 根据权利要求1所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步骤3中，采用前向-反向误差机制和特征点位移统计特性对位移数据异常的目标特征点进行剔除。
  9. 根据权利要求1所述的基于视频的多目标位移跟踪监测方法，其特征在于，在步骤4中，所述物理位移通过计算相机射线与监测目标运动平面的交点获得。
  10. 一种多目标位移跟踪监测装置，其特征在于，包括存储器和一个或多个处理器，所述存储器中存储有可执行代码，所述一个或多个处理器执行所述可执行代码时，用于实现权利要求1-9任一所述的多目标位移跟踪监测方法，其具体步骤如下：获取目标建筑的结构参数与结构三维模型，通过多目标位移跟踪监测方法对指定监测结构进行识别分析，输出目标建筑的实际物理位移。
PCT/CN2023/120925 2022-09-30 2023-09-25 一种基于视频的多目标位移跟踪监测方法及装置 Ceased WO2024067435A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211209467.8 2022-09-30
CN202211209467.8A CN115690150B (zh) 2022-09-30 2022-09-30 一种基于视频的多目标位移跟踪监测方法及装置

Publications (1)

Publication Number Publication Date
WO2024067435A1 true WO2024067435A1 (zh) 2024-04-04

Family

ID=85064448

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/120925 Ceased WO2024067435A1 (zh) 2022-09-30 2023-09-25 一种基于视频的多目标位移跟踪监测方法及装置

Country Status (2)

Country Link
CN (1) CN115690150B (zh)
WO (1) WO2024067435A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118533071A (zh) * 2024-05-22 2024-08-23 湖南大学 基于质心边界框跟踪算法的结构三维位移测量方法及系统
CN118570166A (zh) * 2024-05-30 2024-08-30 智洋创新科技股份有限公司 输电导线的舞动监测点确定方法、舞动监测量化分析方法、舞动监测控制方法及装置
CN118781332A (zh) * 2024-09-11 2024-10-15 浙江大学 基于视频的自适应环境变化的特征点位移监测方法及装置
CN119379782A (zh) * 2024-09-04 2025-01-28 南京理工大学 基于稀疏光流算法的地铁轻量化视频定位方法
CN119832092A (zh) * 2025-03-18 2025-04-15 宁波东方理工大学(暂名) 一种全域超精度应变测量方法
CN119887985A (zh) * 2024-12-30 2025-04-25 成都增强视图科技有限公司 一种基于aigc技术快速生成图像的方法
CN119887852A (zh) * 2024-12-09 2025-04-25 杭州电子科技大学 一种基于改进型角点检测的lk光流目标跟踪方法
CN121459171A (zh) * 2025-11-12 2026-02-03 浙江盛暄电力科技有限公司 一种基于图像分析的输电杆塔倾斜监测方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115690150B (zh) * 2022-09-30 2023-11-03 浙江大学 一种基于视频的多目标位移跟踪监测方法及装置
CN120991723A (zh) * 2025-09-05 2025-11-21 哈尔滨工业大学 建筑结构地震微小位移测量方法、设计及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120093361A1 (en) * 2010-10-13 2012-04-19 Industrial Technology Research Institute Tracking system and method for regions of interest and computer program product thereof
CN105023278A (zh) * 2015-07-01 2015-11-04 中国矿业大学 一种基于光流法的运动目标跟踪方法及系统
CN109035343A (zh) * 2018-07-12 2018-12-18 福州大学 一种基于监控相机的楼层相对位移测量方法
CN112967312A (zh) * 2021-02-07 2021-06-15 湖南桥康智能科技有限公司 一种面向野外刚体目标的实时鲁棒位移监测方法及系统
CN114184127A (zh) * 2021-12-13 2022-03-15 哈尔滨工业大学 一种基于单相机无标靶的建筑物全局位移监测方法
CN115690150A (zh) * 2022-09-30 2023-02-03 浙江大学 一种基于视频的多目标位移跟踪监测方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120093361A1 (en) * 2010-10-13 2012-04-19 Industrial Technology Research Institute Tracking system and method for regions of interest and computer program product thereof
CN105023278A (zh) * 2015-07-01 2015-11-04 中国矿业大学 一种基于光流法的运动目标跟踪方法及系统
CN109035343A (zh) * 2018-07-12 2018-12-18 福州大学 一种基于监控相机的楼层相对位移测量方法
CN112967312A (zh) * 2021-02-07 2021-06-15 湖南桥康智能科技有限公司 一种面向野外刚体目标的实时鲁棒位移监测方法及系统
CN114184127A (zh) * 2021-12-13 2022-03-15 哈尔滨工业大学 一种基于单相机无标靶的建筑物全局位移监测方法
CN115690150A (zh) * 2022-09-30 2023-02-03 浙江大学 一种基于视频的多目标位移跟踪监测方法及装置

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118533071A (zh) * 2024-05-22 2024-08-23 湖南大学 基于质心边界框跟踪算法的结构三维位移测量方法及系统
CN118570166A (zh) * 2024-05-30 2024-08-30 智洋创新科技股份有限公司 输电导线的舞动监测点确定方法、舞动监测量化分析方法、舞动监测控制方法及装置
CN119379782A (zh) * 2024-09-04 2025-01-28 南京理工大学 基于稀疏光流算法的地铁轻量化视频定位方法
CN119379782B (zh) * 2024-09-04 2025-11-21 南京理工大学 基于稀疏光流算法的地铁轻量化视频定位方法
CN118781332A (zh) * 2024-09-11 2024-10-15 浙江大学 基于视频的自适应环境变化的特征点位移监测方法及装置
CN119887852A (zh) * 2024-12-09 2025-04-25 杭州电子科技大学 一种基于改进型角点检测的lk光流目标跟踪方法
CN119887852B (zh) * 2024-12-09 2025-09-26 杭州电子科技大学 一种基于改进型角点检测的lk光流目标跟踪方法
CN119887985A (zh) * 2024-12-30 2025-04-25 成都增强视图科技有限公司 一种基于aigc技术快速生成图像的方法
CN119832092A (zh) * 2025-03-18 2025-04-15 宁波东方理工大学(暂名) 一种全域超精度应变测量方法
CN121459171A (zh) * 2025-11-12 2026-02-03 浙江盛暄电力科技有限公司 一种基于图像分析的输电杆塔倾斜监测方法及系统

Also Published As

Publication number Publication date
CN115690150A (zh) 2023-02-03
CN115690150B (zh) 2023-11-03

Similar Documents

Publication Publication Date Title
CN115690150B (zh) 一种基于视频的多目标位移跟踪监测方法及装置
CN114241298A (zh) 一种激光雷达和图像融合的塔吊环境目标检测方法及系统
Wu et al. Non-contact measurement method of beam vibration with laser stripe tracking based on tilt photography
KR20190142626A (ko) Uav 탑재용 하이브리드 이미지 스캐닝에 기반한 자동화 구조물 균열 평가 시스템 및 그 방법
CN110334701A (zh) 数字孪生环境下基于深度学习和多目视觉的数据采集方法
CN116379937B (zh) 一种输电塔晃动监测方法及装置
CN118333985A (zh) 基于多源图像识别的基坑边坡变形监测方法
CN112685812A (zh) 一种动态监管方法、装置、设备及存储介质
CN115719464A (zh) 一种基于机器视觉的水表耐久性装置漏水监测方法
CN115984768A (zh) 一种基于固定型单目摄像头的多目标行人实时检测定位方法
KR101972582B1 (ko) Ptz 촬상장치 기반 변위 측정 시스템 및 방법
CN114552601A (zh) 双目视觉输电线震荡监测及三维重建方法
CN120147398A (zh) 一种基于Canny-Hough变换和KLT光流的结构大位移估计方法
CN119649225B (zh) 基于数字图像变形处理的大型危岩体识别方法
CN116229365A (zh) 一种基于计算机视觉识别建筑结构振动模态的方法及系统
CN117372872A (zh) 一种基于计算机视觉的外脚手架受力状态监测系统及方法
CN114184127B (zh) 一种基于单相机无标靶的建筑物全局位移监测方法
Cai et al. KCF‐Based Identification Approach for Vibration Displacement of Double‐Column Bents under Various Earthquakes
CN119762756A (zh) 风管拉钩检测方法、装置、电子设备及存储介质
Chen et al. Video camera-based vibration measurement for Condition Assessment of Civil Infrastructure
CN121095199A (zh) 一种基坑支护结构实时安全状态的健康管理系统
CN120259878A (zh) 基于ai图像识别的无人机桥梁边坡病害检测方法及系统
CN119516439A (zh) 一种利用监控视频自动监测结构地震响应的方法
Park et al. Computer vision-based structural health monitoring: A review
CN115993094A (zh) 一种基于ccd相机的大型膜结构屋顶位移监测方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23870666

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23870666

Country of ref document: EP

Kind code of ref document: A1