CN117128865B - Computer vision structure displacement measuring method, system, terminal and storage medium - Google Patents
Computer vision structure displacement measuring method, system, terminal and storage medium Download PDFInfo
- Publication number
- CN117128865B CN117128865B CN202310862680.7A CN202310862680A CN117128865B CN 117128865 B CN117128865 B CN 117128865B CN 202310862680 A CN202310862680 A CN 202310862680A CN 117128865 B CN117128865 B CN 117128865B
- Authority
- CN
- China
- Prior art keywords
- region
- interest
- displacement
- optical flow
- correlation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006073 displacement reaction Methods 0.000 title claims abstract description 178
- 238000000034 method Methods 0.000 title claims abstract description 88
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 230000003287 optical effect Effects 0.000 claims abstract description 112
- 238000005259 measurement Methods 0.000 claims abstract description 38
- 238000004458 analytical method Methods 0.000 claims abstract description 30
- 238000013135 deep learning Methods 0.000 claims abstract description 26
- 230000004927 fusion Effects 0.000 claims abstract description 26
- 238000006243 chemical reaction Methods 0.000 claims abstract description 17
- 238000012937 correction Methods 0.000 claims abstract description 15
- 238000001914 filtration Methods 0.000 claims abstract description 13
- 230000001133 acceleration Effects 0.000 claims abstract description 11
- 238000004364 calculation method Methods 0.000 claims description 22
- 239000003550 marker Substances 0.000 claims description 18
- 238000000691 measurement method Methods 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 9
- 238000012935 Averaging Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 230000033001 locomotion Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000012916 structural analysis Methods 0.000 claims description 6
- 230000007704 transition Effects 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 5
- 238000013136 deep learning model Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 abstract description 17
- 230000007613 environmental effect Effects 0.000 abstract description 6
- 230000008859 change Effects 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 2
- 238000012360 testing method Methods 0.000 description 15
- 239000011229 interlayer Substances 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000036541 health Effects 0.000 description 5
- 239000010410 layer Substances 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000011150 reinforced concrete Substances 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Abstract
The invention discloses a computer vision structure displacement measuring method, a system, a terminal and a storage medium, wherein the computer vision structure displacement measuring method is based on data fusion and comprises the following main contents: after the structure monitoring video is obtained, stripping background components through pre-analysis and selecting an interested region; matching and tracking the displacement of the region of interest through a pyramid accelerated correlation template; performing optical flow estimation through a deep learning optical flow model to obtain the speed of the region of interest; carrying out data fusion by adopting Kalman filtering to obtain more accurate displacement; performing simplified distortion correction and physical quantity conversion to obtain actual displacement of the region of interest; analyzing the overall deformation of the structure. The invention combines a dense optical flow method based on deep learning and a pyramid acceleration correlation template matching method, can eliminate the influence of environmental change and accumulated error on structural displacement measurement, and realizes stable and accurate structural displacement measurement effect.
Description
Technical Field
The invention relates to the technical field of health monitoring of civil engineering structures, in particular to a computer vision structure displacement measuring method, a system, a terminal and a storage medium.
Background
Structural displacement is an important measurement index in structural health monitoring and is an important parameter reflecting the overall safety of a structure. The structural displacement measurement method based on computer vision can acquire the vibration response of the structure by analyzing the structural monitoring video, and becomes one of key technologies for structural displacement measurement by virtue of the advantages of full-field observation and light weight. As shown in fig. 1, the existing computer vision structure displacement measurement method comprises the following steps: selecting a region of interest in the structure monitoring video according to priori knowledge; monitoring the optical flow of the initial frame and the current frame of the video through a traditional dense optical flow model estimation structure; calculating the relative displacement of the region of interest according to the optical flow estimation result; and carrying out video distortion correction and physical quantity conversion to obtain the real displacement of the structure.
The traditional dense optical flow model is used for calculating optical flows of two pictures through a non-deep learning model; optical flow refers to the relative offset of moving pixels in a picture.
The existing computer vision structure displacement measurement method has the following defects:
(1) The existing method selects the region of interest by means of priori knowledge, relies on experience judgment of professionals, has strong subjectivity and is easily interfered by background and non-structural components, and cannot be suitable for batched and automatic structural monitoring video processing;
(2) The displacement measurement accuracy of the existing method is influenced by environmental changes, and displacement measurement failure can be caused by object changes in the environment, so that the method is difficult to be suitable for long-term structural health monitoring of complex changes of the environment;
(3) The current method adopts the traditional dense optical flow model to carry out optical flow estimation, the analysis time of the structural monitoring video is long, and the timeliness requirement of structural health monitoring is difficult to meet.
Patent CN110136168a adopts a method of combining feature point matching and sparse optical flow method to estimate the speed of the object, but its accuracy is limited by the quality of the feature points. The patent CN114757977a performs object trajectory extraction with the object detection network by fusion improvement optical flow method, which is also limited by the quality of the feature points. The patent CN106482711A realizes the positioning of an indoor object by combining the gray feature extraction and the dense optical flow method, but the method uses the traditional dense optical flow model to carry out optical flow estimation, the analysis time is long, the object motion trail depends on the speed integration, the precision is influenced by the integration accumulated error, and the method is difficult to be used for long-term object monitoring.
Disclosure of Invention
In view of this, in order to solve the above-mentioned many disadvantages existing in the existing computer vision structure displacement measurement method, the present invention proposes a computer vision structure displacement measurement method, system, terminal and storage medium, the method uses the dense optical flow model to pre-analyze the monitoring video, strip the background and non-structural components, select the interested region in the structure, obtain the displacement of the interested region through the correlation template matching of pyramid acceleration, obtain the speed of the interested region through the dense optical flow model based on deep learning, and merge two data by adopting Kalman filtering, obtain the accurate position of the interested region; the method eliminates the interference of non-structure and background components on the selection of the interested region through video pre-analysis; the method adopts a dense optical flow model based on deep learning, so that the efficiency of monitoring video analysis is improved; according to the method, the influence of background change on structural displacement measurement is eliminated through data fusion, the robustness of structural displacement estimation is enhanced, the influence of integrated accumulated errors on displacement measurement is avoided, and more accurate structural displacement estimation is realized.
An embodiment of a first aspect of the present application provides a method for measuring displacement of a computer vision structure, including the steps of: 1) Taking the structural vibration video as input to obtain a corresponding structural vibration picture sequence;
2) Extracting part of pictures from all the structure vibration picture sequences at intervals to form picture groups, taking the extracted picture groups as input, adopting a dense optical flow model based on deep learning to carry out integral optical flow estimation of the image, carrying out integral deformation pre-analysis and background stripping on the basis of an optical flow estimation result, and selecting an interested region according to structural deformation characteristics and structural analysis requirements;
3) The initial displacement of the region of interest is obtained by matching and tracking the positions of the region of interest in all the image sequences based on the pyramid acceleration correlation template;
4) Based on the position information of the region of interest, cutting the region of interest in all the image sequences, carrying out optical flow estimation of the region of interest by adopting a dense optical flow model based on deep learning, and averaging the optical flows of all pixels in the region of interest to obtain the speed of the region of interest;
5) Carrying out data fusion on the initial displacement of the region of interest and the speed of the region of interest by adopting a Kalman filtering method to obtain the final displacement of the region of interest;
6) Selecting an object with known engineering length in the picture as a marker, calculating a scale factor, and carrying out camera distortion correction by utilizing the scale factor, so as to respectively calculate image scale factors at different heights and multiply the image scale factors by the final displacement of the region of interest, obtain the displacement of the region of interest after physical quantity conversion, and correct the influence of camera distortion on the video shooting quality;
the calculation formula of the scale factor is as follows:
Wherein alpha is a scale factor, D is the engineering length of the marker, and D is the pixel length of the marker in the picture;
7) And (3) carrying out integral displacement estimation on the structure by adopting the physical quantity converted interested area displacement of different parts of the structure obtained in the step (6), analyzing integral deformation condition of the structure, and evaluating the safety of the structure based on a preset structural deformation threshold.
Optionally, in one embodiment of the present application, the dense optical flow model based on deep learning is a deep learning model, and the dense optical flow model based on deep learning includes a feature extraction module, a feature association module, and a cyclic update module;
the feature extraction module is responsible for compressing the features of the input pictures and extracting the features of the input pictures through the convolution layer;
The feature association module is responsible for extracting the association calculation of the features, and obtains the correlation tensor of the input picture by carrying out the association calculation and dimension reduction on the extracted features;
The loop updating module is responsible for iterative optimization of the optical flow, performs iterative updating of the optical flow field through a loop neural network architecture until the set iteration times are reached, and outputs an optical flow estimation result of the picture.
Optionally, in one embodiment of the present application, the structure integral deformation pre-analysis and background stripping includes:
performing optical flow visualization through the optical flow estimation result to set the color of the pixel according to the direction of the pixel motion in the picture, and setting the shade of the color according to the offset of the pixel motion;
Judging the structural deformation state and the floor with the largest deformation according to the visual result;
And taking a white part in the visual result as a background and taking a part which is overlapped with the structure but has different displacement directions or has offset difference in the picture as an interference component of non-structural vibration according to the visual result so as to avoid the background and the non-structural vibration component in the process of selecting the region of interest.
Optionally, in an embodiment of the present application, the tracking the position of the region of interest in all the image sequences by correlation template matching based on pyramid acceleration, to obtain an initial displacement of the region of interest includes:
3.1 Downsampling the correlation template and the matching image to make the resolution of the matching image become one half of the original resolution;
3.2 On the downsampled matching image, performing correlation template matching by using a downsampled template, and tracking the position of the region of interest; the correlation template matching is carried out by moving the template pixel by pixel on a matching image, calculating the correlation between the template and a coverage area, and taking the position with the maximum correlation as a preliminary tracking result;
The correlation calculation formula is as follows:
Wherein T is the template, I is the template coverage area, R is the correlation coefficient, m is the width of the template, n is the height of the template, T i,j and I i,j are the gray values of the ith row and jth column pixels of the template and the template coverage area respectively, AndRespectively averaging the gray values of the pixels of the template and the coverage area of the template;
3.3 Reverse mapping is carried out on the preliminary tracking position to obtain a corresponding original matching image area; and in the mapping area, performing correlation template matching by using the original template to obtain a tracking result of the region of interest in the original image.
Optionally, in an embodiment of the present application, the performing data fusion on the initial displacement of the region of interest and the velocity of the region of interest by using a kalman filtering method to obtain a final displacement of the region of interest includes:
5.1 Setting a state equation according to the relative relation between the initial displacement of the region of interest and the speed of the region of interest, setting an observation equation according to the relative relation between the actual displacement of the region of interest and the initial displacement of the region of interest obtained based on correlation template matching tracking, and setting initial parameters of the state equation to zero, wherein the state equation and the observation equation are as follows:
xt＝xt-1+vt+nvt，
ut＝xt+ut，
Wherein x t is the actual position state of the region of interest at the time t, the actual displacement of the region of interest, v is the actual speed of the region of interest, v t is the speed of the region of interest at the time t obtained based on an optical flow model, a is a state transition matrix, B is an input control matrix, n vt is corresponding speed measurement noise, u t is the displacement of the region of interest at the time t obtained based on the correlation template matching, H is a displacement observation equation, and n ut is corresponding displacement measurement noise;
5.2 Based on a state equation, carrying out prior estimation on the position state of the region of interest by adopting the speed of the region of interest obtained based on an optical flow model, and calculating corresponding covariance, wherein the prior estimation method and the covariance calculation formula of the position state of the region of interest are as follows:
Wherein, For an a priori estimation of the position state of the region of interest at the current moment,For a posterior estimation of the position state of the region of interest at the previous moment,For the covariance of the prior estimate of the region of interest location state at the current moment,For covariance of the posterior estimation of the position state of the region of interest at the previous moment, A T is a transpose of a state transition matrix, B T is a transpose of an input control matrix, and Q is a variance of noise of the speed of the region of interest estimated based on an optical flow model;
The variance Q of the noise of the speed of the region of interest estimated based on the optical flow model is obtained by calculating variances of all pixel offsets in the region of interest;
5.3 The method comprises the steps of) calculating a Kalman gain coefficient by adopting the displacement of the region of interest and the noise variance obtained based on correlation template matching, updating the position state of the region of interest and the corresponding covariance to obtain the accurate displacement of the region of interest, wherein the calculation formula of the Kalman gain coefficient is as follows:
The update equation for the position state and covariance is:
Wherein K is the Kalman gain coefficient, For a posterior estimation of the position state of the region of interest at the current moment,The covariance estimated for the posterior of the position state of the region of interest at the current moment is obtained, I is an identity matrix, and R is the variance of the noise of the displacement of the region of interest obtained based on correlation template matching;
The variance R of the noise of the displacement of the region of interest obtained based on the correlation template matching is calculated according to the distribution property of n ut, and n ut obeys the average distribution of 0-0.5 pixel based on the randomness of the correlation template matching.
An embodiment of the second aspect of the present application provides a computer vision structure displacement measurement system, comprising: a computer vision structure displacement measurement method for implementing any one of claims 1 to 5, wherein the system comprises:
the picture sequence acquisition module is used for taking the structure vibration video as input to acquire a corresponding structure vibration picture sequence;
The pre-analysis and interested region selection module is used for extracting part of pictures from all the structure vibration picture sequences at intervals to form a picture group, taking the extracted picture group as input, carrying out integral image optical flow estimation by adopting a dense optical flow model based on deep learning, carrying out integral structure deformation pre-analysis and background stripping based on an optical flow estimation result, and selecting an interested region according to structural deformation characteristics and structural analysis requirements;
the pyramid correlation template matching module is used for matching and tracking the positions of the regions of interest in all the image sequences through correlation templates accelerated based on the pyramid to obtain initial displacement of the regions of interest;
The dense optical flow estimation module is used for cutting the region of interest in all the image sequences based on the position information of the region of interest, carrying out optical flow estimation of the region of interest by adopting a dense optical flow model based on deep learning, and averaging the optical flows of all pixels in the region of interest to obtain the speed of the region of interest;
The data fusion module is used for carrying out data fusion on the initial displacement of the region of interest and the speed of the region of interest by adopting a Kalman filtering method to obtain the final displacement of the region of interest;
The distortion correction and displacement conversion module is used for selecting an object with known engineering length in the picture as a marker, calculating a scaling factor, carrying out camera distortion correction by utilizing the scaling factor, respectively calculating image scaling factors at different heights, multiplying the image scaling factors by the final displacement of the region of interest to obtain the displacement of the region of interest after physical quantity conversion, and correcting the influence of camera distortion on video shooting quality;
the calculation formula of the scale factor is as follows:
Wherein alpha is a scale factor, D is the engineering length of the marker, and D is the pixel length of the marker in the picture;
And the structural integral displacement analysis module is used for estimating structural integral displacement by adopting the region of interest displacement obtained by the distortion correction and displacement conversion module and converted by the physical quantity of different parts of the structure, analyzing structural integral deformation condition and evaluating the safety of the structure based on a preset structural deformation threshold.
An embodiment of a third aspect of the present application provides a computer vision structure displacement measurement system terminal, including: the system comprises a memory, a processor and at least one instruction or at least one section of computer program stored on the memory and capable of being loaded and executed on the processor, wherein the processor loads and executes the at least one instruction or the at least one section of computer program to realize the computer vision structure displacement measuring method.
In a fourth aspect, embodiments of the present application provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the described method for measuring displacement of a computer vision structure based on data fusion.
Compared with the prior art, the technical scheme has the following beneficial effects:
(1) According to the displacement measurement method of the computer vision structure, the dense optical flow model is adopted to conduct structure monitoring video pre-analysis, and background and non-structural components are stripped, so that better interested region selection is achieved; compared with the prior art, the method does not need professional personnel to select the region of interest, is more objective, and is beneficial to batch and automatic structure monitoring video processing;
(2) The invention adopts a dense optical flow model based on deep learning to carry out optical flow estimation of an interested region; compared with the prior art, the method can realize the optical flow estimation of the structure in a shorter time, and improves the analysis efficiency of the structure monitoring video;
(3) According to the invention, the deep learning-based dense optical flow model is fused with the pyramid acceleration-based correlation template matching to perform structural displacement measurement, so that the influence of environmental change on structural displacement measurement is eliminated, and more accurate displacement measurement is realized; compared with the prior art, the method has stronger robustness and is suitable for long-term structural health monitoring of complex environmental changes.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flow chart of a conventional computer vision structure displacement measurement method;
FIG. 2 is a flow chart of a method for measuring displacement of a computer vision structure according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a pyramid acceleration-based correlation template matching pursuit scheme according to one embodiment of the application;
FIG. 4 is a flow chart of data fusion using Kalman filtering according to one embodiment of the application;
FIG. 5 is a schematic diagram of a structure and camera layout according to one embodiment of the present application;
FIG. 6 is a schematic diagram of experimental pre-analysis and background stripping according to one embodiment of the application;
FIG. 7 is a comparative schematic diagram of structural interlayer displacement measurement errors obtained using three different methods according to one embodiment of the present application;
FIG. 8 is a photograph of a test piece according to another embodiment of the present application;
FIG. 9 is a schematic view of a test piece and instrument layout according to another embodiment of the present application;
FIG. 10 is a schematic illustration of experimental pre-analysis and background stripping according to another embodiment of the application;
FIG. 11 is a comparative schematic diagram of structural interlayer displacement measurement errors obtained using three different methods according to another embodiment of the present application;
Fig. 12 is a schematic structural diagram of a displacement measurement system of a computer vision structure according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
Specifically, fig. 2 is a schematic flow chart of a method for measuring displacement of a computer vision structure according to an embodiment of the present application.
As shown in fig. 2, the computer vision structure displacement measurement method includes the steps of:
1) Taking the structural vibration video as input to obtain a corresponding structural vibration picture sequence;
2) Extracting part of pictures at intervals from all picture sequences to form picture groups, taking the extracted picture groups as input, adopting a dense optical flow model based on deep learning to carry out integral optical flow estimation of an image, carrying out integral deformation pre-analysis and background stripping on the basis of an optical flow estimation result, and selecting an interested region for subsequent structural displacement tracking according to structural deformation characteristics and structural analysis requirements;
The dense optical flow model based on the deep learning is a deep learning model, is obtained by training an artificially synthesized optical flow data set, and is obtained by manually rendering three-dimensional motions of different objects under a fixed background and then projecting the motions to a two-dimensional plane;
The whole optical flow estimation of the image is to solve the offset of corresponding pixels of a relative moving object of the two images through a dense optical flow model based on deep learning;
analyzing the region with the largest structural deformation and separating the non-structural vibration component and the complex background in the picture;
3) The initial displacement of the region of interest is obtained by matching and tracking the positions of the region of interest in all the image sequences based on the pyramid acceleration correlation template;
The correlation template matching is a matching method based on image gray information, and the position of a tracking target in the image to be matched is judged by moving the template pixel by pixel on the image to be matched and calculating the correlation coefficient of the gray information of the image to be matched and the template;
the pyramid acceleration reduces the calculated amount of a matching algorithm through an image pyramid, and shortens the calculation time;
4) Based on the position information of the region of interest, cutting the region of interest in all the image sequences, carrying out optical flow estimation of the region of interest by adopting a dense optical flow model based on deep learning, and averaging the optical flows of all pixels in the region of interest to obtain the speed of the region of interest;
5) Carrying out data fusion on the initial displacement and the speed of the region of interest by adopting a Kalman filtering method to obtain more accurate final displacement of the region of interest;
The Kalman filtering is an efficient autoregressive filter, and can simultaneously consider measurement errors and uncertainty of the system by estimating the state of the system along with time through combining measurement results and prediction results;
6) Selecting an object with known engineering length in the picture as a marker, and calculating a scale factor, namely an engineering length conversion ratio corresponding to a single pixel; calculating image scale factors at different heights respectively through a simplified camera distortion correction method, multiplying the image scale factors by final displacement of the region of interest to obtain the displacement of the region of interest after physical quantity conversion, and correcting the influence of camera distortion on the video shooting quality; the calculation formula of the scale factor is as follows:
Wherein alpha is a scale factor, D is the engineering length of the marker, and D is the pixel length of the marker in the picture;
7) And (3) carrying out integral displacement estimation on the structure by adopting the region of interest displacement obtained in the step (6) after physical quantity conversion of different parts of the structure, analyzing integral deformation condition of the structure, and evaluating the safety of the structure based on a preset structural deformation threshold.
The dense optical flow model based on deep learning in the step 2) and the step 4) is a deep learning model, a convolutional neural network and cyclic neural network combined architecture is adopted, the model takes two picture groups with the same background as input, optical flows of two pictures are output, namely, the offset of corresponding pixels of a relative moving object in the two pictures mainly comprises a feature extraction module, a feature association module and a cyclic update module; the feature extraction module is responsible for compressing the features of the input pictures and extracting the features of the input picture groups through the convolution layer; the feature association module is responsible for extracting the association calculation of the features, and obtaining the association tensor of the two pictures by carrying out the association calculation and dimension reduction on the extracted features of the two pictures; and the loop updating module is responsible for iterative optimization of the optical flow, performs iterative updating of the optical flow field through a loop neural network architecture until the set iteration times are reached, and outputs an optical flow estimation result of two pictures.
Performing integral deformation pre-analysis and background stripping of the structure in the step 2), and performing optical flow visualization through an optical flow estimation result, namely setting the color of a pixel in a picture according to the direction of the pixel, and setting the depth of the color according to the offset of the pixel motion; judging the structural deformation state and the floor with the largest deformation according to the visual result, and taking the structural deformation state and the floor with the largest deformation as the key point of subsequent analysis; according to the visual result, taking the part without displacement, namely the white part in the visual result, as a background, taking the part which is overlapped with the structure but has different displacement directions or obvious difference in offset in the picture, namely the part with different colors or obvious difference in color shades of the optical flow, as an interference component of the non-structural vibration, and avoiding the background and the non-structural vibration component in the process of selecting the region of interest.
As shown in fig. 3, the correlation template matching based on pyramid acceleration in step 3) is implemented by downsampling an image to be matched with a template, tracking the template in the downsampled image to be matched, and further tracking the template on the original matched image by the downsampled tracking result to obtain the position of the template, which comprises the following specific steps:
3.1 Downsampling the template and the matching image to make the resolution of the template and the matching image become one half of the original resolution;
3.2 On the downsampled matching image, performing correlation template matching by using the downsampled template, and tracking the position of the region of interest; the correlation template matching is carried out by moving the template pixel by pixel on a matching image, calculating the correlation between the template and a coverage area, and taking the position with the maximum correlation as a preliminary tracking result; wherein, the correlation calculation formula is:
Wherein T is a template, I is a template coverage area, R is a correlation coefficient, m is a width of the template, n is a height of the template, T i,j and I i,j are gray values of pixels of ith row and jth column of the template and the template coverage area respectively, AndRespectively averaging the gray values of the pixels of the template and the coverage area of the template;
3.3 Reverse mapping is carried out on the preliminary tracking position to obtain a corresponding original matching image area; and in the mapping area, performing correlation template matching by using the original template to obtain a tracking result of the region of interest in the original image.
As shown in fig. 4, the data fusion based on the kalman filtering in step 5) calculates the prior estimation of the displacement of the region of interest through the velocity of the region of interest based on the optical flow model, and updates the estimation result through the displacement of the region of interest based on the correlation template matching, thereby obtaining more accurate displacement of the region of interest, which comprises the following specific steps:
5.1 Setting a state equation according to the relative relation between the displacement of the region of interest and the speed, setting an observation equation according to the relative relation between the actual displacement of the region of interest and the displacement of interest obtained based on correlation template matching tracking, and setting initial parameters of the state equation to zero, wherein the state equation and the observation equation are as follows:
xt＝xt-1+vt+nvt，
ut＝xt+ut，
Wherein x t is the actual position state of the region of interest at time t, denoted by [ p, v ] T, p is the actual displacement of the region of interest, v is the actual velocity of the region of interest, a is the state transition matrix, B is the input control matrix, v t is the velocity of the region of interest at time t obtained based on the optical flow model, n vt is the corresponding velocity measurement noise, u t is the displacement of the region of interest at time t obtained based on correlation template matching, H is the displacement observation matrix, denoted by [1,0], and n ut is the corresponding displacement measurement noise;
5.2 Based on a state equation, carrying out prior estimation on the position state of the region of interest by adopting the speed of the region of interest obtained based on an optical flow model, and calculating corresponding covariance, wherein the prior estimation method and the covariance calculation formula of the position state of the region of interest are as follows:
Wherein, For an a priori estimation of the position state of the region of interest at the current moment,For a posterior estimation of the position state of the region of interest at the previous moment,For the covariance of the prior estimate of the region of interest location state at the current moment,For covariance of the posterior estimation of the position state of the region of interest at the previous moment, A T is a transpose of a state transition matrix, B T is a transpose of an input control matrix, and Q is a variance of noise of the speed of the region of interest estimated based on an optical flow model;
the variance Q of the noise of the speed of the region of interest estimated based on the optical flow model is obtained by calculating variances of all pixel offsets in the region of interest;
5.3 The method comprises the steps of) calculating a Kalman gain coefficient by adopting the displacement of the region of interest and the noise variance obtained based on correlation template matching, updating the position state of the region of interest and the corresponding covariance to obtain the accurate displacement of the region of interest, wherein the calculation formula of the Kalman gain coefficient is as follows:
The update equation for the position state and covariance is:
Wherein K is a Kalman gain coefficient, H T is a transpose of a displacement observation matrix, For posterior estimation of the position state of the region of interest at the current moment, I is an identity matrix,The covariance of the posterior estimation of the position state of the region of interest at the current moment is obtained, and R is the variance of the noise of the displacement of the region of interest based on correlation template matching;
The variance R of the noise of the displacement of the region of interest obtained based on the correlation template matching is calculated according to the distribution property of n ut, and n ut obeys the average distribution of 0-0.5 pixel based on the randomness of the correlation template matching.
In order to prove the effectiveness of the computer vision structure displacement measurement method based on data fusion, the invention is described in detail through specific embodiments.
Example 1 reinforced concrete Structure vibration table test in indoor scene
As shown in FIG. 5, the present embodiment is an indoor large-size three-layer reinforced concrete structure vibration table test, the test piece layer height is 2.3m, and the plane size is 4.7mX3 m. And arranging a displacement meter in the structure, measuring the interlayer displacement of the structure, and arranging a camera shooting structure vibration process in the short side direction for structural displacement analysis.
The dense optical flow model based on deep learning is adopted to pre-analyze the video, so as to separate interference objects such as a laboratory background, a vibration table surrounding band, a shooting camera and the like which do not vibrate with the structure, and select an interested region according to interlayer displacement analysis requirements, which can be shown as figure 6.
And analyzing the structure vibration video by adopting a computer vision structure displacement measurement method, a correlation template matching method and a dense optical flow method based on data fusion to obtain the interlayer displacement of the structure, wherein the interlayer displacement measurement errors of different methods can be shown in figure 7. The interlayer displacement obtained by the dense optical flow method is subject to the influence of accumulated errors, the errors are increased continuously along with the time, and the root mean square errors in the three interlayer displacement measurements are 8.0mm,12.5mm and 41.5mm respectively. The root mean square error of the correlation template matching method is 1.4mm,2.1mm and 2.2mm respectively. Compared with a dense optical flow method and a correlation template matching method, the method provided by the invention can better measure the interlayer displacement of the structure, has smaller error and root mean square error of 0.8mm,1.3mm and 1.3mm respectively. Compared with a dense optical flow method, the error level of the method is reduced by more than 89%, and compared with a correlation template matching method, the error level of the method is reduced by more than 36%.
Example 2 Cold bent Steel wall Limb vibration table test in outdoor scene
Referring to fig. 8 and 9, this embodiment is an outdoor large-size cold-bent steel wall system vibration table test, in which a test piece is placed on an outdoor vibration table, and the height of the test piece is 2.74m and the width of the test piece is 4.88m. The top of the test piece is provided with a displacement meter for measuring the top displacement of the structure, and the south side of the test piece is provided with a camera shooting test process. The pre-analysis result of the test video can be shown as figure 10, and environmental interference items such as outdoor background, vibration table upright posts and the like are stripped.
And analyzing the structural vibration video by adopting a computer vision structural displacement measurement method, a correlation template matching method and a dense optical flow method based on data fusion to obtain the top horizontal displacement of the structure, wherein the displacement measurement errors of different methods can be shown as shown in fig. 11. In the outdoor test, the root mean square error of the correlation template matching method and the dense optical flow method is 26.6mm and 1.6mm respectively, and the root mean square error of the method provided by the invention is 0.9mm, which is reduced by 96.5% and 41.1% respectively compared with the former two methods. Therefore, the measurement error of the method provided by the invention is obviously smaller than that of other methods, and a more stable and accurate displacement measurement result is obtained.
In summary, the computer vision structure displacement measurement method based on data fusion can effectively peel off the interference of the unstructured part on displacement measurement, avoid the influence of environmental change on displacement measurement, obtain stable and accurate structure displacement measurement effect under different environments, and obviously improve the accuracy and the robustness compared with the existing method.
A computer vision structural displacement measurement system according to an embodiment of the present application will be described next with reference to the accompanying drawings.
FIG. 12 is a schematic diagram of a computer vision structural displacement measurement system according to an embodiment of the present application.
As shown in fig. 12, the computer vision structure displacement measurement system 10 is configured to implement a computer vision structure displacement measurement method, wherein the system 10 includes: the system comprises a picture sequence acquisition module 100, a pre-analysis and region of interest selection module 200, a pyramid correlation template matching module 300, a dense optical flow estimation module 400, a data fusion module 500, a distortion correction and displacement conversion module 600 and a structural whole displacement analysis module 700.
Specifically, the picture sequence obtaining module 100 is configured to obtain a corresponding structure vibration picture sequence by using the structure vibration video as an input.
The pre-analysis and interested region selection module 200 is configured to extract part of pictures from all the structure vibration picture sequences at intervals to form a picture group, take the extracted picture group as input, perform integral image optical flow estimation by adopting a dense optical flow model based on deep learning, perform integral structure deformation pre-analysis and background stripping based on an optical flow estimation result, and select an interested region according to structural deformation characteristics and structural analysis requirements.
The pyramid correlation template matching module 300 is used for matching and tracking the positions of the regions of interest in all the image sequences through correlation templates accelerated based on a pyramid to obtain initial displacement of the regions of interest;
The dense optical flow estimation module 400 is configured to cut the region of interest in all the image sequences based on the position information of the region of interest, perform optical flow estimation of the region of interest by using a dense optical flow model based on deep learning, and average optical flows of all pixels in the region of interest to obtain a speed of the region of interest;
The data fusion module 500 is configured to perform data fusion on the initial displacement of the region of interest and the speed of the region of interest by using a kalman filtering method, so as to obtain a final displacement of the region of interest;
The distortion correction and displacement conversion module 600 is configured to select an object with a known engineering length in a picture as a marker, calculate a scale factor, and perform camera distortion correction by using the scale factor, so as to respectively calculate image scale factors at different heights and multiply the image scale factors with final displacement of an interested region, obtain displacement of the interested region after physical quantity conversion, and correct influence of camera distortion on video shooting quality;
the calculation formula of the scale factor is as follows:
Wherein alpha is a scale factor, D is the engineering length of the marker, and D is the pixel length of the marker in the picture;
The integral structure displacement analysis module 700 is configured to perform integral structure displacement estimation by using the displacements of the region of interest obtained by the distortion correction and displacement conversion module 600 and converted from the physical quantities of different parts of the structure, analyze integral structure deformation conditions, and evaluate the safety of the structure based on a preset structural deformation threshold.
The embodiment also provides a computer vision structure displacement measurement system terminal based on data fusion, which comprises a memory, a processor and at least one instruction or at least one section of computer program stored on the memory and capable of being loaded and operated on the processor, wherein the processor loads and operates the at least one instruction or the at least one section of computer program to realize the steps of the computer vision structure displacement measurement method based on data fusion.
The present embodiment also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for measuring displacement of a computer vision structure based on data fusion.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or N wires, a portable computer cartridge (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (8)
1. The computer vision structure displacement measuring method is characterized by comprising the following main steps:
1) Taking the structural vibration video as input to obtain a corresponding structural vibration picture sequence;
2) Extracting part of pictures from all the structure vibration picture sequences at intervals to form picture groups, taking the extracted picture groups as input, adopting a dense optical flow model based on deep learning to carry out integral optical flow estimation of the image, carrying out integral deformation pre-analysis and background stripping on the basis of an optical flow estimation result, and selecting an interested region according to structural deformation characteristics and structural analysis requirements;
3) The initial displacement of the region of interest is obtained by matching and tracking the positions of the region of interest in all the image sequences based on the pyramid acceleration correlation template;
4) Based on the position information of the region of interest, cutting the region of interest in all the image sequences, carrying out optical flow estimation of the region of interest by adopting a dense optical flow model based on deep learning, and averaging the optical flows of all pixels in the region of interest to obtain the speed of the region of interest;
5) Carrying out data fusion on the initial displacement of the region of interest and the speed of the region of interest by adopting a Kalman filtering method to obtain the final displacement of the region of interest;
6) Selecting an object with known engineering length in the picture as a marker, calculating a scale factor, and carrying out camera distortion correction by utilizing the scale factor, so as to respectively calculate image scale factors at different heights and multiply the image scale factors by the final displacement of the region of interest, obtain the displacement of the region of interest after physical quantity conversion, and correct the influence of camera distortion on the video shooting quality;
the calculation formula of the scale factor is as follows:
Wherein alpha is a scale factor, D is the engineering length of the marker, and D is the pixel length of the marker in the picture;
7) And (3) carrying out integral displacement estimation on the structure by adopting the physical quantity converted interested area displacement of different parts of the structure obtained in the step (6), analyzing integral deformation condition of the structure, and evaluating the safety of the structure based on a preset structural deformation threshold.
2. The method of claim 1, wherein the deep learning-based dense optical flow model is a deep learning model, the deep learning-based dense optical flow model comprising a feature extraction module, a feature correlation module, and a cyclic update module;
the feature extraction module is responsible for compressing the features of the input pictures and extracting the features of the input pictures through the convolution layer;
The feature association module is responsible for extracting the association calculation of the features, and obtains the correlation tensor of the input picture by carrying out the association calculation and dimension reduction on the extracted features;
The loop updating module is responsible for iterative optimization of the optical flow, performs iterative updating of the optical flow field through a loop neural network architecture until the set iteration times are reached, and outputs an optical flow estimation result of the picture.
3. The method according to claim 1 or 2, wherein the structural ensemble deformation pre-analysis and background stripping comprises:
performing optical flow visualization through the optical flow estimation result to set the color of the pixel according to the direction of the pixel motion in the picture, and setting the shade of the color according to the offset of the pixel motion;
Judging the structural deformation state and the floor with the largest deformation according to the visual result;
And taking a white part in the visual result as a background and taking a part which is overlapped with the structure but has different displacement directions or has offset difference in the picture as an interference component of non-structural vibration according to the visual result so as to avoid the background and the non-structural vibration component in the process of selecting the region of interest.
4. A method according to claim 3, wherein said tracking the position of the region of interest in all the image sequences by pyramid acceleration based correlation template matching results in an initial displacement of the region of interest, comprising:
3.1 Downsampling the correlation template and the matching image to make the resolution of the matching image become one half of the original resolution;
3.2 On the downsampled matching image, performing correlation template matching by using a downsampled template, and tracking the position of the region of interest; the correlation template matching is carried out by moving the template pixel by pixel on a matching image, calculating the correlation between the template and a coverage area, and taking the position with the maximum correlation as a preliminary tracking result;
The correlation calculation formula is as follows:
Wherein T is the template, I is the template coverage area, R is the correlation coefficient, m is the width of the template, n is the height of the template, T i,j and I i,j are the gray values of the ith row and jth column pixels of the template and the template coverage area respectively, AndRespectively averaging the gray values of the pixels of the template and the coverage area of the template;
3.3 Reverse mapping is carried out on the preliminary tracking position to obtain a corresponding original matching image area; and in the mapping area, performing correlation template matching by using the original template to obtain a tracking result of the region of interest in the original image.
5. The method of claim 4, wherein the performing data fusion on the initial displacement of the region of interest and the velocity of the region of interest by using a kalman filtering method to obtain the final displacement of the region of interest comprises:
5.1 Setting a state equation according to the relative relation between the initial displacement of the region of interest and the speed of the region of interest, setting an observation equation according to the relative relation between the actual displacement of the region of interest and the initial displacement of the region of interest obtained based on correlation template matching tracking, and setting initial parameters of the state equation to zero, wherein the state equation and the observation equation are as follows:
xt＝Axt-1+Bvt+Bnvt，
ut＝Hxt+nut，
Wherein x t is the actual position state of the region of interest at the time t, p is the actual displacement of the region of interest, v is the actual speed of the region of interest, v t is the speed of the region of interest at the time t obtained based on an optical flow model, A is a state transition matrix, B is an input control matrix, n vt is corresponding speed measurement noise, u t is the displacement of the region of interest at the time t obtained based on the correlation template matching, H is a displacement observation equation, and n ut is corresponding displacement measurement noise;
5.2 Based on a state equation, carrying out prior estimation on the position state of the region of interest by adopting the speed of the region of interest obtained based on an optical flow model, and calculating corresponding covariance, wherein the prior estimation method and the covariance calculation formula of the position state of the region of interest are as follows:
Wherein, For an a priori estimation of the position state of the region of interest at the current moment,For a posterior estimation of the position state of the region of interest at the previous moment,For the covariance of the prior estimate of the region of interest location state at the current moment,For covariance of the posterior estimation of the position state of the region of interest at the previous moment, A T is a transpose of a state transition matrix, B T is a transpose of an input control matrix, and Q is a variance of noise of the speed of the region of interest estimated based on an optical flow model;
The variance Q of the noise of the speed of the region of interest estimated based on the optical flow model is obtained by calculating variances of all pixel offsets in the region of interest;
5.3 The method comprises the steps of) calculating a Kalman gain coefficient by adopting the displacement of the region of interest and the noise variance obtained based on correlation template matching, updating the position state of the region of interest and the corresponding covariance to obtain the accurate displacement of the region of interest, wherein the calculation formula of the Kalman gain coefficient is as follows:
The update equation for the position state and covariance is:
Wherein K is the Kalman gain coefficient, For a posterior estimation of the position state of the region of interest at the current moment,The covariance estimated for the posterior of the position state of the region of interest at the current moment is obtained, I is an identity matrix, and R is the variance of the noise of the displacement of the region of interest obtained based on correlation template matching;
The variance R of the noise of the displacement of the region of interest obtained based on the correlation template matching is calculated according to the distribution property of n ut, and n ut obeys the average distribution of 0-0.5 pixel based on the randomness of the correlation template matching.
6. A computer vision structure displacement measurement system for implementing a computer vision structure displacement measurement method according to any one of claims 1 to 5, wherein the system comprises:
the picture sequence acquisition module is used for taking the structure vibration video as input to acquire a corresponding structure vibration picture sequence;
The pre-analysis and interested region selection module is used for extracting part of pictures from all the structure vibration picture sequences at intervals to form a picture group, taking the extracted picture group as input, carrying out integral image optical flow estimation by adopting a dense optical flow model based on deep learning, carrying out integral structure deformation pre-analysis and background stripping based on an optical flow estimation result, and selecting an interested region according to structural deformation characteristics and structural analysis requirements;
the pyramid correlation template matching module is used for matching and tracking the positions of the regions of interest in all the image sequences through correlation templates accelerated based on the pyramid to obtain initial displacement of the regions of interest;
The dense optical flow estimation module is used for cutting the region of interest in all the image sequences based on the position information of the region of interest, carrying out optical flow estimation of the region of interest by adopting a dense optical flow model based on deep learning, and averaging the optical flows of all pixels in the region of interest to obtain the speed of the region of interest;
The data fusion module is used for carrying out data fusion on the initial displacement of the region of interest and the speed of the region of interest by adopting a Kalman filtering method to obtain the final displacement of the region of interest;
The distortion correction and displacement conversion module is used for selecting an object with known engineering length in the picture as a marker, calculating a scaling factor, carrying out camera distortion correction by utilizing the scaling factor, respectively calculating image scaling factors at different heights, multiplying the image scaling factors by the final displacement of the region of interest to obtain the displacement of the region of interest after physical quantity conversion, and correcting the influence of camera distortion on video shooting quality;
the calculation formula of the scale factor is as follows:
Wherein alpha is a scale factor, D is the engineering length of the marker, and D is the pixel length of the marker in the picture;
And the structural integral displacement analysis module is as follows: the method is used for estimating the integral displacement of the structure by adopting the physical quantity converted interested area displacement of different parts of the structure obtained by the distortion correction and displacement conversion module, analyzing the integral deformation condition of the structure and evaluating the safety of the structure based on a preset structural deformation threshold value.
7. A computer vision structure displacement measurement system terminal comprising a memory, a processor and at least one instruction or at least one computer program stored on the memory and loadable and executable on the processor, characterized in that the processor loads and executes the at least one instruction or the at least one computer program to implement a computer vision structure displacement measurement method as claimed in any one of claims 1 to 5.
8. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements a computer vision structure displacement measurement method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310862680.7A CN117128865B (en) | 2023-07-13 | Computer vision structure displacement measuring method, system, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310862680.7A CN117128865B (en) | 2023-07-13 | Computer vision structure displacement measuring method, system, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117128865A CN117128865A (en) | 2023-11-28 |
| CN117128865B true CN117128865B (en) | 2024-07-02 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109238277A (en) * | 2018-08-01 | 2019-01-18 | 清华大学 | The localization method and device of vision inertial data depth integration |
| CN115638731A (en) * | 2022-09-07 | 2023-01-24 | 清华大学 | Super-resolution-based shaking table test computer vision displacement measurement method |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109238277A (en) * | 2018-08-01 | 2019-01-18 | 清华大学 | The localization method and device of vision inertial data depth integration |
| CN115638731A (en) * | 2022-09-07 | 2023-01-24 | 清华大学 | Super-resolution-based shaking table test computer vision displacement measurement method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11200684B2 (en) | Apparatus and method for measuring flow velocity of stream using optical flow image processing | |
| JP4809291B2 (en) | Measuring device and program | |
| KR101643672B1 (en) | Optical flow tracking method and apparatus | |
| CN104574393B (en) | A kind of three-dimensional pavement crack pattern picture generates system and method | |
| CN109035292B (en) | Moving target detection method and device based on deep learning | |
| JP2006520042A5 (en) | ||
| CN111507132B (en) | Positioning method, device and equipment | |
| CN111127613B (en) | Image sequence three-dimensional reconstruction method and system based on scanning electron microscope | |
| CN114049577A (en) | Fish specification measuring method and system | |
| KR101803340B1 (en) | Visual odometry system and method | |
| CN117128865B (en) | Computer vision structure displacement measuring method, system, terminal and storage medium | |
| CN116778094B (en) | Building deformation monitoring method and device based on optimal viewing angle shooting | |
| CN113034398A (en) | Method and system for eliminating jelly effect in urban surveying and mapping based on artificial intelligence | |
| CN105021573A (en) | Method and device for tracking-based visibility range estimation | |
| CN115830514B (en) | Whole river reach surface flow velocity calculation method and system suitable for curved river channel | |
| CN113917475A (en) | Positioning method, positioning device, electronic equipment and computer readable storage medium | |
| CN117128865A (en) | Computer vision structure displacement measuring method, system, terminal and storage medium | |
| JP2005141655A (en) | Three-dimensional modeling apparatus and three-dimensional modeling method | |
| CN114757848B (en) | Camera jelly effect detection method and system based on artificial intelligence | |
| JP6989819B2 (en) | Data correction device, data correction method, and data correction program | |
| CN114485417B (en) | Structural vibration displacement identification method and system | |
| CN115984592A (en) | Point-line fusion feature matching method based on SuperPoint + SuperGlue | |
| CN116148818A (en) | Assessment method of laser positioning confidence, chip and robot | |
| US20090154771A1 (en) | Automated Determination of Cross-Stream Wind Velocity from an Image Series | |
| CN113916213A (en) | Positioning method, positioning device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |