CN112146834B

CN112146834B - Method and device for measuring structural vibration displacement

Info

Publication number: CN112146834B
Application number: CN202011059970.0A
Authority: CN
Inventors: 王保宪; 王凯; 闫朝勃; 赵维刚; 李义强
Original assignee: Shijiazhuang Tiedao University
Current assignee: Shijiazhuang Tiedao University
Priority date: 2020-09-30
Filing date: 2020-09-30
Publication date: 2022-10-14
Anticipated expiration: 2040-09-30
Also published as: CN112146834A

Abstract

The invention is suitable for the technical field of structural health monitoring, and provides a structural vibration displacement measuring method and a device, wherein the method comprises the following steps: performing video sequence decomposition on the obtained red square target video to obtain each frame of image; extracting red channel data in the current frame image three-channel data; performing local dynamic windowing on the current frame according to the acquired initial position information to obtain four windowed images corresponding to four edges of the red square target respectively; calculating to obtain a boundary line segment intercept value in each windowing image; filtering according to a boundary line segment intercept value in each windowing image corresponding to each frame of a preset frame number adjacent to the current frame to obtain a sideline displacement change measurement sequence value; and calculating the actual displacement change value of the red square target according to the sideline displacement change measurement sequence value. According to the invention, local windowing is carried out on 4 sides of the red square target, so that the horizontal or vertical displacement change of the red square target is obtained, the calculation effect is good, and the system real-time performance is strong.

Description

Method and device for measuring structural vibration displacement

Technical Field

The invention belongs to the technical field of structural health monitoring, and particularly relates to a structural vibration displacement measuring method and device.

Background

The structural health monitoring is generally used for measuring the response of a civil structure under the external action of wind, earthquake or vehicle load, and the like, and the displacement response is the key of the structural health monitoring. The current measurement technology for displacement response mainly comprises a direct method and an indirect method. The direct method is to directly measure the structural vibration displacement by adopting a displacement sensor, and when the direct method is used, a reference point needs to be referenced and a point to be measured needs to be close to or contacted with the reference point. The indirect vibration displacement measurement technology comprises radar interference, a GPS and the like, wherein the radar interference technology is convenient to install and high in measurement precision, but the technology can be applied only by needing a relevant reflection surface of a measured object, and the GPS measurement technology is convenient to install, but is low in measurement precision and within an error range of 5-10 mm.

In recent years, machine vision technology is rapidly developed, and has gradually become a research hotspot for structure displacement measurement due to the advantages of high measurement precision, low cost and the like. The displacement measurement method based on the square target can directly extract line side pixels or column side pixels of the square target in a plane, and can quickly calculate the displacement change of the line side or the column side to calculate the displacement change of the measurement target in the vertical direction or the horizontal direction in the plane. However, currently, 2 problems still exist in the displacement measurement method based on the square target: 1) When the target is interfered by uneven illumination, the target object segmentation effect is poor, so that the number of effective pixels on a row edge or a column edge is reduced, the displacement measurement accuracy is reduced, and even the algorithm is invalid; 2) Image processing and calculation of the global area of the square target object are time-consuming, and the real-time performance of the system is poor.

Disclosure of Invention

In view of this, embodiments of the present invention provide a method and an apparatus for measuring structural vibration displacement, which aim to solve the problems of low accuracy and poor real-time performance of vibration displacement measurement in the prior art.

In order to achieve the above object, a first aspect of an embodiment of the present invention provides a structural vibration displacement measurement method, including:

performing video sequence decomposition on the obtained red square target video to obtain each frame of image, wherein the red square target is a red square target arranged at the center of a black square target, and the red square target video is a red square target video shot when the red square target is arranged on a structure to be detected and the structure to be detected vibrates;

extracting red channel data in the current frame image three-channel data;

acquiring initial position information of the red square target, and performing local dynamic windowing on the current frame according to the initial position information to obtain four windowing images corresponding to four edges of the red square target respectively;

calculating to obtain a boundary line segment intercept value in each windowing image according to the red channel data and the four windowing images;

carrying out image filtering in the windowing and motion filtering of a red square target according to a boundary line segment intercept value in each corresponding windowing image in each frame of a preset frame number adjacent to the current frame to obtain a sideline displacement change measurement sequence value;

and calculating the actual displacement change value of the red square target according to the sideline displacement change measurement sequence value and the initial position information.

As another embodiment of the present application, the performing local dynamic windowing on the current frame according to the initial position information to obtain four windowed images corresponding to four edges of the red square target respectively includes:

acquiring the actual side length of the red square target and the actual maximum displacement change of the red square target in the horizontal direction and the vertical direction;

according to

Calculating the displacement of the red square target in the horizontal direction and the vertical direction; wherein d is ₁ Represents the amount of displacement of the red square target in the horizontal direction, d ₂ Represents the amount of displacement of the red square target in the vertical direction, c _x Represents the actual maximum displacement change of the red square target in the horizontal direction, c _y Represents the maximum change of the actual displacement of the red square target in the vertical direction, l _z Coordinate value, l, representing the left side of the red square target _y Coordinate value, l, representing the right side of the red square target _x Coordinate value, l, representing the lower edge of the red square target _s The coordinate value of the upper side of the red square target is represented, and h represents the actual side length of the red square target;

determining the left side of the left windowing image corresponding to the left side of the red square target to be (l) according to the initial position information and the displacement amounts of the red square target in the horizontal direction and the vertical direction _z -d ₁ ) And the right side is (l) _z -d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the right windowing image corresponding to the right side of the red square target as (l) _y -d ₁ ) And the right side is (l) _y +d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the upper edge windowing image corresponding to the upper edge of the red square target as (l) _z +d ₁ ) And the right side is (l) _y -d ₁ ) The upper side is (l) _s -d ₂ ) And the lower side is (l) _s +d ₂ ) (ii) a Determining the left side of the lower edge windowing image corresponding to the lower edge of the red square target as (l) _z +d ₁ ) The right side is (l) _y -d ₁ ) The upper side is (l) _x -d ₂ ) And the lower side is (l) _x +d ₂ )。

As another embodiment of the present application, the calculating, according to the red channel data and the four windowing images, a boundary line segment intercept value in each windowing image includes:

extracting boundary line segment information in each windowing image by adopting an LSD algorithm according to the red channel data and the four windowing images;

and calculating the intercept value of the boundary line segment in each windowing image according to the information of the boundary line segment in each windowing image.

As another embodiment of the present application, the calculating a boundary line segment intercept value in each windowed image according to the boundary line segment information in each windowed image includes:

calculating the length of each line segment, the length proportion of the line segment and the intercept value of the boundary line segment according to the boundary line segment information in the current windowing image;

removing the line segments with the length not larger than a first preset threshold value, and detecting whether the length ratio of the longest line segment in the remaining line segments is larger than a second preset threshold value;

when the length ratio of the longest line segment is greater than a second preset threshold, determining that the intercept value of the boundary line segment in the current windowing image is the intercept value of the boundary line segment corresponding to the longest line segment, and setting a validity identifier as a first identifier;

when the length proportion of the longest line segment is not greater than a second preset threshold, detecting whether a line segment with the length proportion greater than a third preset threshold exists in the rest line segments;

when no line segment with the length ratio larger than a third preset threshold exists, determining that the intercept value of the boundary line segment in the current windowing image is null, and setting a validity identifier as a second identifier;

when a line segment with the line segment length ratio larger than a third preset threshold exists, reserving the line segment with the line segment length ratio larger than the third preset threshold, recalculating the line segment length ratio of each reserved line segment, calculating a boundary line segment intercept value in the current windowing image according to the sum of products of the line segment length ratio of each reserved line segment and the boundary line segment intercept value of the corresponding line segment, and setting a validity identifier as a first identifier;

and calculating the intercept value of the boundary line segment in each windowing image according to the method.

As another embodiment of the present application, the calculating, according to the boundary line segment information in the current windowed image, the length of each line segment, the length ratio of the line segment, and the intercept value of the boundary line segment includes:

according to s _i ＝‖x _i1 –x _i2 ‖+‖y _i1 –y _i2 II, calculating the length of each line segment; wherein s is _i Represents the length of the ith line segment, (x) _i1 ,y _i1 ) (x) an endpoint coordinate representing the ith line segment _i2 ,y _i2 ) Another end point coordinate representing the ith line segment;

according to

Respectively calculating the length ratio of each line segment(ii) a Wherein, w _i The length ratio of the segment of the ith segment is shown, and n represents the number of the segments;

if the current windowing image is a left windowing image or a right windowing image, determining the cross intercept of the line segment in the current windowing image as an intercept value of the boundary line segment; and if the current windowing image is an upper edge windowing image or a lower edge windowing image, determining the longitudinal intercept of the line segment in the current windowing image as the intercept value of the boundary line segment.

As another embodiment of the present application, before removing the line segment whose length is not greater than the first preset threshold, the method further includes:

if the current windowing image is a left windowing image or a right windowing image, according to a =0.25 · (l) _x -l _s -2·d ₂ ) Calculating a first preset threshold value;

if the current windowing image is an upper edge windowing image or a lower edge windowing image, according to a =0.25 · (l) _y -l _z -2·d ₁ ) Calculating a first preset threshold;

wherein a represents a first preset threshold.

As another embodiment of the present application, the performing, according to a boundary line segment intercept value in each windowed image corresponding to each frame of a preset frame number adjacent to a current frame, an image filtering in a window and a motion filtering of a red square target to obtain a sideline displacement change measurement sequence value includes:

acquiring a boundary line segment intercept value and a corresponding identifier in a corresponding first windowing image in each frame of a preset frame number adjacent to the current frame, wherein the first windowing image is any corresponding windowing image in each frame;

detecting whether a mark corresponding to the intercept value of the boundary line segment in the first windowing image of the current frame is a second mark or not;

if the mark corresponding to the intercept value of the boundary line segment in the first windowing image of the current frame is not the second mark, removing a frame farthest from the current frame from the preset frame number adjacent to the current frame according to a first-in first-out rule, and continuously detecting the next frame of the current frame;

if the identifier corresponding to the boundary segment intercept value in the first windowing image of the current frame is the second identifier, calculating the average value of the boundary segment intercept values of the preset frame number adjacent to the current frame as the boundary segment intercept value of the current frame;

calculating a displacement change sequence value of a corresponding segment in a first windowing image of the current frame according to the boundary segment intercept value of the current frame and each boundary segment intercept value corresponding to a preset frame number adjacent to the current frame;

respectively calculating displacement change sequence values of corresponding line segments in other windowed images of the current frame according to the method for calculating the displacement change sequence values of the corresponding line segments in the first windowed image;

and respectively carrying out windowing image fusion in the horizontal direction and the vertical direction according to the displacement change sequence value of the corresponding line segment in the windowing image of the current frame and the identification corresponding to the line segment to obtain a sideline displacement change measurement sequence value.

As another embodiment of the present application, the performing windowing image fusion in the horizontal direction to obtain a sideline displacement change measurement sequence value includes:

if the mark corresponding to the line segment in the left windowing image of the current frame is a first mark and the mark corresponding to the line segment in the right windowing image is a second mark, taking the displacement change value of the corresponding line segment in the left windowing image as a sideline displacement conversion measurement value;

if the mark corresponding to the line segment in the left windowing image of the current frame is a second mark and the mark corresponding to the line segment in the right windowing image is a first mark, taking the displacement change value of the corresponding line segment in the right windowing image as a sideline displacement conversion measurement value;

and if the identification corresponding to the line segment in the left windowing image of the current frame is the same as the identification corresponding to the line segment in the right windowing image, taking the average value of the displacement change value of the corresponding line segment in the left windowing image and the displacement change value of the corresponding line segment in the right windowing image as a sideline displacement conversion measurement value.

As another embodiment of the present application, the calculating an actual displacement change value of the red square target according to the edge displacement change measurement sequence value and the initial position information includes:

according to

Calculating to obtain a proportional value of the actual displacement and the image coordinate; wherein Pw represents a proportional value of the actual displacement and the image coordinate;

and multiplying the proportional values by corresponding sideline displacement change measurement sequence values of the red square target in the horizontal direction and the vertical direction respectively to obtain an actual displacement change value of the red square target.

A second aspect of an embodiment of the present invention provides a structural vibration displacement measurement apparatus, including:

the decomposition module is used for performing video sequence decomposition on the obtained red square target video to obtain each frame image, the red square target is a red square target arranged at the center of a black square target, and the red square target video is a red square target video shot when the red square target is arranged on a structure to be detected and the structure to be detected vibrates;

the processing module is used for extracting red channel data in the current frame image three-channel data;

the acquisition module is used for acquiring initial position information of the red square target;

the processing module is further configured to perform local dynamic windowing on the current frame according to the initial position information to obtain four windowed images corresponding to four edges of the red square target respectively;

the computing module is used for computing to obtain a boundary line segment intercept value in each windowing image according to the red channel data and the four windowing images;

the fusion filtering module is used for carrying out image filtering in the windowing and motion filtering of the red square target according to the boundary line segment intercept value in each corresponding windowing image in each frame of the preset frame number adjacent to the current frame to obtain a sideline displacement change measurement sequence value;

the calculation module is further configured to calculate an actual displacement change value of the red square target according to the sideline displacement change measurement sequence value and the initial position information.

Compared with the prior art, the embodiment of the invention has the following beneficial effects: compared with the prior art, the method has the advantages that the horizontal or vertical displacement change of the red square target can be obtained by locally windowing the 4 sides of the red square target, the calculation effect is good, and the real-time performance of the system is high; when the local windowing operation is carried out, the local windowing size of 4 side lines of the red square target is determined in a self-adaptive mode, the adaptability of the structure vibration displacement measurement method is improved, and the real-time performance of the system is improved. In this embodiment, by performing line detection on the local windowing image and establishing a determination mode of the intercept value of the boundary line segment based on the line detection result, it can be determined whether a line is successfully detected in the current windowing image region. And considering the interferences of target motion blur, uneven illumination and the like, performing fusion filtering by using the processing results of continuous multi-frame images, and outputting the displacement change of the target to be detected in the horizontal or vertical direction by comprehensively considering the linear detection results of the left side and the right side or the upper side and the lower side.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic flow chart illustrating an implementation of a structural vibration displacement measurement method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a red square target provided by an embodiment of the invention;

FIG. 3 (1) is a schematic diagram of a windowed image provided by an embodiment of the invention;

FIG. 3 (2) is a schematic diagram of a windowed image provided by another embodiment of the present invention;

FIG. 4 is an exemplary graph of calculating boundary line segment intercept values within each windowed image provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram of a structural vibration displacement measurement device provided by an embodiment of the present invention;

fig. 6 is a schematic diagram of a terminal device according to an embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

In order to explain the technical means of the present invention, the following description will be given by way of specific examples.

Fig. 1 is a schematic flow chart illustrating an implementation of a structural vibration displacement measurement method according to an embodiment of the present invention, which is described in detail as follows.

And 101, performing video sequence decomposition on the acquired red square target video to obtain each frame of image.

Optionally, the red square target is arranged at the center of the black square target, and the red square target video is the red square target video shot when the red square target is arranged on the structure to be detected and the structure to be detected vibrates. As the red square target shown in figure 2, the middle influence area is a red square target, the white area is a black square target, and in order to avoid the reflection interference of the target surface, the red square target is made of a frosted acrylic plate. And fixing the whole red square target on a structure to be measured, and completing the displacement measurement of the structure to be measured in the horizontal direction (x direction) and the vertical direction (y direction) in the plane of the red square target.

A camera is placed in a certain distance right in front of the red square target. The camera mounting conditions were: 1) Ensuring that the lens surface of the camera is parallel to the red square target surface; 2) The central axis of the lens of the camera is collinear with the central axis of the red square target. The above 2 conditions can ensure that the red square target image shot by the camera is in the center of the whole image, and the red square target is square.

And shooting a red square target video under the condition that the structure to be detected vibrates by using a camera, and storing the red square target video into the local SD card. The method comprises the steps of reading a red square target video stored on an SD card by adopting computer programming, wherein the red square target video mainly comprises a video file header, a data block and an index block. The file header comprises a video frame number, a data format of each frame of image and the like; the data block contains all the image data streams taken; the index chunk includes a list of data chunks and their locations in the file. With this information, each frame image content of the vibration video data can be arbitrarily accessed.

And 102, extracting red channel data in the current frame image three-channel data.

Optionally, the target in the red square target is a red square target, in the red channel data, the pixel gray scale value of the red square target region is generally larger, and the pixel gray scale value of other regions is generally smaller, so that an obvious boundary line exists between the red square target region and the background region, and therefore, in this embodiment, the red channel data in the three-channel data of the current frame image is directly extracted for subsequent calculation.

103, obtaining initial position information of the red square target, and performing local dynamic windowing on the current frame according to the initial position information to obtain four windowed images corresponding to four edges of the red square target respectively.

Optionally, in this step, the position of the red square target in the red square target is known and thus can be directly obtained, as shown in fig. 3 (1) and 3 (2), the initial position information of the red square target includes the left side l _z Right side l _y Upper edge l _s And the lower side l _x The coordinate values of (2).

Optionally, the actual side length h of the red square target and the actual maximum amount c of displacement change of the red square target in the horizontal direction and the vertical direction are obtained _x 、c _y ；

According to

Calculating the displacement amount of the red square target in the horizontal direction and the vertical direction; wherein d is ₁ Represents the amount of displacement of the red square target in the horizontal direction, d ₂ Represents the amount of displacement of the red square target in the vertical direction, c _x Represents the maximum amount of actual displacement change of the red square target in the horizontal direction, c _y Represents the maximum amount of actual displacement change, l, of the red square target in the vertical direction _z Coordinate value, l, representing the left side of the red square target _y Coordinate value, l, representing the right side of the red square target _x Coordinate value, l, representing the lower edge of the red square target _s The coordinate value of the upper side of the red square target is represented, and h represents the actual side length of the red square target;

determining the left side of the left windowing image corresponding to the left side of the red square target to be (l) according to the initial position information and the displacement amounts of the red square target in the horizontal direction and the vertical direction _z -d ₁ ) And the right side is (l) _z -d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the right windowing image corresponding to the right side of the red square target as (l) _y -d ₁ ) And the right side is (l) _y +d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the upper edge windowing image corresponding to the upper edge of the red square target as (l) _z +d ₁ ) The right side is (l) _y -d ₁ ) The upper side is (l) _s -d ₂ ) And the lower side is (l) _s +d ₂ ) (ii) a Determining that the left side of the lower edge windowing image corresponding to the lower edge of the red square target is (l) _z +d ₁ ) The right side is (l) _y -d ₁ ) The upper side is (l) _x -d ₂ ) And the lower side is (l) _x +d ₂ ). As shown in fig. 3 (1) and 3 (2), the left windowing image 1, the right windowing image 2, and the top windowing image may be sequentially acquired from the red channel dataA window image 3 and a bottom-side windowing image 4.

And 104, calculating to obtain a boundary line segment intercept value in each windowing image according to the red channel data and the four windowing images.

Optionally, in this step, a Line Segment Detector (LSD) algorithm may be used to extract boundary Line Segment information in each windowed image according to the red channel data and the four windowed images; and calculating the intercept value of the boundary line segment in each windowing image according to the information of the boundary line segment in each windowing image.

Optionally, after obtaining the windowing images corresponding to the 4 edge lines of each frame of image, it is known that there is only 1 edge line in each windowing image, and the LSD algorithm may be used to extract the edge line information in each windowing image, where the information mainly includes the number of detected line segments in the current area, the end point coordinates of each line segment, and the like. For the convenience of the following description, it is assumed that n line segments are detected in the windowed image, and the coordinates of the end points of the ith line segment are (x) _i1 ,y _i1 ) And (x) _i2 ,y _i2 )。

Optionally, as shown in fig. 4, calculating an intercept value of the boundary line segment in each windowed image according to the information of the boundary line segment in each windowed image may include the following steps.

Step 401, calculating the length of each line segment, the length ratio of the line segments and the intercept value of the boundary line segment according to the information of the boundary line segment in the current windowing image.

Optionally, according to s _i ＝‖x _i1 –x _i2 ‖+‖y _i1 –y _i2 II, calculating the length of each line segment; wherein s is _i Length of ith line segment, (x) _i1 ,y _i1 ) (x) an endpoint coordinate representing the ith line segment _i2 ,y _i2 ) And represents the coordinates of the other end point of the ith line segment.

Optionally, according to

Respectively calculating the length proportion of each line segment; wherein, w _i The segment length ratio of the ith segment is shown, and n is the number of segments.

Optionally, if the current windowing image is a left windowing image or a right windowing image, determining a cross intercept of a line segment in the current windowing image as an intercept value of a boundary line segment; and if the current windowing image is the upper windowing image or the lower windowing image, determining the longitudinal intercept of the line segment in the current windowing image as the intercept value of the boundary line segment.

Step 402, removing the line segment whose length is not greater than the first preset threshold, and detecting whether the length ratio of the longest line segment in the remaining line segments is greater than the second preset threshold.

Optionally, the line segments in each windowed image are filtered, and before filtering, you determine a first preset threshold value by hand. Optionally, if the current windowing image is a left windowing image or a right windowing image, according to a =0.25 · (l) _x -l _s -2·d ₂ ) Calculating a first preset threshold; if the current windowing image is an upper edge windowing image or a lower edge windowing image, according to a =0.25 · (l) _y -l _z -2·d ₁ ) Calculating a first preset threshold; wherein a represents a first preset threshold.

Optionally, the second preset threshold may be set according to an actual requirement, and a value of the second preset threshold is not limited in this embodiment, for example, the second preset threshold may be 0.8.

And executing step 403 when the length ratio of the longest line segment is greater than a second preset threshold, and executing step 404 when the length ratio of the longest line segment is not greater than the second preset threshold.

Step 403, determining that the intercept value of the boundary line segment in the current windowed image is the intercept value of the boundary line segment corresponding to the longest line segment, and setting a validity flag as a first flag.

Alternatively, the first flag may be 1.

And step 404, detecting whether a line segment with a segment length ratio larger than a third preset threshold exists in the remaining line segments.

Optionally, the third preset threshold may be set according to an actual requirement, and a value of the third preset threshold is not limited in this embodiment, for example, the third preset threshold may be 0.3.

When no line segment with the length ratio larger than a third preset threshold exists, executing step 405; when there is a line segment whose segment length ratio is greater than the third preset threshold, step 406 is executed.

Step 405, determining that the intercept value of the boundary line segment in the current windowing image is null, and setting the validity flag as a second flag.

Alternatively, the second flag may be 0.

And step 406, reserving the line segments with the line segment length proportion larger than the third preset threshold, recalculating the line segment length proportion of each reserved line segment, calculating the intercept value of the boundary line segment in the current windowing image according to the sum of the products of the line segment length proportion of each reserved line segment and the intercept value of the boundary line segment of the corresponding line segment, and setting the validity identifier as the first identifier.

The boundary line segment intercept value in each windowing image is calculated according to the method so as to ensure that one boundary line segment intercept value can be output in each windowing image, but effective line segment characteristics can not be detected sometimes, so that fusion filtering is performed according to the step 105.

And 105, filtering the images in the windows and filtering the motion of the red square target according to the intercept value of the boundary line segment in each corresponding windowed image in each frame of the preset frame number adjacent to the current frame to obtain a sideline displacement change measurement sequence value.

Optionally, in this step, 5 frames are selected according to the preset frame number, that is, after 5 frames of image data are continuously processed, a boundary segment intercept value sequence and a corresponding identifier sequence formed by boundary segment intercept values of adjacent 5 frames can be obtained, and the boundary segment intercept value sequence and the identifier sequence are updated according to a first-in first-out updating principle, that is, an image processing result of a next new frame enters the boundary segment intercept value sequence group in sequence, and an image processing result of an earliest frame in the same time sequence group is removed from the sequence group.

The updating method comprises the following steps: acquiring a boundary line segment intercept value and a corresponding identifier in a corresponding first windowing image in each frame of a preset frame number adjacent to the current frame, wherein the first windowing image is any corresponding windowing image in each frame; detecting whether an identifier corresponding to a boundary line segment intercept value in a first windowing image of the current frame is a second identifier; if the mark corresponding to the intercept value of the boundary line segment in the first windowing image of the current frame is not the second mark, removing a frame farthest from the current frame from the preset frame number adjacent to the current frame according to a first-in first-out rule, and continuously detecting the next frame of the current frame; if the identifier corresponding to the boundary segment intercept value in the first windowing image of the current frame is the second identifier, calculating the average value of the boundary segment intercept values of the preset frame number adjacent to the current frame as the boundary segment intercept value of the current frame; and calculating the displacement change sequence value of the corresponding line segment in the first windowing image of the current frame according to the boundary line segment intercept value of the current frame and each boundary line segment intercept value corresponding to the adjacent preset frame number before the current frame.

The method comprises the steps of judging whether a mark corresponding to a boundary line segment intercept value obtained by detecting a first windowed image of a current frame is 0, if the mark is 0, calculating an average value of side line intercepts of adjacent 5 frames in a boundary line segment intercept value sequence group before updating as the boundary line segment intercept value of the first windowed image of the current frame, and if the mark is 1, directly updating the sequence group according to a first-in first-out rule, so that the boundary line segment intercept value corresponding to each windowed image of each frame of image is guaranteed to be an effective value. Assuming that the sequence of intercept values of the boundary line segments obtained by detecting all the frame images is (b 1, b2, \8230; bm), the 1 st value b1 is subtracted from all the sequence values, and the sequence of pixel displacement change corresponding to the edge line is (bo 1, bo2, \8230; bom). Where m denotes the number of frames.

According to the method for calculating the displacement change sequence value of the corresponding line segment in the first windowing image, the displacement change sequence values of the corresponding line segments in other windowing images of the current frame are respectively calculated. And respectively carrying out windowing image fusion in the horizontal direction and the vertical direction according to the displacement change sequence value of the corresponding line segment in the windowing image of the current frame and the identification corresponding to the line segment to obtain a sideline displacement change measurement sequence value.

Optionally, the horizontal displacement change of the target to be measured can be measured simultaneously by processing the left windowing image and the right windowing image, and the vertical displacement change of the target to be measured can be measured simultaneously by processing the upper windowing image and the lower windowing image. After the intra-filtering of the adjacent 5 frames of windowed images is completed, the borderline displacement change measurement sequence values of the left windowed image and the right windowed image can be fused according to the following rule.

Optionally, if the identifier corresponding to the line segment in the left windowing image of the current frame is a first identifier, and the identifier corresponding to the line segment in the right windowing image is a second identifier, taking the displacement change value of the corresponding line segment in the left windowing image as a sideline displacement conversion measurement value;

if the mark corresponding to the line segment in the left windowing image of the current frame is the second mark and the mark corresponding to the line segment in the right windowing image is the first mark, taking the displacement change value of the corresponding line segment in the right windowing image as a sideline displacement conversion measurement value;

and if the mark corresponding to the line segment in the left windowing image of the current frame is the same as the mark corresponding to the line segment in the right windowing image, taking the average value of the displacement change value of the corresponding line segment in the left windowing image and the displacement change value of the corresponding line segment in the right windowing image as the sideline displacement conversion measurement value.

Similarly, the sideline displacement change measurement sequence values of the upper windowing image and the lower windowing image can be fused according to the method. For example, if the identifier corresponding to the line segment in the upper edge windowing image of the current frame is a first identifier, and the identifier corresponding to the line segment in the lower edge windowing image is a second identifier, taking the displacement change value of the corresponding line segment in the upper edge windowing image as a sideline displacement conversion measurement value;

if the identification corresponding to the line segment in the upper edge windowing image of the current frame is the second identification and the identification corresponding to the line segment in the lower edge windowing image is the first identification, taking the displacement change value of the corresponding line segment in the lower edge windowing image as a sideline displacement conversion measurement value;

and if the identification corresponding to the line segment in the upper edge windowing image of the current frame is the same as the identification corresponding to the line segment in the lower edge windowing image, taking the average value of the displacement change value of the corresponding line segment in the upper edge windowing image and the displacement change value of the corresponding line segment in the lower edge windowing image as the sideline displacement conversion measurement value.

And 106, calculating an actual displacement change value of the red square target according to the sideline displacement change measurement sequence value and the initial position information.

Optionally, the step can be according to

Calculating to obtain a proportional value of the actual displacement and the image coordinate; where Pw represents a proportional value of the actual displacement to the image coordinate.

And multiplying the proportional values by the edge line displacement transformation measurement values corresponding to the red square target in the horizontal direction and the vertical direction respectively to obtain the actual displacement change value of the red square target. The actual displacement variation value may be in units of mm.

According to the structural vibration displacement measurement method, local windowing is performed through the 4 side lines of the red square target, horizontal or vertical displacement changes of the red square target can be obtained, the calculation effect is good, and the system real-time performance is improved. When the local windowing operation is carried out, the local windowing size of 4 side lines of the red square target is determined in a self-adaptive mode, and the adaptability of the structure vibration displacement measurement method is improved. In this embodiment, by performing line detection on the local windowing image and establishing a determination mode of the intercept value of the boundary line segment based on the line detection result, it can be determined whether a line is successfully detected in the current windowing image region. And considering the interferences of target motion blur, uneven illumination and the like, performing fusion filtering by using the processing results of continuous multi-frame images, and outputting the displacement change of the target to be detected in the horizontal or vertical direction by comprehensively considering the linear detection results of the left side and the right side or the upper side and the lower side.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

Fig. 5 is a diagram showing an example of a structure vibration displacement measurement apparatus provided in an embodiment of the present invention, which corresponds to the structure vibration displacement measurement method described in the above embodiment. As shown in fig. 5, the apparatus may include: a decomposition module 501, a processing module 502, an acquisition module 503, a calculation module 504, and a fusion filtering module 505.

The decomposition module 501 is configured to perform video sequence decomposition on the acquired red square target video to obtain each frame of image, where the red square target is a red square target arranged in the center of a black square target, and the red square target video is a red square target video shot when the structure to be detected vibrates and the red square target is arranged on the structure to be detected;

a processing module 502, configured to extract red channel data in the current frame image three-channel data;

an obtaining module 503, configured to obtain initial position information of the red square target;

the processing module 502 is further configured to perform local dynamic windowing on the current frame according to the initial position information to obtain four windowed images corresponding to four edges of the red square target respectively;

a calculating module 504, configured to calculate, according to the red channel data and the four windowing images, a boundary line segment intercept value in each windowing image;

a fusion filtering module 505, configured to perform image filtering in a windowing region and motion filtering of a red square target according to a boundary line segment intercept value in each corresponding windowed image in each frame of a preset frame number adjacent to the current frame, so as to obtain a sideline displacement change measurement sequence value;

the calculating module 504 is further configured to calculate an actual displacement change value of the red square target according to the sideline displacement change measurement sequence value and the initial position information.

Optionally, when the processing module 502 performs local dynamic windowing on the current frame according to the initial position information to obtain four windowed images corresponding to four edges of the red square target, the processing module may be configured to:

acquiring the actual side length of the red square target and the actual displacement change maximum of the red square target in the horizontal direction and the vertical direction;

according to

Calculating the displacement of the red square target in the horizontal direction and the vertical direction; wherein d is ₁ Represents the amount of displacement of the red square target in the horizontal direction, d ₂ Represents the amount of displacement of the red square target in the vertical direction, c _x Represents the maximum amount of actual displacement change of the red square target in the horizontal direction, c _y Represents the maximum change of the actual displacement of the red square target in the vertical direction, l _z Coordinate value, l, representing the left side of the red square target _y A coordinate value, l, representing the right side of the red square target _x Coordinate value, l, representing the lower edge of the red square target _s The coordinate value of the upper side of the red square target is represented, and h represents the actual side length of the red square target;

determining the left side of the left windowing image corresponding to the left side of the red square target to be (l) according to the initial position information and the displacement amounts of the red square target in the horizontal direction and the vertical direction _z -d ₁ ) And the right side is (l) _z -d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the right windowing image corresponding to the right side of the red square target as (l) _y -d ₁ ) And the right side is (l) _y +d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the upper edge windowing image corresponding to the upper edge of the red square target as (l) _z +d ₁ ) And the right side is (l) _y -d ₁ ) The upper side is (l) _s -d ₂ ) And the lower side is (l) _s +d ₂ ) (ii) a Determining that the left side of the lower edge windowing image corresponding to the lower edge of the red square target is (l) _z +d ₁ ) The right side is (l) _y -d ₁ ) The upper side is (l) _x -d ₂ ) And the lower side is (l) _x +d ₂ )。

Optionally, when the calculation module 504 calculates to obtain the boundary line segment intercept value in each windowing image according to the red channel data and the four windowing images, it may be configured to:

Optionally, when the calculating module 504 calculates the boundary line segment intercept value in each windowed image according to the boundary line segment information in each windowed image, it may be configured to:

Optionally, when the calculating module 504 calculates the length of each line segment, the length ratio of the line segment, and the intercept value of the boundary line segment according to the information of the boundary line segment in the current windowing image, it may be configured to:

according to s _i ＝‖x _i1 –x _i2 ‖+‖y _i1 –y _i2 II, calculating the length of each line segment; wherein s is _i Length of ith line segment, (x) _i1 ,y _i1 ) (x) an endpoint coordinate representing the ith line segment _i2 ,y _i2 ) Another endpoint coordinate representing the ith line segment;

according to

Respectively calculating the length proportion of each line segment; wherein w _i The length proportion of the line segments of the ith line segment is shown, and n represents the number of the line segments;

if the current windowing image is a left windowing image or a right windowing image, determining the cross intercept of the line segment in the current windowing image as an intercept value of the boundary line segment; and if the current windowing image is the upper windowing image or the lower windowing image, determining the longitudinal intercept of the line segment in the current windowing image as the intercept value of the boundary line segment.

Optionally, the calculating module 504 is further configured to determine that the current windowing image is a left windowing image or a right windowing image according to a =0.25 · (l) _x -l _s -2·d ₂ ) Calculating a first preset threshold value;

if the current windowing image is an upper windowing image or a lower windowing image, according to a =0.25 · (l) _y -l _z -2·d ₁ ) Calculating a first preset threshold value;

wherein a represents a first preset threshold.

Optionally, the fusion filtering module 505 performs image filtering in the windowing and motion filtering of the red square target according to the boundary line segment intercept value in each windowed image corresponding to each frame of the preset frame number adjacent to the current frame, so as to obtain the edge line displacement change measurement sequence value, and may be configured to:

detecting whether an identifier corresponding to a boundary line segment intercept value in a first windowing image of the current frame is a second identifier;

if the mark corresponding to the intercept value of the boundary line segment in the first windowing image of the current frame is not the second mark, removing a frame farthest from the current frame from the preset frame numbers adjacent to the current frame according to a first-in first-out rule, and continuously detecting the next frame of the current frame;

Optionally, the fusion filtering module 505 performs windowing image fusion in the horizontal direction, and when obtaining the sideline displacement change measurement sequence value, may be configured to:

Optionally, when the calculating module 504 calculates the actual displacement change value of the red square target according to the sideline displacement change measurement sequence value and the initial position information, it may be configured to:

according to

and multiplying the proportional values by the edge line displacement transformation measurement values corresponding to the red square target in the horizontal direction and the vertical direction respectively to obtain the actual displacement change value of the red square target.

Above-mentioned structure vibration displacement measurement device carries out local windowing through 4 sidelines of the square mark target of red, can obtain the level or the vertically displacement change of the square target of red, and the computational effect is better, improves system real-time. When the local windowing operation is carried out, the local windowing size of 4 side lines of the red square target is determined in a self-adaptive mode, and the adaptability of the structure vibration displacement measurement method is improved. In this embodiment, by performing line detection on the local windowing image and establishing a determination mode of the intercept value of the boundary line segment based on the line detection result, it can be determined whether a line is successfully detected in the current windowing image region. And considering the interferences of target motion blur, uneven illumination and the like, performing fusion filtering by using the processing results of continuous multi-frame images, and outputting the displacement change of the target to be detected in the horizontal or vertical direction by comprehensively considering the linear detection results of the left side and the right side or the upper side and the lower side.

Fig. 6 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 6, the terminal device 600 of this embodiment includes: a processor 601, a memory 602, and a computer program 603, such as a structural vibration displacement measurement program, stored in the memory 602 and executable on the processor 601. The processor 601 executes the computer program 603 to implement the steps in the above-mentioned structural vibration displacement measurement method embodiment, such as the steps 101 to 106 shown in fig. 1, or the steps shown in fig. 4, and the processor 601 executes the computer program 603 to implement the functions of the modules in the above-mentioned device embodiments, such as the functions of the modules 501 to 505 shown in fig. 5.

Illustratively, the computer program 603 may be partitioned into one or more program modules, which are stored in the memory 602 and executed by the processor 601 to implement the present invention. The one or more program modules may be a series of computer program instruction segments capable of performing certain functions to describe the execution of the computer program 603 in the structural vibration displacement measurement device or terminal apparatus 600. For example, the computer program 603 may be divided into a decomposition module 501, a processing module 502, an obtaining module 503, a calculating module 504, and a fusion filtering module 505, and specific functions of the modules are shown in fig. 5, which are not described herein again.

The terminal device 600 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 601, a memory 602. Those skilled in the art will appreciate that fig. 6 is merely an example of a terminal device 600, and does not constitute a limitation of terminal device 600, and may include more or fewer components than shown, or some of the components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.

Processor 601 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 602 may be an internal storage unit of the terminal device 600, such as a hard disk or a memory of the terminal device 600. The memory 602 may also be an external storage device of the terminal device 600, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 600. Further, the memory 602 may also include both an internal storage unit and an external storage device of the terminal apparatus 600. The memory 602 is used for storing the computer programs and other programs and data required by the terminal device 600. The memory 602 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the present application. For the specific working processes of the units and modules in the system, reference may be made to the corresponding processes in the foregoing method embodiments, which are not described herein again.

In the above embodiments, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described or recited in any embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one type of logical function division, and other division manners may be available in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments described above may be implemented. . Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer-readable medium may contain suitable additions or subtractions depending on the requirements of legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer-readable media may not include electrical carrier signals or telecommunication signals in accordance with legislation and patent practice.

The above-mentioned embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims

1. A method of measuring vibration displacement of a structure, comprising:

performing video sequence decomposition on the obtained red square target video to obtain each frame image, wherein the red square target is a red square target arranged at the center of a black square target, and the red square target video is a red square target video shot when the red square target is arranged on a structure to be detected and the structure to be detected vibrates;

extracting red channel data in the current frame image three-channel data;

2. The method for measuring structural vibration displacement according to claim 1, wherein the performing local dynamic windowing on the current frame according to the initial position information to obtain four windowed images corresponding to four edges of the red square target respectively comprises:

according to

Calculating the displacement of the red square target in the horizontal direction and the vertical direction; wherein, d ₁ Represents the amount of displacement of the red square target in the horizontal direction, d ₂ Represents the amount of displacement of the red square target in the vertical direction, c _x Represents the maximum amount of actual displacement change of the red square target in the horizontal direction, c _y Represents the maximum amount of actual displacement change, l, of the red square target in the vertical direction _z Coordinate value, l, representing the left side of the red square target _y Coordinate value, l, representing the right side of the red square target _x A coordinate value, l, representing the lower side of the red square target _s The coordinate value of the upper side of the red square target is represented, and h represents the actual side length of the red square target;

according to the initial position information and the displacement amounts of the red square target in the horizontal direction and the vertical direction, determining that the left side of the left windowing image corresponding to the left side of the red square target is (l) _z -d ₁ ) And the right side is (l) _z -d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the right windowing image corresponding to the right side of the red square target as (l) _y -d ₁ ) And the right side is (l) _y +d ₁ ) The upper side is (l) _s +d ₂ ) And the lower side is (l) _x -d ₂ ) (ii) a Determining the left side of the upper edge windowing image corresponding to the upper edge of the red square target as (l) _z +d ₁ ) And the right side is (l) _y -d ₁ ) The upper side is (l) _s -d ₂ ) And the lower side is (l) _s +d ₂ ) (ii) a Determining the left side of the lower edge windowing image corresponding to the lower edge of the red square target as (l) _z +d ₁ ) And the right side is (l) _y -d ₁ ) The upper side is (l) _x -d ₂ ) And the lower side is (l) _x +d ₂ )。

3. The structural vibration displacement measurement method of claim 1 or 2, wherein the calculating a boundary line segment intercept value in each windowed image according to the red channel data and the four windowed images comprises:

4. The method of measuring structural vibration displacement according to claim 3, wherein the calculating a boundary line segment intercept value in each windowed image based on the boundary line segment information in each windowed image comprises:

when no line segment with the length ratio larger than a third preset threshold value exists, determining that the intercept value of the boundary line segment in the current windowing image is null, and setting an effectiveness identifier as a second identifier;

5. The method of measuring structural vibration displacement according to claim 4, wherein the calculating each line segment length, line segment length ratio and boundary line segment intercept value according to the boundary line segment information in the current windowed image comprises:

according to

Respectively calculating the length proportion of each line segment; wherein, w _i The length ratio of the segment of the ith segment is shown, and n represents the number of the segments;

if the current windowing image is a left windowing image or a right windowing image, determining the cross intercept of the line segment in the current windowing image as the intercept value of the boundary line segment; and if the current windowing image is the upper windowing image or the lower windowing image, determining the longitudinal intercept of the line segment in the current windowing image as the intercept value of the boundary line segment.

6. The method of measuring vibration displacement of a structure of claim 4, wherein prior to said removing segments having a segment length not greater than a first predetermined threshold, further comprising:

if the current windowing image is a left windowing image or a right windowing imageAccording to a =0.25 · (l) _x -l _s -2·d ₂ ) Calculating a first preset threshold value;

if the current windowing image is an upper windowing image or a lower windowing image, according to a =0.25 · (l) _y -l _z -2·d ₁ ) Calculating a first preset threshold;

wherein a represents a first preset threshold.

7. The structural vibration displacement measurement method according to claim 4, wherein the obtaining of the borderline displacement change measurement sequence value by performing image filtering in windowing and motion filtering of the red square target according to the boundary line segment intercept value in each of the windowed images corresponding to each of the preset number of frames adjacent to the current frame comprises:

8. The method for measuring structural vibration displacement according to claim 7, wherein the performing horizontal windowing image fusion to obtain the sideline displacement change measurement sequence value comprises:

9. The method as claimed in claim 7, wherein the calculating an actual displacement variation value of the red square target according to the sideline displacement variation measurement sequence value and the initial position information comprises:

according to

10. A structural vibration displacement measurement device, comprising: