CN115717865A - Method for measuring full-field deformation of annular structure - Google Patents

Abstract

The invention discloses a method for measuring the full-field deformation of an annular structure, which takes a single panoramic camera as data acquisition equipment, and the data processing process comprises three steps of panoramic image expansion and calibration, automatic node extraction of the structure and sub-pixel calculation of a node center coordinate. The method comprises the following steps of obtaining a distortion-removed plane image covering the whole field range of an annular structure by a method of decomposing and projecting in multiple directions according to an imaging model of a panoramic camera and a cubic projection method, automatically extracting the range of each node by adopting a deep learning network with a micro target detection layer and an attention mechanism, and accurately calculating the coordinates of the nodes by adopting a perceptual hash method clustering method and a method based on sub-pixel straight line detection and fitting. The method provided by the invention overcomes the defect that the common full-field deformation measurement method depends on a complex multi-camera measurement system, and has the advantages of convenience, easy implementation, low cost and high robustness.

Description

Method for measuring full-field deformation of annular structure

Technical Field

The invention belongs to the field of structural health monitoring and measurement, and particularly relates to a panoramic camera and deep learning assisted positioning based method for measuring the full-field deformation of an annular structure.

Background

For an important large stadium, carrying out load test and simulating special construction working conditions on a scale model of the stadium before construction is an important means for analyzing the bearing capacity of the stadium and reasonably planning the construction sequence. In a load test and a construction working condition simulation, the deformation of the structure in the whole field range needs to be accurately measured so as to analyze the deformation state and the mechanical property of the structure. The traditional deformation measurement method such as pasting a strain gauge, arranging a displacement sensor and the like can accurately measure the structural deformation condition and is widely applied to component experiments (particularly concrete components), but the measurement range of the method is limited, a large number of sensors need to be arranged in advance, and the full-field deformation measurement requirement of a large-size scale model under the condition of large deformation cannot be met. The vision-based measuring method is a low-cost and high-precision measuring method which is widely concerned in recent years, and the method utilizes a camera to capture the pixel change of a structure on an imaging plane and obtains the real deformation of the structure according to the corresponding relation between the imaging plane and a three-dimensional object, so that the deformation condition of a large-range structure can be measured by a simple measuring system which is composed of a few cameras and a computer embedded with an algorithm. Aiming at the problem of measuring the deformation of the whole field of a large-size scale model of an annular stadium, the invention provides a low-cost annular structure deformation rapid measuring method based on a panoramic camera and a machine vision algorithm.

With the rapid improvement of camera performance and the gradual improvement of computer vision related algorithms in recent years, various measurement methods for different measurement objects have been developed, and these methods are applied to deformation measurement of bridges up to kilometers from small to micro-electronics detection, and can be generally divided into measurement methods with targets and measurement methods without targets. The measuring method with the target generally needs to paste specially designed targets, such as coding points with various shapes, infrared targets and active light source targets, on the structure in advance, after the camera collects the image containing the target, the position of the target in the image is automatically identified through a coding point identification algorithm or a lens of an optical filter containing light with special wavelength, so that the displacement condition of the target is calculated. The measuring method without target is to set a feature detector according to the geometrical or textural features of the measured object, and then to calculate the change of the detected features according to the matching relationship between the previous and next images, so as to obtain the pixel displacement. The method comprises the steps of extracting the edge of a target by adopting an edge detection method and a straight line detection method so as to calculate the displacement of the target in two frames of images before and after the target is detected, and a digital image correlation-based method. Digital image correlation has become one of the most common visual measurement methods because of its good stability and accuracy.

Two problems of the full-field deformation measurement of the annular stadium scale model are to be solved urgently by applying the vision measurement method. 1. The field of view of a single camera is limited, and all nodes to be measured of the large-size annular scale model cannot be shot at the same time; 2. the scale model is composed of a slender rod piece and a pull rope, the structure is complicated, the texture of the position of a node to be measured is few, and a target is easily lost or mismatching is generated by adopting the existing visual measurement method in a complicated test environment.

Aiming at the measurement problem of a large-size annular structure, a measurement system consisting of multiple cameras can obtain the large-range full-field deformation of the structure. Some scholars propose two spatial data splicing strategies of a multi-camera digital image correlation system, and discuss feasibility of the two strategies in deformation measurement of large-size industrial devices. The scholars propose a multi-camera digital image correlation method and system for measuring large engineered objects with distributed, non-overlapping regions of interest, and apply it to the three-dimensional deformation measurement of a structure spanning 18 m. Some scholars establish a four-camera vision system, study local calibration and global calibration methods related to multiple cameras and a point cloud correction method for optimizing point cloud stitching, and provide a basis for vision application of the multiple-camera system. Some researchers study the performance of the target-based and calibration-based multi-camera three-dimensional DIC method applied to deformation measurement of the slender component, and a set of measuring system comprising 9 cameras is built and applied to deformation measurement of a concrete beam with the length of 900 mm. The research proves that the multi-camera measuring system can give consideration to the advantages of large field of view and high precision when measuring the deformation of a large-size structure, but the multi-camera system has the problems of complex composition, high cost and the need of completing complex calibration before measurement, the optimal measuring field of view of a single camera is fixed, the number of cameras needs to be multiplied to increase the measuring field of view, for a scale model of an annular stadium with the diameter of more than ten meters, more than 10 cameras are needed to form the multi-camera measuring system with mutually overlapped field of view, and the method is undoubtedly complex and high in cost. A panoramic camera is a camera capable of simultaneously acquiring an environment of a surrounding 360 degrees, and with the increasing progress of imaging technology, a low-cost and highly integrated panoramic camera has been widely used in the fields of photography and VR. In structural health testing, some scholars have also proposed applying panoramic cameras or 360 ° panoramas to overcome the limitations of conventional cameras. Some scholars propose a damage condition evaluation method for post-disaster buildings based on a 360-degree panorama of street view service, which automatically identifies and extracts the buildings in the panorama by applying a convolutional neural network based on regions. Some scholars design a structure surface damage detection network based on panoramic images, and the problem that the traditional deep learning network cannot process the panoramic images with high resolution and high distortion and deformation is solved. The panoramic camera has a view field size far exceeding that of a common camera and high resolution, but research on applying the panoramic camera to structural health detection is very little, and research on applying the panoramic camera to measure the full-field deformation of a structure is not yet researched.

Disclosure of Invention

Based on the current situation, the invention provides a full-field deformation measurement method of an annular structure based on a panoramic camera and deep learning auxiliary positioning. Aiming at the problem of full-field deformation measurement of a large-size annular scale model, the method provides that a single panoramic camera is adopted to obtain a 360-degree full-field image, the panoramic image with obvious distortion is projected to the front, back, left, right, upper and lower six directions by adopting a hexahedral model according to the imaging principle of the panoramic camera, and then a distortion-removed image for measurement is obtained by calibrating each direction; (2) Aiming at the measurement problem of node deformation, an attention mechanism-based improved YOLO v5 model is adopted to solve the range detection problem of small nodes, then a perceptual hash algorithm is adopted to cluster images for a large number of detected node ranges, and finally the coordinates of the central point of the nodes are positioned by a sub-pixel straight line detection fitting method, so that the rapid measurement of the full-field node displacement is realized.

In order to solve the technical problems, the technical scheme of the invention is as follows:

the invention provides a full-field deformation measuring method of an annular structure, which comprises the following steps: s1, unfolding and calibrating a panoramic image; s2, automatically extracting a measurement position range; and S3, calculating the actual displacement of each range position.

The specific steps of S1 are as follows: and obtaining a distortion-removed plane image covering the whole field range of the annular structure according to a method of decomposing and projecting a plurality of directions by a panoramic camera imaging model and a cubic projection method.

In S1, for a projection plane tangent to the panoramic spherical surface, a field angle FOV =90 ° is set, and a projected plan view size is (w, h), and a normalized focal length is first estimated:

for a point (u, v) on the projection plane, the transformation to the spherical coordinate system is carried out

Conversion to polar coordinates (theta, r)

The coordinates (U, V) of the panorama are

The specific steps of S2 are as follows: respectively calibrating the pictures projected and expanded in six directions of the panoramic image by adopting an independent calibration method; and shooting a plurality of images in the front, back, left and right directions of the panoramic camera by using a chessboard pattern calibration plate respectively, and calibrating the images in the four directions by adopting a Zhang calibration method after the obtained panoramic images are subjected to projection segmentation.

The specific steps of S3 are as follows: and (3) accurately calculating the coordinates of the nodes by adopting a perceptual hash method clustering and a method based on sub-pixel straight line detection and fitting.

The panoramic image is expanded and calibrated by converting a spherical projection expansion image recorded by a camera into forward projection images of six surfaces, namely front, back, left, right, upper and lower surfaces according to a cylindrical projection and cubic projection method, and based on the forward projection images, the panoramic camera can be considered to be simplified into a camera with a pinhole model in the directions of the six surfaces, so that the calibration is respectively carried out by adopting a Zhang calibration method, and the calibrated result is used for carrying out distortion correction on the images of the six surfaces. After the operation is finished, when the panoramic camera is used for displacement measurement, the proportion relation between the three-dimensional object to be measured and the projected image of any surface can be obtained by adopting the homography matrix for the calibrated image, so that the displacement of the object with the real scale can be calculated by adopting the image.

The automatic extraction of the measurement position range aims to automatically find the position of a key point required to be measured in an image obtained in the last step, and takes the annular cable network structure as an example, the annular cable network structure comprises dozens of connecting nodes, and the problem of automatically extracting the position of the nodes is to improve the automation degree of the whole method. However, the node position of the cable network structure is small, and the occupied pixels in the whole image are few, so that the key problem of the part is how to realize high-precision identification of the small node target. A YOLO v5 model fused with an attention mechanism is provided for identifying small node targets in an image, a Transformer prediction head is added on the basis of the YOLO v5 model to increase the sensitivity of the model to tiny objects, and each node in a cable net can be accurately identified and framed to a certain extent.

After the calculation range of the node position is determined, the last step is to calculate the displacement amount of each ROI respectively, and the key point is to accurately position the coordinates of the node center from the ROI. Because each node is the intersection point of the intersection of the structural rod pieces, the rod pieces are detected by adopting a straight line detection method, and the intersection points of straight lines are fitted, so that the coordinates of the intersection points can be obtained. And combining the image scale parameters calculated in the previous step to obtain the displacement of each node with a real scale.

According to another aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for full-field deformation measurement of a ring structure of the invention.

According to a further aspect of the invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the method for full field deformation measurement of a ring structure of the invention when executing the program.

Compared with the prior art, the invention at least has the following beneficial effects:

the invention relates to a structure full-field deformation measuring method which is a low-cost and non-contact measuring method. The traditional sensor type deformation measurement method such as pasting of strain gauges and arrangement of displacement sensors can accurately measure the structural deformation, but the measurement range of the method is limited, a large number of sensors need to be arranged in advance, and the requirement of full-field deformation measurement of a large-size scale model under the condition of large deformation cannot be met. However, the existing vision measurement method needs to rely on a complex measurement system consisting of a plurality of cameras for acquiring a full-field image of a structure, and has the difficult problem that the displacement of a small target with low texture is difficult to accurately measure.

Compared with a multi-camera measurement system, the proposed panoramic camera solution has the advantages of small complexity and low cost. The node displacement calculation method improves the precision of the convolutional neural network on small target recognition, and overcomes the defect that the low-texture target is easy to lose by a digital image correlation method.

Drawings

To illustrate the technical solutions of the embodiments of the present invention more clearly, the drawings of the embodiments will be briefly introduced, and it is obvious that the drawings in the following description only relate to some embodiments of the present invention, and are not to limit the present invention.

FIG. 1 is a schematic of the inventive process;

FIG. 2 is a panoramic camera imaging principle of the method of the present invention;

FIG. 3 is a projection expansion method of a panoramic image in six directions according to the method of the present invention;

FIG. 4 is a block diagram of a node detection network improved in the method of the present invention;

FIG. 5 is a flow chart of a method of calculating coordinates of a middle node of the method of the present invention;

FIG. 6 is a graph of the results of a node detection network training in the established data set;

figure 7 is a graph comparing the results of the test in the example of the method of the present invention with the results of the test with a total station.

Detailed Description

The present invention is further illustrated by the following specific examples, which should not be construed as limiting the scope of the invention.

The test example is a scale model of a seat stadium, and the structure of the scale model is divided into a main body part with a fixed outer layer and a movable dome at the top. The movable dome is connected with the fixed main body through a flexible cable structure, and the up-and-down movement of the dome is controlled through 6 groups of supports with pulleys. The outer layer body portion has a length of 18m, a width of 16m and a height of 4.5m, and the dome inner ring at the top of the inner layer has a length of 8.6m and a width of 5.8m.

The test aims at simulating the descending process of the intermediate dome in the using process of a stadium, the experimental working condition is that the dome is gradually descended in two steps, each step is descended by about 10cm, and the descending process is that six pulley blocks are manually controlled to relax a traction chain, so that accurate displacement of each node of the intermediate dome is required to be measured. The descent process measures the displacement of the upper, middle and bottom portions of the middle dome structure for a total of 40 nodes. In the test process, a single panoramic camera measuring system is adopted to measure displacement, and coordinates of nodes measured by the total station are used for verifying panoramic measurement results for comparison.

The method comprises the following specific steps:

firstly, panoramic images are collected at equal time intervals for a model in a test process and are used for analysis. The preprocessing of the panoramic image comprises the expansion and distortion removal of the panoramic image, and projected images in four directions of front, back, left and right are obtained after the step is completed. And then detecting the node range by adopting the trained node detection model, and cutting the nodes from the original image according to the detection result to obtain a large number of node images. Aiming at a large number of node images obtained after shearing, firstly, hash fingerprint images are calculated for all nodes, and Hamming distances are calculated for clustering. And then detecting a straight line by adopting a sub-pixel straight line detection algorithm, obtaining a skeleton line at a node by adopting a skeleton line extraction and center line calculation fitting method, and finally selecting an intersection point closest to the midpoint of the image as the midpoint of the node. The pixel coordinates of each node can be obtained through the processing, and the pixel displacement of each node can be obtained by comparing the image results at different times.

And comparing the measurement result of the proposed method with the point-by-point measurement result of the total station, and analyzing the precision of the proposed method. And selecting a panoramic image at each test stage, calculating the displacement of each measuring node, and comparing the result with the result measured by the total station. In the test, the total station is used for measuring the coordinates of the node in a stable state before measurement, an initial state before the test starts, a first descent and a second descent for four times, so that 4 panoramic photos in four states are correspondingly selected, and the result of each measurement is different from the result of the first time, so that the first step can be obtained: node displacement of an initial state and a stable state, and a second step: the first descending and the node displacement of the initial state, the third step: the second drop is compared with the node displacement of the initial state by three displacement curves, as shown in fig. 7. The displacement measurement results of the nodes show that the measurement results of the proposed method in three stages are well matched with the total station, the vertical displacement of the nodes can be correctly reflected, the error analysis result shows that the average error of the data of the proposed method and the total station is 3.7mm, the maximum error is 8mm, the displacement of the nodes is 272mm when the maximum error occurs, and the measurement error is 2.9%, so that the industrial measurement requirement is met.

Example 2:

the computer-readable storage medium of the present embodiment has stored thereon a computer program that, when executed by a processor, implements the steps in the ring structure full-field deformation measurement method of embodiment 1.

The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, and the like, provided on the terminal; further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the terminal.

The computer-readable storage medium of the present embodiment is used to store a computer program and other programs and data required by the terminal, and may also be used to temporarily store data that has been output or is to be output.

Example 3:

the computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the ring structure full-field deformation measurement method of embodiment 1 when executing the program.

In this embodiment, the processor may be a central processing unit, or may also be other general processors, digital signal processors, application specific integrated circuits, ready-made programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like, where a general processor may be a microprocessor or the processor may also be any conventional processor, and the like; the memory may include both read-only memory and random access memory and provide instructions and data to the processor, and a portion of the memory may also include non-volatile random access memory, e.g., the memory may also store device type information.

It will be appreciated by one skilled in the art that the present disclosure of embodiments may be provided as a method, system, or computer program product. Accordingly, the present solution may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present solution may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

While the present solution has been described with reference to flowchart illustrations and/or block diagrams of methods and computer program products according to embodiments of the solution, it should be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions; these computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-only memory (ROM), a Random Access Memory (RAM), or the like.

The examples described herein are merely illustrative of the preferred embodiments of the present invention and do not limit the spirit and scope of the invention, and various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the design concept of the present invention should fall within the protection scope of the present invention.

Claims (7)

1. A full-field deformation measurement method for an annular structure is characterized by comprising the following steps:

s1, unfolding and calibrating a panoramic image;

s2, automatically extracting a measurement position range;

and S3, calculating the actual displacement of each range position.

2. The method according to claim 1, wherein the specific steps of S1 are: and obtaining a distortion-removed plane image covering the whole field range of the annular structure according to a method of decomposing and projecting a plurality of directions by a panoramic camera imaging model and a cubic projection method.

3. Method according to claim 2, characterized in that in S1, for a projection plane tangent to the panoramic sphere, setting the field angle FOV =90 °, the projected plan size is (w, h), first estimating the normalized focal length:

for a point (u, v) on the projection plane, the transformation to the spherical coordinate system is carried out

Conversion to polar coordinates (theta, r)

The coordinates (U, V) of the panorama are

4. The method according to claim 1, wherein the specific steps of S2 are: respectively calibrating the pictures projected and expanded in six directions of the panoramic image by adopting an independent calibration method; and shooting a plurality of images in the front, back, left and right directions of the panoramic camera by using a chessboard pattern calibration plate respectively, and calibrating the images in the four directions by adopting a Zhang calibration method after the obtained panoramic images are subjected to projection segmentation.

5. The method according to claim 1, wherein the specific steps of S3 are: and accurately calculating the coordinates of the nodes by adopting a perceptual hash method for clustering and a method based on sub-pixel straight line detection and fitting.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that: the program is executed by a processor to implement the steps in the method for measuring the full-field deformation of a ring structure according to any one of claims 1 to 5.

7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps in the method for full-field deformation measurement of a ring structure according to any one of claims 1 to 5 when executing the program.

Images