CN117490579A - Foundation pit displacement monitoring system based on image vision processing - Google Patents

Abstract

The invention discloses a foundation pit displacement monitoring system based on image visual processing, which includes an image sensing module, a data processing module, a monitoring module and an intelligent early warning module. The data processing module includes sub-modules such as image distortion calibration, intelligent division and extraction of image pixels, and high-precision calculation of image pixel displacement. The distortion-calibrated image is processed through a machine vision algorithm based on deep learning to realize the extraction of pixels, regional division, and calculation of the average displacement of the pixel region based on time changes. The foundation pit displacement monitoring system based on image visual processing provided by the present invention adopts an integrated design and integrates image collection, data processing, monitoring visualization and intelligent detection and early warning. It is not only easy to operate and low in cost, but also can monitor intelligently in real time. The displacement inside the foundation pit is an efficient and reliable foundation pit monitoring solution.

Description

A foundation pit displacement monitoring system based on image vision processing

Technical field

The invention belongs to the field of machine vision, and in particular relates to a foundation pit displacement monitoring system based on image vision processing.

Background technique

Geometric deformation monitoring is an important method for discovering abnormal behaviors of buildings and structures. In actual engineering projects in the past, engineers mainly relied on manual monitoring and geodetic measuring instruments to achieve relevant monitoring; currently, in actual projects, engineers often use theodolite, total station and other related professional measuring instruments to monitor foundation pit displacement. However, although manual monitoring is widely used based on the above, it has shortcomings such as cumbersome operation in the actual process, high labor costs, data errors caused by human factors, and the inability to monitor and obtain data in real time.

As engineering sites become increasingly complex and construction requirements become higher and higher, in order to ensure the safety of the construction environment and the surrounding environment, the monitoring requirements of engineering projects are also constantly increasing. With the continuous development of machine vision-related fields, in response to the monitoring requirements required in complex environments, improving monitoring efficiency and accuracy and reducing monitoring costs are the main development directions of actual engineering monitoring. Therefore, a highly intelligent and low-cost monitoring system is needed. system.

Contents of the invention

In order to overcome the above-mentioned defects and shortcomings of existing related monitoring technologies, the present invention provides a foundation pit displacement monitoring system based on image vision processing.

The technical solution of the present invention is:

The invention discloses a foundation pit displacement monitoring system based on image visual processing, which includes an image sensing module, a data processing module, a monitoring module and an intelligent early warning module. The image sensing module uses LED lighting to illuminate the internal surface of the target foundation pit, and captures internal images of the foundation pit through high-precision industrial cameras. These images are then transmitted to the data processing module through wired or wireless means. The data processing module includes sub-modules such as image distortion calibration, intelligent division and extraction of image pixels, and high-precision calculation of image pixel displacement. The distortion-calibrated image is processed through a machine vision algorithm based on deep learning to realize the extraction of pixels, regional division, and calculation of the average displacement of the pixel region based on time changes. In this way, the average displacement value of the foundation pit can be accurately obtained and transmitted to the monitoring module through wired or wireless means for operators to view.

The monitoring module consists of a computer host terminal and a mobile device terminal. Workers can not only conduct visual data analysis through computer host terminals, but also use 5G wireless transmission technology to view data in real time through mobile devices. The intelligent early warning module monitors the average pixel displacement value obtained by the data processing module according to the set pixel displacement threshold range. Once the displacement exceeds the threshold, an early warning will be triggered. If the monitoring data is abnormal, the error module will also be triggered. The intelligent early warning module is equipped with an embedded visualization window in the monitoring module.

The foundation pit displacement monitoring system based on image visual processing provided by the present invention adopts an integrated design and integrates image acquisition, data processing, monitoring visualization and intelligent detection and early warning. It is not only easy to operate and low in cost, but also can intelligently monitor the displacement inside the foundation pit in real time. It is an efficient and reliable foundation pit monitoring solution.

Preferably, the image sensing module uses LED lighting to illuminate the internal surface of the target foundation pit and a high-contrast black and white checkerboard for target identification, and captures the internal images of the foundation pit through an industrial camera. These images are then passed through a wired or Wirelessly transmitted to the data processing module. The image sensing module specifically includes a solar power supply module, an LED lighting sub-module and an industrial camera shooting sub-module. The solar power supply module is composed of solar panels and batteries, and can use solar energy to provide continuous and environmentally friendly power support for the entire monitoring system. The solar panels efficiently convert sunlight into electrical energy, and the battery ensures that the system can operate stably even under no light conditions; the LED lighting sub-module and LED lights have the characteristics of high brightness and low energy consumption, which can ensure that the system can be used in various situations. Under lighting conditions, it can provide uniform and sufficient illumination for the interior of the foundation pit and the target checkerboard markers, thereby ensuring the clarity and accuracy of the captured images; the industrial camera shooting sub-module, including an industrial camera, is specially used to capture the foundation High-quality images of the pit's interior. These images will be used as the main analysis objects of the data processing module for subsequent displacement monitoring and analysis. The use of industrial cameras ensures the details and accuracy of image data, providing a reliable foundation for subsequent data processing.

Further preferably, the data processing module includes an image distortion calibration module, an intelligent division and extraction of image pixels, and an image pixel displacement calculation module.

Further preferably, the image distortion calibration module aims to correct the distortion problem of the captured internal surface image of the foundation pit. This module is based on the position of the LED lighting points and combined with Zhang Zhengyou's checkerboard calibration method to perform distortion calibration on the image. Industrial camera Calibrate based on its own parameters.

Further preferably, in view of the special needs of the foundation pit scene, based on the requirements of Zhang Zhengyou's checkerboard calibration method, a high-contrast black and white two-color checkerboard calibration plate is used to ensure its clarity and recognition rate during the industrial camera capture process. To do this, we settled on a 5×5 checkerboard layout designed to maximize image capture accuracy.

It is further preferred to design the LED lighting distribution plan based on the checkerboard layout. A total of 9 LED lighting lights are set up to ensure that the calibration plate can be fully and evenly illuminated. In this way, the uniformity and effectiveness of the lighting are taken into consideration, and the particularity of the foundation pit environment is also fully considered, ensuring the quality of image capture while maintaining the economy and practicality of the system.

Further preferably, the relevant implementation process and calculation process specifically for image distortion calibration are as follows:

Step 1: Use an industrial camera to capture a black and white two-color checkerboard calibration plate image with LED lighting from a fixed angle. The key to this method lies in the identification and tracking of feature points in the image processing stage. In particular, focus on the checkerboard corner points with LED lighting. LED lighting provides a clear visual marking environment, which greatly facilitates the accurate identification of feature points, thereby improving the accuracy of identifying marked points.

Step 2: Capture the characteristic point coordinates of the LED lighting and substitute them into the distortion parameters of the industrial camera. These parameters are then used to calibrate the image to correct any distortion that may be present. In order to verify the effect of calibration, we select other feature points based on the calibration results for further confirmation.

Image distortion correction steps are:

Step 1: Identify the corner point coordinates of the checkerboard with LED lighting: LED lighting provides more obvious visual markers, helping to identify feature points more accurately;

Step 2: Calculate the distortion parameters of the industrial camera: Use the identified coordinates of the LED lighting feature points to calculate the parameters of the industrial camera, including the internal parameter matrix and distortion coefficient;

Step 3: Perform distortion correction on the pixels of the captured checkerboard image based on the calculated relevant parameters. The calculated internal parameter matrix and distortion parameters are used to accurately correct the distortion of the image pixels.

For step 1, based on the known coordinates of the marker point where the LED lighting is located under normal circumstances, it is necessary to determine the coordinates of the marker point of the LED lighting in the captured image, that is, assuming that I(x,y) represents the image at point (x,y) Brightness value, threshold is set to T. For each detected corner point (x _i ,y _i ), the following calculation is performed:

Calculate the brightness of a corner point: , where N is the number of pixels in the area of the corner point (x _i , y _i );

If L > T, then (x _i ,y _i ) are the coordinates of the LED lighting lamp;

Note: The threshold T needs to be adjusted according to the lighting conditions of the specific environment.

For step 2, after determining the coordinates of the marker points of the LED lighting lamp in the captured image, use the checkerboard image captured by the industrial camera to establish the correspondence between the physical coordinate system and the image coordinate system. First, by analyzing the checkerboard image, the position of each point in the physical space and their corresponding position in the image are determined. Based on this coordinate relationship, the distortion coefficient of the industrial camera can be calculated. The distortion coefficient is a quantitative expression of the inherent distortion of the industrial camera lens, including radial distortion and tangential distortion. Using the calculated distortion coefficients, we perform distortion correction on the captured images.

Among them, the camera internal parameter matrix A:

,

Among them, f _x and f _y are the focal lengths, c _x and c _y are the center coordinates of the image;

The distortion coefficients include: radial distortion coefficients k ₁ , k ₂ , ... and tangential distortion coefficients p ₁ , p ₂ ;

For the distortion coefficient, an optimization algorithm is usually used to calculate it during the camera calibration process. In the present invention, the least squares method is used to obtain the camera distortion coefficient. The calculation formula is as follows:

,

x _ij is the image coordinate of the j-th corner point in the i-th image; n represents the number of images, and m represents the number of corner points;

X _ij is the corresponding physical coordinate;

A is the internal parameter matrix of the camera;

k ₁ , k ₂ are radial distortion coefficients, p ₁ , p ₂ are tangential distortion coefficients;

R,t the rotation and translation matrices of the camera.

Checkerboard image correction based on distortion coefficient, for each pixel (x, y) in the image:

Step 1: Radial Distortion Correction

_xRD = x(1 + k ₁ r ² + k ₂ r ⁴ + …)

y _RD = y(1 + k ₁ r ² + k ₂ r ⁴ + …)

Among them, r ² = x ² + y ² , x, y are the image pixel coordinates before distortion; x _RD , y _RD are the image pixel coordinates after radial distortion calibration; where RD is the abbreviation of Radial Distortion.

Step 2: Tangential distortion correction

x _TD = x _RD + [2p ₁ y _RD + p ₂ (r ² + 2 x ² _RD )]

y _TD = y _RD + [2p ₂ x _RD + p ₁ (r ² + 2 y ² _RD )]

Among them, x _TD and y _TD are the image pixel coordinates after radial distortion calibration; where TD is the abbreviation of tangential distortion (TangentialDistortion).

Further preferably, the image pixel intelligent division and extraction module can operate according to the following principles and calculation steps:

Intelligent division and extraction of pixels:

Step 1: Regional division. The calibrated checkerboard image is still divided into areas according to the previous black and white colors, and is divided into a total of 25 areas. Each area will represent a small grid of the checkerboard.

Step 2: Calculate area size. Assuming that the pixel resolution of the image is W×H (width×height), the pixel size of each region is calculated as (W/5)×(H/5).

Step 3: Area number. These 25 areas are numbered, and the numbering method is from left to right and from top to bottom. For the i-th area, extract all pixels within the coordinate range [((i-1) mod 5)×(W/5), ((i-1) ÷ 5)×(H/5)], Where mod is the modulus operator.

Step 4: Image pixel extraction. For each area, all pixels in the area are extracted based on deep learning machine vision related methods.

For step four, the image pixel points are extracted using deep learning machine vision-related algorithms to extract pixel points in each area of the checkerboard. This process can be divided into several steps: feature learning, area recognition, and pixel extraction. The following are related concepts and extraction formulas:

Feature learning: Use CNN to learn features from images. The formula is as follows:

,

Among them, x is the input image, W is the weight of the convolution kernel, b is the bias term, * represents the convolution operation, and ReLU is the activation function;

Area identification: used to locate each area of the checkerboard, the formula is as follows:

,

Among them, F represents the proposed area, is the output of the convolutional layer; RegionProposalNetwork represents the candidate region generation network;

Pixel extraction: Perform pixel-level processing on each area to extract pixels in the area. The formula is as follows, where SegmentationNetwork represents the segmentation network:

,

Among them, P represents the extracted pixels, and F is the identified area.

Integrating the above step formulas, the specific formula is:

,

Further preferably, in the image pixel displacement calculation module, in order to accurately calculate the foundation pit displacement value, we propose a pixel displacement calculation method based on two images: one is the image taken for the first time after distortion correction, and the other is The images are taken some time after they were first taken. By analyzing these two images, we were able to measure the displacement of the foundation pit. It can be operated according to the following principles and calculation steps:

Step 1: Pixel displacement calculation. For each pixel, we calculate its displacement between the two images, expressed as (△x, △y). △x and △y are the movement distance of the pixel on the X-axis and Y-axis respectively.

Step 2: Calculate the average displacement within the area. For each region in the checkerboard, we calculate the average of the displacements of all pixels in the region. This average will represent the overall pixel displacement in the area.

For the average displacement of the i-th area, the calculation formula is as follows:

average displacement ,/> ,

Among them, n is the total number of pixels in the i-th area, and/> is the displacement of each pixel in the i-th area.

Step 3: Based on the camera calibration and internal parameter matrix, determine the geometric relationship between the camera and the monitoring target, including distance and angle, to establish the relationship between the world coordinate system (actual physical space) and the image coordinate system (pixel space).

Assume that P _world = (X, Y, Z) is a point in the world coordinate system, and P _image = (x, y) is the corresponding pixel point in the image coordinate system. The conversion relationship can be expressed as:

P _image = A · [ R丨t ] · P _world

Among them, A is the internal parameter matrix of the camera, and R丨t is the rotation and translation matrix from the world coordinate system to the camera coordinate system.

Step 4: Judgment of foundation pit displacement. The average displacement of each area is used to represent the overall displacement of the area, and the overall displacement of the foundation pit is judged based on the relationship between the average displacement, the world coordinate system and the image coordinate system.

Further preferably, the monitoring module includes a computer host terminal module and a mobile device terminal module; the computer host terminal module is composed of a wired transmission device, a computer host and a display sub-module. Through wired transmission equipment, specific physical displacement data and analysis results are directly transmitted and stored in the disk of the computer host. Through the connected display, these data and analysis results can be presented to relevant personnel clearly and intuitively; the mobile device terminal module is based on high-speed wireless transmission technology such as 5G. This module allows multiple parties to view relevant data and analysis results anytime and anywhere through mobile devices. This wireless connection increases the flexibility of the monitoring system and improves the convenience of data access.

Further preferably, the intelligent early warning module includes an image pixel displacement threshold setting module, a super-threshold early warning module and a monitoring data mutation error reporting module; the image pixel displacement threshold setting module performs specific physical displacement data based on a preset threshold. Compare and filter. By setting a reasonable displacement threshold, abnormal displacements that may cause structural risks can be effectively identified; the super-threshold early warning module is based on the screening results of the image pixel displacement threshold setting module. When the monitored displacement exceeds the set threshold When the alarm occurs, this module will trigger an early warning prompt; the monitoring data mutation error reporting module is specifically responsible for error reporting of abnormal over-threshold warning data, making it convenient for multiple staff to monitor and repair foundation pits.

Further preferably, a highly intelligent foundation pit displacement acquisition equipment that integrates image acquisition, data processing, monitoring visualization and intelligent detection and early warning can achieve real-time intelligent monitoring of the internal displacement of the foundation pit.

The advantages of the present invention are:

1. The foundation pit displacement monitoring system based on image visual processing provided by the present invention is highly integrated with the system and can provide intelligent and convenient monitoring operations for all relevant staff at all levels. At the same time, the data is lightweight and easy to view anytime and anywhere. Relevant data, understanding the progress of foundation pit related operations and changes in the internal displacement of the foundation pit.

2. The foundation pit displacement monitoring system based on image vision processing provided by the present invention can eliminate the displacement error of the high-definition industrial camera itself due to the external environment through multi-point displacement monitoring.

3. The foundation pit displacement monitoring system based on image vision processing provided by the present invention uses related supporting equipment at low cost, has a higher labor cost advantage than traditional technology, greatly improves work monitoring efficiency, and has a higher overall value.

Description of the drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the invention. Also throughout the drawings, the same reference characters are used to designate the same components. In the attached picture:

Figure 1 is a schematic structural diagram of the foundation pit displacement monitoring system based on image vision processing according to the present invention;

Figure 2 is a schematic diagram of the black and white two-color checkerboard calibration board according to the present invention;

Figure 3 is a flow chart of the image distortion calibration module according to the present invention;

Figure 4 is a flow chart of the pixel extraction module according to the present invention;

Figure 5 is a flow chart of pixel displacement calculation according to the present invention;

Figure 6 is a flow chart of the intelligent early warning module according to the present invention;

Figure 7 is a schematic diagram of the working process of the foundation pit displacement monitoring system based on image vision processing.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a thorough understanding of the disclosure, and to fully convey the scope of the disclosure to those skilled in the art.

As shown in Figure 1, the foundation pit displacement monitoring system based on image vision processing disclosed by the present invention includes an image sensing module, a data processing module, a monitoring module and an intelligent early warning module. The entire system has a high degree of intelligence and low overall cost. , with the advantages of simple operation and real-time monitoring.

The invention provides a foundation pit displacement monitoring system based on image visual processing. The system is composed of an image sensing module, a data processing module, a monitoring module and an intelligent early warning module, and is designed to achieve high-precision and real-time monitoring of foundation pit displacement. .

Image sensing module: This module uses LED light sources to illuminate the internal surface of the target foundation pit, and is equipped with an industrial camera to collect images of the corresponding checkerboard calibration plate inside the foundation pit. The collected picture information is transmitted to the data processing module through wired or wireless means, ensuring the clarity and accuracy of the image information.

Data processing module: This module uses machine vision-based algorithms, including image distortion calibration, image pixel extraction, pixel area division, and calculation of the average displacement of the pixel area based on time changes. The application of these techniques makes it possible to calculate the average displacement value of the pit area from the pictures taken. In addition, the data processing module has the characteristics of lightweight data and can quickly upload monitoring data in real time.

Monitoring module: This module includes a computer host terminal and a mobile device terminal, allowing operators to analyze and view visual data results through the computer host terminal. At the same time, based on 5G wireless transmission technology, operators can also view real-time data through mobile device terminals, increasing the flexibility and convenience of the monitoring system.

Intelligent early warning module: Based on the set pixel displacement threshold range, the intelligent early warning module monitors the average pixel displacement value obtained by the data processing module. Once the monitoring data exceeds the threshold, the early warning module will be triggered. If abnormal data is detected, the error reporting module will also be triggered. The threshold value is determined according to the specific requirements of the specific foundation pit. The intelligent early warning module has an embedded visual window in the monitoring module to provide workers with immediate safety warnings.

Specifically, the method of the foundation pit displacement monitoring system based on image vision according to the present invention includes the following contents. Figure 7 is a schematic diagram of the working process of the foundation pit displacement monitoring system based on image vision processing.

1. Foundation pit displacement monitoring system based on image vision processing. The data processing module includes an image distortion calibration module, an intelligent division and extraction of image pixels, and an image pixel displacement calculation module.

2. The image distortion calibration module aims to correct the distortion problem of the captured foundation pit surface image. This module is based on the position of LED lighting points and combined with Zhang Zhengyou's checkerboard calibration method to perform distortion calibration on the image. The industrial camera is based on its own parameters. Perform calibration.

3. In view of the special needs of the foundation pit scene, based on the requirements of Zhang Zhengyou's checkerboard calibration method, a high-contrast black and white two-color checkerboard calibration plate is used to ensure its clarity and recognition rate during the industrial camera capture process. To do this, we settled on a 5×5 checkerboard layout designed to maximize image capture accuracy.

4. Design the LED lighting layout plan based on the checkerboard layout. A total of 9 LED lighting lights are set up to ensure that the calibration plate can be fully and evenly illuminated. In this way, the uniformity and effectiveness of the lighting are taken into consideration, and the particularity of the foundation pit environment is also fully considered, ensuring the quality of image capture while maintaining the economy and practicality of the system. The black and white two-color checkerboard and LED lighting layout scheme is shown in Figure 2, the schematic diagram of the black and white two-color checkerboard calibration board.

5. The specific implementation and calculation processes for image distortion calibration are as follows:

Step 1: Use an industrial camera to capture a black and white two-color checkerboard calibration plate image with LED lighting from a fixed angle. The key to this method lies in the identification and tracking of feature points in the image processing stage. In particular, focus on the checkerboard corner points with LED lighting. LED lighting provides a clear visual marking environment, which greatly facilitates the accurate identification of feature points, thereby improving the accuracy of identifying marked points.

Step 2: Capture the characteristic point coordinates of the LED lighting and substitute them into the distortion parameters of the industrial camera. These parameters are then used to calibrate the image to correct any distortion that may be present. In order to verify the effect of calibration, we select other feature points based on the calibration results for further confirmation.

Image distortion correction steps are:

Step 1: Identify the corner point coordinates of the checkerboard with LED lighting: LED lighting provides more obvious visual markers, helping to identify feature points more accurately;

Step 2: Calculate the distortion parameters of the industrial camera: Use the identified coordinates of the LED lighting feature points to calculate the parameters of the industrial camera, including the internal parameter matrix and distortion coefficient.

Step 3: Perform distortion correction on the pixels of the captured checkerboard image based on the calculated relevant parameters. The calculated internal parameter matrix and distortion parameters are used to accurately correct the distortion of the image pixels.

For step 1, based on the known coordinates of the marker point where the LED lighting is located under normal circumstances, it is necessary to determine the coordinates of the marker point of the LED lighting in the captured image, that is, assuming that I(x,y) represents the image at point (x,y) Brightness value, the threshold is set to T. For each detected corner point (x _i ,y _i ), the following calculation is performed:

Calculate the brightness of a corner point: , where N is the number of pixels in the area of the corner point (x _i , y _i );

If L > T, then (x _i ,y _i ) are the coordinates of the LED lighting lamp;

Note: The threshold T needs to be adjusted according to the lighting conditions of the specific environment.

For step 2, after determining the coordinates of the marker points of the LED lighting lamp in the captured image, use the checkerboard image captured by the industrial camera to establish the correspondence between the physical coordinate system and the image coordinate system. First, by analyzing the checkerboard image, the position of each point in the physical space and their corresponding position in the image are determined. Based on this coordinate relationship, the distortion coefficient of the industrial camera can be calculated. The distortion coefficient is a quantitative expression of the inherent distortion of the industrial camera lens, including radial distortion and tangential distortion. Using the calculated distortion coefficients, we perform distortion correction on the captured images.

Among them, the camera internal parameter matrix A:

Among them, f _x and f _y are the focal length, c _x and c _y are the center coordinates of the image; distortion coefficients: radial distortion coefficients k ₁ , k ₂ ,...and tangential distortion coefficients p ₁ , p ₂ .

For the distortion coefficient, an optimization algorithm is usually used to calculate it during the camera calibration process. In the present invention, the least squares method is used to obtain the camera distortion coefficient. The calculation formula is as follows:

,

x _ij is the image coordinate of the j-th corner point in the i-th image; n represents the number of images, and m represents the number of corner points;

X _ij is the corresponding physical coordinate;

A is the internal parameter matrix of the camera;

k ₁ , k ₂ are radial distortion coefficients, p ₁ , p ₂ are tangential distortion coefficients;

R,t the rotation and translation matrices of the camera.

Checkerboard image correction based on distortion coefficient, for each pixel (x, y) in the image:

Step 1: Radial Distortion Correction

_xRD = x(1 + k ₁ r ² + k ₂ r ⁴ + …)

y _RD = y(1 + k ₁ r ² + k ₂ r ⁴ + …)

Among them, r ² = x ² + y ² , x, y are the image pixel coordinates before distortion; x _RD , y _RD are the image pixel coordinates after radial distortion calibration; where RD is the abbreviation of Radial Distortion.

Step 2: Tangential distortion correction

x _TD = x _RD + [2p ₁ y _RD + p ₂ (r ² + 2 x ² _RD )]

y _TD = y _RD + [2p ₂ x _RD + p ₁ (r ² + 2 y ² _RD )]

Among them, x _TD and y _TD are the image pixel coordinates after radial distortion calibration; where TD is the abbreviation of tangential distortion (TangentialDistortion).

The specific process is shown in Figure 3. Image distortion module flow chart.

6. The image pixel intelligent division and extraction module can operate according to the following principles and calculation steps:

Intelligent division and extraction of pixels:

Step 1: Regional division. The calibrated checkerboard image is still divided into areas according to the previous black and white colors, and is divided into a total of 25 areas. Each area will represent a small grid of the checkerboard.

Step 2: Calculate area size. Assuming that the pixel resolution of the image is W×H (width×height), the pixel size of each region is calculated as (W/5)×(H/5).

Step 3: Area number. These 25 areas are numbered, and the numbering method is from left to right and from top to bottom. For the i-th area, extract all pixels within the coordinate range [((i-1) mod 5)×(W/5), ((i-1) ÷ 5)×(H/5)], Where mod is the modulus operator.

Step 4: Image pixel extraction. For each area, all pixels in the area are extracted based on deep learning machine vision related methods.

For step four, image pixel extraction uses machine vision-related methods based on deep learning to extract pixels in each area of the checkerboard. This process can be divided into several steps: feature learning, area recognition, and pixel extraction. The following are related concepts and extraction formulas:

Feature learning: Use CNN to learn features from images. The formula is as follows:

,

Among them, x is the input image, W is the weight of the convolution kernel, b is the bias term, * represents the convolution operation, and ReLU is the activation function;

Area identification: used to locate each area of the checkerboard, the formula is as follows:

,

Among them, F represents the proposed area, is the output of the convolutional layer; RegionProposalNetwork represents the candidate region generation network;

Pixel extraction: Perform pixel-level processing on each area to extract pixels in the area. The formula is as follows, where SegmentationNetwork represents the segmentation network:

,

Among them, P represents the extracted pixels, and F is the identified area.

Integrating the above step formulas, the specific formula is:

,

The specific process is shown in Figure 4, the pixel extraction module flow chart.

7. In the image pixel displacement calculation module, in order to accurately calculate the foundation pit displacement value, we propose a pixel displacement calculation method based on two images: one is the image taken for the first time after distortion correction, and the other is An image taken some time after it was first taken. By analyzing these two images, we were able to measure the displacement of the foundation pit. It can be operated according to the following principles and calculation steps:

Step 1: Pixel displacement calculation. For each pixel, we calculate its displacement between the two images, expressed as (△x, △y). △x and △y are the movement distance of the pixel on the X-axis and Y-axis respectively.

Step 2: Calculate the average displacement within the area. For each region in the checkerboard, we calculate the average of the displacements of all pixels in the region. This average will represent the overall pixel displacement in the area.

For the average displacement of the i-th area, the calculation formula is as follows:

average displacement ,/> ,

Among them, n is the total number of pixels in the i-th area, and/> is the displacement of each pixel in the i-th area.

Step 3: Based on the camera calibration and internal parameter matrix, determine the geometric relationship between the camera and the monitoring target, including distance and angle, to establish the relationship between the world coordinate system (actual physical space) and the image coordinate system (pixel space).

Assume that P _world = (X, Y, Z) is a point in the world coordinate system, and P _image = (x, y) is the corresponding pixel point in the image coordinate system. The conversion relationship can be expressed as:

P _image = A · [ R丨t ] · P _world

Among them, A is the internal parameter matrix of the camera, and R丨t is the rotation and translation matrix from the world coordinate system to the camera coordinate system.

Step 4: Judgment of foundation pit displacement. The average displacement of each area is used to represent the overall displacement of the area, and the overall displacement of the foundation pit is judged based on the relationship between the average displacement, the world coordinate system and the image coordinate system.

The specific process is shown in Figure 5, the pixel displacement calculation flow chart.

8. The monitoring module includes a computer host terminal module and a mobile device terminal module; the computer host terminal module is composed of a wired transmission device, a computer host and a display sub-module. Through wired transmission equipment, specific physical displacement data and analysis results are directly transmitted and stored in the disk of the computer host. Through the connected display, these data and analysis results can be presented to relevant personnel clearly and intuitively; the mobile device terminal module is based on high-speed wireless transmission technology such as 5G. This module allows multiple parties to view relevant data and analysis results anytime and anywhere through mobile devices. This wireless connection increases the flexibility of the monitoring system and improves the convenience of data access.

9. The intelligent early warning module includes an image pixel displacement threshold setting module, a super-threshold early warning module and a monitoring data mutation error reporting module; the image pixel displacement threshold setting module compares specific physical displacement data based on preset thresholds. and screening. By setting a reasonable displacement threshold, abnormal displacements that may cause structural risks can be effectively identified; the super-threshold early warning module is based on the screening results of the image pixel displacement threshold setting module. When the monitored displacement exceeds the set threshold When the alarm occurs, this module will trigger an early warning prompt; the monitoring data mutation error reporting module is specifically responsible for error reporting of abnormal over-threshold warning data, making it convenient for multiple staff to monitor and repair foundation pits. The workflow of the intelligent early warning module is shown in Figure 6.

The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention. All substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (10)

1. The utility model provides a foundation ditch displacement monitoring system based on image vision processing which characterized in that: the intelligent early warning system comprises an image sensing module, a data processing module, a monitoring module and an intelligent early warning module;

in the image sensing module, an LED illuminating lamp is used for illuminating the inner surface of a target foundation pit and a black-white double-color checkerboard calibration plate for target identification, and an industrial camera is used for capturing the image of the checkerboard calibration plate placed in the foundation pit and transmitting the inner image to a data processing module;

the data processing module comprises an image distortion calibration module, an image pixel point intelligent dividing and extracting module and an image pixel point displacement calculation module; and the image distortion calibration module performs distortion calibration on the checkerboard calibration plate image in the foundation pit, the image after the distortion calibration is processed through a machine vision algorithm based on deep learning, and the foundation pit displacement is obtained through pixel point extraction, region division and pixel point region average displacement calculation based on time variation.

2. The foundation pit displacement monitoring system of claim 1, wherein: the image distortion calibration module performs distortion calibration on the image based on the LED illuminating lamp point distribution position and by combining a Zhang Zhengyou checkerboard calibration method.

3. The foundation pit displacement monitoring system of claim 1 or 2, wherein: the distortion calibration process of the image is as follows:

step 1: identifying the angular point coordinates of the checkerboard with the LED illuminating lamp:

step 2: calculating distortion parameters of the industrial camera: calculating parameters of the industrial camera, including an internal reference matrix and distortion coefficients, by using the identified LED illuminating lamp characteristic point coordinates;

step 3: and carrying out distortion correction on the photographed checkerboard image pixel points based on the internal reference matrix and the distortion coefficient.

4. A pit displacement monitoring system according to claim 3, wherein:

the step 1 comprises the following steps:

assuming that I (x, y) represents the luminance value of the image at point (x, y), for each detected corner point (x _i ,y _i ) The following calculations were performed:

calculating the brightness of the corner points:wherein N is the corner point (x _i ,y _i ) The number of pixels in the field;

if L > T, then (x _i ,y _i ) The coordinate of the LED illuminating lamp is T which is a preset threshold value.

5. The foundation pit displacement monitoring system of claim 4, wherein:

the steps 2 and 3 comprise:

after the coordinates of the mark points of the LED illuminating lamp in the shot image are determined, a corresponding relation between a physical coordinate system and an image coordinate system is established by utilizing a checkerboard image captured by an industrial camera; determining the position of each point in a physical space and the corresponding position of each point in the image through analysis of the checkerboard image, and calculating the distortion coefficient of the industrial camera based on the corresponding relation between coordinate systems;

wherein, camera internal reference matrix A:

,

wherein f _x ，f _y Is focal length, c _x ，c _y Is the center coordinates of the image;

the distortion coefficients include: coefficient of radial distortion k ₁ ，k ₂ … … and tangential distortion coefficient p ₁ ，p ₂ ；

The camera distortion coefficient is obtained by adopting a least square method, and the calculation formula is as follows:

,

x _ij is the image coordinate of the jth corner in the ith image; n represents the number of images and m represents the number of corner points;

X _ij is the corresponding physical coordinates;

a is an internal reference matrix of the camera;

k ₁ ，k ₂ is the radial distortion coefficient, p ₁ ，p ₂ Is the tangential distortion coefficient;

r, t is the rotation and translation matrix of the camera;

checkerboard image correction based on distortion coefficients, for each pixel point (x, y) in the image:

step 1: radial distortion correction

x _RD = x(1 + k ₁ r ² + k ₂ r ⁴ + …)

y _RD = y(1 + k ₁ r ² + k ₂ r ⁴ + …)

Wherein r is ² = x ² + y ² X, y are the pre-distortion image pixel coordinates; x is x _RD ，y _RD The radial distortion is calibrated image pixel coordinates; where RD is an abbreviation for radial distortion (Radial Distortion);

step 2: tangential distortion correction

x _TD = x _RD + [2p ₁ y _RD + p ₂ (r ² + 2 x ² _RD )]

y _TD = y _RD + [2p ₂ x _RD + p ₁ (r ² + 2 y ² _RD )]

Wherein x is _TD ，y _TD The radial distortion is calibrated image pixel coordinates; wherein TD is an abbreviation for tangential distortion (Tangential Distortion);

in performing image distortion correction, the pixel coordinates are typically first adjusted according to the radial distortion coefficients and then further adjusted according to the tangential distortion coefficients, which are typically performed on the basis of having considered the radial distortion correction.

6. The foundation pit displacement monitoring system of claim 1, wherein: the intelligent dividing and extracting module for the image pixel points operates according to the following steps:

step one: dividing the area: dividing the calibrated checkerboard image into 25 areas according to the previous black-white double colors, wherein each area represents a small checkerboard;

step two: area size calculation: assuming that the pixel resolution of the image is w×h, i.e., width×height, the pixel size of each region is calculated as (W/5) × (H/5);

step three: region number: numbering 25 areas, wherein the numbering mode is that the areas are orderly ordered from left to right and from top to bottom; extracting all pixel points with the coordinate range of [ ((i-1) mod 5) x (W/5) ((i-1) 5) x (H/5) ] for the i-th region, wherein mod is a modulo operator;

step four: extracting image pixel points: for each region, all pixels within the region are extracted based on a deep learning machine vision method.

7. The foundation pit displacement monitoring system of claim 6, wherein:

the deep learning machine vision algorithm comprises the following steps: feature learning, region identification, and pixel point extraction, wherein:

and (3) feature learning: the CNN is used to learn the features from the image, and the formula is as follows:

,

where x is the input image, W is the weight of the convolution kernel, b is the bias term, x represents the convolution operation, and ReLU is the activation function;

and (3) area identification: for locating each region of the checkerboard, the formula is as follows:

,

wherein F represents the proposed area,is the output of the convolutional layer; region proposalnetwork represents a candidate region generation network;

pixel point extraction: and carrying out pixel level processing on each region to extract pixel points in the region, wherein the formula is as follows, and the segmentation network is represented by the following formula:

,

wherein P represents the extracted pixel point, and F is the identified region;

integrating the above step formulas, the specific formulas are as follows:

。

8. the foundation pit displacement monitoring system of claim 1, wherein: the image pixel point displacement calculation module uses a pixel point displacement calculation method based on two images based on the image pixel point coordinates after distortion calibration, wherein one image is the first shot image after distortion correction, and the other image is the image after a period of time after the first shot, and the image pixel point displacement calculation module operates according to the following steps:

step one: calculating pixel displacement: for each pixel, calculating the displacement of the pixel between the two images, denoted as (Deltax, deltay), deltax and Deltay being the movement distance of the pixel on the X-axis and the Y-axis respectively;

step two: calculating average displacement in the region: for each region in the checkerboard, calculating the average value of all pixel point displacements in the region, wherein the average value represents the whole pixel point displacement of the region;

for the average displacement of the i-th region, the calculation formula is as follows:

average displacement，/>,

Where n is the total number of pixels in the ith region,and->Is the displacement of each pixel point in the ith area;

step three: based on camera calibration and an internal reference matrix, determining a geometric relationship between a camera and a monitored target, including a distance and an angle, to establish a relationship between a world coordinate system and an image coordinate system;

step four: and (3) foundation pit displacement judgment: and judging the overall displacement condition of the foundation pit based on the relation among the average displacement, the world coordinate system and the image coordinate system.

9. The foundation pit displacement monitoring system of claim 1, wherein: the monitoring module comprises a computer host terminal module and a mobile equipment terminal module; the computer host terminal module consists of wired transmission equipment, a computer host and a display submodule.

10. The foundation pit displacement monitoring system of claim 1, wherein: the intelligent early warning module comprises an image pixel position displacement threshold setting module, a super threshold early warning module and a monitoring data mutation error reporting module; the image pixel displacement threshold setting module compares and screens the physical displacement data based on a preset threshold to identify abnormal displacement conditions causing structural risks; the super-threshold early warning module triggers early warning prompt when the monitored displacement exceeds a set threshold value based on the screening result of the image pixel displacement threshold value setting module; and the monitoring data mutation error reporting module is used for reporting the abnormal super-threshold early warning data.