CN114820787B - Image correction method and system for large field of view plane vision measurement - Google Patents

Abstract

The invention discloses an image correction method and system for large-view-field plane vision measurement, wherein the method comprises the following steps: acquiring a checkerboard image of a plane to be detected, extracting actual image coordinates of the mark points, reasonably partitioning the image according to the distribution condition of the mark points, constructing ideal image coordinates of each mark point, and acquiring a training database of each partition; training a deep learning network model of each partition by utilizing a training database; according to the trained model, calculating ideal image coordinates of pixel points in each partition, establishing a remap matrix of each partition, generating an undistorted front view of each partition by using the remap matrix, and splicing front views of the respective partitions to generate an undistorted front view of the whole image. The invention adopts the strategy of image partition correction and splicing fusion, can finish the accurate correction of the large-view-field and high-distortion image on the premise of no need of camera calibration, and improves the image correction precision and correction efficiency.

Description

Image correction method and system for large field of view plane vision measurement

Technical Field

The present invention relates to the technical field of image correction, and in particular to an image correction method and system for large-field-of-view plane vision measurement.

Background technique

At present, the traditional image correction method first needs to calibrate the camera, that is, calculate the intrinsic and extrinsic parameters of the camera, and then use the intrinsic and extrinsic parameters of the camera to correct the captured image to obtain an image with less distortion. In this process, the accuracy of the calculation of the intrinsic and extrinsic parameters of the camera under a large field of view will directly affect the correction effect of the image. At the same time, in order to obtain more accurate intrinsic and extrinsic parameters of the camera, the camera calibration under a large field of view requires shooting calibration targets at different positions and postures, which is time-consuming and labor-intensive and often does not produce very good calibration results. Therefore, studying a method that can complete the accurate correction of large-field-of-view, high-distortion images without camera calibration is a technical problem that needs to be solved urgently in this field.

Summary of the invention

The purpose of the present invention is to provide an image correction method and system for large-field-of-view planar vision measurement, which can complete the precise correction of large-field-of-view and high-distortion images without camera calibration, thereby improving the image correction accuracy and correction efficiency.

To achieve the above object, the present invention provides the following solutions:

An image correction method for large-field-of-view plane visual measurement, comprising:

Acquire a checkerboard image of a plane to be measured; a checkerboard calibration plate is placed on the plane to be measured;

Performing landmark point detection on the checkerboard image, extracting actual image coordinates of the landmark points and partitioning the checkerboard image;

Setting ideal image coordinates for the marker points, and establishing a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points;

Establishing a deep learning network model, and using the training database of each partition to train the deep learning network model for each partition, to generate a trained image correction model for each partition;

The image correction model of each partition is used to calculate the ideal image coordinates of all pixels in each partition, and the remap matrix of each partition is calculated; specifically comprising: using the image correction model of each partition to calculate the ideal image coordinates of all pixels in each partition; according to the mapping relationship between the actual image coordinates and the ideal image coordinates of all pixels in each partition, constructing the remap matrix of each partition;

Acquire a plane image to be measured and partition the plane image to be measured;

Generate an undistorted front view of each partition of the plane to be measured using the remap matrix of each partition;

The undistorted front views of the subareas of the plane to be measured are spliced to generate an undistorted front view of the image of the plane to be measured.

Optionally, the obtaining of a checkerboard image of the plane to be measured specifically includes:

A camera equipped with a wide-angle lens is fixed above the plane field of view to be measured, and an LED light source is used to fill in the shooting area;

The checkerboard calibration plate is placed on the plane to be measured, and the camera is controlled by a computer to capture the checkerboard image.

Optionally, the performing marker point detection on the checkerboard image, extracting actual image coordinates of the marker points and partitioning the checkerboard image specifically includes:

Performing landmark point detection on the chessboard image, and extracting corner points of the chessboard as the landmark points;

Extracting the actual image coordinates of the marker points;

The checkerboard image is partitioned according to the actual image coordinate distribution of the marker points; the number of marker points contained in each partition is greater than or equal to 60.

Optionally, setting ideal image coordinates for the marker points and establishing a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points specifically includes:

According to the actual image coordinate distribution of the marker point and the goal of image correction, setting the ideal image coordinates for the marker point;

A training database for each partition is constructed according to the actual image coordinates and ideal image coordinates of all the marker points in each partition.

An image correction system for large-field-of-view plane vision measurement, comprising:

A checkerboard image acquisition module is used to acquire a checkerboard image of a plane to be measured; a checkerboard calibration plate is placed on the plane to be measured;

A landmark point detection and partitioning module, used to perform landmark point detection on the checkerboard image, extract the actual image coordinates of the landmark points and partition the checkerboard image;

A training database establishment module, used to set ideal image coordinates for the marker points, and establish a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points;

An image correction model establishment module is used to establish a deep learning network model, and use the training database of each partition to train the deep learning network model for each partition, so as to generate a trained image correction model for each partition;

A remap matrix establishment module, used to calculate the ideal image coordinates of all pixels in each partition by using the image correction model of each partition, and calculate the remap matrix of each partition; the remap matrix establishment module specifically includes: an ideal image coordinate calculation unit, used to calculate the ideal image coordinates of all pixels in each partition by using the image correction model of each partition; a remap matrix establishment unit, used to construct the remap matrix of each partition according to the mapping relationship between the actual image coordinates and the ideal image coordinates of all pixels in each partition;

A plane image acquisition module to be measured, used to acquire the plane image to be measured and divide the plane image to be measured into different zones;

A partitioned undistorted front view generation module, used to generate an undistorted front view of each partition of the plane to be measured by using the remap matrix of each partition;

The module for generating an undistorted front view of the plane to be measured is used to splice the undistorted front views of the various subareas of the plane to be measured to generate an undistorted front view of the image of the plane to be measured.

Optionally, the chessboard image acquisition module specifically includes:

A camera setting unit, used to fix a camera equipped with a wide-angle lens above the plane field of view to be measured, and use an LED light source to fill in the shooting area;

The checkerboard image acquisition unit is used to place the checkerboard calibration plate on the plane to be measured, and use a computer to control the camera to capture the checkerboard image.

Optionally, the landmark detection and partitioning module specifically includes:

A landmark point detection unit, used to perform landmark point detection on the chessboard image, and extract corner points of the chessboard as the landmark points;

An actual image coordinate extraction unit, used to extract the actual image coordinates of the marker point;

A partitioning unit is used to partition the checkerboard image according to the actual image coordinate distribution of the marker points; the number of marker points contained in each partition is greater than or equal to 60.

Optionally, the training database establishment module specifically includes:

An ideal image coordinate setting unit, used to set ideal image coordinates for the marker point according to the actual image coordinate distribution of the marker point and the target of image correction;

The training database establishing unit is used to construct a training database for each partition according to the actual image coordinates and ideal image coordinates of all the marker points in each partition.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The present invention provides an image correction method and system for large-field-of-view plane visual measurement, the method comprising: obtaining a checkerboard image of a plane to be measured; performing landmark point detection on the checkerboard image, extracting actual image coordinates of the landmark points and partitioning the checkerboard image; setting ideal image coordinates for the landmark points, and establishing a training database for each partition according to the actual image coordinates and ideal image coordinates of the landmark points; establishing a deep learning network model, and using the training database of each partition to train the deep learning network model for each partition, respectively, to generate a trained image correction model for each partition; using the image correction model of each partition to calculate the ideal image coordinates of all pixel points in each partition, and calculating the remap matrix of each partition; obtaining a plane image to be measured and partitioning the plane image to be measured; using the remap matrix of each partition to generate an undistorted front view of each partition of the plane to be measured; splicing the undistorted front views of each partition of the plane to be measured to generate an undistorted front view of the plane image to be measured. The present invention adopts the strategy of image partition correction + splicing fusion to correct images with large field of view and high distortion. The partition strategy reduces the complexity of the deep learning network model and improves the model training efficiency. The trained image correction model is used to generate the remap matrix of the image correction, and multi-threading technology and bilinear interpolation are used to complete the image correction, thereby improving the image correction efficiency. The image correction based on the deep learning model also avoids the calculation of the camera's internal and external parameters and the lens distortion coefficient. Therefore, the image correction method of the present invention can complete the accurate correction of images with large field of view and high distortion without the need for camera calibration, thereby improving the image correction accuracy and correction efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a flow chart of an image correction method for large-field-of-view plane vision measurement provided by the present invention;

FIG2 is a schematic diagram of a chessboard image provided by the present invention;

FIG3 is a schematic diagram of chessboard marker point detection provided by the present invention;

FIG4 is a schematic diagram of chessboard image partitioning provided by the present invention;

FIG5 is a schematic diagram of a deep learning network model provided by the present invention;

6 is a schematic diagram of the distribution of actual coordinates, ideal coordinates, and corrected coordinates of the first partition mark points of the chessboard image provided by the present invention;

FIG7 is a schematic diagram of a corrected chessboard image provided by the present invention;

FIG8 is a diagram illustrating quantitative evaluation parameters of calibration results provided by the present invention;

FIG9 is a schematic diagram of the calculation result of the horizontality of the row mark points provided by the present invention;

FIG10 is a schematic diagram of the calculation results of the verticality of column marker points provided by the present invention;

FIG11 is a schematic diagram of a calculation result of the uniformity of distribution of horizontally adjacent marking points provided by the present invention;

FIG. 12 is a schematic diagram of the calculation results of the uniformity of distribution of vertically adjacent marking points provided by the present invention.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The purpose of the present invention is to provide an image correction method and system for large-field-of-view planar vision measurement, which can complete the precise correction of large-field-of-view and high-distortion images without camera calibration, thereby improving the image correction accuracy and correction efficiency.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

In the large field of view plane measurement based on a monocular camera, in order to complete image correction more conveniently, quickly and accurately to provide effective guarantee for subsequent measurement work, the present invention is based on monocular vision and a partition correction strategy to establish a deep learning network model to complete image correction without calculating the internal and external parameters of the camera. At the same time, a method for evaluating image correction results is proposed for the image correction method of the present invention.

FIG1 is a flow chart of an image correction method for large-field-of-view plane vision measurement provided by the present invention. As shown in FIG1 , an image correction method for large-field-of-view plane vision measurement provided by the present invention comprises:

Step 101: Acquire a checkerboard image of the plane to be measured.

Specifically, a camera equipped with a wide-angle lens is fixed above the field of view of the plane to be measured, and an LED light source is used to fill the shooting area with light to reduce the influence of ambient light on the shooting. A specially made checkerboard calibration plate 1 is placed flat on the plane to be measured 2 as a calibration target, and a computer is used to control the camera to obtain an image of the plane to be measured 2. In the actual shooting process, the exposure time, gain and other shooting parameters of the camera are adjusted in real time according to the average gray value and clarity of the image to obtain an image that meets the mark point detection, that is, the checkerboard image of the plane to be measured 2 shown in Figure 2.

Step 102: perform landmark point detection on the checkerboard image, extract actual image coordinates of the landmark points and partition the checkerboard image.

The acquired image is preprocessed to extract the image coordinates of the marker points of the checkerboard calibration plate, and the marker points of the image are partitioned according to the size of the field of view, the number of marker points and the distribution of the marker points.

Therefore, the step 102 specifically includes:

Step 2.1: Perform landmark point detection on the chessboard image, and extract the corner points of the chessboard as the landmark points.

FIG3 is a schematic diagram of the checkerboard mark point detection provided by the present invention. Specifically, referring to FIG3, the checkerboard image is binarized based on the local average adaptive thresholding method, and the connection of each black block quadrilateral is separated by image expansion to obtain a reduced black block quadrilateral. The black block quadrilateral is detected based on constraints such as aspect ratio, perimeter and area, and each quadrilateral is taken as a unit. The middle point of the line connecting the two opposite points of two diagonally adjacent quadrilaterals is taken as the corner point 3. That is, the four vertices of the black block quadrilateral are the corner points.

Step 2.2: Extract the actual image coordinates of the landmark points.

Step 2.3: partition the checkerboard image according to the actual image coordinate distribution of the marker points, wherein the number of marker points contained in each partition is greater than or equal to 60.

Specifically, in order to improve the accuracy of distortion correction, reduce the complexity of the deep learning network model, and improve the efficiency of model training, the entire chessboard image is partitioned according to the size of the field of view, the number of marker points, and the actual image coordinate distribution of the marker points, and each partition contains no less than 60 marker points. Figure 4 is a schematic diagram of the chessboard image partition provided by the present invention, in which 24 partitions are divided.

Step 103: setting ideal image coordinates for the marker points, and establishing a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points.

The step 103 specifically includes:

Step 3.1: According to the actual image coordinate distribution of the marker point and the goal of image correction, set the ideal image coordinates for the marker point.

Specifically, the purpose of image correction for large field of view plane vision measurement is to obtain an undistorted front view of the plane to be measured. The row marker points and column marker points of the chessboard calibration are orthogonally distributed, and the lateral distance and longitudinal distance of adjacent marker points are equal. Therefore, the ideal image coordinates can be set for each marker point detected in step 102 according to the distribution of the marker points and the goal of image correction.

Step 3.2: Construct a training database for each partition according to the actual image coordinates and ideal image coordinates of all the landmarks in each partition.

Step 104: Establish a deep learning network model, and use the training database of each partition to train the deep learning network model for each partition respectively, to generate a trained image correction model for each partition.

The step 104 specifically includes:

Step 4.1: Build a deep learning network model.

Specifically, FIG5 is a schematic diagram of a deep learning network model provided by the present invention. The present invention uses a DNN deep neural network as shown in FIG5 to build a deep learning network model, and the model includes an input layer, two hidden layers and an output layer. The input layer and the output layer both contain two neurons, which respectively represent the actual image coordinates (col, row) of a marker point and its corresponding ideal image coordinates (col', row'), wherein row and row' represent the row coordinates of the marker point; col and col' represent the column coordinates. That is, the input of the deep learning network model is the actual image coordinates of the marker point, the input of one neuron is the row coordinates of the marker point, and the input of the other neuron is the column coordinates of the marker point. The output of the deep learning network model is the ideal image coordinates corresponding to the marker point, the output of one neuron is the ideal row coordinates corresponding to the marker point, and the output of the other neuron is the ideal column coordinates corresponding to the marker point.

The row and column coordinates of each partition of the image are normalized using formula (1), and the actual image coordinates are (x ₁ , y ₁ ) and the ideal image coordinates are (x ₂ , y ₂ ), as shown in the following formula:

Wherein, z _i is the vector to be normalized; min(Z) is the minimum value of the elements in the vector Z; max(Z) is the maximum value of the elements in the vector Z; and Z′ _i is the vector after normalization.

In the process of normalizing the row and column coordinates of the image, the row coordinates of all the landmarks in each partition are a vector, and the row vector is normalized using formula (1); the column coordinates of all the landmarks are another vector, and the column vector is normalized again using formula (1). The actual image coordinates (x ₁ , y ₁ ) obtained after normalization in each partition and the corresponding ideal image coordinates (x ₂ , y ₂ ) are used as a training sample to constitute the training data of each partition.

Step 4.2: Using the training data of each partition, train a deep learning network model for each partition.

The actual image coordinates and ideal image coordinates of the landmark points are used for training to calculate the model parameters of each partition. During the model training process, the purpose of training is to reduce the gap between the predicted value and the sample label value. The gap is represented by the Euclidean distance of the mean square error, and the loss function is defined as formula (2). Through a series of stages such as model parameter initialization, learning rate adjustment, and weight parameter update, the DNN deep learning network model of each partition is trained separately. The accuracy and effectiveness of the model are then verified through test points to obtain a trained image correction model for each partition. The loss function formula is as follows:

Among them, J(w, b) is the loss function; m is the number of landmarks in each partition; x _i1 is the row coordinate of the actual image coordinates of the i-th landmark; y _i1 is the column coordinate of the actual image coordinates of the i-th landmark; _xi2 is the row coordinate of the ideal image coordinates of the i-th landmark; y _i2 is the column coordinate of the ideal image coordinates of the i-th landmark.

Step 105: Utilize the image correction model of each partition to calculate the ideal image coordinates of all pixels in each partition, and calculate the remap matrix of each partition.

Specifically, the image correction model of each partition is used to calculate the ideal image coordinates of all pixels in each partition, and then according to the mapping relationship between the actual image coordinates and the ideal image coordinates of all pixels in each partition, the remap matrix H _i2i of each partition is constructed as the mapping matrix for image correction.

In order to improve the efficiency of image correction, the image correction model of each partition is no longer used in the subsequent correction of the image to be measured. Instead, the remap matrix _Hi2i is used to calculate the coordinates of the corrected pixels (called correction coordinates). Multi-threading technology and bilinear interpolation are used to correct the image distortion and obtain the standard front view of the plane to be measured, which provides guarantee for high-precision and large-field-of-view plane measurement, reduces the amount of calculation, increases the calculation speed, and thus improves the efficiency of image correction.

In order to verify the validity of the remap matrix H _i2i , the present invention uses the remap matrix H _i2i to calculate the corrected coordinates of all pixels of the checkerboard image for verification.

First, use the remap matrix _Hi2i to calculate the corrected coordinates of the pixels in each partition of the chessboard image. The calculation formula is as follows:

in, is the coordinate vector of the pixel before correction, that is, the vector composed of the original coordinates (or actual coordinates) of the pixel;/> is the coordinate vector of the corresponding pixel point after correction, that is, the vector formed by the pixel point correction coordinates.

FIG6 is a schematic diagram of the distribution of the actual coordinates, ideal coordinates, and corrected coordinates of the marker points of the first partition of the chessboard image provided by the present invention, wherein 4 represents the actual coordinates of the marker points (also called the original coordinates), 5 represents the ideal coordinates (i.e., the ideal image coordinates), and 6 represents the corrected coordinates (also called the corrected coordinates). As shown in FIG6 , the corrected coordinates of the marker points of the chessboard image calculated using the remap matrix H _i2i are highly coincident with the ideal coordinates.

Then, the multi-threading technology and bilinear interpolation method are used to correct the distortion of the chessboard image, and a standard front view of the chessboard image can be obtained, as shown in FIG7 .

The present invention also proposes an image correction evaluation method based on the verticality of column markers, the horizontality of row markers, and the uniformity of the distribution of adjacent markers to verify the accuracy and effectiveness of the image correction method for large-field-of-view plane vision measurement proposed by the present invention. This evaluation method is an effective quantitative evaluation method for image correction for plane measurement.

Specifically, the marker points of the corrected chessboard image shown in FIG7 are extracted, and the verticality of the column marker points, the horizontality of the row marker points, and the uniformity of the distribution of adjacent marker points are calculated. The description of each parameter is shown in FIG8 .

Among them, the calculation formula of the column marker verticality VM is as follows:

Where n is the number of landmarks in each column; m is the number of landmarks in each row; _xij is the x-coordinate of the ith landmark in the jth column; and AVG( _Xj ) represents the average x-coordinate of all landmarks in the jth column.

The formula for calculating the horizontality of the row mark point HM is as follows:

Where _yij is the y-coordinate of the i-th corner point in the j-th row; AVG( _Yj ) represents the average y-coordinate of all corner points in the j-th row.

The calculation formula for the uniformity of distribution of horizontal adjacent landmark points UHM is as follows:

Among them, _xjk is the x-coordinate of the k-th marker point in the j-th row; xjk _-1 is the x-coordinate of the k-1-th marker point in the j-th row.

The calculation formula for the uniformity of vertically adjacent marker points distribution UVM is as follows:

Among them, y _jk is the y-coordinate of the k-th corner point in the j-th column; y _jk-1 is the y-coordinate of the k-1-th corner point in the j-th column.

The better the verticality, horizontality and uniformity, that is, the smaller the element values in the VM, HM, UHM and UVM vectors, the better the image correction effect.

The image correction evaluation method is used to verify the accuracy of the remap matrix _Hi2i of the present invention. Therefore, when correcting the image to be tested later, it is no longer necessary to use the image correction model of each partition. The remap matrix _Hi2i can be used to calculate the coordinates of the corrected pixel points, and the image distortion can be corrected by using multi-threading technology and bilinear interpolation to obtain a standard front view of the plane to be tested.

Step 106: Acquire the plane image to be measured and partition the plane image to be measured.

Specifically, a camera equipped with a wide-angle lens is fixed above the field of view of the plane to be measured, and an LED light source is used to fill the shooting area with light to reduce the influence of ambient light on the shooting. The camera is controlled by a computer to obtain an image of the plane to be measured. The image of the plane to be measured is partitioned according to the partitioning rules of the chessboard image taken under this field of view.

Step 107: Generate an undistorted front view of each partition of the plane to be measured using the remap matrix of each partition.

The remap matrix H _i2i of each partition is used to calculate the corrected coordinates of the pixel points in the partition of the plane image to be measured based on formula (3). The partition image is distorted by bilinear interpolation to obtain the distortion-free front view of the partition.

First, use the remap matrix _Hi2i to calculate the corrected coordinates of the pixels in each partition of the chessboard image. The calculation formula is as follows:

in, is the coordinate vector of the pixel before correction, that is, the vector composed of the original coordinates (or actual coordinates) of the pixel;/> is the coordinate vector of the corresponding pixel point after correction, that is, the vector formed by the pixel point correction coordinates.

Image correction based on the deep learning network model avoids the calculation of camera internal and external parameters and lens distortion coefficients, and integrates the landmark point extraction of traditional image correction and the deep learning method of AI image processing. It does not require image feature extraction through convolution, thus ensuring the image correction accuracy and efficiency to the greatest extent.

In order to realize the large-field-of-view plane measurement, the present invention also provides a conversion matrix between the corrected image front view coordinates and the plane physical coordinates to realize the large-field-of-view plane measurement.

Specifically, the coordinates of the marker points of the standard front view of the plane image to be measured are extracted, the plane physical coordinates of the plane to be measured are constructed, the remap matrix H _i2w is calculated, and the conversion between the marker point image coordinates and the physical coordinates is performed based on H _i2w . The physical coordinate vector calculation formula is as follows:

in, is the image coordinate vector; /> is the corresponding physical coordinate vector.

Step 108: stitching the undistorted front views of the various subareas of the plane to be measured to generate an undistorted front view of the image of the plane to be measured.

The present invention fixes a camera equipped with a wide-angle lens above the field of view of the plane to be measured, places a calibration target on the plane to be measured, and uses a computer to control the camera to obtain an image of the plane to be measured; pre-processes the image, extracts the image coordinates of the mark points of the calibration target, and establishes ideal image coordinates for each mark point according to the actual distribution of the mark; partitions the mark points of the image according to the size of the field of view, the number of mark points, and the distribution of the mark points; establishes a deep learning network model, uses the actual image coordinates and ideal image coordinates of the mark points for training, and calculates the model parameters of each area; uses the trained model, adopts multi-threading technology and bilinear interpolation method to complete the image correction. At the same time, the image correction results are evaluated based on the verticality of the column mark points, the horizontality of the row calibration points, and the uniformity of the distribution of adjacent mark points. In order to improve the efficiency of image correction, the trained network model is used to generate a mapping matrix for image correction. It can be seen that the present invention completes the precise correction of large-field-of-view, high-distortion images suitable for plane vision measurement without relying on the internal and external parameters of the camera.

A specific embodiment of the image correction method for large-field-of-view plane vision measurement of the present invention is provided below.

A specific implementation process of the image correction method of the present invention includes:

1. Get the image of the plane to be measured: Place a special checkerboard calibration plate on the plane to be measured. The physical size of the plane to be measured is 3000mm×2000mm, the physical size of the checkerboard is 50mm×50mm, the number of markers in each row is 63, and the number of markers in each column is 39. Place a camera with a wide-angle lens above the plane to be measured. The camera resolution is 3664×2748. Turn on the real-time acquisition mode of the camera. According to the average gray value and clarity of the image, adjust the exposure time, gain and other shooting parameters of the camera in real time to capture a high-quality checkerboard image, as shown in Figure 2.

2. Landmark point detection and partitioning: Binarize the checkerboard image based on the local average adaptive thresholding method, dilate and separate the connection of each black block quadrilateral, and obtain the reduced black block quadrilateral. Detect the quadrilateral based on the constraints such as aspect ratio, perimeter and area. Take each quadrilateral as a unit, and take the middle point of the line connecting the two opposite points of the two diagonally adjacent quadrilaterals as the corner point, as shown in Figure 3, and use the corner point as the landmark point. According to the distribution of the landmark point image coordinates, divide the entire image into 6×4=24 regions, as shown in Figure 4, and each partition contains no less than 90 landmark points.

3. Generate a training database: In the previous step, 63×39=2457 marker points were detected. The row and column corner points of the chessboard are orthogonally distributed, and the horizontal and vertical distances between adjacent corner points are equal. According to the distribution of corner points and the goal of image correction, set the ideal image coordinates for each marker point. Set the ideal image coordinates of the upper left corner marker point to (127, 329), with the horizontal right direction of the point as the positive direction of the X axis, the vertical downward direction as the positive direction of the Y axis, and a step size of 50 pixels to set the ideal image coordinates for each marker point. Then, use the actual image coordinates and ideal image coordinates of the marker points in each partition to construct a training database, and generate a total of 24 partitioned training databases.

4. Build, train, and deep learning network models: Build a DNN (deep neural network) model for each partition, a total of 24 DNN models, use the training database of each partition to train the corresponding DNN model, and obtain 24 trained image correction models.

5. Correct the image: According to the trained image correction model of each partition, calculate the coordinate mapping relationship between each pixel point in each partition before and after correction, and construct the remap matrix H _i2i . Use bilinear interpolation to correct the image distortion and obtain the front view of each partition of the checkerboard image.

The undistorted front views of each partition of the chessboard image are stitched together to generate an undistorted front view of the entire chessboard image, as shown in FIG7 .

6. Evaluation of image correction results: Extract the coordinates of the marker points of the undistorted front view of the chessboard image, calculate the verticality of the column marker points, the horizontality of the row marker points, and the uniformity of the distribution of adjacent marker points. The corrected image is evaluated based on the verticality of the column marker points, the horizontality of the row calibration points, and the uniformity of the distribution of adjacent marker points to verify the accuracy and effectiveness of the image correction method for large field of view plane vision measurement proposed in the present invention. Figure 9 is a schematic diagram of the calculation results of the horizontality of the row marker points (Horizontal degree of line corner) provided by the present invention. The horizontality of the row marker point HM is a 39×1 vector, as shown in Figure 9, the horizontal coordinate RowIndex is the number of marker rows, the vertical coordinate HM is the horizontality of the row marker points, and the figure shows the horizontality values of 39 row marker points. Figure 10 is a schematic diagram of the calculation results of the verticality of the column marker points (Vertical degree of column corner) provided by the present invention. The verticality VM of the column marker point is a 63×1 vector, as shown in Figure 10, the horizontal coordinate Column Index is the number of marker columns, the vertical coordinate VM is the verticality of the column marker points, and the figure shows the verticality values of 63 column marker points. Figure 11 is a schematic diagram of the calculation results of the distribution uniformity of horizontal corner points provided by the present invention. The distribution uniformity of horizontal adjacent marker points UHM is a 39×1 vector. As shown in Figure 11, the horizontal coordinate Row Index is the number of marker rows, and the vertical coordinate UHM is the distribution uniformity of horizontal adjacent marker points. The figure shows the values of the horizontal adjacent distribution uniformity of 39 rows of marker points. Figure 12 is a schematic diagram of the calculation results of the distribution uniformity of vertical adjacent marker points provided by the present invention. The distribution uniformity of vertically adjacent marker points UVM is a 63×1 vector. As shown in Figure 12, the horizontal coordinate Column Index is the number of marker columns, and the vertical coordinate UVM is the distribution uniformity of vertically adjacent marker points. The figure shows the values of the vertical adjacent distribution uniformity of 63 columns of marker points. Its VM is less than 0.045, HM is less than 0.5, UHM is less than 0.12, and UVM is less than 0.11, which fully verifies the effectiveness of the image correction method of the present invention for large field of view plane visual measurement.

Based on the method provided by the present invention, the present invention also provides an image correction system for large-field-of-view plane visual measurement, the system comprising:

The chessboard image acquisition module is used to acquire a chessboard image of a plane to be measured; a chessboard calibration plate is placed on the plane to be measured.

The landmark point detection and partitioning module is used to perform landmark point detection on the checkerboard image, extract the actual image coordinates of the landmark points and partition the checkerboard image.

The training database establishment module is used to set ideal image coordinates for the marker points, and to establish a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points.

The image correction model establishment module is used to establish a deep learning network model, and use the training database of each partition to train the deep learning network model for each partition respectively, so as to generate a trained image correction model for each partition.

The remap matrix establishment module is used to calculate the ideal image coordinates of all pixels in each partition by using the image correction model of each partition, and calculate the remap matrix of each partition.

The plane image acquisition module to be measured is used to acquire the plane image to be measured and divide the plane image to be measured into different zones.

The partitioned undistorted front view generation module is used to generate an undistorted front view of each partition of the plane to be measured by using the remap matrix of each partition.

The module for generating an undistorted front view of the plane to be measured is used to splice the undistorted front views of the various subareas of the plane to be measured to generate an undistorted front view of the image of the plane to be measured.

Wherein, the chessboard image acquisition module specifically includes:

A camera setting unit, used to fix a camera equipped with a wide-angle lens above the plane field of view to be measured, and use an LED light source to fill in the shooting area;

The checkerboard image acquisition unit is used to place the checkerboard calibration plate on the plane to be measured, and use a computer to control the camera to capture the checkerboard image.

The landmark detection and partitioning module specifically includes:

A landmark point detection unit, used to perform landmark point detection on the chessboard image, and extract corner points of the chessboard as the landmark points;

An actual image coordinate extraction unit, used to extract the actual image coordinates of the marker point;

A partitioning unit is used to partition the checkerboard image according to the actual image coordinate distribution of the marker points; the number of marker points contained in each partition is greater than or equal to 60.

Wherein, the training database establishment module specifically includes:

An ideal image coordinate setting unit, used to set ideal image coordinates for the marker point according to the actual image coordinate distribution of the marker point and the target of image correction;

The training database establishing unit is used to construct a training database for each partition according to the actual image coordinates and ideal image coordinates of all the marker points in each partition.

Wherein, the remap matrix establishment module specifically includes:

An ideal image coordinate calculation unit, used to calculate the ideal image coordinates of all pixels in each partition using the image correction model of each partition;

The remap matrix building unit is used to build the remap matrix of each partition according to the mapping relationship between the actual image coordinates and the ideal image coordinates of all the pixels in each partition.

In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims (8)

1. An image correction method for large field of view plane vision measurement, characterized by comprising: Acquire a checkerboard image of a plane to be measured; a checkerboard calibration plate is placed on the plane to be measured; Performing landmark point detection on the checkerboard image, extracting actual image coordinates of the landmark points and partitioning the checkerboard image; Setting ideal image coordinates for the marker points, and establishing a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points; Establishing a deep learning network model, and using the training database of each partition to train the deep learning network model for each partition, to generate a trained image correction model for each partition; The image correction model of each partition is used to calculate the ideal image coordinates of all pixels in each partition, and the remap matrix of each partition is calculated; specifically comprising: using the image correction model of each partition to calculate the ideal image coordinates of all pixels in each partition; according to the mapping relationship between the actual image coordinates and the ideal image coordinates of all pixels in each partition, constructing the remap matrix of each partition; Acquire a plane image to be measured and partition the plane image to be measured; Generate an undistorted front view of each partition of the plane to be measured using the remap matrix of each partition; The undistorted front views of the subareas of the plane to be measured are spliced to generate an undistorted front view of the image of the plane to be measured. 2. The image correction method according to claim 1, characterized in that the step of obtaining a checkerboard image of the plane to be measured specifically comprises: A camera equipped with a wide-angle lens is fixed above the plane field of view to be measured, and an LED light source is used to fill in the shooting area; The checkerboard calibration plate is placed on the plane to be measured, and the camera is controlled by a computer to capture the checkerboard image. 3. The image correction method according to claim 1, characterized in that the step of detecting the marker points on the checkerboard image, extracting the actual image coordinates of the marker points and partitioning the checkerboard image specifically comprises: Performing landmark point detection on the chessboard image, and extracting corner points of the chessboard as the landmark points; Extracting the actual image coordinates of the marker points; The checkerboard image is partitioned according to the actual image coordinate distribution of the marker points; the number of marker points contained in each partition is greater than or equal to 60. 4. The image correction method according to claim 1, characterized in that the step of setting the ideal image coordinates for the marker points and establishing a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points specifically comprises: According to the actual image coordinate distribution of the marker point and the goal of image correction, setting the ideal image coordinates for the marker point; A training database for each partition is constructed according to the actual image coordinates and ideal image coordinates of all the marker points in each partition. 5. An image correction system for large-field-of-view plane vision measurement, characterized by comprising: A checkerboard image acquisition module is used to acquire a checkerboard image of a plane to be measured; a checkerboard calibration plate is placed on the plane to be measured; A landmark point detection and partitioning module, used to perform landmark point detection on the checkerboard image, extract the actual image coordinates of the landmark points and partition the checkerboard image; A training database establishment module, used to set ideal image coordinates for the marker points, and establish a training database for each partition according to the actual image coordinates and the ideal image coordinates of the marker points; An image correction model establishment module is used to establish a deep learning network model, and use the training database of each partition to train the deep learning network model for each partition, so as to generate a trained image correction model for each partition; A remap matrix establishment module is used to calculate the ideal image coordinates of all pixels in each partition by using the image correction model of each partition, and calculate the remap matrix of each partition; the remap matrix establishment module specifically includes: an ideal image coordinate calculation unit, which is used to calculate the ideal image coordinates of all pixels in each partition by using the image correction model of each partition; a remap matrix establishment unit, which is used to construct the remap matrix of each partition according to the mapping relationship between the actual image coordinates and the ideal image coordinates of all pixels in each partition; A plane image acquisition module to be measured, used to acquire the plane image to be measured and divide the plane image to be measured into different zones; A partitioned undistorted front view generation module, used to generate an undistorted front view of each partition of the plane to be measured by using the remap matrix of each partition; The module for generating an undistorted front view of the plane to be measured is used to splice the undistorted front views of the various subareas of the plane to be measured to generate an undistorted front view of the image of the plane to be measured. 6. The image correction system according to claim 5, characterized in that the checkerboard image acquisition module specifically comprises: A camera setting unit, used to fix a camera equipped with a wide-angle lens above the plane field of view to be measured, and use an LED light source to fill in the shooting area; The checkerboard image acquisition unit is used to place the checkerboard calibration plate on the plane to be measured, and use a computer to control the camera to capture the checkerboard image. 7. The image correction system according to claim 5, characterized in that the landmark detection and partitioning module specifically comprises: A landmark point detection unit, used to perform landmark point detection on the chessboard image, and extract corner points of the chessboard as the landmark points; An actual image coordinate extraction unit, used to extract the actual image coordinates of the marker point; A partitioning unit is used to partition the checkerboard image according to the actual image coordinate distribution of the marker points; the number of marker points contained in each partition is greater than or equal to 60. 8. The image correction system according to claim 5, characterized in that the training database establishment module specifically comprises: An ideal image coordinate setting unit, used to set ideal image coordinates for the marker point according to the actual image coordinate distribution of the marker point and the target of image correction; The training database establishing unit is used to construct a training database for each partition according to the actual image coordinates and ideal image coordinates of all the marker points in each partition.